start_kafka.sh
我有一台安装了 Apache Kafka 的 Ubuntu 16.04 机器。目前,我可以使用包含以下内容的脚本使其完美运行:
JMX_PORT=17264 KAFKA_HEAP_OPTS="-Xms1024M -Xmx3072M" /home/kafka/kafka_2.11-0.10.1.0/bin/kafka-server-start.sh -daemon /home/kafka/kafka_2.11-0.10.1.0/config/server.properties
现在,我想使用supervisor
在进程失败时自动重新启动并在重新启动机器后立即启动。问题是我无法启动supervisor
Kafka。
我supervisor
使用pip
以下位置安装了此配置文件/etc/supervisord.conf
:
; Supervisor config file.
;
; For more information on the config file, please see:
; http://supervisord.org/configuration.html
[unix_http_server]
file=/tmp/supervisor.sock ; (the path to the socket file)
[supervisord]
logfile=/tmp/supervisord.log ; (main log file;default $CWD/supervisord.log)
logfile_maxbytes=50MB ; (max main logfile bytes b4 rotation;default 50MB)
logfile_backups=10 ; (num of main logfile rotation backups;default 10)
loglevel=info ; (log level;default info; others: debug,warn,trace)
pidfile=/tmp/supervisord.pid ; (supervisord pidfile;default supervisord.pid)
nodaemon=false ; (start in foreground if true;default false)
minfds=1024 ; (min. avail startup file descriptors;default 1024)
minprocs=200 ; (min. avail process descriptors;default 200)
; the below section must remain in the config file for RPC
; (supervisorctl/web interface) to work, additional interfaces may be
; added by defining them in separate rpcinterface: sections
[rpcinterface:supervisor]
supervisor.rpcinterface_factory = supervisor.rpcinterface:make_main_rpcinterface
[supervisorctl]
serverurl=unix:///tmp/supervisor.sock ; use a unix:// URL for a unix socket
[program:kafka]
command=/home/kafka/kafka_2.11-0.10.1.0/start_kafka.sh ; the program (relative uses PATH, can take args)
;process_name=%(program_name)s ; process_name expr (default %(program_name)s)
startsecs=10 ; # of secs prog must stay up to be running (def. 1)
startretries=3 ; max # of serial start failures when starting (default 3)
;autorestart=unexpected ; when to restart if exited after running (def: unexpected)
;exitcodes=0,2 ; 'expected' exit codes used with autorestart (default 0,2)
stopsignal=TERM ; signal used to kill process (default TERM)
stopwaitsecs=180 ; max num secs to wait b4 SIGKILL (default 10)
stdout_logfile=NONE ; stdout log path, NONE for none; default AUTO
;environment=A="1",B="2" ; process environment additions (def no adds)
当我尝试启动 Kafka 时,出现以下错误:
# supervisorctl start kafka
kafka: ERROR (spawn error)
主管日志(在/tmp/supervisord.log
)包含以下内容:
2017-01-23 22:10:24,532 INFO spawned: 'kafka' with pid 21311
2017-01-23 22:10:24,536 INFO exited: kafka (exit status 127; not expected)
2017-01-23 22:10:25,542 INFO spawned: 'kafka' with pid 21312
2017-01-23 22:10:25,559 INFO exited: kafka (exit status 127; not expected)
2017-01-23 22:10:27,562 INFO spawned: 'kafka' with pid 21313
2017-01-23 22:10:27,567 INFO exited: kafka (exit status 127; not expected)
2017-01-23 22:10:30,571 INFO spawned: 'kafka' with pid 21314
2017-01-23 22:10:30,576 INFO exited: kafka (exit status 127; not expected)
2017-01-23 22:10:31,578 INFO gave up: kafka entered FATAL state, too many start retries too quickly
必须说,我已经尝试删除标志-daemon
并start_kafka.sh
使用supervisor
但没有成功。
有人知道发生了什么事吗?
答案1
我最终通过两处改变让主管与 Kafka 一起工作:
- 部署 Kafka 时无需
-daemon
标志,因为主管需要非守护进程来管理 - 在主管配置文件中明确定义 Java 路径
这是工作配置:
启动kafka
JMX_PORT=17264 KAFKA_HEAP_OPTS="-Xms1024M -Xmx3072M" /home/kafka/kafka_2.11-0.10.1.0/bin/kafka-server-start.sh /home/kafka/kafka_2.11-0.10.1.0/config/server.properties
监控配置文件
[unix_http_server]
file=/var/run/supervisor.sock ; (the path to the socket file)
chmod=0700 ; sockef file mode (default 0700)
[supervisord]
logfile=/var/log/supervisor/supervisord.log ; (main log file;default $CWD/supervisord.log)
pidfile=/var/run/supervisord.pid ; (supervisord pidfile;default supervisord.pid)
childlogdir=/var/log/supervisor ; ('AUTO' child log dir, default $TEMP)
; the below section must remain in the config file for RPC
; (supervisorctl/web interface) to work, additional interfaces may be
; added by defining them in separate rpcinterface: sections
[rpcinterface:supervisor]
supervisor.rpcinterface_factory = supervisor.rpcinterface:make_main_rpcinterface
[supervisorctl]
serverurl=unix:///var/run/supervisor.sock ; use a unix:// URL for a unix socket
; The [include] section can just contain the "files" setting. This
; setting can list multiple files (separated by whitespace or
; newlines). It can also contain wildcards. The filenames are
; interpreted as relative to this file. Included files *cannot*
; include files themselves.
[include]
files = /etc/supervisor/conf.d/*.conf
[program:kafka]
command=/home/kafka/kafka_2.11-0.10.1.0/start_kafka.sh
directory=/home/kafka/kafka_2.11-0.10.1.0
user=root
autostart=true
autorestart=true
stdout_logfile=/var/log/kafka/stdout.log
stderr_logfile=/var/log/kafka/stderr.log
environment = JAVA_HOME=/usr/lib/jvm/java-8-oracle