I am working with supervisord and I am facing issues in automatic restart of supervisord programs.
##
supervisord.conf
[supervisord]
logfile=/dev/null
pidfile=/tmp/supervisord.pid
nodaemon=true
[unix_http_server]
file = /tmp/supervisor.sock
[rpcinterface:supervisor]
supervisor.rpcinterface_factory = supervisor.rpcinterface:make_main_rpcinterface
[program:worker]
command=./manage.py rq worker %(ENV_QUEUES)s
process_name=%(program_name)s-%(process_num)s
numprocs=%(ENV_WORKERS_COUNT)s
directory=/app
stopsignal=TERM
autostart=true
autorestart=true
startsecs=300
stdout_logfile=/dev/stdout
stdout_logfile_maxbytes=0
stderr_logfile=/dev/stderr
stderr_logfile_maxbytes=0
[eventlistener:worker_healthcheck]
autorestart=true
serverurl=AUTO
command=./manage.py rq healthcheck
stdout_logfile=/dev/stdout
stdout_logfile_maxbytes=0
stderr_logfile=/dev/stderr
stderr_logfile_maxbytes=0
events=TICK_60
## Supervisorctl Status
abc@abc-adhocworker-c89d9667b-9lqbd:/app$ exec supervisorctl -c worker.conf status
worker:worker-0 FATAL Exited too quickly (process log may have details)
worker:worker-1 FATAL Exited too quickly (process log may have details)
worker_healthcheck RUNNING pid 14, uptime 7:43:34
Status shows they are in fatal state but not recovering automatically even I've configured autorestart=true
.
## What else do i need to make sure automatic recovery/restart of programs in case of errors ?
Is it possible that restart is relying on healthcheck eventlistener
and it might be sending false signal i.e. everything is good and thus automatic restart not working for programs ?
My supervisord.conf doesn't have supervisorctl section so not sure if it is added then it can trigger auto restart correctly as without that supervisorctl status
might be failing for supervisord programs ?
Asked by SRJ
(111 rep)
Feb 3, 2023, 01:02 AM