Supervisor usage details

Supervisor usage details

1. Introduction to supervisor

Supervisor is a set of general process management programs developed in Python. It can turn an ordinary command line process into a background daemon, monitor the process status, and automatically restart when it exits abnormally. It starts these managed processes as sub-processes of the supervisor through fork/exec, so as long as the supervisor's configuration file, the path to the executable file of the process to be managed can be written. It also realizes that when the child process hangs, the parent process can accurately obtain the information of the child process hanging, and can choose whether to start and alarm by itself. supervisor also provides a function to set a non-root user for supervisord or each child process, and this user can manage its corresponding process.

Note: This article takes centos7 as an example, supervisor version 3.4.0.

2. supervisor installation

  1. After configuring the yum source, you can install it directly

    yum install supervisor
    
  2. Debian/Ubuntu can be installed via apt

    apt-get install supervisor
    
  3. pip install

    pip install supervisor
    
  4. easy_install install

    easy_install supervisor
    

3. supervisor use

supervisor configuration file: /etc/supervisord.conf

Note: The supervisor's configuration file is incomplete by default, but in most default cases, the basic functions mentioned above have been met.

Subprocess configuration file path: /etc/supervisord.d/

Note: The default subprocess configuration file is in ini format, which can be modified in the supervisor main configuration file.

Fourth, the configuration file description

supervisor.conf configuration file description:

[unix_http_server]
file=/tmp/supervisor.sock   ;UNIX socket document, supervisorctl Will be used
;chmod=0700                 ;socket document mode,Default is 0700
;chown=nobody:nogroup       ;socket document owner,Format: uid:gid
 
;[inet_http_server]         ;HTTP server, providing web management interface
;port=127.0.0.1:9001        ;Web Admin running in the background IP and port, if open to the public network, you need to pay attention to security
;username=user              ;Username for logging in to the management background
;password=123               ;Password to log in to the management background
 
[supervisord]
logfile=/tmp/supervisord.log ;log file, default is $CWD/supervisord.log
logfile_maxbytes=50MB        ;log file size, exceeding the rotate,Default 50 MB,If set to 0, it means unlimited size
logfile_backups=10           ;The default number of log files to keep backups is 10, if set to 0, it means no backup.
loglevel=info                ;log level, default info,other: debug,warn,trace
pidfile=/tmp/supervisord.pid ;pid document
nodaemon=false               ;Whether to start in the foreground, the default is false,i.e. with daemon way to start
minfds=1024                  ;The minimum number of file descriptors that can be opened, default 1024
minprocs=200                 ;The minimum number of processes that can be opened, the default is 200
 
[supervisorctl]
serverurl=unix:///tmp/supervisor.sock ; connect supervisord through UNIX socket, the path is consistent with the file in the unix_http_server part
;serverurl=http://127.0.0.1:9001 ; connect supervisord via HTTP
 
; [program:xx]is the managed process configuration parameter, xx is the name of the process
[program:xx]
command=/opt/apache-tomcat-8.0.35/bin/catalina.sh run  ; program start command
autostart=true       ; exist supervisord Automatically start at startup
startsecs=10         ; If there is no abnormal exit after 10 seconds of startup, it means that the process starts normally. The default is 1 second.
autorestart=true     ; Automatic restart after program exit,Optional values:[unexpected,true,false],The default is unexpected,Indicates that the process was killed unexpectedly before restarting
startretries=3       ; The number of automatic retries for startup failure, the default is 3
user=tomcat          ; Which user to use to start the process, the default is root
priority=999         ; Process startup priority, the default is 999, the smaller the value is, the priority is to start
redirect_stderr=true ; Bundle stderr redirect to stdout,default false
stdout_logfile_maxbytes=20MB  ; stdout log file size, default 50 MB
stdout_logfile_backups = 20   ; stdout The number of log file backups, the default is 10
; stdout log file, it should be noted that when the specified directory does not exist, it cannot be started normally, so you need to manually create the directory ( supervisord A log file is automatically created)
stdout_logfile=/opt/apache-tomcat-8.0.35/logs/catalina.out
stopasgroup=false     ;The default is false,Whether to send messages to this process group when the process is killed stop Signals, including child processes
killasgroup=false     ;The default is false,send to process group kill Signals, including child processes
 
;Include other configuration files
[include]
files = relative/directory/*.ini    ;You can specify one or more.ini end configuration file

Subprocess configuration file description:

Write a configuration file for the child process (program) that needs to be managed, put it in the /etc/supervisor.d/ directory, and use .ini as the extension (the configuration file of each process can be split separately or related scripts can be added put together). For example, an option group (/etc/supervisord.d/test.conf) that arbitrarily defines a script-related project name:

#Item name
[program:blog]
#script directory
directory=/opt/bin
#script execution command
command=/usr/bin/python /opt/bin/test.py

#Whether the supervisor starts at the same time, the default is True
autostart=true
#When the program exit s, the program will not restart automatically. The default is unexpected. To set the automatic restart after the child process hangs, there are three options, false,unexpected and true. If it is false, it will not be restarted under any circumstances. If it is unexpected, only when the exit code of the process is not defined in the following exitcodes
autorestart=false
#This option is how many seconds after the child process is started, if the status is running at this time, we consider the start is successful. Default is 1
startsecs=1

#The user the script runs as 
user = test

#log output 
stderr_logfile=/tmp/blog_stderr.log 
stdout_logfile=/tmp/blog_stdout.log 
#Redirect stderr to stdout, default false
redirect_stderr = true
#stdout log file size, default 50MB
stdout_logfile_maxbytes = 20MB
#Number of stdout log file backups
stdout_logfile_backups = 20
Example of subprocess configuration:
#Description same as above
[program:test] 
directory=/opt/bin 
command=/opt/bin/test
autostart=true 
autorestart=false 
stderr_logfile=/tmp/test_stderr.log 
stdout_logfile=/tmp/test_stdout.log 
#user = test  

Five, supervisor command description

Common commands
supervisorctl status        //View the status of all processes
supervisorctl stop es       //stop es
supervisorctl start es      //start es
supervisorctl restart       //restart es
supervisorctl update        //After the configuration file is modified, use this command to load the new configuration
supervisorctl reload        //Restart all programs in the configuration

Note: Replace es with all to manage all processes in the configuration. Enter supervisorctl directly to enter the shell interface of supervisorctl. At this time, the above command can be used directly without supervisorctl.

Precautions

Start supervisord before using the supervisor process management command, otherwise the program will report an error.
Start with the command supervisord -c /etc/supervisord.conf.
If centos7:

systemctl start supervisord.service     //Start supervisor and load default configuration file
systemctl enable supervisord.service    //Add supervisor to startup items
common problem
  1. unix:///var/run/supervisor.sock no such file
    Description of the problem: The error reported by using supervisorctl directly without starting the service after installing the supervisor
    Solution: supervisord -c /etc/supervisord.conf

  2. The process specified in command is up, but the supervisor keeps restarting
    Description of the problem: The startup method in the command is background startup, which causes the pid to not be recognized, and then restarts continuously. Here, elasticsearch is used, and the command specifies $path/bin/elasticsearch -d
    Solution: Supervisor cannot detect the pid of the background startup process, and the supervisor itself is a background startup daemon, so don't worry about this

  3. Multiple supervisord services are started, resulting in failure to shut down the service normally
    Description of the problem: Before running supervisord -c /etc/supervisord.conf, running supervisord -c /etc/supervisord.d/xx.conf directly caused some processes to be managed by multiple supervisors and could not be shut down normally.
    Solution: Use ps -fe | grep supervisord to view all started supervisord services and kill related processes.

 

Tags: Python Celery

Posted by MesaFloyd on Thu, 12 May 2022 04:36:47 +0300