Linux Process Management Tool Supervisor

Linux / hypervisor installation and management tools
Supervisor(http://supervisord.org )It is a client/server service developed in Python. It is a process management tool under Linux/Unix system and does not support Windows system. It can easily monitor, start, stop and restart one or more processes. For a process managed by Supervisor, when a process is accidentally killed, Supervisor will automatically pull it up after listening to the death of the process. It is very convenient to achieve the function of automatic recovery of the process, and there is no need to write your own shell script to control it.

Because the Supervisor is developed in Python, check whether Python 2.0 is installed in the system before installation 4 or above. The following is centos7 6,Python2. In the environment of version 7.5, the installation and configuration steps of Supervisor are introduced:

Experimental environment

system platform

cat /etc/redhat-release
CentOS Linux release 7.6.1810 (Core)

Python version

python -V
Python 2.7.5

If the version of python is lower than 2.6, please upgrade. An installation of python 3.0 is posted below 6.8 installation example of

yum install zlib-devel bzip2-devel openssl-devel ncurses-devel sqlite-devel readline-devel tk-devel gdbm-devel db4-devel libpcap-devel xz-devel -y libffi-devel
wget https://www.python.org/ftp/python/3.6.8/Python-3.6.8.tar.xz
tar xf Python-3.6.8.tar.xz
cd Python-3.6.8
./configure --prefix=/usr/local/python368
make && make install
echo 'export PATH=/usr/local/python368/bin:$PATH' >> /etc/profile
source /etc/profile
python3 -V

Install Supervisor

There are many ways to install Supervisor. The following three methods are introduced, and the third one I use here

1,easy_install install supervisor

Install Python package management tool (easy_install)
easy_install is a command in the setuptools package. Use easy_install is actually calling setuptools to complete the installation of modules, so you can install setuptools:

wget https://pypi.io/packages/source/s/setuptools/setuptools-33.1.1.zip
unzip setuptools-33.1.1.zip
cd setuptools-33.1.1
python setup.py install

easy_install supervisor

2. pip install supervisor

Use pip to install. The prerequisite is to ensure that the PIP version is greater than 2.6

pip install supervisor

3. Install supervisor using Yum EPEL release

yum install -y epel-release && yum install -y supervisor

superviso command

After supervisor is installed, three execution programs will be generated: supervisortd, supervisorctl and echo_supervisord_conf:

Supervisor RTD: used to manage the supervisor service itself
supervisorctl: used to manage the services we need to delegate to the superviso rctl tool
echo_ superviso rd_ Conf: configuration file used to generate Supervisor
supervisor's daemon service (used to receive process management commands)
Client (used to communicate with the daemon and send instructions to manage the process)

[root@Jumpserver /]# which supervisord
/bin/supervisord
[root@Jumpserver /]# which supervisorctl
/bin/supervisorctl
[root@Jumpserver /]# which echo_supervisord_conf
/bin/echo_supervisord_conf

Configure Supervisor

By running echo_ supervisord_ The conf program generates the initialization configuration file of supervisor

If you use yum to install, this step is omitted and you can modify the configuration file directly

mkdir /etc/supervisord.d
echo_supervisord_conf > /etc/supervisord.conf

Modify profile

There are many contents in the configuration file of supervisor, but many of them can be used without modification. I only modified the following two items here

#Modify the mode of the socket file. The default is 0700
sed -i 's/;chmod=0700/chmod=0766/g' /etc/supervisord.conf   

#Add the following two lines at the end of the configuration file to include the / etc / Supervisor directory
sed -i '$a [include] \
files = /etc/supervisord.d/*.conf' /etc/supervisord.conf

Write processes that need to be managed by the Supervisor

The Supervisor can only manage non daemon processes. For example, redis runs in the foreground by default, and Tomcat is actually startup sh shutdown. SH to call Catalina SH runs in the background. Catalina is the default SH is a program running in the foreground and cannot manage non dameon processes like Nginx

Tomcat is managed by Supervisor

Tomcat is installed as follows:

wget http://us.mirrors.quenda.co/apache/tomcat/tomcat-8/v8.5.47/bin/apache-tomcat-8.5.47.tar.gz
yum install java-1.8.0-openjdk.x86_64 java-1.8.0-openjdk-devel.x86_64 -y
tar xf apache-tomcat-8.5.47.tar.gz  -C /usr/local/
mv /usr/local/apache-tomcat-8.5.47 /usr/local/tomcat

If you want our application to be managed by the supervisor, you need to write a configuration file in the / etc / Supervisor directory. The Tomcat case is as follows:

vim /etc/supervisord.d/tomcat.conf
[program:tomcat] #Program unique name
directory=/usr/local/tomcat #Program path
command=/usr/local/tomcat/bin/catalina.sh run #Command to run the program
autostart=true #Is tomcat started after supervisor is started
startsecs=10 #If there is no abnormal exit after 10 seconds of startup, it means that the process is started normally. The default is 1 second
autorestart=true #The program will restart automatically after exiting. The optional values are: [unexpected,true,false]. The default value is unexpected, which means that the process will restart only after being killed accidentally; It means that if the process is not closed by supervisor, it is considered to be closed improperly. Supervisor will start the process again and can only use the supervisor CTL to close, start and restart
startretries=3 #The number of automatic retries after startup failure. The default is 3
user=root #Which user is used to start the process? The default is root
priority=999 #The process startup priority is 999 by default. If supervisor needs to manage multiple processes, the one with a small value will be started first
stopsignal=INT
redirect_stderr=true #Redirect stderr to stdout standard output. The default is false
stdout_logfile_maxbytes=200MB #stdout standard output log file size. When the log file size reaches 200M, it will be cut, and the cut log file will be marked as Catalina out1,catalina. out2,catalina. out3...， Default 50MB
stdout_logfile_backups = 100 #stdout standard outputs the number of log file backups. Save 100 200MB log files. If there are more than 100, the old ones will be deleted. The default is 10. Save 10
stdout_logfile=/usr/local/tomcat/logs/catalina.out #Standard log output location. If the output location does not exist, it will fail to start
stopasgroup=false #The default is false. When a process is killed, whether to send a stop signal to this process group, including child processes
killasgroup=false #The default value is false. It sends a kill signal to the process group, including child processes

Start process
After using supervisor management to start, when you use / usr / local / Tomcat / shutdown When sh or kill $PID, the supervisor will consider it as an accidental shutdown and will automatically pull up the process again, unless it is closed by using the supervisor command

#Supervisor start
supervisord -c /etc/supervisord.conf                    #Start the supervisor process. We set the autostart=true parameter in the configuration file. When supervisor starts, tomcat will also start
ps -ef|grep java                                        #Check whether tomcat is started

Program management

supervisorctl status tomcat                             #tomcat status
supervisorctl stop tomcat                               #Stop tomcat
supervisorctl start tomcat                              #Start tomcat
supervisorctl restart tomcat                            #Restart tomcat
supervisorctl reoload tomcat                            #Heavy tomcat

Redis is managed by the Supervisor

By default, redis does not add the daemon yes parameter to the configuration file, but it is started by the foreground, so it can also be managed by our Supervisor
The redis configuration file is as follows:

cat redis6001.conf
port 6001
bind 192.168.31.230
protected-mode yes
pidfile "/usr/local/redis/run/redis6001.pid"
loglevel notice
logfile "/usr/local/redis/logs/redis6001.log"
save 900 1
stop-writes-on-bgsave-error yes
rdbcompression yes
rdbchecksum  yes
dbfilename dump.rdb
dir "/usr/local/redis/data/rdb/"
timeout 0
tcp-keepalive 300

Write a case where redis is managed by the Supervisor

vim /etc/supervisord.d/redis.conf
[program:redis]
directory=/usr/local/redis
command=/usr/local/redis/bin/redis-server /usr/local/redis/etc/redis6001.conf
autostart=true
startsecs=10
autorestart=true
startretries=3
user=root
priority=999
stopsignal=INT
redirect_stderr=true
stdout_logfile_maxbytes=200MB
stdout_logfile_backups = 100
stdout_logfile=/usr/local/redis/logs/redis6001.log
stopasgroup=false
killasgroup=false

Start redis with super

#Close tomcat
supervisorctl stop tomcat
tomcat: stopped

#Kill Supervisor
ps -ef|grep supervisord
root     26927     1  0 10:47 ?        00:00:00 /usr/bin/python /bin/supervisord -c /etc/supervisord.conf
root     27549 27402  0 11:07 pts/2    00:00:00 grep --color=auto super
kill -9 26927

#Restart supervisor to reload the configuration file. Supervisor will pull redis and tomcat together by default
supervisord -c /etc/supervisord.conf

Program management

supervisorctl status redis                              #redis status
supervisorctl stop redis                                #Stop redis
supervisorctl start redis                               #Start redis
supervisorctl restart reids                             #Restart redis
supervisorctl reoload redis                             #Reload redis

Program management

supervisorctl status all                            #View all process status
supervisorctl stop   all                            #Stop all processes
supervisorctl start  all                            #Start all processes
supervisorctl restart all                           #Restart all processes
supervisorctl reoload all                           #Reload all processes

Hypervisord enable startup configuration

vim /usr/lib/systemd/system/supervisord.service
[Unit]
Description=Process Monitoring and Control Daemon
After=rc-local.service nss-user-lookup.target

[Service]
Type=forking
ExecStart=/usr/bin/supervisord -c /etc/supervisord.conf

[Install]
WantedBy=multi-user.target

systemctl enable supervisord
systemctl is-enabled supervisord

I haven't shared resource benefits with you for a while. I looked at my own folder and sorted out some Python learning materials that I think are better. I believe this set of information can be helpful to your advanced senior engineer

Learning tools