Detailed explanation and deployment of Redis master-slave replication and sentinel mode under Centos

This article will introduce the Redis master-slave replication and sentinel mode, and then set up the master-slave replication and sentinel modes with the help of different ports of a single host

The actual production environment requires different hosts to deploy in the same way. You only need to modify the IP address in the configuration file

The appendix is the automation code

preface

Master-slave replication: master-slave replication is mainly used to realize multi machine backup of data, load balancing for read operation and simple fault recovery
Disadvantages: the high availability of the primary node cannot be guaranteed, and the load balancing of master write is not solved

Sentinel mode: Based on master-slave replication, it is a distributed system used to monitor master-slave replication; Achieve automatic failover
Disadvantages:

1. The master-slave mode switching takes time and data will be lost, which does not solve the load balancing problem written by the master
2. Automatic failover cannot be performed on the slave node. If the slave node fails in the scenario of read-write separation, it will be unreadable, and the slave node needs additional monitoring and switching operations

Master-slave replication

1. Concept

The replication function of Redis allows users to copy data from one Redis server to other Redis servers. The former is the master node and the latter is the slave node

Data replication is unidirectional and can only be from master node to slave node; Data replication is divided into full synchronization and incremental synchronization

The master node and the slave node have a one to many relationship. The master node can have multiple slave nodes or no slave nodes, but the slave node can only have one master node. Redis's master-slave structure can adopt one master-slave or cascade structure

2. Replication process

Redis uses asynchronous replication by default, which is characterized by low latency and high performance. Master slave replication relies on three main mechanisms:

• When the connection between a master and a slave is normal, the master will send a series of command streams to keep the slave updated, so as to copy the changes of its own data set to the slave, including client writing, expiration or expulsion of key s, etc
• After the connection between the master and slave is disconnected, because of network problems or the master and slave realize that the connection times out, the slave reconnects to the master and attempts partial resynchronization: this means that it will try to obtain only the command flow lost during the disconnection
• When partial resynchronization cannot be performed, the slave will request full resynchronization. This involves a more complex process. For example, the master needs to create a snapshot of all data and send it to the slave, and then continuously send the command flow to the slave when the dataset changes

Redis identifies the exact version of a master dataset through a pair of given replication IDs and offset s.

• Replication ID: a large pseudo-random string that marks a given dataset
• Offset: when the master sends the copy stream to the slave, the offset of itself will increase according to the number of bytes of data sent; In other words, the offset records the modification of master data

When slave is connected to master, PSYNC command is used to send the old master replication ID they recorded and the processed offset; In this way, the master sends only the incremental data required by the slave. However, if there are not enough command backlog buffer records in the master buffer, or the slave references a nonexistent or expired Replication ID, full resynchronization will be performed

Full replication

Incremental replication

Sentinel mode

Redis sentinel is mainly used to manage multiple redis servers. It is a distributed monitoring redis master-slave server system, which can automatically cope with all kinds of failure events for failover without human intervention. Its four functions are as follows:

• Monitoring: Sentinel constantly checks whether the master and slave instances are working as expected
• Notification: Sentinel can notify the system administrator or other applications through an api that the monitored Redis instance has some problems
• Automatic failover: if a master node does not work as expected, Sentinel will start the failover process, promote a slave node to the master node, reconfigure other slave nodes to use the new master node, and the application using Redis service will also be notified of the new address when connecting
• Configuration provider: Sentinel provides service discovery for the client. Source: for a given service, the client connects to sentinels to find the address of the current master node. Sentinels will report the new address when a failover occurs

Redis Sentinel's distributed advantages:

1. When multiple sentinels agree that a master is no longer available, fault detection is performed. This significantly reduces the probability of error
2. Even if not all Sentinel are working, Sentinel can work normally. This feature makes the system very healthy

Redis multi sentry mode

1) Subjective offline
The Sentinel node regularly monitors the node status. Within the specified time (down after milliseconds), the Sentinel node sends a ping command to the redis server for a heartbeat check. If the reply is wrong, the Sentinel node will judge the server as "subjective offline". When more than half of Sentinel nodes judge the subjective offline of the Master, it is "objective offline"

2) Objective offline
It is only applicable to the Master server. Sentinel 1 will ask other sentinel nodes to judge the status of the Master server through corresponding commands when it finds that the Master server has failed. If more than half of sentinel nodes think that the primary server is down, Sentinel1 node determines that the primary service is "objective offline"
3) Vote
When the master server fails, Sentinel nodes will jointly elect a failover and notification through the Raft algorithm (election algorithm). Master node election:

• Filter out unhealthy (offline) slave nodes that do not reply to the sentry ping response.
• Select the highest priority configuration from the node in the configuration file. (replica priority, the default value is 100)
• Select the slave node with the largest copy offset, that is, the most complete copy.

Summary of the whole process:

Sentinel is responsible for monitoring the "health" status of the master and slave nodes. When the master node hangs up, an optimal slave node is automatically selected to switch to the master node. When the client connects to the Redis cluster, it will first connect to sentinel, query the address of the master node through sentinel, and then connect to the master node for data interaction. When the primary node fails, the client will ask sentinel for the address again, and sentinel will tell the client the latest primary node address. Therefore, the application can automatically complete the master-slave node switching without restarting.

Configuration process (single machine multi port)

The later deployment process is deployed in single machine multi port mode. If it is multi machine deployment, you only need to modify / etc / redis Conf and / etc / redis sentinel After setting the configuration items in the two files, restart the service through service redis restart and service redis sentinel restart

Redis server master-slave

1. install

    yum install redis -y
    # Soft connection
    ln -s /usr/local/redis/bin/* /usr/bin/
   

2. Modify redis Conf file

    # backups
    cp /etc/redis.conf /etc/redis.conf.backup
    # Modify / etc / redis The IP bindings in the conf file are consistent because they are the same machine 
    bind 127.0.0.1 172.16.30.16
    # Copy file
    cp /etc/redis.conf /etc/redis_6379.conf
    cp /etc/redis.conf /etc/redis_6380.conf
    cp /etc/redis.conf /etc/redis_6381.conf
    # Differentiated modification, master redis_6379.conf
    port 6379 
    dbfilename dump-6379.rdb 
    requirepass new2020 # password
    pidfile /var/run/redis_6379.pid
    logfile  /var/log/redis/redis_6379.log
    # From redis_6380.conf redis_6381.conf
    port 6380 # 6381
    dbfilename dump-6380.rdb # 6381
    requirepass new2020 # password
    pidfile /var/run/redis_6380.pid # 6381
    logfile  /var/log/redis/redis_6380.log  # 6381
   
    masterauth new2020
    replicaof 172.16.30.16 6379
   

3. start-up

    redis-server /etc/redis-6379.conf & # 6380 6381
   

Redis sentinel sentry

1. Modify redis sentinel conf

    # backups
    cp /etc/redis-sentinel.conf /etc/redis-sentinel.conf.backup
    # Configure the monitoring master node and password / etc / redis sentinel conf
    sentinel monitor mymaster 172.16.30.16 6379 2
    sentinel auth-pass mymaster new2020
    # Copy, sentinel mode is the way of election, so there are at least three sentinels
    cp /etc/redis-sentinel.conf /etc/redis-sentinel_26379.conf 
    cp /etc/redis-sentinel.conf /etc/redis-sentinel_26380.conf 
    cp /etc/redis-sentinel.conf /etc/redis-sentinel_26381.conf 
    # Modify / etc / redis Sentinel_ (26379 26380 26381). Conf port, pid, and log file
    port 26379 # 26380 26381
    pidfile "/var/run/redis-sentinel_26379.pid" # 26380 26381
    logfile "/var/log/redis/sentinel_26379.log" # 26380 26381
   

2. start-up

    redis-sentinel /etc/redis-sentinel_26379.conf & # 26380 26381
   

3. Check the log file and the effect of startup

    3918:X 11 Apr 2022 19:57:05.500 # Sentinel ID is 03c0c9665f6f6e977b07581f194dee5e8bd069a2
    3918:X 11 Apr 2022 19:57:05.501 # +monitor master mymaster 172.16.30.16 6379 quorum 2
    3918:X 11 Apr 2022 19:57:05.501 * +slave slave 127.0.0.1:6380 127.0.0.1 6380 @ mymaster 172.16.30.16 6379
    3918:X 11 Apr 2022 19:57:05.503 * +slave slave 127.0.0.1:6381 127.0.0.1 6381 @ mymaster 172.16.30.16 6379
   

   6379 port is master and has two slave ports
4. View sentinel information

    redis-cli -p 26379
    127.0.0.1:26379> info Sentinel
   
    # The following message appears
    sentinel_masters:1
    sentinel_tilt:0
    sentinel_running_scripts:0
    sentinel_scripts_queue_length:0
    sentinel_simulate_failure_flags:0
    master0:name=mymaster,status=ok,address=172.16.30.16:6379,slaves=2,sentinels=3 # Indicates that there are two slave and three sentinel
   

appendix

1,redis.conf configuration file remarks

# Whether the redis process runs as a daemon. Yes is yes and no is no (if it does not run as a daemon, it will occupy a terminal). 
daemonize no 
# Specify the PID file storage location of redis process 
pidfile /var/run/redis.pid 
# Port number of redis process 
port 6379 
#Whether to enable the protection mode. It is enabled by default. If the bind and password are not specified in the configuration. When redis is enabled, only external access will be denied. If the password and bind are enabled, they can be enabled. Otherwise, it is better to turn off and set it to no. 
protected-mode yes 
# Bound host address 
bind 127.0.0.1 
# After the client is idle for how long, close the connection. By default, this parameter is 0, that is, close this function 
timeout 300 
# redis log level. The available levels are debug verbose. notice. warning 
loglevel verbose 
# Log file output location. If the process runs as a daemon and the output file is set to stdout here, the log information will be output to /dev/null 
logfile stdout 
# Set the number of databases. The default is 0. You can use the select < dbid > command to specify the database id on the connection 
databases 16 
# Specifies how many times the data will be synchronized to the data file in how long 
save <seconds> <changes> 
# Specifies whether to compress the file when storing to the local database. The default value is yes, which means that storage is enabled 
rdbcompression yes 
# Specify the local database file name 
dbfilename dump.db 
# Specify local data storage location 
dir ./ 
# Specify the IP address and port of the master service when the local machine is a slave service. When redis is started, it will automatically synchronize data with the master; The old version was slaveof
replicaof <masterip> <masterport> 
# When password protection is set for the master, the slave service uses the password to connect to the master 
masterauth <master-password> 
# Set the redis connection password. If the connection password is configured, the client needs to provide the password through the auth < password > command when connecting to redis. It is closed by default 
requirepass footbared 
# Set the maximum number of customer connections at the same time. There is no limit by default. The number of clients that redis can connect to at the same time is the maximum file descriptor that redis can open. If maxclients 0 is set, it means there is no limit. When the number of client connections reaches the limit, redis will close the new connection and return the max number of clients reached error message to the client 
maxclients 128 
# Specify the maximum memory limit of redis. When redis starts, it will load data into memory. After reaching the maximum memory, redis will first try to clear the expired or about to expire keys. After this method is processed, the maximum memory setting is still reached, and the write operation can no longer be carried out, but the read operation can still be carried out. Redis's new vm mechanism will store the Key in memory and the Value in the swap area 
maxmemory<bytes> 
# Specify whether to log after each update operation. Redis writes data to disk asynchronously by default. If it is not enabled, it may cause data loss for a period of time during power failure. Because redis synchronizes data files according to the above save conditions, some data will only exist in memory for a period of time. The default is no. 
appendonly no 
# Specify the new log file name. The default is appendonly aof 
appendfilename appendonly.aof 
# Specify the conditions for updating the log. There are three optional parameters - no: it means that the operating system will synchronize the data cache to the disk (fast), always: it means that fsync() is called manually to write the data to the disk after each update operation (slow, safe), and everysec: it means to synchronize once per second (compromise, default); 
appendfsync everysec

2,redis-sentinel.conf configuration file remarks

The myid in the sentry's configuration file cannot be the same. If it is deleted and started, it will be automatically assigned

# Simplest configuration
sentinel monitor mymaster 127.0.0.1 6379 2 # Redis monitors a master node called mymaster. The address is 127.0.0.1, the port number is 6379, and there are two arbitration machines
sentinel down-after-milliseconds mymaster 5000 # 5000 milliseconds, 5 seconds, so if we can't receive a reply within this time, the main node will find it failed.
sentinel failover-timeout mymaster 6000 # If mysater still fails to start after 6 seconds, start failover  
sentinel parallel-syncs mymaster 1 # When performing failover, up to one slave server synchronizes the new master server at the same time
# pid and log
pidfile "/var/run/redis-sentinel.pid" # PID file storage location of the process 
logfile "/var/log/redis/sentinel.log" # log file output location

3. Single machine multi port automation script

#!/bin/bash

# Redis node private IP and Port, 6379 is master
REDIS_IP="172.16.30.16"
REDIS_PORT=$(echo {6379..6381})
REDIS_PASSWD="123456"
# Redis master node port and passwd 
REDIS_MASTER_PORT=6379
# Is Redis the master node? 1 is the master node and 0 is the slave node
REDIS_IS_MASTER=1 
# Redis slave node Port
REDIS_SLAVE_PORT=$(echo {6380..6381})
# Redis Sentinel Port
REDIS_SENTINEL_PORT=$(echo {26379..26381})
# Whether Redis is a master-slave. 1 is a master-slave, and 0 is a single machine
REDIS_SLAVE_ENABLE=0 
# Is Redis sentinel? 1 is sentinel mode, 0 is not
REDIS_SENTINEL_ENABLE=0
# Number of sentinel arbitration machines
REDIS_SLAVE_NUM=2

echo "begin"
function initRedis()
{
    yum install redis -y
    if [ $? -ne 0 ]; then
        echo "yum install redis failed."
        return 1
    fi
    ln -s /usr/local/redis/bin/* /usr/bin/
    cp /etc/redis.conf /etc/redis.conf.backup 
    cp /etc/redis-sentinel.conf /etc/redis-sentinel.conf.backup
    # General configuration
    for redis_port in $REDIS_PORT
    do
        echo $redis_port
        cp -f /etc/redis.conf /etc/redis_$redis_port.conf
        
        sed -i 's/appendfsync everysec/appendfsync no/g' /etc/redis_$redis_port.conf
        sed -i 's/^save /#save /g' /etc/redis_$redis_port.conf
        sed -i "s/^bind.*/bind 127.0.0.1 ${REDIS_IP}/g" /etc/redis_$redis_port.conf
        sed -i "s/^port.*/port ${redis_port}/g" /etc/redis_$redis_port.conf
        sed -i "s#^pidfile.*#pidfile  /var/run/redis_${redis_port}.pid#g" /etc/redis_$redis_port.conf
        echo $redis_port
        sed -i "s/^dbfilename.*/dbfilename  dump_${redis_port}.rdb/g" /etc/redis_$redis_port.conf
        sed -i "s#^logfile.*#logfile  /var/log/redis/redis_${redis_port}.log#g" /etc/redis_$redis_port.conf
        # If the password is different, you can configure it here
        if [ `grep "^requirepass " /etc/redis_$redis_port.conf | wc -l` -eq 0 ]; then 
            echo "requirepass ${REDIS_PASSWD}" >> /etc/redis_$redis_port.conf
        else
            sed -i "s/^requirepass.*/requirepass ${REDIS_PASSWD}/g" /etc/redis_$redis_port.conf
        fi 
    done
    if [ $REDIS_SLAVE_ENABLE -eq 1 ]; then
        # slave configuration
        for redis_port in $REDIS_SLAVE_PORT
        do
            if [ `grep "^masterauth " /etc/redis_$redis_port.conf | wc -l` -eq 0 ]; then 
                echo "masterauth ${REDIS_PASSWD}" >> /etc/redis_$redis_port.conf
            else
                sed -i "s/^masterauth.*/masterauth ${REDIS_PASSWD}/g" /etc/redis_$redis_port.conf
            fi 
            if [ `grep "^replicaof " /etc/redis_$redis_port.conf | wc -l` -eq 0 ]; then 
                echo "replicaof ${REDIS_IP} ${REDIS_MASTER_PORT}" >> /etc/redis_$redis_port.conf
            else
                sed -i "s/^replicaof.*/replicaof ${REDIS_IP} ${REDIS_MASTER_PORT}/g" /etc/redis_$redis_port.conf
            fi
        done

        for redis_port in $REDIS_PORT
        do
            redis-server /etc/redis_$redis_port.conf &
            if [ $? -ne 0 ];then
                echo "redis $redis_port start failed!"
                return 1
            fi 
        done
        # sentinel configuration
        if [ $REDIS_SENTINEL_ENABLE -eq 1 ]; then
            cat > /etc/redis-sentinel.conf << EOF
port 26380
daemonize no
pidfile "/var/run/redis-sentinel.pid"
logfile "/var/log/redis/sentinel.log"

sentinel deny-scripts-reconfig yes
sentinel monitor mymaster ${REDIS_IP} ${REDIS_MASTER_PORT} ${REDIS_SLAVE_NUM}
sentinel down-after-milliseconds mymaster 5000
sentinel failover-timeout mymaster 6000
sentinel parallel-syncs mymaster 1

sentinel auth-pass mymaster ${REDIS_PASSWD}
EOF
            for port in $REDIS_SENTINEL_PORT
            do 
                cp -f /etc/redis-sentinel.conf /etc/redis-sentinel_$port.conf
                sed -i "s/^port.*/port ${port}/g" /etc/redis-sentinel_$port.conf
                sed -i "s#^pidfile \"/var/run/redis-sentinel.pid\"#pidfile \"/var/run/redis-sentinel_$port.pid\"#g" /etc/redis-sentinel_$port.conf
                sed -i "s#^logfile \"/var/log/redis/sentinel.log\"#logfile \"/var/log/redis/sentinel_$port.log\"#g" /etc/redis-sentinel_$port.conf
                
                redis-sentinel /etc/redis-sentinel_$port.conf &
                if [ $? -ne 0 ];then
                    echo "redis-sentinel $port start failed!"
                    return 1
                fi 
                sleep 30 # Each sentinels needs to be added at least 30 seconds apart
            done 
        fi
    else 
        redis-server /etc/redis_$REDIS_MASTER_PORT.conf &
        if  [ $? -ne 0 ];then
            echo "redis-server start fail!"
        fi
    fi
    echo "end"
}

initRedis

4. Same configuration script for multiple machines

#!/bin/bash

# Redis IP,port,passwd
REDIS_IP="172.16.30.16"
REDIS_PORT=6379
REDIS_PASSWD="123456" # Password of current node
# Is Redis the master node? 1 is the master node and 0 is the slave node
REDIS_IS_MASTER=1 
# Redis master node IP and PORT
REDIS_MASTER_IP="172.16.30.16"
REDIS_MASTER_PORT=6379
# Redis Sentinel Port
REDIS_SENTINEL_PORT=26379
# Whether Redis is a master-slave. 1 is a master-slave, and 0 is a single machine
REDIS_SLAVE_ENABLE=0 
# Is Redis sentinel? 1 is sentinel mode, 0 is not
REDIS_SENTINEL_ENABLE=0 
# Number of sentinel arbitration machines
REDIS_SLAVE_NUM=2
# Redis server or sentinel, 1 is redis server and 0 is redis Sentinel
REDIS_SERVER = 1 

func redisServer(){
    cp /etc/redis.conf /etc/redis.conf.backup 
    ln -s /usr/local/redis/bin/* /usr/bin/
    sed -i 's/appendfsync everysec/appendfsync no/g' /etc/redis.conf
    sed -i 's/^save /#save /g' /etc/redis.conf
    sed -i "s/^bind.*/bind 127.0.0.1 ${REDIS_IP}/g" /etc/redis.conf
    sed -i "s/^port.*/port ${REDIS_PORT}/g" /etc/redis.conf
    if [ `grep "^requirepass " /etc/redis.conf | wc -l` -eq 0 ]; then 
        echo "requirepass ${REDIS_PASSWD}" >> /etc/redis.conf
    else
        sed -i "s/^requirepass.*/requirepass ${REDIS_PASSWD}" /etc/redis.conf
    fi 
    if [ $REDIS_SLAVE_ENABLE -eq 1 ]; then
        if [ $REDIS_IS_MASTER -ne 1 ]; then 
            if [ `grep "^masterauth " /etc/redis.conf | wc -l` -eq 0 ]; then 
                echo "masterauth ${REDIS_PASSWD}" >> /etc/redis.conf
            else
                sed -i "s/^masterauth.*/masterauth ${REDIS_PASSWD}" /etc/redis.conf
            fi 
            if [ `grep "^replicaof " /etc/redis.conf | wc -l` -eq 0 ]; then 
                echo "replicaof ${REDIS_MASTER_IP} ${REDIS_MASTER_PORT}" >> /etc/redis.conf
            else
                sed -i "s/^replicaof.*/replicaof ${REDIS_MASTER_IP} ${REDIS_MASTER_PORT}" /etc/redis.conf
            fi 
        fi 
        service redis restart 
        if [ $? -ne 0 ]; then
            echo "redis restart failed!"
            return 1
        fi 
    else 
        service redis start 
    fi
}

func sentinel(){
    if [ $REDIS_SLAVE_ENABLE -eq 1 ]; then
        if [ $REDIS_SENTINEL_ENABLE -eq 1 ]; then
            cp /etc/redis-sentinel.conf /etc/redis-sentinel.conf.backup
            sed -i "s/^sentinel myid/#sentinel myid/g" /etc/redis-sentinel.conf
            sed -i "s/^port.*/port ${REDIS_SENTINEL_PORT}/g" /etc/redis-sentinel.conf
            sed -i "s/^sentinel monitor.*/sentinel monitor mymaster ${REDIS_MASTER_IP} ${REDIS_MASTER_PORT} ${REDIS_SLAVE_NUM}/g" /etc/redis-sentinel.conf
            sed -i "s/^sentinel auth-pass mymaster/#sentinel auth-pass mymaster/g" /etc/redis-sentinel.conf
            sed -i "s/^sentinel down-after-milliseconds mymaster/#sentinel down-after-milliseconds mymaster/g" /etc/redis-sentinel.conf
            sed -i "s/^sentinel failover-timeout mymaster/#sentinel failover-timeout mymaster/g" /etc/redis-sentinel.conf
            sed -i "s/^sentinel parallel-syncs mymaster/#sentinel parallel-syncs mymaster/g" /etc/redis-sentinel.conf
            
            cat >> /etc/redis-sentinel.conf <<EOF
asentinel auth-pass mymaster ${REDIS_PASSWD}
sentinel down-after-milliseconds mymaster 5000
sentinel failover-timeout mymaster 60000
sentinel parallel-syncs mymaster 1
EOF
            # redis-sentinel /etc/redis-sentinel.conf
            # service uses / etc / redis sentinel.com by default Conf start
            service redis-sentinel restart
            if [ $? -ne 0 ];then
                echo "redis-sentinel restart failed!"
                return 1
            fi 
        fi
    fi
}

function initRedis()
{
    yum install redis -y
    if [ $? -ne 0 ]; then
        echo "yum install redis failed."
        return 1
    fi
    
    ln -s /data/redis /usr/local/redis

    if [ $REDIS_SERVER -eq 1 ]; then
        redisServer
    else
        sentinel
    fi
}

## Execution function
initRedis

reference resources

[1] Redis master-slave mode setup
[2] Redis replication
[3] Redis sentry - achieve high availability of redis