Interviewer: talk about redis6 Three ways to build cluster in version 0

Friends interested in PHP back-end technology and PHP architecture technology, my official group Click here , study together and discuss with each other.

There are already managers in the group to sort out the knowledge system (source code, learning videos and other materials). Welcome to add the group for free.

1, Single node instance

The single node instance is still relatively simple. Usually do a test and write a small program. If you need to use the cache, start one

Redis is still very easy. It is also competent as a key/value database

2, Master slave mode (Master / slave)

redis master-slave mode configuration

Master slave mode:

The master-slave mode of redis uses asynchronous replication. The slave node asynchronously copies data from the master node

The node provides read-write services, while the slave node only provides read-write services (this is the default configuration, which can be modified by modifying the configuration file)

Slave read only control). The master node can have multiple slave nodes. Configuring a slave node only requires

redis. You can specify slaveof master IP master port in the conf file.

There are three ways to enable master-slave replication from a slave node:

configuration file

Add slaveof < masterip > < masterport > to the configuration file of the slave server

Start command

Add slaveof < masterip > < masterport > after the redis server starts the command

Client command

After the Redis server is started, execute the command directly through the client: slaveof < masterip > < masterport >, then the Redis

The instance becomes a slave node.

The above three methods are equivalent. Let's take the client command as an example to see Redis after slaveof is executed

Changes in master and slave nodes.

This example: a master node has two slave nodes

to configure:

1,cd /usr/local/redis/redis-4.0.2

Switch to the current redis installation path

2, mkdir config

Create a new folder to store redis configuration files

3. Under config, create three configuration files as follows:

cd config
vi master-6739.conf
bind 0.0.0.0
port 6379
logfile "6379.log"
dbfilename "dump-6379.rdb"
daemonize yes
rdbcompression yes
vi slave-6380.confbind 0.0.0.0
port 6380
logfile "6380.log"
dbfilename "dump-6380.rdb"
daemonize yes
rdbcompression yes
slaveof 192.168.81.135 6379
vi slave-6381.conf
bind 0.0.0.0
port 6381
logfile "6381.log"
dbfilename "dump-6381.rdb"
daemonize yes
rdbcompression yes
slaveof 192.168.81.135 6379

master-6739.conf, master node configuration file, slave-6380 conf,slave-6381.conf is the slave node configuration file

In the configuration file of the slave node, use: slaveof to specify the master node

4. Start three reids services

[root@localhost redis-4.0.2]# ./src/redis-server config/master-6379.conf  
[root@localhost redis-4.0.2]# ./src/redis-server config/slave-6380.conf  
[root@localhost redis-4.0.2]# ./src/redis-server config/slave-6381.conf

Check the redis service

Test master-slave mode:

a. First connect three redis services respectively, obtain the value of key as name, and specify the redis service connected to that port through - p

[root@localhost redis-4.0.2]# ./src/redis-cli -p 6379 
127.0.0.1:6379> get name 
(nil) 
[root@localhost redis-4.0.2]# ./src/redis-cli -p 6380 
127.0.0.1:6380> get name 
(nil) 
[root@localhost redis-4.0.2]# ./src/redis-cli -p 6381 
127.0.0.1:6381> get name 
(nil) 
#The obtained values are empty

b. set a key for the master node

[root@localhost redis-4.0.2]# ./src/redis-cli -p 6379
127.0.0.1:6379> set name cmy
OK
127.0.0.1:6379> get name
"cmy"

c. The slave node directly reads the value whose key is name

[root@localhost redis-4.0.2]# ./src/redis-cli -p 6380
127.0.0.1:6380> get name
"cmy"
[root@localhost redis-4.0.2]# ./src/redis-cli -p 6381
127.0.0.1:6381> get name
"cmy"

d. The slave node only provides read services and cannot write operations

127.0.0.1:6381> set age 23
(error) READONLY You can't write against a read only slave.

be careful

When using the master-slave mode, you should pay attention to the persistence operation of the matser node. If the matser node does not use persistence, please refer to the details

In case of downtime and automatic restart of service, the slave server will lose data.

First, disable the persistence of the matser service

127.0.0.1:6379> CONFIG SET save ""

OK

set a value in the master node

127.0.0.1:6379> set age 23

OK

The slave node can get the value of age

127.0.0.1:6380> get age 
"23"

Turn off the master node service

127.0.0.1:6379> shutdown 
not connected>

The slave node can still get the value of age at this time

127.0.0.1:6380> get age 
"23"

Restart the master service, and the value of age cannot be obtained at this time

[root@localhost redis-4.0.2]# ./src/redis-server config/master-6379.conf  
[root@localhost redis-4.0.2]# ./src/redis-cli -p 6379 
127.0.0.1:6379> get age 
(nil)

At this time, the slave node obtains the value of age, which is null, and the data is lost

[root@localhost redis-4.0.2]# ./src/redis-cli -p 6380 
127.0.0.1:6380> get age 
(nil)

Cause of data loss: after the master service is suspended and the service is restarted, the slave node will communicate with the master node

A complete resynchronization operation. Because the master node is not persistent, the data on the slave node will be lost

Lost. Therefore, when the master-slave mode of Redis is configured, the persistence function of the master server should be turned on.

At this point, the master-slave mode of redis has been completed

Talk about the necessity of master-slave mode in my opinion:

One function of master-slave mode is to back up data, so that when a node is damaged (referring to unrecoverable hardware damage), the data will be recovered

Because there is backup, it can be easily restored.

Another function is load balancing. If all clients access one node, it will certainly affect Redis's work efficiency. With master-slave

Later, the query operation can be completed by querying the slave node.

Necessary understanding of master-slave mode (the conclusion has been verified and can be verified by yourself):

A Master can have more than one slave

Under the default configuration, the master node can read and write, while the slave node can only read and write, and the write operation is prohibited

Do not modify the configuration so that the slave node supports write operations. This is meaningless. First, the written data will not be synchronized to other nodes

Node; Second, when the master node modifies the same piece of data, the slave node's data will be overwritten

If the slave node hangs, it will not affect the reading of other slave nodes and the reading and writing of the master node. After restarting, the data will be transferred from the slave node to the master node

master node synchronization

After the master node hangs, it will not affect the read of the slave node. Redis will no longer provide write services. The master node starts

Redis will provide external write service again after.

After the master node hangs, a new master will not be selected for the slave node

For the case with a password, when the master node sets a password:

The client needs a password to access the master

You need a password to start the slave. You can configure it in the configuration

The client does not need a password to access the slave

Disadvantages of master-slave nodes

The disadvantages of master-slave mode can be seen from the above description:

After the master node hangs, redis cannot provide external write services because the remaining slave cannot become the master

This disadvantage has a great impact, especially for the production environment, the service can not be stopped for a moment, so the general production environment

The production environment will not only have the master-slave mode. So we have the following sentinel pattern. 3, Sentinel mode

Redis sentinel mode, in popular words, can be said to be a "sentinel robot", which compares the "sentinel robot"

After the configuration, the "robot" can work 7 * 24 hours. It can automatically help you do some things, such as monitoring

Control, remind, automatically handle faults, etc.

Introduction to redis Sentinel

Redis sentinel is the author of redis antirez. Because the redis cluster is used by major companies, each company needs to write its own cluster management tool, so antirez spent several weeks writing redis sentinel.

Redis Sentinel system is used to manage multiple redis servers (instance s). Redis Sentinel provides redis with high availability. Use Sentinel mode to create a redis deployment that can deal with various failures without human intervention.

The system performs the following three tasks:

Monitoring: Sentinel will constantly check whether your master server and slave server are allowed to work normally.

Notification: when a monitored Redis server has a problem, Sentinel can send a notification to the administrator or other applications through the API.

Automatic failover: (1) Sentinel will start when a primary server cannot work normally

When starting an automatic failover operation, it will upgrade one of the failed primary servers from the server to a new primary server,

And change other slave servers of the failed master server to copy the new master server; (2) The client tried to connect to the failed master server

Server, the cluster will also return the address of the new master server to the customer service end. Yes, the cluster can use the new master server instead of the failed server

The server.

Distributed characteristics of sentinel

Redis Sentinel is a distributed system,

You can run multiple Sentinel processes (progress) in one architecture. These processes use gossip protocols to receive information about whether the master server is offline, and use agreement protocols to decide whether to perform automatic failover and which slave server to choose as the new master server.

It is unreliable for a single sentinel process to monitor the redis cluster. When the sentinel process goes down (sentinel itself has a single point of failure), the whole cluster system will not run as expected. So it is necessary to

sentinel cluster has several advantages:

  • Some sentinel processes are down, and the active / standby switch of redis cluster can still be carried out;
  • If there is only one sentinel process, redis cannot be implemented if the process runs incorrectly or the network is blocked

Active / standby switching of cluster (single point problem);

If there are multiple sentinels, redis clients can freely connect to any sentinel to obtain information about the redis cluster

A robust deployment requires at least three sentinel instances.

The three sentinel instances should be placed in the computer or virtual machine where the customer confirms the failure in an independent way. For example, different physics

Virtual machines or virtual machines in different available areas. [this explanation is built on a machine, and multi-level is the same background

In recent project requirements, I came into contact with the construction of Redis, and simply recorded the pit encountered in the construction process

Overall configuration

192.168.1.100:6379 -> master 
192.168.1.101:6379 -> slave 
192.168.1.102:6379 -> slave 
192.168.1.100:26379 -> sentinel 
192.168.1.101:26379 -> sentinel 
192.168.1.102:26379 -> sentinel

Construction steps

1. Install redis

# decompression 
tar -xvf /usr/local/redis-3.2.11.tar.gz 
mkdir -p /usr/local/redis/bin 
cp  
/usr/local/redis/src/{redis-benchmark,redis-check-aof,redis-check-rdb,redis-cli,redis-sentinel,redi 
s-server,redis-trib.rb} /usr/local/redis/bin 
mkdir -p /u01/redis/{6379/{log,data,pid,conf},26379/{log,data,pid,conf} 
# Add environment variable 
echo "export PATH=/usr/local/redis/bin:$PATH" >> /etc/profile 
source /etc/profile

2.redis-6379 configuration

The redis node configuration is basically as follows. The following configurations are cp to the three virtual machines

/u01/redis/6379/conf/redis_6379.conf 
bind 0.0.0.0 
protected-mode no 
daemonize yes 
pidfile "/u01/redis/6379/pid/redis_6379.pid" 
port 6379 
tcp-backlog 511 
timeout 0tcp-keepalive 0 
loglevel notice 
logfile "/u01/redis/6379/log/redis_6379.log" 
databases 16 
stop-writes-on-bgsave-error yes 
rdbcompression yes 
rdbchecksum yes 
dbfilename "dump.rdb" 
dir "/u01/redis/6379/data" 
slave-serve-stale-data yes 
slave-read-only yes 
repl-diskless-sync no 
repl-diskless-sync-delay 5 
repl-disable-tcp-nodelay no 
slave-priority 100 
min-slaves-to-write 1 
min-slaves-max-lag 10 
appendonly no 
appendfilename "appendonly.aof" 
appendfsync everysec 
no-appendfsync-on-rewrite no 
auto-aof-rewrite-percentage 100 
auto-aof-rewrite-min-size 64mb 
aof-load-truncated yes 
lua-time-limit 5000 
slowlog-log-slower-than 10000 
slowlog-max-len 128 
latency-monitor-threshold 0 
notify-keyspace-events "" 
hash-max-ziplist-entries 512 
hash-max-ziplist-value 64 
list-max-ziplist-entries 512

Start service

# Execute on three virtual machines respectively 
redis-server /u01/redis/6379/conf/redis_6379.conf

Establish master-slave relationship

# At 192.168.1.101 
redis-cli -p 6379 SLAVEOF 192.168.1.100 6379 
# At 192.168.1.102 
redis-cli -p 6379 SLAVEOF 192.168.1.100 6379 see Replication 
192.168.1.101:6379> info replication 
# Replication 
role:master 
connected_slaves:2 
min_slaves_good_slaves:2 
slave0:ip=192.168.1.102,port=6379,state=online,offset=9577826,lag=1 
slave1:ip=192.168.1.103,port=6379,state=online,offset=9577965,lag=0 
master_repl_offset:9577965 
repl_backlog_active:1 
repl_backlog_size:1048576 
repl_backlog_first_byte_offset:8529390 
repl_backlog_histlen:1048576 
192.168.1.102:6379> info replication 
# Replication 
role:slave 
master_host:192.168.1.101 
master_port:6379 
master_link_status:up 
master_last_io_seconds_ago:0 
master_sync_in_progress:0 
slave_repl_offset:9600220 
slave_priority:100 
slave_read_only:1 
connected_slaves:0 
min_slaves_good_slaves:0 
master_repl_offset:0 
repl_backlog_active:0 
repl_backlog_size:1048576 
repl_backlog_first_byte_offset:0 
repl_backlog_histlen:0 
192.168.1.103:6379> info replication 
# Replication 
role:slave 
master_host:192.168.1.101 
master_port:6379 
master_link_status:up 
master_last_io_seconds_ago:0 
master_sync_in_progress:0 
slave_repl_offset:9612675slave_priority:100 
slave_read_only:1 
connected_slaves:0 
min_slaves_good_slaves:0 
master_repl_offset:0 
repl_backlog_active:0 
repl_backlog_size:1048576 
repl_backlog_first_byte_offset:0 
repl_backlog_histlen:0

3.sentinel-6379 configuration

The sentinel node configuration is basically as follows. The following configurations are cp applied to the three virtual machines

/u01/redis/26379/conf/sentinel_26379.conf

sentinel monitor mymaster monitors the master node in redis, that is, 192.168.1.100, so

This file is the same on three machines

port 26379 
bind 0.0.0.0 
daemonize yes 
protected-mode no 
dir "/u01/redis/26379/tmp" 
logfile "/u01/redis/26379/log/sentinel_26379.log" 
sentinel monitor mymaster 192.168.1.100 6379 1

Observe / u01 / redis / 26379 / conf / sentinel after startup_ File 26conf.379 changes

View sentinel status with info sentinel

redis-cli -h 192.168.1.100 -p 26379 info sentinel 
# Sentinel 
sentinel_masters:1 
sentinel_tilt:0 
sentinel_running_scripts:0 
sentinel_scripts_queue_length:0 
sentinel_simulate_failure_flags:0 
master0:name=zhuanche01,status=ok,address=192.168.1.100:6379,slaves=2,sentinels=3

summary

When I was building, I encountered an error in a period of time after the sentinel on 192.168.1.101 and 192.168.1.102 was started,

Later, it was found that the master was not monitored

Then there's a problem. Look at the log

4, The emergence of cluster mode is to solve the problem of limited capacity of single Redis, and distribute Redis data according to certain rules

To multiple machines. Some understanding of cluster:

Cluster can be said to be a combination of sentinel and master-slave mode. Master-slave and master reselection can be realized through cluster

Yes, so if you configure two replicas and three partitions, you need six Redis instances.

Because Redis data is allocated to different machines in the cluster according to certain rules, when the amount of data is too large, it can be added

Machine capacity expansion

This mode is suitable for caching requirements with a large amount of data. When the amount of data is not large, sentinel can be used.

I hope to help you with the above contents. Many PHPer will encounter some problems and bottlenecks when upgrading. There is no sense of direction when writing too much business code. I don't know where to start to improve. For this, I have sorted out some materials, including but not limited to: distributed architecture, high scalability, high performance, high concurrency, server performance tuning, TP6, laravel, YII2, Redis, Swoole, Swoft, Kafka, Mysql optimization, shell script Docker, microservices, Nginx and other knowledge points. Advanced dry goods can be shared with you free of charge. The official group Click here

Partial data display

Learning materials, documents, interview questions, videos, click the link to get Shimo for free im

 

 

Tags: PHP Programming MySQL Redis Programmer

Posted by alex.hughson on Fri, 20 May 2022 17:24:46 +0300