Redis advanced practice

Redis advanced practice

transaction management

Redis transactions can execute multiple commands at once, and have the following three important guarantees:

  • The batch operation is put into the queue cache before sending the EXEC command.
  • After receiving the EXEC command, the transaction is executed. Any command in the transaction fails to be executed, and the remaining commands are still executed.
  • During transaction execution, command requests submitted by other clients will not be inserted into the transaction execution command sequence.

A transaction goes through the following three stages from start to execution:

  • Start the transaction.
  • I ordered you to join the team.
  • Execute the transaction.

Serial number:

Command and description

1

DISCARD Cancel the transaction and abort the execution of all commands in the transaction block.

2

EXEC Execute commands within all transaction blocks.

3

MULTI Marks the beginning of a transaction block.

4

UNWATCH Cancel the monitoring of all key s by the WATCH command.

5

[WATCH key key ...] Monitor one (or more) key. If this (or these) key is changed by other commands before the transaction is executed, the transaction will be interrupted.

  • Transaction in redis is a similar concept to transaction in relational database, but it is different.
  • If the transaction of the relational database fails to execute, the subsequent sql will not be executed. If one command in redis fails to execute, the remaining commands will be executed as usual;
  • The redis start transaction command is different
Comparison between redis and mysql

Mysql

redis

open

start transaction /begin

multi

sentence

Common sql

Common command

fail

rollback

discard cancel (it is not called rollback, because the commands in the queue are not executed, and the tasks in the queue are not executed at all, instead of being executed, they can be revoked back)

success

commit

exec

redis slow log query

slow log is a log system used by redis to record query execution time. It is stored in memory and has fast reading and writing speed

983:# slow log. When a new command is logged the oldest one is removed from the
987:# to one second. Note that a negative number disables the slow log, while
989:slowlog-log-slower-than 10000
992:# You can reclaim memory used by the slow log with SLOWLOG RESET.
993:slowlog-max-len 128

redis high availability and cluster

Although redis can realize data persistence of a single machine, neither RDB nor AOF can solve the problem of single point downtime. That is, once a single redis server has a system failure or hardware failure, it will directly cause data loss. Therefore, it needs to use other technologies to solve the single point problem.

redis master-slave replication

Synchronization mechanism
  • Asynchronous replication
  • The master and slave are just connected and fully synchronized. After full synchronization, partial synchronization is performed. If necessary, slave can initiate master synchronization at any time;
  • The redis policy will attempt to perform partial synchronization at any time. If it is not successful, it will initiate full synchronization and start BGSAVE... After BGSAVE is completed, the RDB file will be transferred. If the file is successful, it will perform partial synchronization from the server and transfer the data in the backlog space

Master slave replication principle
  1. The slave server sends a SYNC command to the master server
  2. After receiving the SYNC instruction, the master calls BGSAVE, generates an RDB file, and records all subsequent write commands in the buffer
  3. When the master server executes BGSAVE, it will send RDB files like the slave server, and the slave server will accept and load the RDB files
  4. The master server sends all write commands stored in the buffer to the slave server for execution.
Command propagation

After the master-slave synchronization is completed, every time the master server executes a write command, it will send the executed command to the slave server. This operation is command propagation

Command propagation is a continuous process. As long as the replication continues, the command propagation will continue, so that the status of the master and slave servers can always be consistent.

SYNC and PSYNC

Before redis 2.8, the reconnected slave server after disconnection always performs a full resynchronization operation. After 2.8, redis uses PSYNC command instead of SYNC. The biggest improvement of PSYNC over SYNC is that PSYNC realizes partial resync hronization. Features: when the slave server is disconnected and reconnected, PSYNC can enable the master server to synchronize only the data lost during the disconnection to the slave server, without re synchronizing the entire database to the slave server, as long as conditions permit.

Consistency of redis replication

redis replication security improvement

Redis master-slave replication practice

Modify the following two parameters from the server: redis.conf

replicaof 192.168.3.49 6379

masterauth redis

################################# REPLICATION #################################

# Master-Replica replication. Use replicaof to make a Redis instance a copy of
# another Redis server. A few things to understand ASAP about Redis replication.
#
#   +------------------+      +---------------+
#   |      Master      | ---> |    Replica    |
#   | (receive writes) |      |  (exact copy) |
#   +------------------+      +---------------+
#
# 1) Redis replication is asynchronous, but you can configure a master to
#    stop accepting writes if it appears to be not connected with at least
#    a given number of replicas.
# 2) Redis replicas are able to perform a partial resynchronization with the
#    master if the replication link is lost for a relatively small amount of
#    time. You may want to configure the replication backlog size (see the next
#    sections of this file) with a sensible value depending on your needs.
# 3) Replication is automatic and does not need user intervention. After a
#    network partition replicas automatically try to reconnect to masters
#    and resynchronize with them.
#
# replicaof <masterip> <masterport>

# If the master is password protected (using the "requirepass" configuration
# directive below) it is possible to tell the replica to authenticate before
# starting the replication synchronization process, otherwise the master will
# refuse the replica request.
#
# masterauth <master-password>

https://blog.csdn.net/qq_42606357/article/details/118742333

Redis HA practice

redis sentinel

Redis Sentinel (translated as "sentinel") is a high availability (HA) solution recommended by redis officials. When redis is used as a master slave high availability solution, if the master server goes down, redis itself (including many of its clients) does not realize automatic master / standby switching, and Redis Sentinel itself is an independent process that can monitor multiple master slave clusters, After the master is found to be down, it can be switched automatically.

Subjective offline and objective offline: ping method is adopted. A single sentinel in the subjective offline cannot Ping, and multiple sentinels in the objective offline cannot Ping

redis sentinel efficacy

Monitoring: Sentinel will constantly check whether your master server and slave server are working properly.

Notification: when a monitored Redis server has a problem, Sentinel can send a notification to the administrator or other applications through the API.

Automatic failover: when a master server fails to work normally, Sentinel will start an automatic failover operation. It will upgrade one of the slave servers of the failed master server to a new master server, and change other slave servers of the failed master server to copy the new master server; When the client tries to connect to the failed primary server, the cluster will also return the address of the new primary server to the client, so that the cluster can use the new primary server to replace the failed server.

Redis Sentinel is a distributed system. You can run multiple Sentinel processes in one architecture. These processes use gossip protocols to receive information about whether the master server is offline, and use agreement protocols to decide whether to perform automatic failover and select which slave server is the new master server.

One Sentinel process can connect with other Sentinel processes. Each Sentinel process can check each other's availability and exchange information

port 26379
pidfile "/var/run/redis-sentinel.pid"
logfile "./sentinel26379.log"
dir "/local/apps/redis"
sentinel myid 6fbefcfa4c14e9c2075ad733365f9a88dd84f054
sentinel deny-scripts-reconfig yes
# If the mymaster does not respond within 3s, it is considered that the mymaster is down
sentinel monitor mymaster 127.0.0.1 6380 1
#If myster is still not started after 10 seconds, start failover
sentinel auth-pass mymaster redis
# When performing failover, at most one slave server synchronizes with the new master server at the same time
sentinel config-epoch mymaster 5
# Generated by CONFIG REWRITE
daemonize yes
protected-mode no
sentinel leader-epoch mymaster 6
sentinel known-replica mymaster 192.168.3.49 6381
sentinel known-replica mymaster 127.0.0.1 6381
sentinel known-replica mymaster 127.0.0.1 6379
sentinel known-replica mymaster 192.168.3.49 6380
sentinel current-epoch 6



# sentinel monitor [master-name] [master-ip] [master-port] [quorum]
# The [Master name] here can be customized, but the parameters related to [Master name] must be the same
sentinel monitor mymaster 127.0.0.1 6380 1

The explanation of quorum is as follows:

(1) At least how many sentinels should agree that the master process is down, or the slave process is down, or a failover operation should be started
(2) quorum is used to identify failures. When the failover is actually performed, the election should be performed in the sentinel cluster, and a sentinel process should be elected to perform the failover operation
(3) Suppose there are 5 sentinels and quorum is set to 2, then if 2 of the 5 sentinels think that the master is dead; One of the two sentinels will make an election and elect one sentinel to perform failover; If three of the five sentinels are running, failover is allowed

Redis cluster

Redis cluster is an assembly that provides data sharing among multiple redis.

distributed and fault tolerant redis memory K/V service

16384 hash slots

The redis Cluster requires at least three master and three slave redis servers

Linux + nginx + php + redis

# Add to cluster
redis-cli --cluster create --cluster-replicas 1 192.168.65.100:6379 192.168.65.101:6379 192.168.65.102:6379 192.168.65.103:6379 192.168.65.104:6379 192.168.65.105:6379   
# Login cluster
redis-cli -c -p 6379
#View cluster information
cluster nodes
#View master-slave information
info replication
#Redis cli cluster help command
redis-cli --cluster help
#Add new master node
redis-cli --cluster add-node new_host:new_port existing_host:existing_port   # new_host:new_port: to add a new master node (existing) to the cluster:_ host:existing_ Port: any node in the original cluster

#Add new slave node
redis-cli --cluster add-node new_host:new_port existing_host:existing_port --cluster-slave --cluster-master-id node_id  # new_host:new_port: to add a new master node (existing) to the cluster:_ host:existing_ Port: any node in the original cluster_ id is the primary node to add to****

#Delete node
redis-cli --cluster del-node host:port node_id

#Hash slot Reallocation: after adding a new node, you need to reallocate the hash slot of the newly added master node. At this time, the master node can store data. There are 16384 slots in Redis
redis-cli --cluster reshard host:port  #Configure according to prompts
redis-cli --cluster reshard host:port --cluster-from node_id --cluster-to node_id --cluster-slots <arg> --cluster-yes
  • Each time the key value is entered and queried in the redis cli, redis will calculate the slot that the key should be sent to. If it is not the slot of the server corresponding to the client, redis will report an error and inform the redis instance address and port that should be sent

  • The redis cli client provides the – c parameter to realize automatic redirection

  • In the cluster, multiple key commands such as mset and mget cannot be used

  • The concept of group can be defined through {}, so that the key value pairs of the same content in {} in the key can be placed in a slot

  • If the master node goes offline, the slave node can be automatically promoted to the master node, but if it exceeds 15 seconds, it will fail.

  • After the master node is down and then restored, it will automatically become a slave node.

  • If all the master and slave nodes of a certain slot are down, if the configuration file redis If the cluster require full coverage in conf is yes, the entire cluster will hang up; If the configuration file redis If the cluster require full coverage in conf is no, the data in this slot cannot be used or stored.

Advantages and disadvantages of Redis cluster

  • advantage:

    • Realize capacity expansion.
    • Share the pressure.
    • The centerless configuration is relatively simple.
  • Disadvantages:

    • Multi key operation is not supported.
    • Multi key Redis transactions are not supported. lua script is not supported.
    • Due to the late appearance of the cluster scheme, many companies have adopted other cluster schemes, while the proxy or client segmentation scheme needs to be migrated to the redis cluster as a whole rather than gradually, which is complicated.
redisdump data export and import

Redis dump data import and export

Tags: Java C Database Redis Cache hdfs

Posted by ben2.0 on Sat, 27 Aug 2022 14:01:57 +0300