Teach you how to build a RocketMQ cluster from 0 to 1

Hello, I'm brother Jun. Today, let's share how to build a RocketMQ cluster.

The environment used by the cluster in this paper is 2 virtual machines. The architecture is as follows:

At 47.xx.xx XX and 39.xx.xx A Name Server and a Broker master node are respectively deployed on XX. The slave node is not built here. That is, the 2m nosave construction mode introduced on the official website.

Note: there are three configuration modes on the official website: 2m-2s-sync, 2m-2s-async and 2m noslave. These three configurations all use the asynchronous disk brushing strategy: ASYNC_FLUSH.

After the cluster is successfully built, the Broker will register the Topic routing information with the Name Server, and the Producer and Consumer will obtain the routing information from the Name Server. Then the Producer will send messages to the Broker according to the routing information, and the Consumer will pull messages from the Broker according to the routing information.

(1) preparation

To install RocketMQ, you first need to install the Java environment, Maven environment, on the virtual machine. I won't give a detailed introduction here.

After the Java and Maven environments are installed, download the latest version from the official website at the following address:

https://dlcdn.apache.org/rocketmq/4.9.4/rocketmq-all-4.9.4-bin-release.zip

After downloading, decompress and compile, and execute the following commands:

> unzip rocketmq-all-4.9.4-source-release.zip
> cd rocketmq-all-4.9.4-source-release/
> mvn -Prelease-all -DskipTests clean install -U
> cd distribution/target/rocketmq-4.9.4/rocketmq-4.9.4

When you see the following output, the compilation is successful:

(2) startup

2.1 start Name Server

Because the memory of my virtual machine is small, and the JVM heap memory required by RocketMQ to start the Name Server is 4g by default, the default heap memory needs to be modified. Modify the file as:

rocketmq-all-4.9.4-source-release/distribution/target/rocketmq-4.9.4/rocketmq-4.9.4/bin/runserver.sh

The modified configuration is as follows:

JAVA_OPT="${JAVA_OPT} -server -Xms256m -Xmx256m -Xmn128m -XX:MetaspaceSize=128m -XX:MaxMetaspaceSize=320m"

After modification, execute the following command:

nohup sh bin/mqnamesrv &

After successful startup, nohup The following logs are printed in the out file:

The Name Server boot success. serializeType=JSON

After setting up one machine, set up the second machine in the same way.

2.2 start Broker

Similarly, the JVM heap memory required by RocketMQ to start Broker is 8g by default, so the heap memory needs to be modified. Modify the file as:

rocketmq-all-4.9.4-source-release/distribution/target/rocketmq-4.9.4/rocketmq-4.9.4/bin/runbroker.sh

The modified configuration is as follows:

JAVA_OPT="${JAVA_OPT} -server -Xms512m -Xmx512m"

Create the conf.properties file as follows:

brokerClusterName = MyTestCluster
brokerName = broker-a
brokerId = 0
deleteWhen = 04
fileReservedTime = 48
brokerRole = ASYNC_MASTER
flushDiskType = ASYNC_FLUSH
namesrvAddr=127.0.0.1:9876;47.xx.xx.xx:9876

After the configuration is completed, start the broker-a node. The start command is as follows:

nohup bash bin/mqbroker -c conf/broker.properties &

After the Broker is started successfully, the log of the Name Server is as follows:

Start broker-b, conf The properties file is configured as follows:

#Cluster name
brokerClusterName = MyTestCluster
#broker name
brokerName = broker-b
brokerId = 0
#The time to delete the expired file of message storage is 4:00 a.m
deleteWhen = 04
#The retention time of the message storage file that has not been updated is 48 hours. It will expire after 48 hours and will be deleted
fileReservedTime = 48
brokerRole = ASYNC_MASTER
flushDiskType = ASYNC_FLUSH
namesrvAddr=127.0.0.1:9876;xx.xx.xx.xx:9876

After successful startup, execute the mqadmin command to view the cluster information:

[root@rocketmq-4.9.4]# ./bin/mqadmin topicList -n 127.0.0.1:9876
MyTestCluster
RMQ_SYS_TRANS_HALF_TOPIC
BenchmarkTest
OFFSET_MOVED_EVENT
iZ8vbfyc7hfzygtpgapblyZ
TBW102
SELF_TEST_TOPIC
DefaultCluster
iZ2ze4m2ri7irk6au5mbfgZ
SCHEDULE_TOPIC_XXXX
DefaultCluster_REPLY_TOPIC
broker-b
broker-a
MyTestCluster_REPLY_TOPIC

Note: to execute the above command, you need to modify bin / tools The JVM parameters in SH are configured as follows:

JAVA_OPT="${JAVA_OPT} -server -Xms256m -Xmx256m -Xmn128m -XX:MetaspaceSize=128m -XX:MaxMetaspaceSize=128m"

3 dashboard

3.1 installation

To install dashboard, you need to compile the source code. The github address of the source code is as follows:

https://github.com/apache/rocketmq-dashboard

After downloading, modify the application YML parameter. I only modified the Name Server address here:

rocketmq:
  config:
    namesrvAddrs:
      - 127.0.0.1:9876
      - 47.xx.xx.xx:9876

Then execute the Maven package command and print out the jar package:

mvn clean package -Dmaven.test.skip=true

After typing the jar package, upload it to the server and execute the following command to start:

java -jar rocketmq-dashboard-1.0.1-SNAPSHOT.jar

After successful startup, output the following log:

3.2 test

Browser input http://39.xx.xx.xx:8080/ The interface is as follows:

Use the following command to create a Topic named MyTestTopic:

sh bin/mqadmin updateTopic -n 127.0.0.1:9876\;47.92.71.246:9876  -c MyTestCluster  -t MyTestTopic

This is the Topic you can see on the interface, as shown in the following figure:

Send a message using the command:

[root@rocketmq-4.9.4]# sh bin/mqadmin sendMessage -n 127.0.0.1:9876 -t MyTestTopic -p 'hello test message'
#Broker Name                      #QID  #Send Result            #MsgId
broker-a                          1     SEND_OK                 7F000001D3C3339097527B50D1740000

View the message on the interface, as shown in the following figure:

(4) summary

This article mainly describes the construction of RocketMQ cluster. I hope it can help you understand RocketMQ.

Tags: message queue RocketMQ cluster

Posted by maqmus on Sun, 28 Aug 2022 01:05:12 +0300