1.RocketMQ-Cluster集群搭建
1.1.说明
RocketMQ是一款分布式消息中间件,其各方面的性能都比目前已有的消息队列要好。RocketMQ默认采用长轮询的拉模式,单机支持千万级别的消息堆积,可以非常好的应用在海量消息系统中。
集群的模式有四种,单master模式、多master模式、多master多slave异步复制模式、多master多slave同步复制模式,推荐生产环境使用多master多slave同步复制模式,本文也使用此模式进行搭建集群。该模式下每个Master配置一个Slave,有多对Master-Slave,采用同步双写方式,即只有主备都写成功,才向应用返回成功,数据与服务都无单点故障,Master宕机情况下,消息无延迟,服务可用性与数据可用性都非常高。
1.2.环境配置
1.2.1.准备
1.下载rocketmq-all-4.2.0-bin-release.zip,下载地址:
http://rocketmq.apache.org/dowloading/releases/。
解压后并将文件移动到指定位置/usr/local/。
unzip rocketmq-all-4.2.0-bin-release.zip -d rocketmq-all-4.2.0-bin-release
mv rocketmq-all-4.2.0-bin-release /usr/local/
可使用安装命令:
yum install java-1.8.0-openjdk*
2.关闭防火墙
本次搭建使用虚拟机,为了防止出现防火墙问题,直接关闭了集群中节点服务器的防火墙,如果是在生产环境不允许关闭防火墙,那么就开放集群中用到的nameserver和broker端口号即可。
firewall-cmd --zone=public --add-port=(端口号)/tcp --permanent & firewall-cmd --reload(开启指定端口)
或
systemctl stop firewalld.service (禁用防火墙)
systemctl disable firewalld.service(停止并禁用开机启动)
3.配置环境变量
配置JAVA_HOME(本机jdk安装后的地址为/usr/lib/jvm):
echo 'export JAVA_HOME=/usr/lib/jvm/java-1.8.0-openjdk-1.8.0.292.b10-0.uelc20.x86_64' >> /etc/profile
source /etc/profile
echo $JAVA_HOME
配置rocketmq的环境变量:ROCKETMQ_HOME,值为rocketmq解压安装文件夹的路径,然后将ROCKETMQ_HOME/bin添加到PATH中去。
vim /etc/profile
在profile 文件末尾添加
set rocketmq
ROCKETMQ_HOME=/usr/local/rocketmq-all-4.2.0-bin-release
PATH=$PATH:$ROCKETMQ_HOME/bin
export ROCKETMQ_HOME PATH
#保存退出并生效
:wq
source /etc/profile
1.3.搭建集群
1.3.1.修改vim /etc/hosts
nameserver
10.100.20.91 rocketmq-nameserver1
10.100.20.92 rocketmq-nameserver2
broker
10.100.20.91 rocketmq-master1
10.100.20.91 rocketmq-slave2
10.100.20.92 rocketmq-master2
10.100.20.92 rocketmq-slave1
1.3.2.重启网卡
systemctl restart NetworkManager
1.3.3.创建消息存储路径
10.100.20.91服务器
mkdir -p /usr/local/rocketmq/store/broker-a
mkdir -p /usr/local/rocketmq/store/broker-a/commitlog
mkdir -p /usr/local/rocketmq/store/broker-a/consumequeue
mkdir -p /usr/local/rocketmq/store/broker-a/index
mkdir -p /usr/local/rocketmq/store/broker-b-s
mkdir -p /usr/local/rocketmq/store/broker-b-s/commitlog
mkdir -p /usr/local/rocketmq/store/broker-b-s/consumequeue
mkdir -p /usr/local/rocketmq/store/broker-b-s/index
10.100.20.92服务器
mkdir -p /usr/local/rocketmq/store/broker-b
mkdir -p /usr/local/rocketmq/store/broker-b/commitlog
mkdir -p /usr/local/rocketmq/store/broker-b/consumequeue
mkdir -p /usr/local/rocketmq/store/broker-b/index
mkdir -p /usr/local/rocketmq/store/broker-a-s
mkdir -p /usr/local/rocketmq/store/broker-a-s/commitlog
mkdir -p /usr/local/rocketmq/store/broker-a-s/consumequeue
mkdir -p /usr/local/rocketmq/store/broker-a-s/index
1.3.4.修改broker配置文件
10.100.20.91服务器 master
vim /usr/local/rocketmq-all-4.2.0-bin-release/conf/2m-2s-sync/broker-a.properties
修改配置如下:
#暴露的外网IP
brokerIP1=1号服务器公网IP
brokerIP2=1号服务器公网IP
#所属集群名字
brokerClusterName=rocketmq-cluster
#broker名字,注意此处不同的配置文件填写的不一样
brokerName=broker-a
#0 表示 Master,>0 表示 Slave
brokerId=0
#nameServer地址,分号分割
namesrvAddr=rocketmq-nameserver1:9876;rocketmq-nameserver2:9876
#在发送消息时,自动创建服务器不存在的topic,默认创建的队列数
defaultTopicQueueNums=4
#是否允许 Broker 自动创建Topic,建议线下开启,线上关闭
autoCreateTopicEnable=true
#是否允许 Broker 自动创建订阅组,建议线下开启,线上关闭
autoCreateSubscriptionGroup=true
#Broker 对外服务的监听端口
listenPort=10911
#删除文件时间点,默认凌晨 4点
deleteWhen=04
#文件保留时间,默认 48 小时
fileReservedTime=120
#commitLog每个文件的大小默认1G
mapedFileSizeCommitLog=1073741824
#ConsumeQueue每个文件默认存30W条,根据业务情况调整
mapedFileSizeConsumeQueue=300000
#destroyMapedFileIntervalForcibly=120000
#redeleteHangedFileInterval=120000
#检测物理文件磁盘空间
diskMaxUsedSpaceRatio=88
#存储路径
storePathRootDir=/usr/local/rocketmq/store/broker-a
#commitLog 存储路径
storePathCommitLog=/usr/local/rocketmq/store/broker-a/commitlog
#消费队列存储路径存储路径
storePathConsumeQueue=/usr/local/rocketmq/store/broker-a/consumequeue
#消息索引存储路径
storePathIndex=/usr/local/rocketmq/store/broker-a/index
#checkpoint 文件存储路径
storeCheckpoint=/usr/local/rocketmq/store/broker-a/checkpoint
#abort 文件存储路径
abortFile=/usr/local/rocketmq/store/broker-a/abort
#限制的消息大小
maxMessageSize=65536
#flushCommitLogLeastPages=4
#flushConsumeQueueLeastPages=2
#flushCommitLogThoroughInterval=10000
#flushConsumeQueueThoroughInterval=60000
#Broker 的角色
#- ASYNC_MASTER 异步复制Master
#- SYNC_MASTER 同步双写Master
#- SLAVE
brokerRole=SYNC_MASTER
#刷盘方式
#- ASYNC_FLUSH 异步刷盘
#- SYNC_FLUSH 同步刷盘
flushDiskType=SYNC_FLUSH
#checkTransactionMessageEnable=false
#发消息线程池数量
#sendMessageThreadPoolNums=128
#拉消息线程池数量
#pullMessageThreadPoolNums=128
10.100.20.91服务器 slave
vim /usr/local/rocketmq-all-4.2.0-bin-release/conf/2m-2s-sync/broker-b-s.properties
修改配置如下:
#所属集群名字
brokerClusterName=rocketmq-cluster
brokerName=broker-b
brokerId=1
namesrvAddr=rocketmq-nameserver1:9876;rocketmq-nameserver2:9876
defaultTopicQueueNums=4
autoCreateTopicEnable=true
autoCreateSubscriptionGroup=true
listenPort=11011
deleteWhen=04
fileReservedTime=120
mapedFileSizeCommitLog=1073741824
mapedFileSizeConsumeQueue=300000
#destroyMapedFileIntervalForcibly=120000
#redeleteHangedFileInterval=120000
diskMaxUsedSpaceRatio=88
storePathRootDir=/usr/local/rocketmq/store/broker-b-s
storePathCommitLog=/usr/local/rocketmq/store/broker-b-s/commitlog
storePathConsumeQueue=/usr/local/rocketmq/store/broker-b-s/consumequeue
storePathIndex=/usr/local/rocketmq/store/broker-b-s/index
storeCheckpoint=/usr/local/rocketmq/store/broker-b-s/checkpoint
abortFile=/usr/local/rocketmq/store/broker-b-s/abort
maxMessageSize=65536
#flushCommitLogLeastPages=4
#flushConsumeQueueLeastPages=2
#flushCommitLogThoroughInterval=10000
#flushConsumeQueueThoroughInterval=60000
brokerRole=SLAVE
flushDiskType=ASYNC_FLUSH
#checkTransactionMessageEnable=false
#sendMessageThreadPoolNums=128
#pullMessageThreadPoolNums=128
10.100.20.92服务器 master
vim /usr/local/rocketmq-all-4.2.0-bin-release/conf/2m-2s-sync/broker-b.properties
修改配置如下:
brokerIP1=2号服务器公网IP
brokerIP2=2号服务器公网IP
brokerClusterName=rocketmq-cluster
brokerName=broker-b
brokerId=0
namesrvAddr=rocketmq-nameserver1:9876;rocketmq-nameserver2:9876
defaultTopicQueueNums=4
autoCreateTopicEnable=true
autoCreateSubscriptionGroup=true
listenPort=10911
deleteWhen=04
fileReservedTime=120
mapedFileSizeCommitLog=1073741824
mapedFileSizeConsumeQueue=300000
#destroyMapedFileIntervalForcibly=120000
#redeleteHangedFileInterval=120000
diskMaxUsedSpaceRatio=88
storePathRootDir=/usr/local/rocketmq/store/broker-b
storePathCommitLog=/usr/local/rocketmq/store/broker-b/commitlog
storePathConsumeQueue=/usr/local/rocketmq/store/broker-b/consumequeue
storePathIndex=/usr/local/rocketmq/store/broker-b/index
storeCheckpoint=/usr/local/rocketmq/store/broker-b/checkpoint
abortFile=/usr/local/rocketmq/store/broker-b/abort
maxMessageSize=65536
#flushCommitLogLeastPages=4
#flushConsumeQueueLeastPages=2
#flushCommitLogThoroughInterval=10000
#flushConsumeQueueThoroughInterval=60000
brokerRole=SYNC_MASTER
flushDiskType=SYNC_FLUSH
#checkTransactionMessageEnable=false
#sendMessageThreadPoolNums=128
#pullMessageThreadPoolNums=128
10.100.20.92服务器 slave
vim /usr/local/rocketmq-all-4.2.0-bin-release/conf/2m-2s-sync/broker-a-s.properties
修改配置如下:
brokerClusterName=rocketmq-cluster
brokerName=broker-a
brokerId=1
namesrvAddr=rocketmq-nameserver1:9876;rocketmq-nameserver2:9876
defaultTopicQueueNums=4
autoCreateTopicEnable=true
autoCreateSubscriptionGroup=true
listenPort=11011
deleteWhen=04
fileReservedTime=120
mapedFileSizeCommitLog=1073741824
mapedFileSizeConsumeQueue=300000
#destroyMapedFileIntervalForcibly=120000
#redeleteHangedFileInterval=120000
diskMaxUsedSpaceRatio=88
storePathRootDir=/usr/local/rocketmq/store/broker-a-s
storePathCommitLog=/usr/local/rocketmq/store/broker-a-s/commitlog
storePathConsumeQueue=/usr/local/rocketmq/store/broker-a-s/consumequeue
storePathIndex=/usr/local/rocketmq/store/broker-a-s/index
storeCheckpoint=/usr/local/rocketmq/store/broker-a-s/checkpoint
abortFile=/usr/local/rocketmq/store/broker-a-s/abort
maxMessageSize=65536
#flushCommitLogLeastPages=4
#flushConsumeQueueLeastPages=2
#flushCommitLogThoroughInterval=10000
#flushConsumeQueueThoroughInterval=60000
brokerRole=SLAVE
flushDiskType=ASYNC_FLUSH
#checkTransactionMessageEnable=false
#sendMessageThreadPoolNums=128
#pullMessageThreadPoolNums=128
1.3.5.修改启动脚本文件
vim /usr/local/rocketmq-all-4.2.0-bin-release/bin/runbroker.sh
JAVA_OPT="${JAVA_OPT} -server -Xms256m -Xmx256m -Xmn128m"
vim /usr/local/rocketmq-all-4.2.0-bin-release/bin/runserver.sh
JAVA_OPT="${JAVA_OPT} -server -Xms256m -Xmx256m -Xmn128m -XX:MetaspaceSize=128m -XX:MaxMetaspaceSize=320m"
1.3.6.服务启动
1.启动NameServe集群(分别在服务器启动)
cd /usr/local/rocketmq-all-4.2.0-bin-release/bin
nohup sh mqnamesrv &
2.启动Broker集群
10.100.90.91服务器master1:
cd /usr/local/rocketmq-all-4.2.0-bin-release/bin
nohup sh mqbroker -c /usr/local/rocketmq-all-4.2.0-bin-release/conf/2m-2s-sync/broker-a.properties &
10.100.90.91服务器slave1:
cd /usr/local/rocketmq-all-4.2.0-bin-release/bin
nohup sh mqbroker -c /usr/local/rocketmq-all-4.2.0-bin-release/conf/2m-2s-sync/broker-b-s.properties &
10.100.90.92服务器master2:
cd /usr/local/rocketmq-all-4.2.0-bin-release/bin/
nohup sh mqbroker -c /usr/local/rocketmq-all-4.2.0-bin-release/conf/2m-2s-sync/broker-b.properties &
10.100.90.92服务器slave2:
cd /usr/local/rocketmq-all-4.2.0-bin-release/bin/
nohup sh mqbroker -c /usr/local/rocketmq-all-4.2.0-bin-release/conf/2m-2s-sync/broker-a-s.properties &
1.3.7.查看进程状态
启动后通过JPS查看启动进程
查看nameServer日志
tail -500f ~/logs/rocketmqlogs/namesrv.log
查看broker日志
tail -500f ~/logs/rocketmqlogs/broker.log
在集群中任意一个服务器上,执行以下命令查看该集群上所启动的各实例信息(namesrvAddr只写该节点的nameserver地址即可):
mqadmin clusterList --namesrvAddr=10.100.20.91:9876
1.3.8.Rocketmq-console控制台连接集群
RocketMQ-Console是RocketMQ项目的扩展插件,是一个图形化管理控制台,提供Broker集群状态查看,Topic管理,Producer、Consumer状态展示,消息查询等常用功能,这个功能在安装好RocketMQ后需要额外单独安装、运行。
1.rocketmq-console下载
进入rocketmq-externals项目的GitHub地址,如下图,可看到RocketMQ项目的诸多扩展项目,其中就包含我们需要下载的rocketmq-console。
克隆项目到本地
git clone https://github.com/apache/rocketmq-externals.git
#比较慢的话,可以用国内的gitee
git clone https://gitee.com/mirrors/RocketMQ-Externals.git
cd rocketmq-externals/rocketmq-console //切换到目录
2.修改rocketmq-console中配置namesrv集群地址
vim src/main/resources/application.properties
内容改为
rocketmq.config.namesrvAddr=10.100.20.91:9876;10.100.20.92:9876
3.打包(跳过测试打包)
mvn clean package -Dmaven.test.skip=true
若没有mvn则:
yum install maven
4.启动rocketmq-console:(其中一台服务器上运行都可以)
cd target/
java -jar rocketmq-console-ng-2.0.0.jar
5.访问工作台(访问地址10.100.20.91:8080/#/)