docker集群 emq docker集群动态扩容

转载

mob64ca1414098d 2023-08-18 14:52:49

文章标签 docker集群 emq redis docker 5e 文章分类 Docker 云计算

1、集群规划(3主3从)：

name	ip	port
redis-node-1	192.168.150.110	6381
redis-node-2	192.168.150.110	6382
redis-node-3	192.168.150.110	6383
redis-node-4	192.168.150.110	6384
redis-node-5	192.168.150.110	6385
redis-node-6	192.168.150.110	6386

2、安装docker：

5、1 安装需要的软件包

$ yum install -y yum-utils

5、2 设置stable镜像仓库：

$ yum-config-manager --add-repo https://mirrors.aliyun.com/docker-ce/linux/centos/docker-ce.repo

5、3 生成缓存：

$ yum makecache

5、4 安装DOCKER CE：

$ yum install docker-ce docker-ce-cli containerd.io docker-compose-plugin

5、5 配置阿里云镜像加速：

$ mkdir -p /etc/docker
$ tee /etc/docker/daemon.json <<-'EOF'
{
   "registry-mirrors": ["https://fxt824bw.mirror.aliyuncs.com"]
}
EOF

5、6 验证docker是否安装：

$ docker -v
Docker version 20.10.18, build b40c2f6

5、7 启动docker：

$ systemctl start docker

3、拉取redis镜像：

$ docker pull redis:6.0.8

4、运行redis容器实例：

docker run -d --name redis-node-1 --net host --privileged=true -v /redis/share/redis-node-1:/data -v /redis/redis.conf redis:6.0.8 --cluster-enabled yes --appendonly yes --port 6381 
docker run -d --name redis-node-2 --net host --privileged=true -v /redis/share/redis-node-2:/data -v /redis/redis.conf redis:6.0.8 --cluster-enabled yes --appendonly yes --port 6382 
docker run -d --name redis-node-3 --net host --privileged=true -v /redis/share/redis-node-3:/data -v /redis/redis.conf redis:6.0.8 --cluster-enabled yes --appendonly yes --port 6383 
docker run -d --name redis-node-4 --net host --privileged=true -v /redis/share/redis-node-4:/data -v /redis/redis.conf redis:6.0.8 --cluster-enabled yes --appendonly yes --port 6384 
docker run -d --name redis-node-5 --net host --privileged=true -v /redis/share/redis-node-5:/data -v /redis/redis.conf redis:6.0.8 --cluster-enabled yes --appendonly yes --port 6385 
docker run -d --name redis-node-6 --net host --privileged=true -v /redis/share/redis-node-6:/data -v /redis/redis.conf redis:6.0.8 --cluster-enabled yes --appendonly yes --port 6386

查看六个redis容器实例是否运行

$ docker ps
CONTAINER ID   IMAGE                    COMMAND                  CREATED         STATUS          PORTS                                                                                        NAMES
1aa258d0a1ab   redis:6.0.8              "docker-entrypoint.s…"   5 minutes ago   Up 5 minutes                                                                                                     redis-node-6
b6b0d8674fba   redis:6.0.8              "docker-entrypoint.s…"   5 minutes ago   Up 5 minutes                                                                                                     redis-node-5
f84b41abdda8   redis:6.0.8              "docker-entrypoint.s…"   5 minutes ago   Up 5 minutes                                                                                                     redis-node-4
69585693a06a   redis:6.0.8              "docker-entrypoint.s…"   5 minutes ago   Up 5 minutes                                                                                                     redis-node-3
7ae9867a244f   redis:6.0.8              "docker-entrypoint.s…"   5 minutes ago   Up 5 minutes                                                                                                     redis-node-2
b3192a6c160b   redis:6.0.8              "docker-entrypoint.s…"   6 minutes ago   Up 6 minutes

可以看到容器实例已经运行。

5、部署redis集群

5、1 进入容器内部

docker exec -it redis-node-1 /bin/bash

5、2 构建主从关系

$ redis-cli --cluster create 192.168.150.110:6381 192.168.150.110:6382 192.168.150.110:6383 192.168.150.110:6384 192.168.150.110:6385 192.168.150.110:6386 --cluster-replicas 1

# 参数说明：
--cluster-replicas 1： 为每个master创建一个slave节点

5、3 查看节点状态

root@gr:/data# redis-cli -p 6381
127.0.0.1:6381>
127.0.0.1:6381>
127.0.0.1:6381> cluster nodes
1f46effff69e974a1586b67082570e70961b9abe 192.168.150.110:6385@16385 slave 8b3a4a57bff74c389e7a3e4edbdb3d5edf2a613d 0 1667581564000 2 connected
aa5ed45e1c2b5aeb6b3ed34337af9b90ace91e51 192.168.150.110:6383@16383 master - 0 1667581559000 3 connected 10923-16383
2dee6046534340e3f7db45e32fe02cf94b7b4e32 192.168.150.110:6386@16386 slave aa5ed45e1c2b5aeb6b3ed34337af9b90ace91e51 0 1667581562000 3 connected
4d4404303675e0fb887a73dccd0ae9e5f10edaf5 192.168.150.110:6384@16384 slave 8b12adc7fff6e626fddbfe46b69b0a04051148c8 0 1667581563442 1 connected
8b3a4a57bff74c389e7a3e4edbdb3d5edf2a613d 192.168.150.110:6382@16382 master - 0 1667581564446 2 connected 5461-10922
8b12adc7fff6e626fddbfe46b69b0a04051148c8 192.168.150.110:6381@16381 myself,master - 0 1667581563000 1 connected 0-5460



127.0.0.1:6381> cluster info
cluster_state:ok
cluster_slots_assigned:16384
cluster_slots_ok:16384
cluster_slots_pfail:0
cluster_slots_fail:0
cluster_known_nodes:6
cluster_size:3
cluster_current_epoch:6
cluster_my_epoch:1
cluster_stats_messages_ping_sent:172
cluster_stats_messages_pong_sent:172
cluster_stats_messages_sent:344
cluster_stats_messages_ping_received:167
cluster_stats_messages_pong_received:172
cluster_stats_messages_meet_received:5
cluster_stats_messages_received:344

5、4 测试数据的写入和读取

# 测试写入
root@gr:/data# redis-cli -p 6381 -c
127.0.0.1:6381>
127.0.0.1:6381>
127.0.0.1:6381> set k1 v1
-> Redirected to slot [12706] located at 192.168.150.110:6383
OK
192.168.150.110:6383> set k2 v2
-> Redirected to slot [449] located at 192.168.150.110:6381
OK

# 测试读取
192.168.150.110:6381> get k1
-> Redirected to slot [12706] located at 192.168.150.110:6383
"v1"
192.168.150.110:6383> get k2
-> Redirected to slot [449] located at 192.168.150.110:6381
"v2"

可以看到数据可以正常写入和读取。

5、5 测试集群的容错迁移

# 停掉主机redis-node-1
docker stop redis-node-1

再次查看集群信息

$ docker exec -it redis-node-2 bash
root@gr:/data# redis-cli -p 6382 -c
127.0.0.1:6382>
127.0.0.1:6382>
127.0.0.1:6382> cluster nodes
aa5ed45e1c2b5aeb6b3ed34337af9b90ace91e51 192.168.150.110:6383@16383 master - 0 1667582456000 3 connected 10923-16383
2dee6046534340e3f7db45e32fe02cf94b7b4e32 192.168.150.110:6386@16386 slave aa5ed45e1c2b5aeb6b3ed34337af9b90ace91e51 0 1667582457414 3 connected
8b3a4a57bff74c389e7a3e4edbdb3d5edf2a613d 192.168.150.110:6382@16382 myself,master - 0 1667582456000 2 connected 5461-10922
4d4404303675e0fb887a73dccd0ae9e5f10edaf5 192.168.150.110:6384@16384 master - 0 1667582455000 7 connected 0-5460
8b12adc7fff6e626fddbfe46b69b0a04051148c8 192.168.150.110:6381@16381 master,fail - 1667582395917 1667582390000 1 disconnected
1f46effff69e974a1586b67082570e70961b9abe 192.168.150.110:6385@16385 slave 8b3a4a57bff74c389e7a3e4edbdb3d5edf2a613d 0 1667582456405 2 connected

可以看到redis-node-1的从节点变成了主节点，至此，redis3主3从集群搭建成功。

6、redis集群扩容

6、1 新建两个redis容器实例

$ docker run -d --name redis-node-7 --net host --privileged=true -v /redis/share/redis-node-7:/data -v /redis/redis.conf redis:6.0.8 --cluster-enabled yes --appendonly yes --port 6387

$ docker run -d --name redis-node-8 --net host --privileged=true -v /redis/share/redis-node-8:/data -v /redis/redis.conf redis:6.0.8 --cluster-enabled yes --appendonly yes --port 6388

6、2 进入新增的端口为6387的容器实例内部

$ docker exec -it redis-node-7

6、3 将新增的6387节点(空槽号)作为master节点加入原集群

$ redis-cli --cluster add-node 192.168.150.110:6387 192.168.150.110:6381

6、4 检查集群情况

root@gr:/data# redis-cli --cluster check 192.168.150.110:6381
192.168.150.110:6384 (4d440430...) -> 1 keys | 5461 slots | 1 slaves.
192.168.150.110:6382 (8b3a4a57...) -> 0 keys | 5462 slots | 1 slaves.
192.168.150.110:6387 (1d2a6729...) -> 0 keys | 0 slots | 0 slaves.
192.168.150.110:6383 (aa5ed45e...) -> 1 keys | 5461 slots | 1 slaves.

可以看到新加入的节点redis-node-7此时并没有slot槽位。

6、5 重新分配槽位

$ redis-cli --cluster reshard 192.168.150.110:6381
How many slots do you want to move (from 1 to 16384)? 4096
What is the receiving node ID? 新加入节点的id
Please enter all the source node IDs.
  Type 'all' to use all the nodes as source nodes for the hash slots.
  Type 'done' once you entered all the source nodes IDs.
Source node #1: all

6、6 再次检查集群情况

$ redis-cli --cluster check 192.168.150.110:6381
192.168.150.110:6384 (4d440430...) -> 0 keys | 4096 slots | 1 slaves.
192.168.150.110:6382 (8b3a4a57...) -> 0 keys | 4096 slots | 1 slaves.
192.168.150.110:6387 (1d2a6729...) -> 1 keys | 4096 slots | 0 slaves.
192.168.150.110:6383 (aa5ed45e...) -> 1 keys | 4096 slots | 1 slaves.

可以看到新加入的节点已经分配了4096个槽位。

6、7 为新加入的主节点分配从节点

$ redis-cli --cluster add-node 192.168.150.110:6388 192.168.150.110:6387 --cluster-slave --cluster-master-id 新加入集群节点的id

6、8 再次查看集群情况

$ redis-cli --cluster check 192.168.150.110:6381
192.168.150.110:6384 (4d440430...) -> 0 keys | 4096 slots | 1 slaves.
192.168.150.110:6382 (8b3a4a57...) -> 0 keys | 4096 slots | 1 slaves.
192.168.150.110:6387 (1d2a6729...) -> 1 keys | 4096 slots | 1 slaves.
192.168.150.110:6383 (aa5ed45e...) -> 1 keys | 4096 slots | 1 slaves.
[OK] 2 keys in 4 masters.
0.00 keys per slot on average.
>>> Performing Cluster Check (using node 192.168.150.110:6381)
S: 8b12adc7fff6e626fddbfe46b69b0a04051148c8 192.168.150.110:6381
   slots: (0 slots) slave
   replicates 4d4404303675e0fb887a73dccd0ae9e5f10edaf5
M: 4d4404303675e0fb887a73dccd0ae9e5f10edaf5 192.168.150.110:6384
   slots:[1365-5460] (4096 slots) master
   1 additional replica(s)
S: af0234e0d13117a44c8dc5c619bccfa1fdb4ff7a 192.168.150.110:6388
   slots: (0 slots) slave
....

可以看到从节点已经加入集群中，至此，redis集群的扩容已经成功。

7、redis集群缩容：

将加入集群的节点redis-node-7和redis-node-8从集群中删除。

7、1 删除从节点redis-node-8

$ redis-cli --cluster del-node 192.168.150.110:6388 redis-node-8的id

7、2 检查是否删除成功

$ redis-cli --cluster check 192.168.150.110:6382
192.168.150.110:6382 (8b3a4a57...) -> 0 keys | 4096 slots | 1 slaves.
192.168.150.110:6387 (1d2a6729...) -> 1 keys | 4096 slots | 0 slaves.
192.168.150.110:6383 (aa5ed45e...) -> 1 keys | 4096 slots | 1 slaves.
192.168.150.110:6384 (4d440430...) -> 0 keys | 4096 slots | 1 slaves.
[OK] 2 keys in 4 masters.
0.00 keys per slot on average.
>>> Performing Cluster Check (using node 192.168.150.110:6382)
M: 8b3a4a57bff74c389e7a3e4edbdb3d5edf2a613d 192.168.150.110:6382
   slots:[6827-10922] (4096 slots) master
   1 additional replica(s)
M: 1d2a6729c7c582d3e725e978c9511865e75a5e2f 192.168.150.110:6387
   slots:[0-1364],[5461-6826],[10923-12287] (4096 slots) master
M: aa5ed45e1c2b5aeb6b3ed34337af9b90ace91e51 192.168.150.110:6383
   slots:[12288-16383] (4096 slots) master
   1 additional replica(s)
S: 2dee6046534340e3f7db45e32fe02cf94b7b4e32 192.168.150.110:6386
   slots: (0 slots) slave
   replicates aa5ed45e1c2b5aeb6b3ed34337af9b90ace91e51
M: 4d4404303675e0fb887a73dccd0ae9e5f10edaf5 192.168.150.110:6384
   slots:[1365-5460] (4096 slots) master
   1 additional replica(s)
S: 8b12adc7fff6e626fddbfe46b69b0a04051148c8 192.168.150.110:6381
   slots: (0 slots) slave
   replicates 4d4404303675e0fb887a73dccd0ae9e5f10edaf5
S: 1f46effff69e974a1586b67082570e70961b9abe 192.168.150.110:6385
   slots: (0 slots) slave
   replicates 8b3a4a57bff74c389e7a3e4edbdb3d5edf2a613d

可以看到redis-node-8已经成功删除。

7、3 清空主节点redis-node-7的槽位(可以分批分配出去，这里我直接分配给node4节点)

$ redis-cli --cluster reshard 192.168.150.110:6384
How many slots do you want to move (from 1 to 16384)? 4096
What is the receiving node ID? node4节点的Id
Please enter all the source node IDs.
  Type 'all' to use all the nodes as source nodes for the hash slots.
  Type 'done' once you entered all the source nodes IDs.
Source node #1: 要删除节点的id
Source node #2: done

7、4 查看要删除节点的槽位是否被分配

$ redis-cli --cluster check 192.168.150.110:6382
192.168.150.110:6382 (8b3a4a57...) -> 0 keys | 4096 slots | 1 slaves.
192.168.150.110:6387 (1d2a6729...) -> 0 keys | 0 slots | 0 slaves.
192.168.150.110:6383 (aa5ed45e...) -> 1 keys | 4096 slots | 1 slaves.
192.168.150.110:6384 (4d440430...) -> 1 keys | 8192 slots | 1 slaves.

可以看到槽位已经被分配。

7、5 删除节点

$ redis-cli --cluster del-node 192.168.150.110:6387 要删除节点的id

7、6 检查是否集群节点是否删除

$ redis-cli --cluster check 192.168.150.110:6382
192.168.150.110:6382 (8b3a4a57...) -> 0 keys | 4096 slots | 1 slaves.
192.168.150.110:6383 (aa5ed45e...) -> 1 keys | 4096 slots | 1 slaves.
192.168.150.110:6384 (4d440430...) -> 1 keys | 8192 slots | 1 slaves.

可以看到节点已经被删除了。

至此，redis3主3从集群搭建和集群扩缩容已经实现。

本文章为转载内容，我们尊重原作者对文章享有的著作权。如有内容错误或侵权问题，欢迎原作者联系我们进行内容更正或删除文章。