写在前面的话:

对于redis来说,它有四种部署模式,分别是单机模式、主从模式、哨兵模式和集群模式,他们的使用场景有些区别,当然也是越来越复杂,可靠性越来越高。

本文从实际操作的角度,来介绍和讲解下,这几种模式的特点,鉴于篇幅的问题,文章分成两篇,一篇用来介绍:单机模式、主从模式和哨兵模式;本篇文章是,另外一个模式的介绍:集群模式。

一、集群模式解决什么问题呢?

集群模式用来解决Redis的在线扩容的问题,原因是:Redis容量受限于单机配置的问题,并且前面的几种模式并不能很好的做到这一点。

集群模式有三种方式:客户端实现、Proxy代理层、服务端实现。

本文章主要讲解服务端实现方式,服务端的实现方式就是标准的集群(分区分片)模式,RedisCluster是Redis在3.0版本后推出的分布式解决方案。

Cluster模式实现了Redis的分布式存储,即每台节点存储不同的内容,来解决在线扩容的问题。

二、Cluster模式的工作机制是什么?

在Redis的每个节点上,都有一个插槽(slot),总共16384个哈希槽,取值范围为0-16383。采用Slot的设计(一个集群有多个主从节点,一个主从节点上会分配多个Slot槽,每个槽点上存的是Key-Value数据):

当我们存取key的时候,Redis会根据CRC16的算法得出一个结果,然后把结果对16384求余数,这样每个key都会对应一个编号在0-16383之间的哈希槽,通过这个值,去找到对应的插槽所对应的节点,然后直接自动跳转到这个对应的节点上进行存取操作。如图所示:

redis生产集群部署 redis集群部署模式_分布式

1.为了保证高可用,Cluster模式也引入主从复制模式,一个主节点对应一个或者多个从节点,
当主节点宕机的时候,就会启用从节点。(备注:这里的从节点只是做备份来用,不作为读写分离来操作。)
2.当其它主节点ping一个主节点A时,如果半数以上的主节点与A通信超时,那么认为主节点A宕机了。
如果主节点A和它的从节点都宕机了,那么该集群就无法再提供服务了。


备注:Cluster模式集群节点最小配置6个节点(3主3从,因为需要半数以上)

三、集群部署实际操作

1. 配置和启动集群

1)修改配置文件,这里分别启动7001~7006六个redis服务:

#修改成自己对应的端口号
port 7001 
#指定了记录日志的文件。
logfile 
#数据目录,数据库的写入会在这个目录。rdb、aof文件也会写在这个目录
dir 
#是否开启集群
cluster-enabled
#集群配置文件的名称,每个节点都有一个集群相关的配置文件,持久化保存集群的信息。
#这个文件并不需要手动配置,这个配置文件由Redis生成并更新,每个Redis集群节点需要一个单独的配置文件,请确保与实例运行的系统中配置文件名称不冲突(建议配对应端口号)
cluster-config-file nodes-7001.conf
#节点互连超时的阀值。集群节点超时毫秒数
cluster-node-timeout 15000 
#默认redis使用的是rdb方式持久化,这种方式在许多应用中已经足够用了。
#但是redis如果中途宕机,会导致可能有几分钟的数据丢失,根据save来策略进行持久化,Append Only File是另一种持久化方式,可以提供更好的持久化特性。
#Redis会把每次写入的数据在接收后都写入 appendonly.aof 文件,每次启动时Redis都会先把这个文件的数据读入内存里,先忽略RDB文件。
appendonly yes
protected-mode no #保护模式 yes改为no
bind 127.0.0.1 #本机试验,使用这个ip,这里也可以指定为集群的IP
daemonize yes   #用来指定redis是否要用守护线程的方式启动,yes表示后台启动

2)启动redis-server:

按照下面的命令来分别启动7001~7006六个服务:

$ redis-server /usr/local/etc/redis7005.conf
14903:C 05 Jan 2021 11:50:05.601 # oO0OoO0OoO0Oo Redis is starting oO0OoO0OoO0Oo
14903:C 05 Jan 2021 11:50:05.601 # Redis version=5.0.5, bits=64, commit=00000000, modified=0, pid=14903, just started
14903:C 05 Jan 2021 11:50:05.601 # Configuration loaded

显示启动的redis-server:

$ ps -ef | grep redis
  501 14867     1   0 11:45上午 ??         0:00.66 redis-server 127.0.0.1:7001 [cluster]
  501 14884     1   0 11:48上午 ??         0:00.17 redis-server 127.0.0.1:7002 [cluster]
  501 14887     1   0 11:49上午 ??         0:00.08 redis-server 127.0.0.1:7003 [cluster]
  501 14890     1   0 11:49上午 ??         0:00.06 redis-server 127.0.0.1:7004 [cluster]
  501 28799     1   0 171120  ??        45:20.39 redis-server 127.0.0.1:7005 [cluster]
  501 28801     1   0 171120  ??        47:39.12 redis-server 127.0.0.1:7006 [cluster]
  501 13669   841   0 10:11上午 ttys002    0:00.01 redis-cli -h 127.0.0.1 -p 7001
  501 14897 14516   0 11:49上午 ttys007    0:00.00 grep redis

3)设置为cluster集群模式:

命令:    redis-cli --cluster create 127.0.0.1:7001 127.0.0.1:7002 127.0.0.1:7003 127.0.0.1:7004 127.0.0.1:7005 127.0.0.1:7006 --cluster-replicas 1

# 1表示每个主节点至少一个备份
$ redis-cli --cluster create 127.0.0.1:7001 127.0.0.1:7002 127.0.0.1:7003 127.0.0.1:7004 127.0.0.1:7005 127.0.0.1:7006 --cluster-replicas 1>>> Performing hash slots allocation on 6 nodes...
Master[0] -> Slots 0 - 5460
Master[1] -> Slots 5461 - 10922
Master[2] -> Slots 10923 - 16383
Adding replica 127.0.0.1:7005 to 127.0.0.1:7001
Adding replica 127.0.0.1:7006 to 127.0.0.1:7002
Adding replica 127.0.0.1:7004 to 127.0.0.1:7003
>>> Trying to optimize slaves allocation for anti-affinity
[WARNING] Some slaves are in the same host as their master
M: 4359ede424a92c7f249fb6524c6cc3f6dc46fb60 127.0.0.1:7001
   slots:[0-5460] (5461 slots) master
M: 2c779d5cc0b786af908d1c76dd3d4968e15796b3 127.0.0.1:7002
   slots:[5461-10922] (5462 slots) master
M: 62c2ac721e402a27f017be49589bc06532548cdd 127.0.0.1:7003
   slots:[10923-16383] (5461 slots) master
S: 2b51a786e6f204c292dc349b45aa8b515637382c 127.0.0.1:7004
   replicates 62c2ac721e402a27f017be49589bc06532548cdd
S: 357221f17c5f5bf11d0551506fbff245ed0c047b 127.0.0.1:7005
   replicates 4359ede424a92c7f249fb6524c6cc3f6dc46fb60
S: 2ecce47670970e13a034a37dd425154d298dc480 127.0.0.1:7006
   replicates 2c779d5cc0b786af908d1c76dd3d4968e15796b3
Can I set the above configuration? (type 'yes' to accept): yes
>>> Nodes configuration updated
>>> Assign a different config epoch to each node
>>> Sending CLUSTER MEET messages to join the cluster
Waiting for the cluster to join
.....
>>> Performing Cluster Check (using node 127.0.0.1:7001)
M: 4359ede424a92c7f249fb6524c6cc3f6dc46fb60 127.0.0.1:7001
   slots:[0-5460] (5461 slots) master
   1 additional replica(s)
S: 2ecce47670970e13a034a37dd425154d298dc480 127.0.0.1:7006
   slots: (0 slots) slave
   replicates 2c779d5cc0b786af908d1c76dd3d4968e15796b3
M: 2c779d5cc0b786af908d1c76dd3d4968e15796b3 127.0.0.1:7002
   slots:[5461-10922] (5462 slots) master
   1 additional replica(s)
S: 2b51a786e6f204c292dc349b45aa8b515637382c 127.0.0.1:7004
   slots: (0 slots) slave
   replicates 62c2ac721e402a27f017be49589bc06532548cdd
M: 62c2ac721e402a27f017be49589bc06532548cdd 127.0.0.1:7003
   slots:[10923-16383] (5461 slots) master
   1 additional replica(s)
S: 357221f17c5f5bf11d0551506fbff245ed0c047b 127.0.0.1:7005
   slots: (0 slots) slave
   replicates 4359ede424a92c7f249fb6524c6cc3f6dc46fb60
[OK] All nodes agree about slots configuration.
>>> Check for open slots...
>>> Check slots coverage...
[OK] All 16384 slots covered.

备注:这一步碰到的问题:

$ redis-cli --cluster create 127.0.0.1:7001 127.0.0.1:7002 127.0.0.1:7003 127.0.0.1:7004 127.0.0.1:7005 127.0.0.1:7006 --cluster-replicas 1
[ERR] Node 127.0.0.1:7003 is not empty. Either the node already knows other nodes (check with CLUSTER NODES) or contains some key in database 0.

解决方法:

1)将每个节点下aof、rdb、nodes.conf本地备份文件删除;
2)172.168.63.201:7001> flushdb #清空当前数据库(可省略)

$redis-cli -p 7000
127.0.0.1:7000> flushall
127.0.0.1:7000> cluster reset
127.0.0.1:7000> exit

3)之后再执行脚本,成功执行

详情可以参考:

https://www.jianshu.com/p/338bc2a74300

https://stackoverflow.com/questions/37206993/redis-server-cluster-not-working

2. 集群检查

1) 查看集群信息以及slot分配信息:

$ redis-cli -h 127.0.0.1 -p 7001
127.0.0.1:7001> cluster node
(error) ERR Unknown subcommand or wrong number of arguments for 'node'. Try CLUSTER HELP.
127.0.0.1:7001> cluster nodes
2ecce47670970e13a034a37dd425154d298dc480 127.0.0.1:7006@17006 slave 2c779d5cc0b786af908d1c76dd3d4968e15796b3 0 1609823130000 6 connected
2c779d5cc0b786af908d1c76dd3d4968e15796b3 127.0.0.1:7002@17002 master - 0 1609823129000 2 connected 5461-10922
2b51a786e6f204c292dc349b45aa8b515637382c 127.0.0.1:7004@17004 slave 62c2ac721e402a27f017be49589bc06532548cdd 0 1609823128000 4 connected
4359ede424a92c7f249fb6524c6cc3f6dc46fb60 127.0.0.1:7001@17001 myself,master - 0 1609823128000 1 connected 0-5460
62c2ac721e402a27f017be49589bc06532548cdd 127.0.0.1:7003@17003 master - 0 1609823130975 3 connected 10923-16383
357221f17c5f5bf11d0551506fbff245ed0c047b 127.0.0.1:7005@17005 slave 4359ede424a92c7f249fb6524c6cc3f6dc46fb60 0 1609823129000 5 connected
127.0.0.1:7001>

介绍:可以根据以上得出总共有3个master节点,每个节点有一个slave节点,三个master节点分别分配了0-5460、5461-10922、10923-16383三个slot范围,7001这个master节点的slave节点为7005节点,其id为4359ede424a92c7f249fb6524c6cc3f6dc46fb60

2)添加数据,数据会根hash然后存到对应的slot槽,会存到slot槽对应的节点上

$ redis-cli -c -h 127.0.0.1 -p 7001 // -c 是集群方式启动
127.0.0.1:7001> set 122 11 // hash在本机7001
OK
127.0.0.1:7001> set 1221 222 // hash到了7003
-> Redirected to slot [13341] located at 127.0.0.1:7003
OK
127.0.0.1:7003> set 12233 222 // hash到了7002
-> Redirected to slot [6669] located at 127.0.0.1:7002
OK
127.0.0.1:7002> set 12233 111 // hash到了7003
OK
127.0.0.1:7002> keys *
1) "12233"
127.0.0.1:7002>

备注:

127.0.0.1:7002> set 6666 6 
(error) MOVED 1491 127.0.0.1:7001 // 提示这个错误是因为没有按照集群启动redis-cli

3.宕机测试

测试步骤:关掉7001,查看集群,发现7001的slave变成master,重启7001,发现7001变成7005的salve。

1)关掉7001

$ kill 14867
$ ps -ef | grep redis
  501 14884     1   0 11:48上午 ??         0:14.18 redis-server 127.0.0.1:7002 [cluster]
  501 14887     1   0 11:49上午 ??         0:13.94 redis-server 127.0.0.1:7003 [cluster]
  501 14890     1   0 11:49上午 ??         0:14.02 redis-server 127.0.0.1:7004 [cluster]
  501 14904     1   0 11:50上午 ??         0:13.84 redis-server 127.0.0.1:7005 [cluster]
  501 14907     1   0 11:50上午 ??         0:13.65 redis-server 127.0.0.1:7006 [cluster]
  501 13669   841   0 10:11上午 ttys002    0:00.01 redis-cli -h 127.0.0.1 -p 7001
  501 15295  7048   0  1:26下午 ttys004    0:00.02 redis-cli -c -h 127.0.0.1 -p 7001
  501 15319 14516   0  1:32下午 ttys007    0:00.00 grep redis

2) 7005变成master

127.0.0.1:7003> cluster nodes
2b51a786e6f204c292dc349b45aa8b515637382c 127.0.0.1:7004@17004 slave 62c2ac721e402a27f017be49589bc06532548cdd 0 1609824784402 4 connected
357221f17c5f5bf11d0551506fbff245ed0c047b 127.0.0.1:7005@17005 master - 0 1609824786425 7 connected 0-5460
2c779d5cc0b786af908d1c76dd3d4968e15796b3 127.0.0.1:7002@17002 master - 0 1609824785000 2 connected 5461-10922
62c2ac721e402a27f017be49589bc06532548cdd 127.0.0.1:7003@17003 myself,master - 0 1609824780000 3 connected 10923-16383
4359ede424a92c7f249fb6524c6cc3f6dc46fb60 127.0.0.1:7001@17001 master,fail - 1609824748365 1609824746000 1 disconnected
2ecce47670970e13a034a37dd425154d298dc480 127.0.0.1:7006@17006 slave 2c779d5cc0b786af908d1c76dd3d4968e15796b3 0 1609824785413 6 connected
127.0.0.1:7003>

3)重启7001,变成7005的salve

$ redis-server /usr/local/etc/redis7001.conf
15354:C 05 Jan 2021 13:39:21.856 # oO0OoO0OoO0Oo Redis is starting oO0OoO0OoO0Oo
15354:C 05 Jan 2021 13:39:21.856 # Redis version=5.0.5, bits=64, commit=00000000, modified=0, pid=15354, just started
15354:C 05 Jan 2021 13:39:21.856 # Configuration loaded
$ ps -ef | grep redis
  501 14884     1   0 11:48上午 ??         0:16.23 redis-server 127.0.0.1:7002 [cluster]
  501 14887     1   0 11:49上午 ??         0:15.99 redis-server 127.0.0.1:7003 [cluster]
  501 14890     1   0 11:49上午 ??         0:16.05 redis-server 127.0.0.1:7004 [cluster]
  501 14904     1   0 11:50上午 ??         0:15.88 redis-server 127.0.0.1:7005 [cluster]
  501 14907     1   0 11:50上午 ??         0:15.70 redis-server 127.0.0.1:7006 [cluster]
  501 15355     1   0  1:39下午 ??         0:00.02 redis-server 127.0.0.1:7001 [cluster]
  501 13669   841   0 10:11上午 ttys002    0:00.01 redis-cli -h 127.0.0.1 -p 7001
  501 15295  7048   0  1:26下午 ttys004    0:00.02 redis-cli -c -h 127.0.0.1 -p 7001
  501 15360 14516   0  1:39下午 ttys007    0:00.00 grep red

cluster nodes查看:

127.0.0.1:7003> cluster nodes
2b51a786e6f204c292dc349b45aa8b515637382c 127.0.0.1:7004@17004 slave 62c2ac721e402a27f017be49589bc06532548cdd 0 1609825212544 4 connected
357221f17c5f5bf11d0551506fbff245ed0c047b 127.0.0.1:7005@17005 master - 0 1609825211000 7 connected 0-5460
2c779d5cc0b786af908d1c76dd3d4968e15796b3 127.0.0.1:7002@17002 master - 0 1609825209000 2 connected 5461-10922
62c2ac721e402a27f017be49589bc06532548cdd 127.0.0.1:7003@17003 myself,master - 0 1609825208000 3 connected 10923-16383
4359ede424a92c7f249fb6524c6cc3f6dc46fb60 127.0.0.1:7001@17001 slave 357221f17c5f5bf11d0551506fbff245ed0c047b 0 1609825210000 7 connected
2ecce47670970e13a034a37dd425154d298dc480 127.0.0.1:7006@17006 slave 2c779d5cc0b786af908d1c76dd3d4968e15796b3 0 1609825211533 6 connected
127.0.0.1:7003>

4.水平拓展

场景是集群能力不够的时候,扩容用的,下面新增7007,7008来演示下。

1) 新增集群配置 7007和7008,按照前面方法操作,启动如下:

$ ps -ef | grep redis
  501 14884     1   0 11:48上午 ??         0:19.85 redis-server 127.0.0.1:7002 [cluster]
  501 14887     1   0 11:49上午 ??         0:19.64 redis-server 127.0.0.1:7003 [cluster]
  501 14890     1   0 11:49上午 ??         0:19.64 redis-server 127.0.0.1:7004 [cluster]
  501 14904     1   0 11:50上午 ??         0:19.51 redis-server 127.0.0.1:7005 [cluster]
  501 14907     1   0 11:50上午 ??         0:19.29 redis-server 127.0.0.1:7006 [cluster]
  501 15355     1   0  1:39下午 ??         0:03.61 redis-server 127.0.0.1:7001 [cluster]
  501 15426     1   0  1:54下午 ??         0:00.61 redis-server 127.0.0.1:7007 [cluster]
  501 15430     1   0  1:54下午 ??         0:00.59 redis-server 127.0.0.1:7008 [cluster]
  501 13669   841   0 10:11上午 ttys002    0:00.01 redis-cli -h 127.0.0.1 -p 7001
  501 15295  7048   0  1:26下午 ttys004    0:00.02 redis-cli -c -h 127.0.0.1 -p 7001
  501 15446 14516   0  1:57下午 ttys007    0:00.00 grep redis

尽管7007,7008启动了,但是集群中并没有这两个服务

127.0.0.1:7003> cluster nodes
2b51a786e6f204c292dc349b45aa8b515637382c 127.0.0.1:7004@17004 slave 62c2ac721e402a27f017be49589bc06532548cdd 0 1609827727127 4 connected
357221f17c5f5bf11d0551506fbff245ed0c047b 127.0.0.1:7005@17005 master - 0 1609827726116 7 connected 0-5460
2c779d5cc0b786af908d1c76dd3d4968e15796b3 127.0.0.1:7002@17002 master - 0 1609827724000 2 connected 5461-10922
62c2ac721e402a27f017be49589bc06532548cdd 127.0.0.1:7003@17003 myself,master - 0 1609827724000 3 connected 10923-16383
4359ede424a92c7f249fb6524c6cc3f6dc46fb60 127.0.0.1:7001@17001 slave 357221f17c5f5bf11d0551506fbff245ed0c047b 0 1609827725104 7 connected
2ecce47670970e13a034a37dd425154d298dc480 127.0.0.1:7006@17006 slave 2c779d5cc0b786af908d1c76dd3d4968e15796b3 0 1609827724094 6 connected
127.0.0.1:7003>

2)7007加入到集群中,并把他变为master

增加主节点7007,其中7002是集群里面随便选择的一个master节点。

$ redis-cli --cluster add-node 127.0.0.1:7007 127.0.0.1:7002
>>> Adding node 127.0.0.1:7007 to cluster 127.0.0.1:7002
>>> Performing Cluster Check (using node 127.0.0.1:7002)
M: 2c779d5cc0b786af908d1c76dd3d4968e15796b3 127.0.0.1:7002
   slots:[5461-10922] (5462 slots) master
   1 additional replica(s)
S: 2b51a786e6f204c292dc349b45aa8b515637382c 127.0.0.1:7004
   slots: (0 slots) slave
   replicates 62c2ac721e402a27f017be49589bc06532548cdd
M: 62c2ac721e402a27f017be49589bc06532548cdd 127.0.0.1:7003
   slots:[10923-16383] (5461 slots) master
   1 additional replica(s)
S: 4359ede424a92c7f249fb6524c6cc3f6dc46fb60 127.0.0.1:7001
   slots: (0 slots) slave
   replicates 357221f17c5f5bf11d0551506fbff245ed0c047b
M: 357221f17c5f5bf11d0551506fbff245ed0c047b 127.0.0.1:7005
   slots:[0-5460] (5461 slots) master
   1 additional replica(s)
S: 2ecce47670970e13a034a37dd425154d298dc480 127.0.0.1:7006
   slots: (0 slots) slave
   replicates 2c779d5cc0b786af908d1c76dd3d4968e15796b3
[OK] All nodes agree about slots configuration.
>>> Check for open slots...
>>> Check slots coverage...
[OK] All 16384 slots covered.
>>> Send CLUSTER MEET to node 127.0.0.1:7007 to make it join the cluster.
[OK] New node added correctly.

检查7007是否成功加入,但是发现7007没有index

$ redis-cli -c -h 127.0.0.1 -p 7001
127.0.0.1:7001> cluster nodes
357221f17c5f5bf11d0551506fbff245ed0c047b 127.0.0.1:7005@17005 master - 0 1609827919000 7 connected 0-5460
62c2ac721e402a27f017be49589bc06532548cdd 127.0.0.1:7003@17003 master - 0 1609827920739 3 connected 10923-16383
2ecce47670970e13a034a37dd425154d298dc480 127.0.0.1:7006@17006 slave 2c779d5cc0b786af908d1c76dd3d4968e15796b3 0 1609827917706 6 connected
2c779d5cc0b786af908d1c76dd3d4968e15796b3 127.0.0.1:7002@17002 master - 0 1609827919728 2 connected 5461-10922
2b51a786e6f204c292dc349b45aa8b515637382c 127.0.0.1:7004@17004 slave 62c2ac721e402a27f017be49589bc06532548cdd 0 1609827918000 4 connected
8dd6f8799e364e86e3df2acbbc42885482e5f526 127.0.0.1:7007@17007 master - 0 1609827917000 0 connected
4359ede424a92c7f249fb6524c6cc3f6dc46fb60 127.0.0.1:7001@17001 myself,slave 357221f17c5f5bf11d0551506fbff245ed0c047b 0 1609827918000 1 connected
127.0.0.1:7001>

分配hash槽位给7007:

命令:$redis-cli --cluster reshard 127.0.0.1:7002 // 7002是随机选的一个master

[OK] All 16384 slots covered.
How many slots do you want to move (from 1 to 16384)? 2000 // 这里设置7007所需要的 槽位数量
What is the receiving node ID? 8dd6f8799e364e86e3df2acbbc42885482e5f526 // 7007对应的id
Please enter all the source node IDs.
  Type 'all' to use all the nodes as source nodes for the hash slots.
  Type 'done' once you entered all the source nodes IDs.
Source node #1: all // 类型选择的是每一个master平均分配的方式。

检查分结果:

$ redis-cli -c -h 127.0.0.1 -p 7001
127.0.0.1:7001> cluster nodes
357221f17c5f5bf11d0551506fbff245ed0c047b 127.0.0.1:7005@17005 master - 0 1609828313000 7 connected 666-5460
62c2ac721e402a27f017be49589bc06532548cdd 127.0.0.1:7003@17003 master - 0 1609828315610 3 connected 11589-16383
2ecce47670970e13a034a37dd425154d298dc480 127.0.0.1:7006@17006 slave 2c779d5cc0b786af908d1c76dd3d4968e15796b3 0 1609828314000 6 connected
2c779d5cc0b786af908d1c76dd3d4968e15796b3 127.0.0.1:7002@17002 master - 0 1609828314596 2 connected 6128-10922
2b51a786e6f204c292dc349b45aa8b515637382c 127.0.0.1:7004@17004 slave 62c2ac721e402a27f017be49589bc06532548cdd 0 1609828313000 4 connected
8dd6f8799e364e86e3df2acbbc42885482e5f526 127.0.0.1:7007@17007 master - 0 1609828314000 8 connected 0-665 5461-6127 10923-11588
4359ede424a92c7f249fb6524c6cc3f6dc46fb60 127.0.0.1:7001@17001 myself,slave 357221f17c5f5bf11d0551506fbff245ed0c047b 0 1609828312000 1 connected

分析:发现每一个master都分配了一部分槽位给7007,如下:0-665 5461-6127 10923-11588

3) 7008加入到集群中,并把他变为salve

添加7008到集群:$ redis-cli --cluster add-node 127.0.0.1:7008 127.0.0.1:7002

[OK] All nodes agree about slots configuration.
>>> Check for open slots...
>>> Check slots coverage...
[OK] All 16384 slots covered.
>>> Send CLUSTER MEET to node 127.0.0.1:7008 to make it join the cluster.
[OK] New node added correctly.
$ redis-cli -c -h 127.0.0.1 -p 7001
127.0.0.1:7001> cluster nodes
558579e037a19ecc2fc541c121df379bba9ab853 127.0.0.1:7008@17008 master - 0 1609828830000 0 connected
357221f17c5f5bf11d0551506fbff245ed0c047b 127.0.0.1:7005@17005 master - 0 1609828831852 7 connected 666-5460
62c2ac721e402a27f017be49589bc06532548cdd 127.0.0.1:7003@17003 master - 0 1609828828000 3 connected 11589-16383
2ecce47670970e13a034a37dd425154d298dc480 127.0.0.1:7006@17006 slave 2c779d5cc0b786af908d1c76dd3d4968e15796b3 0 1609828830000 6 connected
2c779d5cc0b786af908d1c76dd3d4968e15796b3 127.0.0.1:7002@17002 master - 0 1609828829000 2 connected 6128-10922
2b51a786e6f204c292dc349b45aa8b515637382c 127.0.0.1:7004@17004 slave 62c2ac721e402a27f017be49589bc06532548cdd 0 1609828830000 4 connected
8dd6f8799e364e86e3df2acbbc42885482e5f526 127.0.0.1:7007@17007 master - 0 1609828830843 8 connected 0-665 5461-6127 10923-11588
4359ede424a92c7f249fb6524c6cc3f6dc46fb60 127.0.0.1:7001@17001 myself,slave 357221f17c5f5bf11d0551506fbff245ed0c047b 0 1609828828000 1 connected
127.0.0.1:7001>

指定7008的master为7007,命令:

cluster replicate 8dd6f8799e364e86e3df2acbbc42885482e5f526

$ redis-cli -c -h 127.0.0.1 -p 7008
127.0.0.1:7008> cluster replicate 8dd6f8799e364e86e3df2acbbc42885482e5f526
OK
127.0.0.1:7008> cluster nodes
558579e037a19ecc2fc541c121df379bba9ab853 127.0.0.1:7008@17008 myself,slave 8dd6f8799e364e86e3df2acbbc42885482e5f526 0 1609829022000 0 connected
4359ede424a92c7f249fb6524c6cc3f6dc46fb60 127.0.0.1:7001@17001 slave 357221f17c5f5bf11d0551506fbff245ed0c047b 0 1609829024000 7 connected
8dd6f8799e364e86e3df2acbbc42885482e5f526 127.0.0.1:7007@17007 master - 0 1609829023000 8 connected 0-665 5461-6127 10923-11588
357221f17c5f5bf11d0551506fbff245ed0c047b 127.0.0.1:7005@17005 master - 0 1609829020000 7 connected 666-5460
62c2ac721e402a27f017be49589bc06532548cdd 127.0.0.1:7003@17003 master - 0 1609829022956 3 connected 11589-16383
2b51a786e6f204c292dc349b45aa8b515637382c 127.0.0.1:7004@17004 slave 62c2ac721e402a27f017be49589bc06532548cdd 0 1609829023000 3 connected
2c779d5cc0b786af908d1c76dd3d4968e15796b3 127.0.0.1:7002@17002 master - 0 1609829024977 2 connected 6128-10922
2ecce47670970e13a034a37dd425154d298dc480 127.0.0.1:7006@17006 slave 2c779d5cc0b786af908d1c76dd3d4968e15796b3 0 1609829023966 2 connected
127.0.0.1:7008>

5. 节点下线

场景:与水平扩展相反的是节点下线,一旦不需要这么多服务或者节点坏掉,我们可以操作节点下线,并从集群中移除。

1)移除salve节点,命令:redis-cli --cluster del-node 127.0.0.1:7008 558579e037a19ecc2fc541c121df379bba9ab853

$ redis-cli --cluster del-node 127.0.0.1:7008 558579e037a19ecc2fc541c121df379bba9ab853
>>> Removing node 558579e037a19ecc2fc541c121df379bba9ab853 from cluster 127.0.0.1:7008
>>> Sending CLUSTER FORGET messages to the cluster...
>>> SHUTDOWN the node.
$ redis-cli -c -h 127.0.0.1 -p 7001
127.0.0.1:7001> cluster nodes
357221f17c5f5bf11d0551506fbff245ed0c047b 127.0.0.1:7005@17005 master - 0 1609829451017 7 connected 666-5460
62c2ac721e402a27f017be49589bc06532548cdd 127.0.0.1:7003@17003 master - 0 1609829449000 3 connected 11589-16383
2ecce47670970e13a034a37dd425154d298dc480 127.0.0.1:7006@17006 slave 2c779d5cc0b786af908d1c76dd3d4968e15796b3 0 1609829449000 6 connected
2c779d5cc0b786af908d1c76dd3d4968e15796b3 127.0.0.1:7002@17002 master - 0 1609829446000 2 connected 6128-10922
2b51a786e6f204c292dc349b45aa8b515637382c 127.0.0.1:7004@17004 slave 62c2ac721e402a27f017be49589bc06532548cdd 0 1609829450009 4 connected
8dd6f8799e364e86e3df2acbbc42885482e5f526 127.0.0.1:7007@17007 master - 0 1609829448000 8 connected 0-665 5461-6127 10923-11588
4359ede424a92c7f249fb6524c6cc3f6dc46fb60 127.0.0.1:7001@17001 myself,slave 357221f17c5f5bf11d0551506fbff245ed0c047b 0 1609829447000 1 connected
127.0.0.1:7001>

2)移除master节点

回收hash槽位给集群:

$ redis-cli --cluster reshard 127.0.0.1:7002 //将7007的槽位回收给7005
>>> Performing Cluster Check (using node 127.0.0.1:7007)
M: 8dd6f8799e364e86e3df2acbbc42885482e5f526 127.0.0.1:7007
   slots:[0-665],[5461-6127],[10923-11588] (1999 slots) master
S: 2b51a786e6f204c292dc349b45aa8b515637382c 127.0.0.1:7004
   slots: (0 slots) slave
   replicates 62c2ac721e402a27f017be49589bc06532548cdd
M: 62c2ac721e402a27f017be49589bc06532548cdd 127.0.0.1:7003
   slots:[11589-16383] (4795 slots) master
   1 additional replica(s)
S: 2ecce47670970e13a034a37dd425154d298dc480 127.0.0.1:7006
   slots: (0 slots) slave
   replicates 2c779d5cc0b786af908d1c76dd3d4968e15796b3
S: 4359ede424a92c7f249fb6524c6cc3f6dc46fb60 127.0.0.1:7001
   slots: (0 slots) slave
   replicates 357221f17c5f5bf11d0551506fbff245ed0c047b
M: 2c779d5cc0b786af908d1c76dd3d4968e15796b3 127.0.0.1:7002
   slots:[6128-10922] (4795 slots) master
   1 additional replica(s)
M: 357221f17c5f5bf11d0551506fbff245ed0c047b 127.0.0.1:7005
   slots:[666-5460] (4795 slots) master
   1 additional replica(s)
[OK] All nodes agree about slots configuration.
>>> Check for open slots...
>>> Check slots coverage...
[OK] All 16384 slots covered.
How many slots do you want to move (from 1 to 16384)? 2000
What is the receiving node ID? 357221f17c5f5bf11d0551506fbff245ed0c047b
Please enter all the source node IDs.
  Type 'all' to use all the nodes as source nodes for the hash slots.
  Type 'done' once you entered all the source nodes IDs.
Source node #1: 8dd6f8799e364e86e3df2acbbc42885482e5f526
Source node #2: done

查看hash槽位结果,7007变为了0:

$ redis-cli -c -h 127.0.0.1 -p 7001
127.0.0.1:7001> cluster nodes
357221f17c5f5bf11d0551506fbff245ed0c047b 127.0.0.1:7005@17005 master - 0 1609830719272 9 connected 0-8578 10923-12363
62c2ac721e402a27f017be49589bc06532548cdd 127.0.0.1:7003@17003 master - 0 1609830720283 3 connected 12364-16383
2ecce47670970e13a034a37dd425154d298dc480 127.0.0.1:7006@17006 slave 2c779d5cc0b786af908d1c76dd3d4968e15796b3 0 1609830721294 6 connected
2c779d5cc0b786af908d1c76dd3d4968e15796b3 127.0.0.1:7002@17002 master - 0 1609830717248 2 connected 8579-10922
2b51a786e6f204c292dc349b45aa8b515637382c 127.0.0.1:7004@17004 slave 62c2ac721e402a27f017be49589bc06532548cdd 0 1609830719000 4 connected
8dd6f8799e364e86e3df2acbbc42885482e5f526 127.0.0.1:7007@17007 master - 0 1609830718000 8 connected
4359ede424a92c7f249fb6524c6cc3f6dc46fb60 127.0.0.1:7001@17001 myself,slave 357221f17c5f5bf11d0551506fbff245ed0c047b 0 1609830718000 1 connected
127.0.0.1:7001>

下线7008,命令 redis-cli -p 7007 shutdown

$ redis-cli -p 7007 shutdown
$ redis-cli -c -h 127.0.0.1 -p 7001
127.0.0.1:7001> cluster nodes
357221f17c5f5bf11d0551506fbff245ed0c047b 127.0.0.1:7005@17005 master - 0 1609830849507 9 connected 0-8578 10923-12363
62c2ac721e402a27f017be49589bc06532548cdd 127.0.0.1:7003@17003 master - 0 1609830850515 3 connected 12364-16383
2ecce47670970e13a034a37dd425154d298dc480 127.0.0.1:7006@17006 slave 2c779d5cc0b786af908d1c76dd3d4968e15796b3 0 1609830850000 6 connected
2c779d5cc0b786af908d1c76dd3d4968e15796b3 127.0.0.1:7002@17002 master - 0 1609830846472 2 connected 8579-10922
2b51a786e6f204c292dc349b45aa8b515637382c 127.0.0.1:7004@17004 slave 62c2ac721e402a27f017be49589bc06532548cdd 0 1609830848000 4 connected
8dd6f8799e364e86e3df2acbbc42885482e5f526 127.0.0.1:7007@17007 master - 1609830839496 1609830838387 8 disconnected
4359ede424a92c7f249fb6524c6cc3f6dc46fb60 127.0.0.1:7001@17001 myself,slave 357221f17c5f5bf11d0551506fbff245ed0c047b 0 1609830848000 1 connected

参考文档:

redis的几种模式:
client的几种连接方式:
哨兵模式:
redis官方文档:https://godoc.org/gopkg.in/redis.v5
redis安装:
redis-cli命令:
redis命令:http://redisdoc.com/string/mset.html