周氏一族,整理技术文档,给下一代留点教程......


redis 3.0 cluster 安装篇,请看 http://zhoushouby.blog.51cto.com/9150272/1560400


本篇,是在 "redis 3.0 cluster 集群 学习之路篇 [2]" 的基础上面实现 “实际生产中Redis服务器部署” 安装调试以及测试。


六台服务器

server21      10.1.3.21         

server22      10.1.3.22

server23      10.1.3.23

server24      10.1.3.24

server25      10.1.3.25

server26      10.1.3.26

---------------------------------每一台/etc/hosts里面都有添加一一对应记录



redis.conf 配置如下

port  6379

daemonize yes

cluster-enabled yes

cluster-config-file /usr/local/redis/nodes.conf  

cluster-node-timeout 5000

dir /usr/local/redis

细心看,你会发现,上面多了一个路径,对的,个人习惯,我给redis创建了一个单独的目录,方便以后查找。


六台机器,所有配置一模一样,统一配置。


一路下来,完全没问题,每一台都能独立的运行redis服务

redis-server /usr/local/redis/redis.conf > /usr/local/redis/redis.log 2>&1 &


但是,当我要来给他们开始集群的时候,就发生下面这么一幕情况

[root@server21 src]# ./redis-trib.rb create --replicas 1 10.1.3.21:6379 10.1.3.22:6379 10.1.3.26:6379 10.1.3.25:6379 10.1.3.24:6379 10.1.3.23:6379

>>> Creating cluster

Connecting to node 10.1.3.21:6379: OK

Connecting to node 10.1.3.22:6379: OK

Connecting to node 10.1.3.26:6379: OK

Connecting to node 10.1.3.25:6379: OK

Connecting to node 10.1.3.24:6379: OK

Connecting to node 10.1.3.23:6379: OK

>>> Performing hash slots allocation on 6 nodes...

Using 3 masters:

10.1.3.23:6379

10.1.3.24:6379

10.1.3.25:6379

Adding replica 10.1.3.26:6379 to 10.1.3.23:6379

Adding replica 10.1.3.22:7379 to 10.1.3.24:6379

Adding replica 10.1.3.21:7379 to 10.1.3.25:6379

S: fe32667413d75aa9dcebf28aa53fe4ed0df91bba 10.1.3.21:6379

   replicates 7f4c3a0ea64c9f85e7d239c8968c94818decb9a8

S: 3b10b61ebf7ff4e618a945aa353e9e113541557a 10.1.3.22:6379

   replicates f6c6fac2ada651819fac2ddc0daa91f54062a261

S: 0f9a72b4cb6b1ec2d20cc5f0d82be29925c8a050 10.1.3.26:6379

   replicates 96dda21bf7dc9ba81c160375b39a59aedf78733d

M: 7f4c3a0ea64c9f85e7d239c8968c94818decb9a8 10.1.3.25:6379

   slots:10923-16383 (5461 slots) master

M: f6c6fac2ada651819fac2ddc0daa91f54062a261 10.1.3.24:6379

   slots:0-10922 (10923 slots) master

M: 96dda21bf7dc9ba81c160375b39a59aedf78733d 10.1.3.23:6379

   slots:0-10922 (10923 slots) master

Can I set the above configuration? (type 'yes' to accept): yes

/usr/lib64/ruby/gems/1.8/gems/redis-3.0.7/lib/redis/client.rb:97:in `call': ERR Slot 16011 is already busy (Redis::CommandError)

        from /usr/lib64/ruby/gems/1.8/gems/redis-3.0.7/lib/redis.rb:2432:in `method_missing'

        from /usr/lib64/ruby/gems/1.8/gems/redis-3.0.7/lib/redis.rb:37:in `synchronize'

        from /usr/lib/ruby/1.8/monitor.rb:242:in `mon_synchronize'

        from /usr/lib64/ruby/gems/1.8/gems/redis-3.0.7/lib/redis.rb:37:in `synchronize'

        from /usr/lib64/ruby/gems/1.8/gems/redis-3.0.7/lib/redis.rb:2431:in `method_missing'

        from ./redis-trib.rb:203:in `flush_node_config'

        from ./redis-trib.rb:629:in `flush_nodes_config'

        from ./redis-trib.rb:628:in `each'

        from ./redis-trib.rb:628:in `flush_nodes_config'

        from ./redis-trib.rb:964:in `create_cluster_cmd'

        from ./redis-trib.rb:1340:in `send'

        from ./redis-trib.rb:1340

[root@server21 src]# 


人生,从此不再寂寞了,开启蛋疼模式


开始找问题,蛋疼的就是他一个错误日志都没有,没法下手。

没办法之下,只好问度娘了,但是,网上看得我眼花缭乱,在大陆,只能找到这么一出玩意

http://blog.csdn.net/moxiaomomo/article/details/19333547


感觉有点道理,因为我的生产环境,之前是redis 2.x的版本,今天更新到3.0,可能会是我redis没卸载干净导致的。

好,二话不说,把服务器再清理一遍,然后重启一下。因为安装步骤都是按照我之前的来,没出现过问题,而且之前在VM里面装的时候,VM的系统是全新的,所以觉得他这个有点道理。开始捣腾起来了。


半个小时候过后,问题依旧,又来到这个问题了。


最终没办法,翻一下墙,泡一下老外redis开源问答区,碰见这个问题的,还不少,比中国大陆好多了,至少搜索下来,有那么10几篇文章,都是在咨询这个问题,但是没有最终的答案


别人不行,那就只能靠自己了。

/usr/lib64/ruby/gems/1.8/gems/redis-3.0.7/lib/redis/client.rb:97:

他提示我说97行这里不行,那就编辑他,gg到87行这里定位一下


终于,总算,找到问题所在了。这个是因为port和nodes文件的问题,不能设置成相同nodes名字的,而且端口也设置成不一样。


晒一下,我最终的配置

[root@server21 ~]# ll /usr/local/redis/

total 60

-rw-r--r-- 1 root root    89 Oct  3 20:02 appendonly.aof

-rw-r--r-- 1 root root    18 Oct  3 20:17 dump.rdb

-rw-r--r-- 1 root root   733 Oct  3 19:30 nodes-8021.conf

-rw-r--r-- 1 root root   115 Oct  3 17:27 nodes.conf

-rw-r--r-- 1 root root 37019 Oct  3 18:48 redis.conf

-rw-r--r-- 1 root root    22 Oct  3 18:56 redis.log

[root@server21 ~]# 


server21      port  8021       nodes-8021.conf

server22      port  8022       nodes-8022.conf

server23      port  8023       nodes-8023.conf

server24      port  8024       nodes-8024.conf

server25      port  8025       nodes-8025.conf

server26      port  8026       nodes-8026.conf


折腾了快两个小时,最后一次通过,这种感觉超爽!!!

[root@server21 src]# ./redis-trib.rb create --replicas 1 10.1.3.21:8021 10.1.3.22:8022 10.1.3.23:8023 10.1.3.24:8024 10.1.3.25:8025 10.1.3.26:8026

>>> Creating cluster

Connecting to node 10.1.3.21:8021: OK

Connecting to node 10.1.3.22:8022: OK

Connecting to node 10.1.3.23:8023: OK

Connecting to node 10.1.3.24:8024: OK

Connecting to node 10.1.3.25:8025: OK

Connecting to node 10.1.3.26:8026: OK

>>> Performing hash slots allocation on 6 nodes...

Using 3 masters:

10.1.3.26:8026

10.1.3.25:8025

10.1.3.24:8024

Adding replica 10.1.3.23:8023 to 10.1.3.26:8026

Adding replica 10.1.3.22:8022 to 10.1.3.25:8025

Adding replica 10.1.3.21:8021 to 10.1.3.24:8024

S: 690bb1a31fff14557483e1e6dea60e5e748fff41 10.1.3.21:8021

   replicates 89efca75e220cdeb159e1e4954b953337563683f

S: ae258d0a8e31d2cac64aaad62327ec85c3ce12d7 10.1.3.22:8022

   replicates da00c6e9cf5f572396d7cfb006cbcb98e69928ee

S: 8d45084d0ec34c4547c7e57228a5db0ba7eeeba2 10.1.3.23:8023

   replicates e25ffd0663bf13d5b6aa6a397ffb0c21e4b11e05

M: 89efca75e220cdeb159e1e4954b953337563683f 10.1.3.24:8024

   slots:10923-16383 (5461 slots) master

M: da00c6e9cf5f572396d7cfb006cbcb98e69928ee 10.1.3.25:8025

   slots:5461-10922 (5462 slots) master

M: e25ffd0663bf13d5b6aa6a397ffb0c21e4b11e05 10.1.3.26:8026

   slots:0-5460 (5461 slots) master

Can I set the above configuration? (type 'yes' to accept): yes

>>> Nodes configuration updated

>>> Assign a different config epoch to each node

>>> Sending CLUSTER MEET messages to join the cluster

Waiting for the cluster to join...

>>> Performing Cluster Check (using node 10.1.3.21:8021)

M: 690bb1a31fff14557483e1e6dea60e5e748fff41 10.1.3.21:8021

   slots: (0 slots) master

   replicates 89efca75e220cdeb159e1e4954b953337563683f

M: ae258d0a8e31d2cac64aaad62327ec85c3ce12d7 10.1.3.22:8022

   slots: (0 slots) master

   replicates da00c6e9cf5f572396d7cfb006cbcb98e69928ee

M: 8d45084d0ec34c4547c7e57228a5db0ba7eeeba2 10.1.3.23:8023

   slots: (0 slots) master

   replicates e25ffd0663bf13d5b6aa6a397ffb0c21e4b11e05

M: 89efca75e220cdeb159e1e4954b953337563683f 10.1.3.24:8024

   slots:10923-16383 (5461 slots) master

M: da00c6e9cf5f572396d7cfb006cbcb98e69928ee 10.1.3.25:8025

   slots:5461-10922 (5462 slots) master

M: e25ffd0663bf13d5b6aa6a397ffb0c21e4b11e05 10.1.3.26:8026

   slots:0-5460 (5461 slots) master

[OK] All nodes agree about slots configuration.

>>> Check for open slots...

>>> Check slots coverage...

[OK] All 16384 slots covered.

[root@server21 src]# ./redis-trib.rb check 10.1.3.21:8021

Connecting to node 10.1.3.21:8021: OK

Connecting to node 10.1.3.23:8023: OK

Connecting to node 10.1.3.26:8026: OK

Connecting to node 10.1.3.25:8025: OK

Connecting to node 10.1.3.24:8024: OK

Connecting to node 10.1.3.22:8022: OK

>>> Performing Cluster Check (using node 10.1.3.21:8021)

S: 690bb1a31fff14557483e1e6dea60e5e748fff41 10.1.3.21:8021

   slots: (0 slots) slave

   replicates 89efca75e220cdeb159e1e4954b953337563683f

S: 8d45084d0ec34c4547c7e57228a5db0ba7eeeba2 10.1.3.23:8023

   slots: (0 slots) slave

   replicates e25ffd0663bf13d5b6aa6a397ffb0c21e4b11e05

M: e25ffd0663bf13d5b6aa6a397ffb0c21e4b11e05 10.1.3.26:8026

   slots:0-5460 (5461 slots) master

   1 additional replica(s)

M: da00c6e9cf5f572396d7cfb006cbcb98e69928ee 10.1.3.25:8025

   slots:5461-10922 (5462 slots) master

   1 additional replica(s)

M: 89efca75e220cdeb159e1e4954b953337563683f 10.1.3.24:8024

   slots:10923-16383 (5461 slots) master

   1 additional replica(s)

S: ae258d0a8e31d2cac64aaad62327ec85c3ce12d7 10.1.3.22:8022

   slots: (0 slots) slave

   replicates da00c6e9cf5f572396d7cfb006cbcb98e69928ee

[OK] All nodes agree about slots configuration.

>>> Check for open slots...

>>> Check slots coverage...

[OK] All 16384 slots covered.

[root@server21 src]# 


写下来,给后来者留点脚印......