一、redis集群特点
数据 在多个Redis节点之间自动分片
sentinel特点:
它的主要功能有以下几点
不时地监控redis是否按照预期良好地运行;
如果发现某个redis节点运行出现状况,能够通知另外一个进程(例如它的客户端);
能够进行自动切换。当一个master节点不可用时,能够选举出master的多个slave(如果有超过一个slave的话)中的一个来作为新的master,其它的slave节点会将它所追随的master的地址改为被提升为master的slave的新地址。
sentinel配置文件详解
主节点down了,从节点选举机制如下
二、redis端口
对于每个节点,要使Redis群集正常工作,您需要:
为客户端提供服务的普通Redis TCP端口,例如6379
集群总线例如:16379(客户端端口+ 10000)
三、安装
3.1 多台机器 创建pv且大于60g,pv.yaml(ps:不能在一台机器运行所有,因为每个pod都有podAntiAffinity:实现每台机器只能有一个redis)
apiVersion: v1
kind: PersistentVolume
metadata:
name: data
labels:
app: redis-ha #pv和pvc测试过,不需要name和labels识别,调度器会为pvc自动选择合适容量的pv
spec:
capacity:
storage: 60Gi
accessModes:
- ReadWriteOnce
hostPath:
path: /opt/data
kubectl create -f pv.yaml
安装启动redis集群
#修改本机path路径,授权用户(1000)具备创建目录权限
helm install --set hostPath.path=/opt --name=redis-ha stable/redis-ha
3.2 单机实现redis集群,需要取消affinity
redis-ha-values-custom.yaml
## Node labels, affinity, and tolerations for pod assignment
## ref: https://kubernetes.io/docs/concepts/configuration/assign-pod-node/#nodeselector
## ref: https://kubernetes.io/docs/concepts/configuration/assign-pod-node/#taints-and-tolerations-beta-feature
## ref: https://kubernetes.io/docs/concepts/configuration/assign-pod-node/#affinity-and-anti-affinity
# Just for local develop environment
affinity: {}
启动
helm install --set hostPath.path=/opt --name=redis-ha -f ./redis-ha-values-custom.yaml stable/redis-ha #通过取消affinity单机实现redis集群
看看service文件,每个redis通过本svc识别其他redis
apiVersion: v1
kind: Service
metadata:
labels:
app: redis-ha
chart: redis-ha-3.3.3
name: redis-ha-announce-0
namespace: default
spec:
clusterIP: 10.102.130.11
ports:
- name: server
port: 6379
protocol: TCP
targetPort: redis
- name: sentinel
port: 26379
protocol: TCP
targetPort: sentinel
type: ClusterIP
selector:
app: redis-ha
release: redis-ha
statefulset.kubernetes.io/pod-name: redis-ha-server-0
四、测试
kubectl get svc |grep redis,如下
redis-ha ClusterIP None <none> 6379/TCP,26379/TCP 37m
redis-ha-announce-0 ClusterIP 10.102.130.11 <none> 6379/TCP,26379/TCP 37m #默认这个为redis主
redis-ha-announce-1 ClusterIP 10.105.223.197 <none> 6379/TCP,26379/TCP 37m
redis-ha-announce-2 ClusterIP 10.106.29.241 <none> 6379/TCP,26379/TCP 37m
kubectl get pod -o wide|grep redis,如下
redis-ha-server-0 2/2 Running 0 41m 10.244.0.11 hostname-test
redis-ha-server-1 2/2 Running 0 37m 10.244.0.12 hostname-test
redis-ha-server-2 2/2 Running 0 35m 10.244.0.13 hostname-test
进入pod为redis-ha-server-1,容器为sentinel
kubectl exec -it redis-ha-server-1 -c sentinel sh
容器里面$ redis-cli -h redis-ha-announce-0 -p 26379
容器里面$ redis-cli -h redis-ha-announce-0 -p 6379
故障转移实验
停止主redis:
#1、在主上故障转移测试,使主down掉
redis-cli -h redis-ha-announce-0 -p 6379 debug segfault
#2、然后进入redis容器或sentinel容器
kubectl exec -it redis-ha-server-2 -c redis sh
kubectl exec -it redis-ha-server-0 -c sentinel sh
#3、容器里面使用redis客户端连接redis服务端
redis-cli -h redis-ha-announce-1 -p 6379
redis-cli -h redis-ha-announce-2 -p 6379
#容器里面使用redis客户端连接Sentinel服务端
redis-cli -h redis-ha-announce-0 -p 26379
#4、redis查看主从状态
info replication
#sentinel查看状态
INFO Sentinel
#5、查看日志
kubectl logs -f redis-ha-server-0 -c redis
kubectl logs -f redis-ha-server-1 -c redis
kubectl logs -f redis-ha-server-2 -c redis
#主前任主上查看redis主从状态如下,10.106.29.241是第三台redis的ip,说转换成功了
#测试添加一台redis实验,如下:
1、修改文件redis-ha-values-upgrade.yaml
replicas: 4
affinity: {}
hostPath:
path:/opt
2、创建pv4
apiVersion: v1
kind: PersistentVolume
metadata:
name: data03
spec:
capacity:
storage: 10Gi
accessModes:
- ReadWriteOnce
hostPath:
path: /opt/data03
ps:如果单机部署集群,每个redis的pv的路径不能相同,否则会报错
3、升级
helm upgrade -f redis-ha-values-upgrade.yaml redis-ha stable/redis-ha
4、报错可以回滚
helm rollback redis-ha 1
五、删除
helm ls --all
helm del --purge redis-ha