一、Keepalived详解
1、什么是keepalived
keepalived是个基于VRRP协议来实现的LVS服务高可用方案,可以用来避免单点故障。一个LVS服务包含两台运行着keepalived服务的服务器,一台为主服务器(MASTER),一台为备份服务器(BACKUP),但是对外只表现为一个虚拟IP(VIP),主服务器会发送特定的消息给备份服务器。如果主服务器宕机时,备份服务器就收不到这个消息,此时备份服务器就会接管虚拟IP,继续提供服务,以实现高可用。
2、VRRP协议
在现实网络中,两台主机之间并没有直接的物理连接,在这种情况下,该如何进行路由选择呢?通常有两种方式:第一,在主机上配置动态路由协议,比如RIP,OSPF等;第二,在主机上配置静态路由。在主机上配置动态路由协议成本较高,所以静态路由协议就比较流行。但路由器往往会成为单点故障。
VRRP的目的就是为了解决静态路由单点故障问题,VRRP通过竞选(election)协议,动态的将路由任务交给虚拟路由器中的某台VRRP路由器。
3、VRRP工作机制
在一个VRRP虚拟路由器中,有多台物理的VRRP路由器,但是这多台的物理的机器并不能同时工作,而是由一台称为MASTER的负责路由工作,其它的都是BACKUP。MASTER并非一成不变,VRRP让每个VRRP路由器参与竞选,最终获胜的就是MASTER。MASTER拥有一些特权,比如,拥有虚拟路由器的IP地址,我们的主机就是用这个IP地址作为静态路由的。拥有特权的MASTER要负责转发发送给网关地址的包和响应ARP请求。
VRRP通过竞选协议来实现虚拟路由器的功能,所有的协议报文都是通过多播(multicast)(多播地址224.0.0.18)形式发送的。虚拟路由器由VRID(范围0-255)和一组IP地址组成,对外表现为一个MAC地址。所以,在一个虚拟路由 器中,不管谁是MASTER,对外都是相同的MAC和IP(即VIP)。
在一个虚拟路由器中,只有作为MASTER的VRRP路由器会一直发送VRRP通告信息,BACKUP不会抢MASTER,除非它的优先级(priority)更高。当MASTER不可用时(BACKUP收不到通告信息), 多台BACKUP中优先级最高的这台会被抢占为MASTER。这种抢占是非常快速的(<1s),以保证服务的连续性。
4、VRRP工作流程
(1)初始化
路由器启动时,如果路由器的优先级是255(最高优先级,路由器拥有路由器地址),要发送VRRP通告信息,并发送广播ARP信息通告路由器IP地址对应的MAC地址为路由虚拟MAC,设置通告信息定时器准备定时发送VRRP通告信息,转为MASTER状态;否则进入BACKUP状态,设置定时器检查定时检查是否收到MASTER的通告信息。
(2)Master
二、实验环境
1.使用Centos6.6系统,在虚拟机上克隆了4台。分别为:
Node1:192.168.1.119 作为RS1
Node2:192.168.1.120 作为RS2
Master:192.168.1.121
Slave:192.168.1.122
VIP:192.168.1.215
2.时间同步
node1
[root@bogon ~]#ntpdate 202.120.2.101
node2
[root@bogon ~]#ntpdate 202.120.2.101
Master
[root@bogon ~]#ntpdate 202.120.2.101
Slave
[root@bogon ~]#ntpdate 202.120.2.101
3.主机名解析
node1
[root@bogon ~]# cat /etc/hosts 127.0.0.1 localhost localhost.localdomain localhost4 localhost4.localdomain4 ::1 localhost localhost.localdomain localhost6 localhost6.localdomain6 172.16.0.1 server.magelinux.com server 192.168.1.119 node1.zhoushuyu.com node1 192.168.1.120 node2.zhoushuyu.com node2
把该配置文件传递给node2
[root@bogon ~]# scp /etc/hosts192.168.1.120:/etc/
三、实现过程
node1
1.安装httpd
[root@bogon ~]# yum install httpd
2.配置httpd
[root@bogon ~]# vim /var/www/html/index.html <h1>RS1</h1>
3.启动httpd服务
[root@bogon ~]# service httpd start
4.测试
5.设置开机自启动
[root@bogon~]# chkconfig httpd on [root@bogon~]# chkconfig httpd --list httpd 0:off 1:off 2:on 3:on 4:on 5:on 6:off
6.配置node1
[root@bogon ~]#mkdir src [root@bogon ~]#cd src [root@bogonsrc]# vim realserver.sh #!/bin/bash . /etc/rc.d/init.d/functions VIP=192.168.1.215 host=`/bin/hostname` case"$1" in start) /sbin/ifconfig lo down /sbin/ifconfig lo up echo 1 >/proc/sys/net/ipv4/conf/lo/arp_ignore echo 2 >/proc/sys/net/ipv4/conf/lo/arp_announce echo 1 >/proc/sys/net/ipv4/conf/all/arp_ignore echo 2 >/proc/sys/net/ipv4/conf/all/arp_announce /sbin/ifconfig lo:0 $VIP broadcast $VIPnetmask 255.255.255.255 up /sbin/route add -host $VIP dev lo:0 ;; stop) /sbin/ifconfig lo:0 down echo 0 >/proc/sys/net/ipv4/conf/lo/arp_ignore echo 0 >/proc/sys/net/ipv4/conf/lo/arp_announce echo 0 >/proc/sys/net/ipv4/conf/all/arp_ignore echo 0 >/proc/sys/net/ipv4/conf/all/arp_announce ;; status) islothere=`/sbin/ifconfig lo:0 | grep$VIP` isrothere=`netstat -rn | grep"lo:0" | grep $VIP` if [ ! "$islothere" -o !"$isrothere" ];then echo "RS Stopped" else echo "RS Running" fi ;; *) echo "$0: Usage:$0{start|status|stop}" exit 1 ;; esac [root@bogonsrc]# chmod +x realserver.sh [root@bogonsrc]# ll total 4 -rwxr-xr-x 1root root 985 Oct 21 12:03 realserver.sh [root@bogonsrc]# ./realserver.sh start
7.查看配置
[root@bogonsrc]# ifconfig eth1 Link encap:Ethernet HWaddr 00:0C:29:DC:10:96 inet addr:192.168.1.119 Bcast:192.168.1.255 Mask:255.255.255.0 inet6 addr:fe80::20c:29ff:fedc:1096/64 Scope:Link UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1 RX packets:4706 errors:0 dropped:0overruns:0 frame:0 TX packets:2293 errors:0 dropped:0overruns:0 carrier:0 collisions:0 txqueuelen:1000 RX bytes:2440828 (2.3 MiB) TX bytes:302434 (295.3 KiB) lo Link encap:Local Loopback inet addr:127.0.0.1 Mask:255.0.0.0 inet6 addr: ::1/128 Scope:Host UP LOOPBACK RUNNING MTU:65536 Metric:1 RX packets:0 errors:0 dropped:0overruns:0 frame:0 TX packets:0 errors:0 dropped:0overruns:0 carrier:0 collisions:0 txqueuelen:0 RX bytes:0 (0.0 b) TX bytes:0 (0.0 b) lo:0 Link encap:Local Loopback inet addr:192.168.1.215 Mask:255.255.255.255 UP LOOPBACK RUNNING MTU:65536 Metric:1 [root@bogonsrc]# route -n Kernel IProuting table Destination Gateway Genmask Flags Metric Ref Use Iface 168.1.215 0.0.0.0 255.255.255.255 UH 0 0 0 lo 168.1.0 0.0.0.0 255.255.255.0 U 1 0 0 eth1 0.0.0 192.168.1.1 0.0.0.0 UG 0 0 0 eth1 [root@bogon src]# cat /proc/sys/net/ipv4/conf/lo/arp_ignore 1 [root@bogon src]# cat /proc/sys/net/ipv4/conf/lo/arp_announce 2 [root@bogon src]# cat /proc/sys/net/ipv4/conf/all/arp_ignore 1 [root@bogon src]# cat /proc/sys/net/ipv4/conf/all/arp_announce 2
node2
1.配置httpd,并启动服务
[root@bogon ~]# vim /var/www/html/index.html <h1>RS2</h1> [root@bogon ~]# service httpd start
2.测试
3.设置开机自启动
[root@bogon~]# chkconfig httpd on [root@bogon~]# chkconfig httpd --list httpd 0:off 1:off 2:on 3:on 4:on 5:on 6:off
4.配置node2
[root@bogon~]# mkdir src [root@bogon ~]# cd src 在node1上将realserver.sh传过来 [root@bogon src]# scp realserver.sh192.168.1.120:/root/src/ [root@bogonsrc]# ./realserver.sh start
5.查看配置
[root@bogonsrc]# ifconfig eth1 Link encap:Ethernet HWaddr 00:0C:29:CB:6B:06 inet addr:192.168.1.120 Bcast:192.168.1.255 Mask:255.255.255.0 inet6 addr: fe80::20c:29ff:fecb:6b06/64Scope:Link UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1 RX packets:2986 errors:0 dropped:0overruns:0 frame:0 TX packets:1398 errors:0 dropped:0overruns:0 carrier:0 collisions:0 txqueuelen:1000 RX bytes:2292870 (2.1 MiB) TX bytes:143620 (140.2 KiB) lo Link encap:Local Loopback inet addr:127.0.0.1 Mask:255.0.0.0 inet6 addr: ::1/128 Scope:Host UP LOOPBACK RUNNING MTU:65536 Metric:1 RX packets:0 errors:0 dropped:0overruns:0 frame:0 TX packets:0 errors:0 dropped:0overruns:0 carrier:0 collisions:0 txqueuelen:0 RX bytes:0 (0.0 b) TX bytes:0 (0.0 b) lo:0 Link encap:Local Loopback inet addr:192.168.1.215 Mask:255.255.255.255 UP LOOPBACKRUNNING MTU:65536 Metric:1 [root@bogonsrc]# route -n Kernel IProuting table Destination Gateway Genmask Flags Metric Ref Use Iface 168.1.215 0.0.0.0 255.255.255.255 UH 0 0 0 lo 168.1.0 0.0.0.0 255.255.255.0 U 1 0 0 eth1 0.0.0 192.168.1.1 0.0.0.0 UG 0 0 0 eth1 [root@bogonsrc]# cat /proc/sys/net/ipv4/conf/lo/arp_announce 2 [root@bogonsrc]# cat /proc/sys/net/ipv4/conf/lo/arp_ignore 1 [root@bogonsrc]# cat /proc/sys/net/ipv4/conf/all/arp_ignore 1 [root@bogonsrc]# cat /proc/sys/net/ipv4/conf/all/arp_announce 2
master(192.168.1.121)和slave(192.168.1.222)
1.安装keepalived和ipvsadm
yum install keepalived ipvsadm -y
2.修改配置文件
!Configuration File for keepalived global_defs{ notification_email { root@localhost } notification_email_fromAlexandre.Cassen@firewall.loc smtp_server 172.0.0.1 smtp_connect_timeout 30 router_id LVS_DEVEL } vrrp_instanceVI_1 { state MASTER interface eth1 virtual_router_id 51 priority 100 advert_int 1 authentication { auth_type PASS auth_pass f6de2fd2 } virtual_ipaddress { 192.168.1.215 } } virtual_server192.168.1.215 80 { delay_loop 6 lb_algo rr lb_kind DR nat_mask 255.255.255.0 #persistence_timeout 50 protocol TCP real_server 192.168.1.119 80 { weight 1 HTTP_GET { url { path / status_code 200 } connect_timeout 2 nb_get_retry 3 delay_before_retry 1 } } real_server 192.168.1.120 80 { weight 1 HTTP_GET { url { path / status_code 200 } connect_timeout 2 nb_get_retry 3 delay_before_retry 1 } } }
3.将配置文件传递给192.168.1.122
[root@bogonkeepalived]# scp keepalived.conf 192.168.1.122:/etc/keepalived/
4.修改一下.122主机上的配置文件
state BACKUP priority 99
5.启动Master和Slave上的keepalived服务
[root@bogon ~]# service keepalived start
6.查看LVS状态
[root@bogon~]# ipvsadm -L -n IP VirtualServer version 1.2.1 (size=4096) ProtLocalAddress:Port Scheduler Flags -> RemoteAddress:Port Forward Weight ActiveConn InActConn TCP 192.168.1.215:80 rr -> 192.168.1.119:80 Route 1 0 0 ->192.168.1.120:80 Route 1 0 0
7.使用ip addr list 查看,此处VIP在Master主机上
模拟故障
1.停止node1
[root@bogonsrc]# service httpd stop Stoppinghttpd: [ OK ]
2.在master上查看lvs
[root@bogon~]# ipvsadm -L -n IPVirtual Server version 1.2.1 (size=4096) ProtLocalAddress:Port Scheduler Flags -> RemoteAddress:Port Forward Weight ActiveConn InActConn TCP 192.168.1.215:80 rr ->192.168.1.120:80 Route 1 0 0
3.测试
此时只能在slave上请求到node2
重启node1
[root@bogon src]# service httpd start Starting httpd: httpd: Could not reliablydetermine the server's fully qualified domain name, using 211.98.71.195 forServerName [ OK ]
再次查看lvs状态
4.关闭master上的keepalived
[root@bogon ~]#service keepalived stop Stoppingkeepalived: [ OK ]
5.查看slave的状态
slave已经拿到了VIP192.168.1.215
在master上执行如下命令
说明slave已经接替了master
配置sorry server
当所有的RS都宕机后,客户端访问时会出现错误页面,这将使很不友好的;所以我们要提供一个维护页面来提醒用户。
1.在master和slave上安装httpd
[root@bogon ~]# yum install httpd
2.配置维护页面
[root@bogon ~]# vim /var/www/html/index.html
3.启动httpd服务并测试
[root@bogon ~]# service httpd start
4.修改配置文件,在文件的末尾加入一行
sorry_server 127.0.0.1 80
5.关闭所有RS
[root@bogon src]# service httpd stop 重启master和slave的keepalived [root@bogon ~]# service keepalived restart
6.查看一下master和slave上的lvs
[root@bogon~]# ipvsadm -L -n IPVirtual Server version 1.2.1 (size=4096) ProtLocalAddress:Port Scheduler Flags -> RemoteAddress:Port Forward Weight ActiveConn InActConn TCP 192.168.1.215:80 rr ->127.0.0.1:80 Local 1 0 0 这是slave上的: [root@bogon~]# ipvsadm -L -n IPVirtual Server version 1.2.1 (size=4096) ProtLocalAddress:Port Scheduler Flags -> RemoteAddress:Port Forward Weight ActiveConn InActConn TCP 192.168.1.215:80 rr -> 127.0.0.1:80 Local 1 0 0 You have new mail in /var/spool/mail/root
7.测试
如何完成维护模式下keepalived的切换?
1.修改配置文件
在global_defs {} 和vrrp_instanceVI_1{}之间加入如下内容 vrrp_scriptchk_schedown { svript "[ -e /etc/keepalived/down ]&& exit 1 || exit 0" interval 1 weight -5 fall 2 rise 1 } 在vrrp_instance VI_1{}中加入如下内容 track_script{ chk_schedown }
2.测试
在master上创建一个down,然后ip addr list 查看VIp是否已经转移到slave上。 [root@bogonkeepalived]# touch down [root@bogonkeepalived]# ip addr list 1: lo:<LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN link/loopback 00:00:00:00:00:00 brd00:00:00:00:00:00 inet 127.0.0.1/8 scope host lo inet6 ::1/128 scope host valid_lft forever preferred_lft forever 2: eth1:<BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen1000 link/ether 00:0c:29:27:c6:65 brdff:ff:ff:ff:ff:ff inet 192.168.1.121/24 brd 192.168.1.255scope global eth1 inet6 fe80::20c:29ff:fe27:c665/64 scopelink valid_lft forever preferred_lft forever 3: pan0:<BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN link/ether e6:55:94:ce:b7:1d brd ff:ff:ff:ff:ff:ff
在slave上
这样就完成了通过监控脚本实现维护模式的切换