keepalived简介:
keepalived是集群管理中保证高可用的一个服务软件,用来防止单点故障。
keepalived的作用是监测服务器的状态,如果有一台服务器岩机,并将有故障的服务器从系统中剔除,同时使用其他服务器代替该服务器工作。
keepalived服务介绍:
起初的设计是专门给LVS服务的,专门用来监控LVS集群系统中各个服务节点的状态,后来又加入了VRRP(Virtual Router
Redundancy
Protocol)虚拟路由器冗余协议,VRRP是为了解决静态路由单点故障,它有两大功能:健康检查与故障切换(主备之间快速切换) HA的功能:
在主备LB之间相互监控运行状态,一旦主LB宕机,备LB会立马接收主LB的所有资源(IP资源与VIP资源),然后接管主LB来运行LB的功能,一旦主LB恢复,它将资源还给主LB
健康检查: 主要是对RS节点的运行健康性检查,一旦有节点宕机,它会不再将主求发给这个RS节点
keepalived工作原理:
keepalived是以VRRP为实现基础的,即虚拟路由冗余协议;
虚拟路由冗余协议:可以认为是实现路由器高可用的协议,即将多个提供相同功能的路由器组成一个路由器组,这个组里面有master和backup,master上面有一个对外服务的vip地址,master会发组播,当backup收不到vrrp包时就认为master岩机了,这时就需要根据vrrp的优先级选举出一个backup当master,这样就实现了高可用。
keepalived主要有三个模块是core、check和vrrp。core模块为keepalived的核心,负责主进程的启动、维护以及全局配置文件的加载和解析。check负责健康检查,包括常见的各种检查方式。vrrp模块是来实现VRRP协议的。
keepalived相关组件:
vrrp stack 实现vrrp虚拟ip
checkers 负责后端服务器的健康监测。
ipvs wrapper-----ipvs 对lvs的实现高可用架构。能够根据配置文件自动生成ipvs规则。
keepalived的配置文件介绍:在这我只选择了前两段进行详细说明。
! Configuration File for keepalived
global_defs { 全局段,主要是配置故障发生时的通知对象及主机标识
notification_email { 告警邮件。
acassen@firewall.loc 这有三个实列,表示将告警邮件发给谁。
failover@firewall.loc
sysadmin@firewall.loc
}
notification_email_from Alexandre.Cassen@firewall.loc 表示发件人。
smtp_server 192.168.200.1 表示发件人使用的服务器。
smtp_connect_timeout 30 表示连接服务器的超时时间。
router_id LVS_DEVEL 表示标识唯一主机,通常为hostname
}
vrrp_instance VI_1 { vrrp_instance区域用来定义对外提供服务的vip区域及相关属性。
state MASTER 表示这台主机的虚拟IP为主节点。
interface eth0
virtual_router_id 51 虚拟地址id标识。
priority 100 虚拟ip的在这台主机的优先级。
advert_int 1 每个多长时间发送心跳信息广播包,告诉备份主机的存活状态
authentication { 认证模式。
auth_type PASS
auth_pass 8ab998cd 认证密码可随机生成。
}
virtual_ipaddress {
192.168.200.16 这些都是用于游离的虚拟地址。
192.168.200.17
192.168.200.18
}
}
**keepalived的实现配置。
配置前提:
1,本机的主机名,要和hostname(uname -n)获得的名称保持一致。;
centos6:/etc/sysconfig/network
centos7:hostnamectl set-hostname hostname
2,各节点要互相解析主机名;一般建议通过hosts文件进行解析。
3,确保iptables及selinux不会成为服务阻碍;
两台主机,两台主机尽量时间同步。
node3 | node1 |
10.5.100.183 | 10.2.100.207 |
NODE3节点
一:安装keepalived软件包
[root@node3 ~]# yum install keepalived -y
[root@node3 ~]# cd /etc/keepalived/
[root@node3 keepalived]# ls
keepalived.back keepalived.conf
[root@node3 keepalived]# vim keepalived.conf
[root@node3 keepalived]#
! Configuration File for keepalived
global_defs {
root@localhost
}
notification_email_from kaadmin@localhost
smtp_server 127.0.0.1
smtp_connect_timeout 30
router_id node3.yan.com
vrrp_mcast_group 224.18.0.100 设置组播地址,只能在这个组播域内
}
vrrp_instance VI_1 {
state MASTER
interface enp4s0
virtual_router_id 51
priority 100
advert_int 1
authentication {
auth_type PASS
auth_pass 8ab998cd
}
virtual_ipaddress {
10.5.100.80/24
}
}
[root@node3 keepalived]# scp keepalived.conf node1:/etc/keepalived/
[root@node3 ~]# systemctl restart keepalived
NODE1节点:
[root@node1 ~]# vim /etc/keepalived/keepalived.conf
! Configuration File for keepalived
global_defs {
notification_email {
root@localhost
}
notification_email_from kaadmin@localhost
smtp_server 127.0.0.1
smtp_connect_timeout 30
router_id node1.yan.com 修改主机名称
vrrp_mcast_group 224.18.0.100
}
vrrp_instance VI_1 {
state backup
interface eno16777736 修改网卡设备
virtual_router_id 51
priority 98 修改成backup变为备用主机
advert_int 1
authentication {
auth_type PASS
auth_pass 8ab998cd
}
virtual_ipaddress {
10.5.100.80/24
}
}
[root@node3 ~]# systemctl restart keepalived 重启keepalived服务
验证keepalived的高可用架构。
[root@node3 ~]# ip a
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
inet 127.0.0.1/8 scope host lo
valid_lft forever preferred_lft forever
inet6 ::1/128 scope host
valid_lft forever preferred_lft forever
2: enp4s0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP group default qlen 1000
link/ether 74:27:ea:e6:54:d7 brd ff:ff:ff:ff:ff:ff
inet 10.5.100.183/24 brd 10.5.100.255 scope global noprefixroute dynamic enp4s0
valid_lft 585886sec preferred_lft 585886sec
inet 10.5.100.80/24 scope global secondary enp4s0 虚拟ip地址已经在node3主节点。
valid_lft forever preferred_lft forever
inet6 fe80::477d:49ca:2e17:2bf1/64 scope link noprefixroute
valid_lft forever preferred_lft forever
[root@node3 ~]#
node1是没有虚拟游离ip地址的。
[root@node1 ~]# ip a
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
inet 127.0.0.1/8 scope host lo
valid_lft forever preferred_lft forever
inet6 ::1/128 scope host
valid_lft forever preferred_lft forever
2: eno16777736: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000
link/ether 00:0c:29:d0:83:ae brd ff:ff:ff:ff:ff:ff
inet 10.5.100.207/24 brd 10.5.100.255 scope global dynamic eno16777736
valid_lft 580437sec preferred_lft 580437sec
inet6 fe80::20c:29ff:fed0:83ae/64 scope link
valid_lft forever preferred_lft forever
现在我把NODE3节点keepalived挂掉。看游离地址的转换。
[root@node3 ~]# systemctl stop keepalived
[root@node3 ~]# ip a
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
inet 127.0.0.1/8 scope host lo
valid_lft forever preferred_lft forever
inet6 ::1/128 scope host
valid_lft forever preferred_lft forever
2: enp4s0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP group default qlen 1000
link/ether 74:27:ea:e6:54:d7 brd ff:ff:ff:ff:ff:ff
inet 10.5.100.183/24 brd 10.5.100.255 scope global noprefixroute dynamic enp4s0
valid_lft 583952sec preferred_lft 583952sec
inet6 fe80::477d:49ca:2e17:2bf1/64 scope link noprefixroute
valid_lft forever preferred_lft forever
[root@node3 ~]#
[root@node1 ~]# ip a
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
inet 127.0.0.1/8 scope host lo
valid_lft forever preferred_lft forever
inet6 ::1/128 scope host
valid_lft forever preferred_lft forever
2: eno16777736: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000
link/ether 00:0c:29:d0:83:ae brd ff:ff:ff:ff:ff:ff
inet 10.5.100.207/24 brd 10.5.100.255 scope global dynamic eno16777736
valid_lft 578546sec preferred_lft 578546sec
inet 10.5.100.80/24 scope global secondary eno16777736 查看虚拟地址ip已经转化过来了
valid_lft forever preferred_lft forever
inet6 fe80::20c:29ff:fed0:83ae/64 scope link
valid_lft forever preferred_lft forever
当我们在把node3节点启动时,游离虚拟ip地址又会转换成到node3节点,因为node3节点优先级高。
[root@node3 ~]# systemctl restart keepalived
[root@node3 ~]# ip a
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
inet 127.0.0.1/8 scope host lo
valid_lft forever preferred_lft forever
inet6 ::1/128 scope host
valid_lft forever preferred_lft forever
2: enp4s0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP group default qlen 1000
link/ether 74:27:ea:e6:54:d7 brd ff:ff:ff:ff:ff:ff
inet 10.5.100.183/24 brd 10.5.100.255 scope global noprefixroute dynamic enp4s0
valid_lft 583700sec preferred_lft 583700sec
inet 10.5.100.80/24 scope global secondary enp4s0
valid_lft forever preferred_lft forever
inet6 fe80::477d:49ca:2e17:2bf1/64 scope link noprefixroute
valid_lft forever preferred_lft forever
[root@node1 ~]# ip a
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
inet 127.0.0.1/8 scope host lo
valid_lft forever preferred_lft forever
inet6 ::1/128 scope host
valid_lft forever preferred_lft forever
2: eno16777736: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000
link/ether 00:0c:29:d0:83:ae brd ff:ff:ff:ff:ff:ff
inet 10.5.100.207/24 brd 10.5.100.255 scope global dynamic eno16777736
valid_lft 578273sec preferred_lft 578273sec
inet6 fe80::20c:29ff:fed0:83ae/64 scope link
valid_lft forever preferred_lft forever
设定脚本在不关闭keepalived服务情况下自动切换主备节点。
[root@node3 ~]# vim /etc/keepalived/keepalived.conf
! Configuration File for keepalived
global_defs {
root@localhost
}
notification_email_from kaadmin@localhost
smtp_server 127.0.0.1
smtp_connect_timeout 30
router_id node3.yan.com
vrrp_mcast_group 224.18.0.100
}
vrrp_script chk_yan { 脚本设定区域
script "[[ -f /etc/keepalived/down ]] && exit 1 || exit 0" 判断语句当down文件存在时
interval 1 表示测试一次。
weight -2 文件存在时就自动降低优先级2个成为备节点。
}
vrrp_instance VI_1 {
state MASTER
interface enp4s0
virtual_router_id 51
priority 100
advert_int 1
authentication {
auth_type PASS
auth_pass 8ab998cd
}
virtual_ipaddress {
10.5.100.80/24
}
track_script { 调用这个脚本。
chk_yan
}
}
[root@node3 ~]# systemctl restart keepalived
NODE1节点配置脚本,跟上述节点配置一样。
[root@node1 ~]# vim /etc/keepalived/keepalived.conf
! Configuration File for keepalived
global_defs {
root@localhost
}
notification_email_from kaadmin@localhost
smtp_server 127.0.0.1
smtp_connect_timeout 30
router_id node3.yan.com
vrrp_mcast_group 224.18.0.100
}
vrrp_script chk_yan {
script "[[ -f /etc/keepalived/down ]] && exit 1 || exit 0"
interval 1
weight -2
}
vrrp_instance VI_1 {
state MASTER
interface enp4s0
virtual_router_id 51
priority 99
advert_int 1
authentication {
auth_type PASS
auth_pass 8ab998cd
}
virtual_ipaddress {
10.5.100.80/24
}
track_script {
chk_yan
}
}
[root@node3 ~]# systemctl restart keepalived
创建和删除down文件,在node3节点进行验证。
[root@node3 keepalived]# ip a
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
inet 127.0.0.1/8 scope host lo
valid_lft forever preferred_lft forever
inet6 ::1/128 scope host
valid_lft forever preferred_lft forever
2: enp4s0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP group default qlen 1000
link/ether 74:27:ea:e6:54:d7 brd ff:ff:ff:ff:ff:ff
inet 10.5.100.183/24 brd 10.5.100.255 scope global noprefixroute dynamic enp4s0
valid_lft 581140sec preferred_lft 581140sec
inet 10.5.100.80/24 scope global secondary enp4s0
valid_lft forever preferred_lft forever
inet6 fe80::477d:49ca:2e17:2bf1/64 scope link noprefixroute
valid_lft forever preferred_lft forever
[root@node3 keepalived]# touch down
[root@node3 keepalived]# ls
down keepalived.back keepalived.conf
[root@node3 keepalived]# ip a
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
inet 127.0.0.1/8 scope host lo
valid_lft forever preferred_lft forever
inet6 ::1/128 scope host
valid_lft forever preferred_lft forever
2: enp4s0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP group default qlen 1000
link/ether 74:27:ea:e6:54:d7 brd ff:ff:ff:ff:ff:ff
inet 10.5.100.183/24 brd 10.5.100.255 scope global noprefixroute dynamic enp4s0
valid_lft 582177sec preferred_lft 582177sec
inet6 fe80::477d:49ca:2e17:2bf1/64 scope link noprefixroute
valid_lft forever preferred_lft forever
[root@node3 keepalived]# rm -rf down
[root@node3 keepalived]# ls
keepalived.back keepalived.conf
[root@node3 keepalived]# ip a
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
inet 127.0.0.1/8 scope host lo
valid_lft forever preferred_lft forever
inet6 ::1/128 scope host
valid_lft forever preferred_lft forever
2: enp4s0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP group default qlen 1000
link/ether 74:27:ea:e6:54:d7 brd ff:ff:ff:ff:ff:ff
inet 10.5.100.183/24 brd 10.5.100.255 scope global noprefixroute dynamic enp4s0
valid_lft 582168sec preferred_lft 582168sec
inet 10.5.100.80/24 scope global secondary enp4s0
valid_lft forever preferred_lft forever
inet6 fe80::477d:49ca:2e17:2bf1/64 scope link noprefixroute
valid_lft forever preferred_lft forever
在两个节点定义一个脚本实现,在主节点和备节点发生切换的时候各自发送状态邮件。
NODE1节点
[root@node1 keepalived]# vim notify.sh
#!/bin/bash
vip=10.5.100.88
contact='root@localhost'
notify() {
mailsubject="`hostname` to be $1: $vip floating"
mailbody="`date '+%F %H:%M:%S'`: vrrp transition, `hostname` change to be $1"
echo $mailbody | mail -s "$mailsubject" $contact
}
case "$1" in
master)
notify master
exit 0
;;
backup)
notify backup
exit 0
;;
fault)
notify fault
exit 0
;;
*)
echo 'usage:`basename $0` {master|backup|fault}'
exit 1
;;
esac
[root@node1 keepalived]# scp notify.sh node3:/etc/keepalived/ 将脚本传送给备用节点。
二:定义好脚本在配置文件中虚拟实例中调用脚本即可。
! Configuration File for keepalived
global_defs {
notification_email {
root@localhost
}
notification_email_from admin@localhost
smtp_server 127.0.0.1
smtp_connect_timeout 30
router_id node1.yan.com
vrrp_mcast_group 224.18.0.100
}
vrrp_instance VI_1 {
state BACKUP
interface eno16777736
virtual_router_id 51
priority 99
advert_int 1
authentication {
auth_type PASS
auth_pass 0dc5e628
}
virtual_ipaddress {
10.5.100.88/24
}
notify_master "/etc/keepalived/notify.sh master"
notify_backup "/etc/keepalived/notify.sh backup"
notify_fault "/etc/keepalived/notify.sh fault"
}
测试:当我们正常启动keepalived功能时,主从节点都会相应发送邮件。
[root@node1 keepalived]# mail 现在表示node1有可能成为备份节点,因为node1优先级低
Heirloom Mail version 12.5 7/5/10. Type ? for help.
"/var/spool/mail/root": 44 messages 8 new 37 unread
U 30 root Wed Jan 8 00:47 19/694 "node1 to be master: 10.5.100.88 floating"
U 31 root Wed Jan 8 00:50 19/694 "node1 to be backup: 10.5.100.88 floating"
U 32 root Wed Jan 8 00:50 19/694 "node1 to be master: 10.5.100.88 floating"
U 33 root Wed Jan 8 00:52 19/694 "node1 to be backup: 10.5.100.88 floating"
U 34 root Wed Jan 8 00:52 19/694 "node1 to be master: 10.5.100.88 floating"
35 root Wed Jan 8 01:04 19/695 "node1 to be backup: 10.5.100.88 floating"
36 root Wed Jan 8 01:04 19/695 "node1 to be master: 10.5.100.88 floating"
>N 37 root Wed Jan 8 01:08 18/684 "node1 to be backup: 10.5.100.88 floating"
N 38 root Wed Jan 8 01:08 18/684 "node1 to be master: 10.5.100.88 floating"
N 39 root Wed Jan 8 01:09 18/684 "node1 to be backup: 10.5.100.88 floating"
N 40 root Wed Jan 8 01:09 18/684 "node1 to be master: 10.5.100.88 floating"
N 41 root Wed Jan 8 01:11 18/684 "node1 to be backup: 10.5.100.88 floating"
N 42 root Wed Jan 8 01:11 18/684 "node1 to be master: 10.5.100.88 floating"
N 43 root Wed Jan 8 01:12 18/684 "node1 to be backup: 10.5.100.88 floating"
N 44 root Wed Jan 8 01:12 18/684 "node1 to be master: 10.5.100.88 floating"
& 37
Message 37:
From root@node1.localdomain Wed Jan 8 01:08:45 2020
Return-Path: <root@node1.localdomain>
X-Original-To: root@localhost
Delivered-To: root@localhost.localdomain
Date: Wed, 08 Jan 2020 01:08:45 -0800
To: root@localhost.localdomain
Subject: node1 to be backup: 10.5.100.88 floating
User-Agent: Heirloom mailx 12.5 7/5/10
Content-Type: text/plain; charset=us-ascii
From: root@node1.localdomain (root)
Status: R
2020-01-08 01:08:45: vrrp transition, node1 change to be backup
NODE3与NODE1同理即可。
[root@node3 ~]# vim /etc/keepalived/notify.sh
#!/bin/bash
vip=10.5.100.88
contact='root@localhost'
notify() {
mailsubject="`hostname` to be $1: $vip floating"
mailbody="`date '+%F %H:%M:%S'`: vrrp transition, `hostname` change to be $1"
echo $mailbody | mail -s "$mailsubject" $contact
}
case "$1" in
master)
notify master
exit 0
;;
backup)
notify backup
exit 0
;;
fault)
notify fault
exit 0
;;
*)
echo 'usage:`basename $0` {master|backup|fault}'
exit 1
;;
esac
定义好脚本在虚拟实例中调用:
! Configuration File for keepalived
global_defs {
notification_email {
root@localhost
}
notification_email_from admin@localhost
smtp_server 127.0.0.1
smtp_connect_timeout 30
router_id node3.yan.com
vrrp_mcast_group 224.18.0.10
}
vrrp_instance VI_1 {
state MASTER
interface enp4s0
virtual_router_id 77
priority 200
advert_int 1
authentication {
auth_type PASS
auth_pass 0dc5e628
}
virtual_ipaddress {
10.5.100.88/24
}
notify_master "/etc/keepalived/notify.sh master"
notify_backup "/etc/keepalived/notify.sh backup"
notify_fault "/etc/keepalived/notify.sh fault"
}
[root@node3 ~]# mail
Heirloom Mail version 12.5 7/5/10. Type ? for help.
"/var/spool/mail/root": 10 messages 4 new 9 unread
U 1 root Mon Jan 6 18:37 23/858 "node3 to be master: 10.5.100.88 floating"
U 2 root Mon Jan 6 18:41 23/858 "node3 to be master: 10.5.100.88 floating"
U 3 root Mon Jan 6 18:46 23/858 "node3 to be master: 10.5.100.88 floating"
4 root Wed Jan 8 17:48 23/859 "node3 to be master: 10.5.100.88 floating"
U 5 root Wed Jan 8 17:51 23/858 "node3 to be master: 10.5.100.88 floating"
U 6 root Wed Jan 8 18:04 23/858 "node3 to be master: 10.5.100.88 floating"
>N 7 root Wed Jan 8 18:10 22/848 "node3 to be master: 10.5.100.88 floating"
N 8 root Wed Jan 8 18:12 22/848 "node3 to be backup: 10.5.100.88 floating"
N 9 root Wed Jan 8 18:12 22/848 "node3 to be master: 10.5.100.88 floating"
N 10 root Wed Jan 8 18:13 22/848 "node3 to be master: 10.5.100.88 floating"
& 7
Message 7:
From root@node3.localdomain Wed Jan 8 18:10:05 2020
Return-Path: <root@node3.localdomain>
X-Original-To: root@node3
Delivered-To: root@node3.localdomain
From: root <root@node3.localdomain>
Date: Wed, 08 Jan 2020 18:09:04 +0900
To: root@node3.localdomain
Subject: node3 to be master: 10.5.100.88 floating
User-Agent: Heirloom mailx 12.5 7/5/10
Content-Type: text/plain; charset=us-ascii
Status: R
2020-01-08 18:09:04: vrrp transition, node3 change to be master
总结:到这里定义脚本完成了,总结一下keepalived高可用:keepalive利用虚拟vip地址使得地址可以游离,
一个虚拟实列可以对应多个地址,一般都对应一个地址,住备节点配置要一模一样,除了“主机名和网卡设备不同”,keepalived的配置文件中我们可以定义脚本,使得我们不用关闭keepalived就能实现地址游离,也可以调用外部脚本,当我们的主节点或备节点发生状态切换时,可以发送邮件告知我们。这就是简简单单的基本keepalived高可用实现。下次见。