LVS + Keepalived 高可用高性能负载均衡案例实战

  • ​​前言​​
  • ​​一、架构设计​​
  • ​​二、环境准备​​
  • ​​三、案例部署​​
  • ​​四、测试​​
  • ​​1、Keepalived 健康检查​​
  • ​​2、Keepalived MASTER/BACKUP 切换​​

前言

Keepalived 作为一个高性能的集群高可用解决方案。提供了集群节点心跳检测、健康检查以及故障切换的功能。原生支持 LVS 负载均衡集群。接下来，我将详细介绍。

一、架构设计

负载均衡方案系统架构拓扑图

Web集群案例实战 -- LVS + Keepalived 高可用高性能负载均衡案例实战_nginx


两台负载均衡器部署详细拓扑图

Web集群案例实战 -- LVS + Keepalived 高可用高性能负载均衡案例实战_运维_02

二、环境准备

role

host

ip

software installed

OS

LVS Dir、Keepalive MASTER

node01

192.168.5.11

LVS、keepalived-2.0.12

Centos 7.8

LVS Dir、Keepalive BACKUP

node02

192.168.5.12

LVS、keepalived-2.0.12

Centos 7.8

nginx web server1、LVS RS

node03

192.168.5.13

Nginx-1.18.0

Centos 7.8

nginx web server1、LVS RS

node04

192.168.5.14

Nginx-1.18.0

Centos 7.8

Client

node05

192.168.5.15

----

Centos 7.8

三、案例部署

配置前端 keeapalived + lvs

---node01
[root@node01 ~]# vim /etc/keepalived/keepalived.conf
! Configuration File for keepalived

global_defs {
   notification_email {
     acassen@firewall.loc
     failover@firewall.loc
     sysadmin@firewall.loc
   }
   notification_email_from Alexandre.Cassen@firewall.loc
   smtp_server 192.168.5.10
   smtp_connect_timeout 30
   router_id LVS_DEVEL1
}


vrrp_instance VI_1 {
    state MASTER
    interface ens33
    virtual_router_id 51
    priority 200
    advert_int 1
    authentication {
        auth_type PASS
        auth_pass 1111
    }
    virtual_ipaddress {
        192.168.5.20
    }
}

virtual_server 192.168.5.20 80 {
    delay_loop 6
    lb_algo rr
    lb_kind DR
    #persistence_timeout 50
    protocol TCP

    real_server 192.168.5.13 80 {
        weight 1
        TCP_CHECK {
            connect_timeout 3
            nb_get_retry 3
            delay_before_retry 3
            connect_port 80
        }
    }

    real_server 192.168.5.14 80 {
        weight 1
        TCP_CHECK {
            connect_timeout 3
            nb_get_retry 3
            delay_before_retry 3
            connect_port 80
        }
    }
}
[root@node01 ~]# systemctl restart keepalived.service


---node02
! Configuration File for keepalived

global_defs {
   notification_email {
     acassen@firewall.loc
     failover@firewall.loc
     sysadmin@firewall.loc
   }
   notification_email_from Alexandre.Cassen@firewall.loc
   smtp_server 192.168.5.10
   smtp_connect_timeout 30
   router_id LVS_DEVEL2
}

vrrp_instance VI_1 {
    state BACKUP
    interface ens33
    virtual_router_id 51
    priority 100
    advert_int 1
    authentication {
        auth_type PASS
        auth_pass 1111
    }

    virtual_ipaddress {
        192.168.5.20
    }
}

virtual_server 192.168.5.20 80 {
    delay_loop 6
    lb_algo rr
    lb_kind DR
    #persistence_timeout 50
    protocol TCP

    real_server 192.168.5.13 80 {
        weight 1
        TCP_CHECK {
            connect_timeout 3
            nb_get_retry 3
            delay_before_retry 3
            connect_port 80
        }
    }

    real_server 192.168.5.14 80 {
        weight 1
        TCP_CHECK {
            connect_timeout 3
            nb_get_retry 3
            delay_before_retry 3
            connect_port 80
        }
    }
}
[root@node02 ~]# systemctl restart keepalived.service

配置后端 web 服务

---node03
[root@node03 ~]# yum install nginx-1.18.0-1.el7.ngx.x86_64.rpm -y
[root@node03 ~]# echo "`hostname -I` web test page..." > /usr/share/nginx/html/index.html
[root@node03 ~]# systemctl enable --now nginx


----node04
[root@node04 ~]# yum install nginx-1.18.0-1.el7.ngx.x86_64.rpm -y
[root@node04 ~]# echo "`hostname -I` web test page..." > /usr/share/nginx/html/index.html
[root@node04 ~]# systemctl enable --now nginx

配置后端 lvs

---node03
[root@node03 ~]# vim /etc/init.d/lvs_dr_rs
#!/bin/sh
# 
# Startup script handle the initialisation of LVS
# chkconfig: - 28 72
# description: Initialise the Linux Virtual Server for DR
# 
### BEGIN INIT INFO
# Provides: ipvsadm
# Required-Start: $local_fs $network $named
# Required-Stop: $local_fs $remote_fs $network
# Short-Description: Initialise the Linux Virtual Server
# Description: The Linux Virtual Server is a highly scalable and highly
#   available server built on a cluster of real servers, with the load
#   balancer running on Linux.
# description: start LVS of DR-RIP
LOCK=/var/lock/ipvsadm.lock
VIP=192.168.5.20
. /etc/rc.d/init.d/functions
start() {
    PID=`ifconfig | grep lo:100 | wc -l`
    if [ $PID -ne 0 ];
    then
        echo "The LVS-DR-RIP Server is already running !"
    else
       /sbin/ifconfig lo:100 $VIP netmask 255.255.255.255 broadcast $VIP up
       /sbin/route add -host $VIP dev lo:100
       echo "1" >/proc/sys/net/ipv4/conf/lo/arp_ignore
       echo "2" >/proc/sys/net/ipv4/conf/lo/arp_announce
       echo "1" >/proc/sys/net/ipv4/conf/ens33/arp_ignore
       echo "2" >/proc/sys/net/ipv4/conf/ens33/arp_announce
       echo "1" >/proc/sys/net/ipv4/conf/all/arp_ignore
       echo "2" >/proc/sys/net/ipv4/conf/all/arp_announce
       /bin/touch $LOCK
       echo "starting LVS-DR-RIP server is ok !"
    fi
    } 
    
stop() {
       /sbin/route del -host $VIP dev lo:100
       /sbin/ifconfig lo:100 down >/dev/null
       echo "0" >/proc/sys/net/ipv4/conf/lo/arp_ignore
       echo "0" >/proc/sys/net/ipv4/conf/lo/arp_announce
       echo "0" >/proc/sys/net/ipv4/conf/ens33/arp_ignore
       echo "0" >/proc/sys/net/ipv4/conf/ens33/arp_announce
       echo "0" >/proc/sys/net/ipv4/conf/all/arp_ignore
       echo "0" >/proc/sys/net/ipv4/conf/all/arp_announce
       rm -rf $LOCK
       echo "stopping LVS-DR-RIP server is ok !"
}

status() {
     if [ -e $LOCK ];
     then
        echo "The LVS-DR-RIP Server is already running !"
     else
        echo "The LVS-DR-RIP Server is not running !"
     fi
} 

case "$1" in
   start)
        start
        ;;
   stop)
        stop
        ;;
   restart)
        stop
        start
        ;;
   status)
        status
        ;;
   *)
        echo "Usage: $1 {start|stop|restart|status}"
        exit 1
esac
exit 0

[root@node03 ~]# chmod +x /etc/init.d/lvs_dr_rs
[root@node03 ~]# chkconfig --add lvs_dr_rs
[root@node03 ~]# chkconfig lvs_dr_rs on
[root@node03 ~]# systemctl enable --now lvs_dr_rs



---node04
[root@node04 ~]# vim /etc/init.d/lvs_dr_rs
#!/bin/sh
# 
# Startup script handle the initialisation of LVS
# chkconfig: - 28 72
# description: Initialise the Linux Virtual Server for DR
# 
### BEGIN INIT INFO
# Provides: ipvsadm
# Required-Start: $local_fs $network $named
# Required-Stop: $local_fs $remote_fs $network
# Short-Description: Initialise the Linux Virtual Server
# Description: The Linux Virtual Server is a highly scalable and highly
#   available server built on a cluster of real servers, with the load
#   balancer running on Linux.
# description: start LVS of DR-RIP
LOCK=/var/lock/ipvsadm.lock
VIP=192.168.5.20
. /etc/rc.d/init.d/functions
start() {
    PID=`ifconfig | grep lo:100 | wc -l`
    if [ $PID -ne 0 ];
    then
        echo "The LVS-DR-RIP Server is already running !"
    else
       /sbin/ifconfig lo:100 $VIP netmask 255.255.255.255 broadcast $VIP up
       /sbin/route add -host $VIP dev lo:100
       echo "1" >/proc/sys/net/ipv4/conf/lo/arp_ignore
       echo "2" >/proc/sys/net/ipv4/conf/lo/arp_announce
       echo "1" >/proc/sys/net/ipv4/conf/ens33/arp_ignore
       echo "2" >/proc/sys/net/ipv4/conf/ens33/arp_announce
       echo "1" >/proc/sys/net/ipv4/conf/all/arp_ignore
       echo "2" >/proc/sys/net/ipv4/conf/all/arp_announce
       /bin/touch $LOCK
       echo "starting LVS-DR-RIP server is ok !"
    fi
    } 
    
stop() {
       /sbin/route del -host $VIP dev lo:100
       /sbin/ifconfig lo:100 down >/dev/null
       echo "0" >/proc/sys/net/ipv4/conf/lo/arp_ignore
       echo "0" >/proc/sys/net/ipv4/conf/lo/arp_announce
       echo "0" >/proc/sys/net/ipv4/conf/ens33/arp_ignore
       echo "0" >/proc/sys/net/ipv4/conf/ens33/arp_announce
       echo "0" >/proc/sys/net/ipv4/conf/all/arp_ignore
       echo "0" >/proc/sys/net/ipv4/conf/all/arp_announce
       rm -rf $LOCK
       echo "stopping LVS-DR-RIP server is ok !"
}

status() {
     if [ -e $LOCK ];
     then
        echo "The LVS-DR-RIP Server is already running !"
     else
        echo "The LVS-DR-RIP Server is not running !"
     fi
} 

case "$1" in
   start)
        start
        ;;
   stop)
        stop
        ;;
   restart)
        stop
        start
        ;;
   status)
        status
        ;;
   *)
        echo "Usage: $1 {start|stop|restart|status}"
        exit 1
esac
exit 0

[root@node04 ~]# chmod +x /etc/init.d/lvs_dr_rs
[root@node04 ~]# chkconfig --add lvs_dr_rs
[root@node04 ~]# chkconfig lvs_dr_rs on
[root@node04 ~]# systemctl enable --now lvs_dr_rs

查看lvs集群

查看集群状态

Web集群案例实战 -- LVS + Keepalived 高可用高性能负载均衡案例实战_运维_03


客户端访问 VIP

Web集群案例实战 -- LVS + Keepalived 高可用高性能负载均衡案例实战_LVS 、Keepalived_04


实现web服务负载均衡！

四、测试

1、Keepalived 健康检查

动态监控 ipvsadm 列表

Web集群案例实战 -- LVS + Keepalived 高可用高性能负载均衡案例实战_Web集群_05


模拟后端服务故障

[root@node04 ~]# systemctl stop nginx

Web集群案例实战 -- LVS + Keepalived 高可用高性能负载均衡案例实战_Web集群_06


node05 访问 VIP

Web集群案例实战 -- LVS + Keepalived 高可用高性能负载均衡案例实战_Web集群_07


后端故障恢复

[root@node04 ~]# systemctl start nginx

Web集群案例实战 -- LVS + Keepalived 高可用高性能负载均衡案例实战_Server_08


node05 访问 VIP

Web集群案例实战 -- LVS + Keepalived 高可用高性能负载均衡案例实战_nginx_09


Web集群正常提供相应！

2、Keepalived MASTER/BACKUP 切换

查看keeapalived ip地址状况

node01

Web集群案例实战 -- LVS + Keepalived 高可用高性能负载均衡案例实战_nginx_10


node02

Web集群案例实战 -- LVS + Keepalived 高可用高性能负载均衡案例实战_运维_11


模拟 Keepalived MASTER 故障

[root@node01 ~]# systemctl stop keepalived.service

node01

Web集群案例实战 -- LVS + Keepalived 高可用高性能负载均衡案例实战_nginx_12


node02

Web集群案例实战 -- LVS + Keepalived 高可用高性能负载均衡案例实战_Web集群_13


web 服务访问不受影响

Web集群案例实战 -- LVS + Keepalived 高可用高性能负载均衡案例实战_nginx_14

模拟 Keepalived MASTER 故障恢复

[root@node01 ~]# systemctl start keepalived.service

node01

Web集群案例实战 -- LVS + Keepalived 高可用高性能负载均衡案例实战_Server_15

node02

Web集群案例实战 -- LVS + Keepalived 高可用高性能负载均衡案例实战_运维_16

keeapalived 实现 VIP 漂移 ！