keepalived绑定vip keepalived的vip怎么设置

文章目录

• 项目名称：基于keepalived的双vip高可用web集群
• 架构图
• 项目环境：CentOS 8.2（8台 1核1G），ansible 2.9.17，keepalived 2.0.10，Nginx 1.19.7，NFS
• 项目描述：

•         构建一个高并发，高可用性的web集群项目，融入ansible实现自动化运维，使用Nginx做负载均衡器，keepalived实现高可用。ansible服务器（1台）、firewall服务器（1台）、堡垒机（1台）、负载均衡器（2台）、web服务器（3台，其中1台是NFS+web服务器）；

• 项目步骤：

•         1.安装好8台CentOS 8.2的系统，部署好ansible服务器，在所有的机器之间配置ssh免密通道；
•         2.部署好堡垒机，配置firewall服务器的NAT策略，用于发布内网NFS服务器的ssh服务；
•         3.编写好部署Nginx和keepalived的脚本，编写ansible的playbook实现批量部署；
•         4.部署好后端的3台web服务器，其中1台搭载NFS服务，实现访问相同内容的功能；
•         5.模拟中台系统，通过对2台负载均衡器 Nginx配置文件的修改，实现基于url的负载均衡，并实现对不同的url进行转发，例如/download 实现下载功能；
•         6.对负载均衡器进行配置，实现基于keepalived双vip的高可用性，让MASTER和BACKUP资源充分利用，不闲置；
•         7.优化Linux和Nginx的相关参数，达到自定义目的：如文件描述符，并发连接数，状态统计功能，流量限速等；

• 项目心得：

•         1.提前规划好整个集群的架构，可以提高项目开展时效率；
•         2.对本地hosts文件进行DNS集群域名解析记录，效果并不明显，考虑在前面加一个负载均衡器，实现论询效果；
•         3.对基于Nginx的集群和高可用有了深入的理解，同时对脑裂现象也有了更加深刻的体会和分析；
•         4.锻炼了自己细心和用专业角度解决问题的能力；

---

项目名称：基于keepalived的双vip高可用web集群

---

架构图

---

项目环境：CentOS 8.2（8台 1核1G），ansible 2.9.17，keepalived 2.0.10，Nginx 1.19.7，NFS

项目描述：

---

项目步骤：

# ansible服务器
[root@ansible ~]# ssh-keygen -t ecdsa					#注：生成密钥对
[root@ansible ~]# ssh-copy-id -i ~/.ssh/id_ecdsa.pub 192.168.1.110	#注：firewall
[root@ansible ~]# ssh-copy-id -i ~/.ssh/id_ecdsa.pub 192.168.1.130	#注：堡垒机
[root@ansible ~]# ssh-copy-id -i ~/.ssh/id_ecdsa.pub 192.168.1.141	#注：load_balancer-1
[root@ansible ~]# ssh-copy-id -i ~/.ssh/id_ecdsa.pub 192.168.1.142	#注：load_balancer-2
[root@ansible ~]# ssh-copy-id -i ~/.ssh/id_ecdsa.pub 192.168.1.151	#注：web-1
[root@ansible ~]# ssh-copy-id -i ~/.ssh/id_ecdsa.pub 192.168.1.152	#注：web-2
[root@ansible ~]# ssh-copy-id -i ~/.ssh/id_ecdsa.pub 192.168.1.153	#注：web-3

# 配置firewall服务器的NAT策略为了发布内网的ssh服务器（即web-1服务器）
[root@firewall ~]# cat iptables_open_ssh-server.sh 
#!/bin/bash
  
#停止firewalld服务
service firewalld stop
#清除下iptables里的规则
iptables -t nat -F
iptables -F
#disabled selinux
setenforce 0
sed -i '/^SELINUX=/ s/enforcing/disabled/' /etc/sysconfig/selinux

#开启路由功能
echo 1 >/proc/sys/net/ipv4/ip_forward

#配置SNAT策略，实现snat功能,将所有内网是192.168.1.0/24这个网段的ip包的源ip修改为192.168.1.110
iptables -t nat -A POSTROUTING -s 192.168.1.0/24 -o ens33 -j SNAT --to-source 192.168.1.110
#配置DNAT策略，发布内网的ssh服务器192.168.1.151(即web-1服务器)，访问firewall服务器的2233端口转发到内网的192.158.1.151的22号端口
iptables -t nat -A PREROUTING -d 192.168.1.110 -p tcp --dport 2233 -i ens33 -j DNAT --to-destination 192.168.1.151:22

[root@firewall ~]# bash iptables_open_ssh-server.sh

效果：192.168.1.110:2233 ssh可以连到web1 192.168.1.151:22

效果图

# ansible服务器
[root@ansible ~]# yum install epel-release -y
[root@ansible ~]# yum install ansible -y

[root@ansible ~]# cat /etc/ansible/hosts 
[nginx-servers]
192.168.1.141
192.168.1.142
192.168.1.151
192.168.1.152
192.168.1.153

[keepalived-servers]
192.168.1.141
192.168.1.142

[root@ansible ~]# cat ansible_playbook.yaml
- hosts: nginx-servers
  remote_user: root
  tasks:
  - name: deploy server-web
    script: ~/onekey_install_nginx.sh
- hosts: keepalived-servers
  remote_user: root
  tasks:
  - name: deploy server-lb
    script: ~/onekey_install_keepalived.sh

[root@ansible ~]# ansible-playbook ansible_playbook.yaml

效果图

Nginx编译安装脚本

[root@ansible ~]# cat onekey_install_nginx.sh 
#!/bin/bash

#解决软件的依赖关系，需要安装的软件包
yum -y install zlib zlib-devel openssl openssl-devel pcre pcre-devel gcc gcc-c++ autoconf automake make
#useradd cPen
id cPen || useradd -s /sbin/nologin cPen
#download nginx
mkdir -p /nginx
cd /nginx
curl -O http://nginx.org/download/nginx-1.19.7.tar.gz
#解压 下载的nginx的源码包
tar xf nginx-1.19.7.tar.gz
cd nginx-1.19.7
#生成编译前配置工作 --> Makefile
./configure --prefix=/usr/local/nginx  --user=cPen  --group=cPen --with-threads --with-http_ssl_module  --with-http_realip_module  --with-http_v2_module --with-file-aio  --with-http_stub_status_module --with-stream
#编译
make -j 2
#编译安装 --> 将编译好的二进制程序安装到指定目录 /usr/local/nginx1
make install

#修改PATH变量
echo "PATH=$PATH:/usr/local/nginx/sbin" >>/root/.bashrc
#执行修改了环境变量的脚本
source /root/.bashrc

#启动nginx
/usr/local/nginx/sbin/nginx

#firewalld and selinux

#stop firewalld和设置下次开机不启动firewalld
service firewalld stop
systemctl disable firewalld

#临时停止selinux和永久停止selinux
setenforce 0
sed -i '/^SELINUX=/ s/enforcing/disabled/' /etc/sysconfig/selinux

#开机自启
echo "/usr/local/nginx/sbin/nginx" >>/etc/rc.local
chmod +x /etc/rc.d/rc.local

Keepalived部署脚本

[root@ansible ~]# cat onekey_install_keepalived.sh 
#!/bin/bash
  
#安装keepalived
yum install keepalived -y

#关闭防火墙
service firewalld stop
systemctl disable firewalld

web-1: 192.168.1.151    nfs+web服务器
web-2: 192.168.1.152    web服务器
web-3: 192.168.1.153    web服务器

1.安装

[root@web-1 ~]# yum install nfs-utils -y

[root@web-2 ~]# yum install nfs-utils -y

[root@web-3 ~]# yum install nfs-utils -y

2.添加exports配置

[root@web-1 ~]# vim /etc/exports
/usr/local/nginx/html 192.168.1.0/24(rw)

3.重启nfs服务

[root@web-1 ~]# service nfs-server restart

4.web-1配置好nfs服务，开启nfs服务，然后在web-2、web-3两台上挂载网络文件

[root@web-2 ~]# mount -t nfs 192.168.1.151:/usr/local/nginx/html /usr/local/nginx/html

[root@web-3 /]# mount -t nfs 192.168.1.151:/usr/local/nginx/html /usr/local/nginx/html

[root@web-1 ~]# echo "this is the nfs_test; from web-1 index" >/usr/local/nginx/html/index.html

效果图

# 负载均衡器 负载均衡配置
[root@load_balancer-1 ~]# vim /usr/local/nginx/conf/nginx.conf
    upstream myweb {
        server 192.168.1.151;
        server 192.168.1.152;
        server 192.168.1.153;
    }
    server {
        listen       80;
        server_name  www.web_test.com;

        location / {
            proxy_pass  http://myweb;
        }

        location /download {
            proxy_pass  http://192.168.1.151;
        }
    }

# 负载均衡器 load balancer-2 配置相同

# 中台系统（负载均衡器）的配置
[root@load_balancer-1 ~]# vim /usr/local/nginx/conf/nginx.conf
    server {
        listen       80;
        server_name  www.web_test.com;

        location /download {
            proxy_pass  http://192.168.1.151;
        }

[root@load_balancer-1 ~]# nginx -s reload

# 中台系统 load balancer-2 配置同理

# web-1服务器的配置
# autoindex on 指令放在 location / 下，实现下载功能
[root@web-1 download]# cat /usr/local/nginx/conf/nginx.conf
……
        location / {
            root   html;
            index  index.html index.htm;
            autoindex on;
        }
……

[root@web-1 ~]# nginx -s reload

[root@web-1 ~]# mkdir /usr/local/nginx/html/download
[root@web-1 ~]# cd /usr/local/nginx/html/download/
[root@web-1 download]# cp /nginx/nginx-1.19.7 . -r
[root@web-1 download]# touch download_test1.txt
[root@web-1 download]# touch download_test2.txt
[root@web-1 download]# touch download_test3.txt

效果图

如何实现双vip的效果？
        dns域名集群解析记录+双vip实现，master和backup都不闲置

C:\Windows\System32\drivers\etc\hosts

192.168.1.181  www.web_test.com
192.168.1.182  www.web_test.com

在这里插入代码片

负载均衡器 负载均衡配置
[root@load_balancer-1 ~]# vim /usr/local/nginx/conf/nginx.conf
    upstream myweb {
        server 192.168.1.151;
        server 192.168.1.152;
        server 192.168.1.153;
    }
    server {
        listen       80;
        server_name  www.web_test.com;

        location / {
            proxy_pass  http://myweb;
        }

        location /download {
            proxy_pass  http://192.168.1.151;
        }
    }

# 负载均衡器 load balancer-2 配置同理

负载均衡器 keepalived高可用性 双主模式配置

# load balancer-1服务器上的配置
[root@load_balancer-1 ~]# cd /etc/keepalived/
[root@load_balancer-1 keepalived]# cat keepalived.conf 
! Configuration File for keepalived

global_defs {
   notification_email {
     [e-mail]
     [e-mail]
     [e-mail]
   }
   notification_email_from [e-mail]
   smtp_server 192.168.200.1
   smtp_connect_timeout 30
   router_id LVS_DEVEL
   vrrp_skip_check_adv_addr
   #vrrp_strict
   vrrp_garp_interval 0
   vrrp_gna_interval 0
}

vrrp_instance VI_1 {
    state MASTER
    interface ens33
    virtual_router_id 199
    priority 110
    advert_int 1
    authentication {
        auth_type PASS
        auth_pass 123456
    }
    virtual_ipaddress {
        192.168.1.181
    }
}

vrrp_instance VI_2 {
    state BACKUP
    interface ens33
    virtual_router_id 200
    priority 100
    advert_int 1
    authentication {
        auth_type PASS
        auth_pass 123456
    }
    virtual_ipaddress {
        192.168.1.182
    }
}

# load balancer-2服务器上的配置
[root@load_balancer-2 ~]# cd /etc/keepalived/
[root@load_balancer-2 keepalived]# cat keepalived.conf 
! Configuration File for keepalived

global_defs {
   notification_email {
     [e-mail]
     [e-mail]
     [e-mail]
   }
   notification_email_from [e-mail]
   smtp_server 192.168.200.1
   smtp_connect_timeout 30
   router_id LVS_DEVEL
   vrrp_skip_check_adv_addr
   #vrrp_strict
   vrrp_garp_interval 0
   vrrp_gna_interval 0
}

vrrp_instance VI_1 {
    state BACKUP
    interface ens33
    virtual_router_id 199
    priority 100
    advert_int 1
    authentication {
        auth_type PASS
        auth_pass 123456
    }
    virtual_ipaddress {
        192.168.1.181
    }
}

vrrp_instance VI_2 {
    state MASTER
    interface ens33
    virtual_router_id 200
    priority 110
    advert_int 1
    authentication {
        auth_type PASS
        auth_pass 123456
    }
    virtual_ipaddress {
        192.168.1.182
    }
}

重启keepalived服务

[root@load_balancer-1 keepalived]# service keepalived restart

[root@load_balancer-2 keepalived]# service keepalived restart

效果

# 负载均衡器load_balancer-1
[root@load_balancer-1 ~]# ip add
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
    inet6 ::1/128 scope host 
       valid_lft forever preferred_lft forever
2: ens33: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc fq_codel state UP group default qlen 1000
    link/ether 00:0c:29:38:10:f7 brd ff:ff:ff:ff:ff:ff
    inet 192.168.1.141/24 brd 192.168.1.255 scope global noprefixroute ens33
       valid_lft forever preferred_lft forever
    inet 192.168.1.181/32 scope global ens33
       valid_lft forever preferred_lft forever
    inet6 2409:8a20:b86f:aac0:20c:29ff:fe38:10f7/64 scope global dynamic mngtmpaddr 
       valid_lft 259016sec preferred_lft 172616sec
    inet6 fe80::20c:29ff:fe38:10f7/64 scope link 
       valid_lft forever preferred_lft forever

# 负载均衡器load_balancer-2
[root@load_balancer-2 ~]# ip add
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
    inet6 ::1/128 scope host 
       valid_lft forever preferred_lft forever
2: ens33: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc fq_codel state UP group default qlen 1000
    link/ether 00:0c:29:33:ee:80 brd ff:ff:ff:ff:ff:ff
    inet 192.168.1.142/24 brd 192.168.1.255 scope global noprefixroute ens33
       valid_lft forever preferred_lft forever
    inet 192.168.1.182/32 scope global ens33
       valid_lft forever preferred_lft forever
    inet6 2409:8a20:b86f:aac0:20c:29ff:fe33:ee80/64 scope global dynamic mngtmpaddr 
       valid_lft 258985sec preferred_lft 172585sec
    inet6 fe80::20c:29ff:fe33:ee80/64 scope link 
       valid_lft forever preferred_lft forever

效果图

# 示例1：状态统计功能
[root@web-1 ~]# vim /usr/local/nginx/conf/nginx.conf
        location = /status {
            stub_status;
        }
     
[root@web-1 ~]# nginx -s reload

效果图

# 示例2：限速
[root@web-1 ~]# vim /usr/local/nginx/conf/nginx.conf
    server {
        listen       80;
        limit_rate_after 10k;
        limit_rate 5k;
……

[root@web-1 ~]# nginx -s reload

效果图

短时间多次刷新网址后

# 示例：文件描述符+并发连接数
[root@web-1 ~]# vim /usr/local/nginx/conf/nginx.conf
    limit_conn_zone $binary_remote_addr zone=perip:10m;
    server {
        listen       80;
        limit_conn perip 1;
……

[root@web-1 ~]# nginx -s reload

[root@web-1 ~]# ulimit -n 1000000

---