接着上一篇复制文档,搭建keepalived
环境介绍:
两台mysql服务器(mysql 5.6.27)
MySQL01 192.168.11.129(默认:3306端口)
MySQL02 192.168.11.140(默认:3306端口)
VIP:192.168.11.100
关闭防火墙和selinux
一 Keepalived安装
1.安装源码包
# rpm -ivh http://mirrors.ustc.edu.cn/fedora/epel/6/x86_64/epel-release-6-8.noarch.rpm
# yum install keepalived
# yum install MySQL-python
2.默认配置文件路径
/etc/keepalived/keepalived.conf
3.日志目录
keepalived默认日志目录:/var/log/messages
清空日志 cat /dev/null > /var/log/messages
4.创建一个监控用户:
GRANT REPLICATION CLIENT ON *.* TO 'monitor'@'%' IDENTIFIED BY 'monitor';
flush privileges;
5.安装keepalived可能出现的问题
报错1:
configure: error:
!!! OpenSSL is not properly installed on your system. !!!
!!! Can not include OpenSSL headers files. !!!
解决:
yum install -y openssl openssl-devel
报错2:
configure: error: Popt libraries is required
解决：
yum install popt-devel
报错3:
# /etc/keepalived/checkMySQL.py -h 127.0.0.1 -P 3306
Traceback (most recent call last):
File "/etc/keepalived/checkMySQL.py", line 8, in <module>
import MySQLdb
File "/usr/local/lib/python2.7/site-packages/MySQLdb/__init__.py", line 19, in <module>
import _mysql
ImportError: libmysqlclient.so.18: cannot open shared object file: No such file or directory
解决：
/etc/ld.so.conf.d/mysql-x86_64.conf中添加
/usr/local/mysql/lib/
然后
# ldconfig
二 配置文件(主从配置文件一致)
# cat /etc/keepalived/keepalived.conf
vrrp_script vs_mysql_100 {
script "/etc/keepalived/checkMySQL.py -h 127.0.0.1 -P 3306"
#script "/etc/keepalived/check_mysql.sh"
interval 60 #运行上面的检测脚本时间间隔
}
vrrp_instance VI_100 {
state BACKUP #刚开始先处于backup状态
nopreempt #不抢占，只在优先级高的机器上设置即可，优先级低的机器不设置
interface eth2 #绑定的网卡
virtual_router_id 100 #组ID，要和其他组分开(0~255)
priority 100 #优先级，另一台可以改为90
advert_int 5
authentication {
auth_type PASS
auth_pass 1111
}
track_script {
vs_mysql_100 #脚本返回0表示正常,返回非0都不正常,会发生VIP切换(释放本地VIP)
}
#notify_master /etc/keepalived/notify_master_mysql.sh
virtual_ipaddress {
192.168.11.100
}
}
建议:
1.virtual_router_id 的值和VIP 最后一位保持一致，方便管理
2.希望VIP在哪台机器上,就先在哪个机器上启动keepavlived如下脚本用来确认本地的MySQL是否可以连接上去，相当于做了一个Connect ping.主从都执行,返回值为0表示配置正常.
# /etc/keepalived/checkMySQL.py -h 127.0.0.1 -P 3306
# echo $?
0
checkMySQL.py 脚本
# cat /etc/keepalived/checkMySQL.py
#!/usr/bin/python
#coding: utf-8
import sys
import os
import getopt
import MySQLdb
import logging
dbhost='localhost'
dbport=3306
dbuser='monitor'
dbpassword='monitor'
def checkMySQL():
global dbhost
global dbport
global dbuser
global dbpassword
shortargs='h:P:'
opts, args=getopt.getopt(sys.argv[1:],shortargs)
for opt, value in opts:
if opt=='-h':
dbhost=value
elif opt=='-P':
dbport=value
#print "host : %s, port: %d, user: %s, password: %s" % (dbhost, int(dbport), dbuser, dbpassword)
db = instanceMySQL(dbhost, dbport, dbuser, dbpassword)
st = db.ishaveMySQL()
#if ( db.connect() != 0 ):
# return 1
#db.disconnect()
return st
class instanceMySQL:
conn = None
def __init__(self, host=None,port=None, user=None, passwd=None):
self.dbhost= host
self.dbport = int(port)
self.dbuser = user
self.dbpassword = passwd
def ishaveMySQL(self):
cmd="ps -ef | egrep -i \"mysqld\" | grep %s | egrep -iv \"mysqld_safe\" | grep -v grep | wc -l" % self.dbport
mysqldNum = os.popen(cmd).read()
cmd ="netstat -tunlp | grep \":%s\" | wc -l" % self.dbport
mysqlPortNum= os.popen(cmd).read()
#print mysqldNum, mysqlPortNum
if ( int(mysqldNum) <= 0):
print "error"
return 1
if ( int(mysqldNum) > 0 and mysqlPortNum <= 0):
return 1
return 0
def connect(self):
# print "in db conn"
# print "host : %s, port: %d, user: %s, password: %s" % (self.dbhost, self.dbport, self.dbuser, self.dbpassword)
try:
self.conn=MySQLdb.connect(host="%s"%self.dbhost, port=self.dbport,user="%s"%dbuser, passwd="%s"%self.dbpassword)
except Exception, e:
# print " Error"
print e
return 1
return 0
def disconnect(self):
if (self.conn):
self.conn.close()
self.conn = None
if __name__== "__main__":
st=checkMySQL()
sys.exit(st)三 启动keepalived
keepalived启动流程：
keepalived主进程
healthcheck进程
vrrp进程
先进入backup state,运行一次vrrp_script 成功后，发现没有主 ,-> master state -> 拉起VIP 完成启动
切换流程：
原keepalived master 节点，运行检查脚本异常，则keepalived进入FAULT state ,释放VIP，原backup的keepalived 会接管VIP
注意：如果节点是FAULT state，则不具备接管条件。
如果原keepalived mysql 又启动了，则会进入backup state,具备接管条件。
如果主节点直接挂了 poweroff ,则backup 的节点会直接接管，不会等待。
主服务器先启动keepalived
# /etc/init.d/keepalived start
Starting keepalived: [ OK ]
# ps -ef |grep keepalived
root 2472 1 0 20:56 ? 00:00:00 /usr/sbin/keepalived -D
root 2473 2472 0 20:56 ? 00:00:00 /usr/sbin/keepalived -D
root 2474 2472 0 20:56 ? 00:00:00 /usr/sbin/keepalived -D
root 2495 2182 0 20:57 pts/0 00:00:00 grep keepalived
查看VIP是否绑定在主服务器上,看到VIP 192.168.11.100已绑定成功
# ip addr show
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 16436 qdisc noqueue state UNKNOWN
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
inet 127.0.0.1/8 scope host lo
inet6 ::1/128 scope host
valid_lft forever preferred_lft forever
2: eth2: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000
link/ether 00:0c:29:80:03:15 brd ff:ff:ff:ff:ff:ff
inet 192.168.11.129/24 brd 192.168.11.255 scope global eth2
inet 192.168.11.100/32 scope global eth2
inet6 fe80::20c:29ff:fe80:315/64 scope link
valid_lft forever preferred_lft forever查看主服务器keepalived启动日志
# cat /var/log/messages
Jul 8 17:05:56 centos02 Keepalived[3146]: Starting Keepalived v1.2.13 (03/19,2015)
Jul 8 17:05:57 centos02 Keepalived[3147]: Starting Healthcheck child process, pid=3148
Jul 8 17:05:57 centos02 Keepalived[3147]: Starting VRRP child process, pid=3149
Jul 8 17:05:57 centos02 Keepalived_vrrp[3149]: Netlink reflector reports IP 192.168.11.129 added
Jul 8 17:05:57 centos02 Keepalived_vrrp[3149]: Netlink reflector reports IP fe80::20c:29ff:fe80:315 added
Jul 8 17:05:57 centos02 Keepalived_vrrp[3149]: Registering Kernel netlink reflector
Jul 8 17:05:57 centos02 Keepalived_vrrp[3149]: Registering Kernel netlink command channel
Jul 8 17:05:57 centos02 Keepalived_vrrp[3149]: Registering gratuitous ARP shared channel
Jul 8 17:05:57 centos02 kernel: IPVS: Registered protocols (TCP, UDP, SCTP, AH, ESP)
Jul 8 17:05:57 centos02 kernel: IPVS: Connection hash table configured (size=4096, memory=64Kbytes)
Jul 8 17:05:57 centos02 kernel: IPVS: ipvs loaded.
Jul 8 17:05:57 centos02 Keepalived_healthcheckers[3148]: Netlink reflector reports IP 192.168.11.129 added
Jul 8 17:05:57 centos02 Keepalived_healthcheckers[3148]: Netlink reflector reports IP fe80::20c:29ff:fe80:315 added
Jul 8 17:05:57 centos02 Keepalived_healthcheckers[3148]: Registering Kernel netlink reflector
Jul 8 17:05:57 centos02 Keepalived_healthcheckers[3148]: Registering Kernel netlink command channel
Jul 8 17:05:57 centos02 Keepalived_vrrp[3149]: Opening file '/etc/keepalived/keepalived.conf'.
Jul 8 17:05:57 centos02 Keepalived_vrrp[3149]: Configuration is using : 62852 Bytes
Jul 8 17:05:57 centos02 Keepalived_vrrp[3149]: Using LinkWatch kernel netlink reflector...
Jul 8 17:05:57 centos02 Keepalived_vrrp[3149]: VRRP_Instance(VI_100) Entering BACKUP STATE
Jul 8 17:05:57 centos02 Keepalived_vrrp[3149]: VRRP sockpool: [ifindex(2), proto(112), unicast(0), fd(10,11)]
Jul 8 17:05:58 centos02 Keepalived_healthcheckers[3148]: Opening file '/etc/keepalived/keepalived.conf'.
Jul 8 17:05:58 centos02 Keepalived_healthcheckers[3148]: Configuration is using : 5150 Bytes
Jul 8 17:05:58 centos02 Keepalived_healthcheckers[3148]: Using LinkWatch kernel netlink reflector...
Jul 8 17:05:58 centos02 Keepalived_vrrp[3149]: VRRP_Script(vs_mysql_100) succeeded
Jul 8 17:06:13 centos02 Keepalived_vrrp[3149]: VRRP_Instance(VI_100) Transition to MASTER STATE
Jul 8 17:06:18 centos02 Keepalived_vrrp[3149]: VRRP_Instance(VI_100) Entering MASTER STATE
Jul 8 17:06:18 centos02 Keepalived_vrrp[3149]: VRRP_Instance(VI_100) setting protocol VIPs.
Jul 8 17:06:18 centos02 Keepalived_vrrp[3149]: VRRP_Instance(VI_100) Sending gratuitous ARPs on eth2 for 192.168.11.100
Jul 8 17:06:18 centos02 Keepalived_healthcheckers[3148]: Netlink reflector reports IP 192.168.11.100 added
Jul 8 17:06:23 centos02 Keepalived_vrrp[3149]: VRRP_Instance(VI_100) Sending gratuitous ARPs on eth2 for 192.168.11.100从服务器启动keepalived
# /etc/init.d/keepalived start
从服务器启动keepalived日志
# cat /var/log/messages
Jul 9 01:16:22 centos03 Keepalived[3086]: Starting Keepalived v1.2.13 (03/19,2015)
Jul 9 01:16:22 centos03 Keepalived[3087]: Starting Healthcheck child process, pid=3089
Jul 9 01:16:22 centos03 Keepalived[3087]: Starting VRRP child process, pid=3090
Jul 9 01:16:22 centos03 Keepalived_vrrp[3090]: Netlink reflector reports IP 192.168.11.140 added
Jul 9 01:16:22 centos03 Keepalived_healthcheckers[3089]: Netlink reflector reports IP 192.168.11.140 added
Jul 9 01:16:22 centos03 Keepalived_vrrp[3090]: Netlink reflector reports IP fe80::20c:29ff:fe78:d7d1 added
Jul 9 01:16:22 centos03 Keepalived_vrrp[3090]: Registering Kernel netlink reflector
Jul 9 01:16:22 centos03 Keepalived_vrrp[3090]: Registering Kernel netlink command channel
Jul 9 01:16:22 centos03 Keepalived_vrrp[3090]: Registering gratuitous ARP shared channel
Jul 9 01:16:22 centos03 Keepalived_healthcheckers[3089]: Netlink reflector reports IP fe80::20c:29ff:fe78:d7d1 added
Jul 9 01:16:22 centos03 Keepalived_healthcheckers[3089]: Registering Kernel netlink reflector
Jul 9 01:16:22 centos03 Keepalived_healthcheckers[3089]: Registering Kernel netlink command channel
Jul 9 01:16:23 centos03 Keepalived_vrrp[3090]: Opening file '/etc/keepalived/keepalived.conf'.
Jul 9 01:16:23 centos03 Keepalived_healthcheckers[3089]: Opening file '/etc/keepalived/keepalived.conf'.
Jul 9 01:16:23 centos03 Keepalived_healthcheckers[3089]: Configuration is using : 5150 Bytes
Jul 9 01:16:23 centos03 Keepalived_vrrp[3090]: Configuration is using : 62852 Bytes
Jul 9 01:16:23 centos03 Keepalived_vrrp[3090]: Using LinkWatch kernel netlink reflector...
Jul 9 01:16:23 centos03 Keepalived_healthcheckers[3089]: Using LinkWatch kernel netlink reflector...
Jul 9 01:16:23 centos03 Keepalived_vrrp[3090]: VRRP_Instance(VI_100) Entering BACKUP STATE
Jul 9 01:16:23 centos03 Keepalived_vrrp[3090]: VRRP sockpool: [ifindex(2), proto(112), unicast(0), fd(10,11)]
Jul 9 01:16:23 centos03 Keepalived_vrrp[3090]: VRRP_Script(vs_mysql_100) succeeded
使用VIP连接mysql
# mysql -h192.168.11.100 -uroot -p
root@192.168.11.100:3306 01:18:26 [(none)]>show variables like '%server_id%';
+----------------+---------+
| Variable_name | Value |
+----------------+---------+
| server_id | 1763306 | #为MySQL01 的server_id,说明mysql服务当前在主节点上
| server_id_bits | 32 |
+----------------+---------+
2 rows in set (0.11 sec)到此keepalived搭建成功,至于切换下节再讲.
设置从库为只读(注意当从库变成主库时,要关闭只读属性)
mysql> set global read_only=on;
四 关于脑裂
假设VIP在MySQL01上,由于网络问题(或者其他问题)MySQL02可能认为MySQL01死掉了 ,MySQL02也会接管VIP.此时两台服务器上都有了VIP,如果网络好了,谁的VIP生效呢?
就看谁最后更新了ARP列表而且更新成功了,谁就生效.
简单的检查办法:
# ssh vip 看连到哪台机器
1 什么情况下会遇到脑裂
a.由于防火墙启着,使VRRP通信中断,导致脑裂
b.网络中断,由于两台机器在不同的交换机下,如果交换机之间网络不通,但是都能连的出去.也会导致脑裂
2 怎么解决脑裂
a.关闭防火墙
b.使得机器在同一个交换机下或者在同一个局域网下
3 脚本要做的,每次先ping网关
"/etc/keepalived/checkMySQL.py"脚本中添加 ping网关的操作,如果网关都ping不通,return 非0,退出抢夺VIP,state直接进入FAULT状态
