1. 前言




        MHA可以在较短的时间内实现自己主动故障检測和故障转移,通常在10-30秒以内;在复制框架中,MHA可以非常好地解决复制过程中的数据一致性问题,因为不须要在现有的replication中加入额外的server,仅须要一个manager节点,而一个Manager能管理多套复制。所以能大大地节约server的数量;另外,安装简单,无性能损耗,以及不须要改动现有的复制部署也是它的优势之处。

         MHA还提供在线主库切换的功能,可以安全地切换当前执行的主库到一个新的主库中(通过将从库提升为主库),大概0.5-2秒内就可以完毕。

MHA提供了上述功能。使得其在适用于对高可用性。数据完整性要求高的场合。还有要求差点儿non-stop的主库维护。

         这次主要是解说MHA的高速部署和简单配置。兴许补充高级特性以及相关架构吧。


2. 环境介绍


环境有限。我使用的是vmware虚拟机,搭建三台server,1主2从结构。

当中manager管理节点放在当中一台从库中。


1)server1(node节点)


OS:CentOS 6.4 X64


IPADDR:172.16.21.12(eth0);192.168.8.6(eth1)


HOSTNAME:mysql-mha01


DB:Mysql 5.5.43-log MySQL Community Server (GPL)


SOFTWARE:mha4mysql-node-0.56-0.el6.noarch.rpm


防火墙关闭;selinux关闭。





2)server2(node节点)


OS:CentOS 6.4 X64


IPADDR:172.16.21.13(eth2)。192.168.8.7(eth3)


HOSTNAME:mysql-mha02


DB:Mysql 5.5.43-log MySQL Community Server (GPL)


SOFTWARE:mha4mysql-node-0.56-0.el6.noarch.rpm


防火墙关闭。selinux关闭;




3)server3(node节点。manager节点)


OS:CentOS 6.4 X64


IPADDR:172.16.21.15(eth2);192.168.8.8(eth3)


HOSTNAME:mysql-mha01


DB:Mysql 5.5.43-log MySQL Community Server (GPL)


SOFTWARE:mha4mysql-node-0.56-0.el6.noarch.rpm。mha4mysql-manager-0.56-0.el6.noarch.rpm


防火墙关闭;selinux关闭;



MHA软件下载地址 ​​https://code.google.com/p/mysql-master-ha/wiki/Downloads?tm=2​

3. 软件安装

3.1 Node节点安装


在3个节点上安装node软件包。首先安装依赖包



yum install perl-DBD-MySQL


安装node软件包


rpm -ivh mha4mysql-node-0.56-0.el6.noarch.rpm


3.2 Manager节点安装


安装依赖包

yum install perl-Params-Validate perl-LWP-Simple perl-Config-Tiny perl-Time-HiRes perl-Log-Dispatch perl-Parallel-ForkManager


我当时配置的163的yum源没有perl-Log-Dispatch和perl-Parallel-ForkManager,所以就自己下载的rpm包安装,大家能够到​​http://pkgs.repoforge.org/​​ 下载自己须要的包。



rpm -ivh perl-Log-Dispatch-2.26-1.el6.rf.noarch.rpm perl-Parallel-ForkManager-0.7.5-2.2.el6.rf.noarch.rpm


安装manager软件包


rpm -ivh mha4mysql-manager-0.56-0.el6.noarch.rpm


4. MHA配置


4.1 Mysql复制配置


        此处就不具体说明了,大家能够參考我曾经的文章。​​javascript:void(0)​​ 和 ​​javascript:void(0)​​。


我搭建的是半同步复制的一主双从结构。当中mysql-mha02为master节点,其它为slave节点。

复制账户passwordrepl/slavepass。(PS我们公司环境是5.5的所以这里的虚拟机也是用到的mysql5.5。如今MHA支持5.6的GTID复制。因此大家能够考虑使用5.6的gtid搭建。能够方便master因为崩溃恢复后又一次增加集群管理)




4.2 MHA配置


在管理节点mysql-mha02中建立配置文件



mkdir -p /etc/MHA
touch /etc/MHA/MHAmanager.conf

查看manager配置文件


[server default]
# mysql user and password
user=root
password=123456
ssh_user=root

# mysql replication user and password
repl_user=repl
repl_password=slavepass

# working directory on the manager
manager_workdir=/var/log/masterha/manager

# working directory on MySQL servers
remote_workdir=/var/log/masterha/node

#Master Binlog dir
master_binlog_dir=/var/lib/mysql/logs

#secondary route to detect the reachable of master
secondary_check_script= masterha_secondary_check -s mysql-mha03 --user=root --master_host=mysql-mha02 --master_ip=172.16.21.13 --master_port=3306

#define the manager log
manager_log=/var/log/masterha/manager.log

[server1]
hostname=mysql-mha01

[server2]
hostname=mysql-mha02

[server3]
hostname=mysql-mha03


MHA其中具体參数大家能够參考 ​​https://code.google.com/p/mysql-master-ha/wiki/Parameters​​。

master_binlog_dir标志着master节点binlog的存放位置。假设大家没有使用默认的/var/lib/mysql的话须要单独制定。MHA在故障切换的时候假设master节点能够ssh联通的话会自己主动到该文件夹抓取日志文件来防止slave节点日志不完整。


workdir是manager节点和node节点的工作文件夹。在MHA管理启动时候。假设文件夹不存在,会通过ssh来进行创建。请保证ssh用户的权限。


4.3 主机信任


MHA环境须要保证manager和node之间的信任。



ssh-keygen -t rsa 
ssh-copy-id -i /root/.ssh/id_rsa.pub root@192.168.8.6
ssh-copy-id -i /root/.ssh/id_rsa.pub root@192.168.8.7
ssh-copy-id -i /root/.ssh/id_rsa.pub root@192.168.8.8


验证机器信任


[root@mysql-mha03 ~]# masterha_check_ssh --conf=/etc/MHA/MHAmanager.conf 
Thu Jul 23 14:33:26 2015 - [warning] Global configuration file /etc/masterha_default.cnf not found. Skipping.
Thu Jul 23 14:33:26 2015 - [info] Reading application default configuration from /etc/MHA/MHAmanager.conf..
Thu Jul 23 14:33:26 2015 - [info] Reading server configuration from /etc/MHA/MHAmanager.conf..
Thu Jul 23 14:33:26 2015 - [info] Starting SSH connection tests..
Thu Jul 23 14:33:26 2015 - [debug]
Thu Jul 23 14:33:26 2015 - [debug] Connecting via SSH from root@mysql-mha01(192.168.8.6:22) to root@mysql-mha02(192.168.8.7:22)..
Thu Jul 23 14:33:26 2015 - [debug] ok.
Thu Jul 23 14:33:26 2015 - [debug] Connecting via SSH from root@mysql-mha01(192.168.8.6:22) to root@mysql-mha03(192.168.8.8:22)..
Thu Jul 23 14:33:26 2015 - [debug] ok.
Thu Jul 23 14:33:27 2015 - [debug]
Thu Jul 23 14:33:26 2015 - [debug] Connecting via SSH from root@mysql-mha02(192.168.8.7:22) to root@mysql-mha01(192.168.8.6:22)..
Thu Jul 23 14:33:26 2015 - [debug] ok.
Thu Jul 23 14:33:26 2015 - [debug] Connecting via SSH from root@mysql-mha02(192.168.8.7:22) to root@mysql-mha03(192.168.8.8:22)..
Thu Jul 23 14:33:27 2015 - [debug] ok.
Thu Jul 23 14:33:27 2015 - [debug]
Thu Jul 23 14:33:27 2015 - [debug] Connecting via SSH from root@mysql-mha03(192.168.8.8:22) to root@mysql-mha01(192.168.8.6:22)..
Thu Jul 23 14:33:27 2015 - [debug] ok.
Thu Jul 23 14:33:27 2015 - [debug] Connecting via SSH from root@mysql-mha03(192.168.8.8:22) to root@mysql-mha02(192.168.8.7:22)..
Thu Jul 23 14:33:27 2015 - [debug] ok.
Thu Jul 23 14:33:27 2015 - [info] All SSH connection tests passed successfully.


4.4 开启MHA


验证复制环境



[root@mysql-mha03 ~]# masterha_check_repl --conf=/etc/MHA/MHAmanager.conf 
Thu Jul 23 14:40:27 2015 - [warning] Global configuration file /etc/masterha_default.cnf not found. Skipping.
Thu Jul 23 14:40:27 2015 - [info] Reading application default configuration from /etc/MHA/MHAmanager.conf..
Thu Jul 23 14:40:27 2015 - [info] Reading server configuration from /etc/MHA/MHAmanager.conf..
Thu Jul 23 14:40:27 2015 - [info] MHA::MasterMonitor version 0.56.
Thu Jul 23 14:40:28 2015 - [info] GTID failover mode = 0
Thu Jul 23 14:40:28 2015 - [info] Dead Servers:
Thu Jul 23 14:40:28 2015 - [info] Alive Servers:
Thu Jul 23 14:40:28 2015 - [info] mysql-mha01(192.168.8.6:3306)
Thu Jul 23 14:40:28 2015 - [info] mysql-mha02(192.168.8.7:3306)
Thu Jul 23 14:40:28 2015 - [info] mysql-mha03(192.168.8.8:3306)
Thu Jul 23 14:40:28 2015 - [info] Alive Slaves:
Thu Jul 23 14:40:28 2015 - [info] mysql-mha01(192.168.8.6:3306) Version=5.5.43-log (oldest major version between slaves) log-bin:enabled
Thu Jul 23 14:40:28 2015 - [info] Replicating from 192.168.8.7(192.168.8.7:3306)
Thu Jul 23 14:40:28 2015 - [info] mysql-mha03(192.168.8.8:3306) Version=5.5.43-log (oldest major version between slaves) log-bin:enabled
Thu Jul 23 14:40:28 2015 - [info] Replicating from 192.168.8.7(192.168.8.7:3306)
Thu Jul 23 14:40:28 2015 - [info] Current Alive Master: mysql-mha02(192.168.8.7:3306)
Thu Jul 23 14:40:28 2015 - [info] Checking slave configurations..
Thu Jul 23 14:40:28 2015 - [info] read_only=1 is not set on slave mysql-mha01(192.168.8.6:3306).
Thu Jul 23 14:40:28 2015 - [info] read_only=1 is not set on slave mysql-mha03(192.168.8.8:3306).
Thu Jul 23 14:40:28 2015 - [info] Checking replication filtering settings..
Thu Jul 23 14:40:28 2015 - [info] binlog_do_db= , binlog_ignore_db=
Thu Jul 23 14:40:28 2015 - [info] Replication filtering check ok.
Thu Jul 23 14:40:28 2015 - [info] GTID (with auto-pos) is not supported
Thu Jul 23 14:40:28 2015 - [info] Starting SSH connection tests..
Thu Jul 23 14:40:29 2015 - [info] All SSH connection tests passed successfully.
Thu Jul 23 14:40:29 2015 - [info] Checking MHA Node version..
Thu Jul 23 14:40:29 2015 - [info] Version check ok.
Thu Jul 23 14:40:29 2015 - [info] Checking SSH publickey authentication settings on the current master..
Thu Jul 23 14:40:30 2015 - [info] HealthCheck: SSH to mysql-mha02 is reachable.
Thu Jul 23 14:40:30 2015 - [info] Master MHA Node version is 0.56.
Thu Jul 23 14:40:30 2015 - [info] Checking recovery script configurations on mysql-mha02(192.168.8.7:3306)..
Thu Jul 23 14:40:30 2015 - [info] Executing command: save_binary_logs --command=test --start_pos=4 --binlog_dir=/var/lib/mysql/logs --output_file=/var/log/masterha/node/save_binary_logs_test --manager_version=0.56 --start_file=mysql-bin.000013
Thu Jul 23 14:40:30 2015 - [info] Connecting to root@192.168.8.7(mysql-mha02:22)..
Creating /var/log/masterha/node if not exists.. ok.
Checking output directory is accessible or not..
ok.
Binlog found at /var/lib/mysql/logs, up to mysql-bin.000013
Thu Jul 23 14:40:30 2015 - [info] Binlog setting check done.
Thu Jul 23 14:40:30 2015 - [info] Checking SSH publickey authentication and checking recovery script configurations on all alive slave servers..
Thu Jul 23 14:40:30 2015 - [info] Executing command : apply_diff_relay_logs --command=test --slave_user='root' --slave_host=mysql-mha01 --slave_ip=192.168.8.6 --slave_port=3306 --workdir=/var/log/masterha/node --target_version=5.5.43-log --manager_version=0.56 --relay_log_info=/var/lib/mysql/relay-log.info --relay_dir=/var/lib/mysql/ --slave_pass=xxx
Thu Jul 23 14:40:30 2015 - [info] Connecting to root@192.168.8.6(mysql-mha01:22)..
Checking slave recovery environment settings..
Opening /var/lib/mysql/relay-log.info ... ok.
Relay log found at /var/lib/mysql, up to mysql-mha01-relay-bin.000005
Temporary relay log file is /var/lib/mysql/mysql-mha01-relay-bin.000005
Testing mysql connection and privileges.. done.
Testing mysqlbinlog output.. done.
Cleaning up test file(s).. done.
Thu Jul 23 14:40:30 2015 - [info] Executing command : apply_diff_relay_logs --command=test --slave_user='root' --slave_host=mysql-mha03 --slave_ip=192.168.8.8 --slave_port=3306 --workdir=/var/log/masterha/node --target_version=5.5.43-log --manager_version=0.56 --relay_log_info=/var/lib/mysql/relay-log.info --relay_dir=/var/lib/mysql/ --slave_pass=xxx
Thu Jul 23 14:40:30 2015 - [info] Connecting to root@192.168.8.8(mysql-mha03:22)..
Checking slave recovery environment settings..
Opening /var/lib/mysql/relay-log.info ... ok.
Relay log found at /var/lib/mysql, up to mysql-mha03-relay-bin.000005
Temporary relay log file is /var/lib/mysql/mysql-mha03-relay-bin.000005
Testing mysql connection and privileges.. done.
Testing mysqlbinlog output.. done.
Cleaning up test file(s).. done.
Thu Jul 23 14:40:30 2015 - [info] Slaves settings check done.
Thu Jul 23 14:40:30 2015 - [info]
mysql-mha02(192.168.8.7:3306) (current master)
+--mysql-mha01(192.168.8.6:3306)
+--mysql-mha03(192.168.8.8:3306)

Thu Jul 23 14:40:30 2015 - [info] Checking replication health on mysql-mha01..
Thu Jul 23 14:40:30 2015 - [info] ok.
Thu Jul 23 14:40:30 2015 - [info] Checking replication health on mysql-mha03..
Thu Jul 23 14:40:30 2015 - [info] ok.
Thu Jul 23 14:40:30 2015 - [warning] master_ip_failover_script is not defined.
Thu Jul 23 14:40:30 2015 - [warning] shutdown_script is not defined.
Thu Jul 23 14:40:30 2015 - [info] Got exit code 0 (Not master dead).

MySQL Replication Health is OK.


启动mha manager


[root@mysql-mha03 ~]# nohup masterha_manager --conf=/etc/MHA/MHAmanager.conf &
[1] 5552



查看启动日志



Thu Jul 23 14:49:01 2015 - [info] MHA::MasterMonitor version 0.56.
Thu Jul 23 14:49:01 2015 - [info] GTID failover mode = 0
Thu Jul 23 14:49:01 2015 - [info] Dead Servers:
Thu Jul 23 14:49:01 2015 - [info] Alive Servers:
Thu Jul 23 14:49:01 2015 - [info] mysql-mha01(192.168.8.6:3306)
Thu Jul 23 14:49:01 2015 - [info] mysql-mha02(192.168.8.7:3306)
Thu Jul 23 14:49:01 2015 - [info] mysql-mha03(192.168.8.8:3306)
Thu Jul 23 14:49:01 2015 - [info] Alive Slaves:
Thu Jul 23 14:49:01 2015 - [info] mysql-mha01(192.168.8.6:3306) Version=5.5.43-log (oldest major version between slaves) log-bin:enabled
Thu Jul 23 14:49:01 2015 - [info] Replicating from 192.168.8.7(192.168.8.7:3306)
Thu Jul 23 14:49:01 2015 - [info] mysql-mha03(192.168.8.8:3306) Version=5.5.43-log (oldest major version between slaves) log-bin:enabled
Thu Jul 23 14:49:01 2015 - [info] Replicating from 192.168.8.7(192.168.8.7:3306)
Thu Jul 23 14:49:01 2015 - [info] Current Alive Master: mysql-mha02(192.168.8.7:3306)
Thu Jul 23 14:49:01 2015 - [info] Checking slave configurations..
Thu Jul 23 14:49:01 2015 - [info] read_only=1 is not set on slave mysql-mha01(192.168.8.6:3306).
Thu Jul 23 14:49:01 2015 - [info] read_only=1 is not set on slave mysql-mha03(192.168.8.8:3306).
Thu Jul 23 14:49:01 2015 - [info] Checking replication filtering settings..
Thu Jul 23 14:49:01 2015 - [info] binlog_do_db= , binlog_ignore_db=
Thu Jul 23 14:49:01 2015 - [info] Replication filtering check ok.
Thu Jul 23 14:49:01 2015 - [info] GTID (with auto-pos) is not supported
Thu Jul 23 14:49:01 2015 - [info] Starting SSH connection tests..
Thu Jul 23 14:49:02 2015 - [info] All SSH connection tests passed successfully.
Thu Jul 23 14:49:02 2015 - [info] Checking MHA Node version..
Thu Jul 23 14:49:03 2015 - [info] Version check ok.
Thu Jul 23 14:49:03 2015 - [info] Checking SSH publickey authentication settings on the current master..
Thu Jul 23 14:49:03 2015 - [info] HealthCheck: SSH to mysql-mha02 is reachable.
Thu Jul 23 14:49:03 2015 - [info] Master MHA Node version is 0.56.
Thu Jul 23 14:49:03 2015 - [info] Checking recovery script configurations on mysql-mha02(192.168.8.7:3306)..
Thu Jul 23 14:49:03 2015 - [info] Executing command: save_binary_logs --command=test --start_pos=4 --binlog_dir=/var/lib/mysql/logs --output_file=/var/log/masterha/node/save_binary_logs_test --manager_version=0.56 --start_file=mysql-bin.000013
Thu Jul 23 14:49:03 2015 - [info] Connecting to root@192.168.8.7(mysql-mha02:22)..
Creating /var/log/masterha/node if not exists.. ok.
Checking output directory is accessible or not..
ok.
Binlog found at /var/lib/mysql/logs, up to mysql-bin.000013
Thu Jul 23 14:49:03 2015 - [info] Binlog setting check done.
Thu Jul 23 14:49:03 2015 - [info] Checking SSH publickey authentication and checking recovery script configurations on all alive slave servers..
Thu Jul 23 14:49:03 2015 - [info] Executing command : apply_diff_relay_logs --command=test --slave_user='root' --slave_host=mysql-mha01 --slave_ip=192.168.8.6 --slave_port=3306 --workdir=/var/log/masterha/node --target_version=5.5.43-log --manager_version=0.56 --relay_log_info=/var/lib/mysql/relay-log.info --relay_dir=/var/lib/mysql/ --slave_pass=xxx
Thu Jul 23 14:49:03 2015 - [info] Connecting to root@192.168.8.6(mysql-mha01:22)..
Checking slave recovery environment settings..
Opening /var/lib/mysql/relay-log.info ... ok.
Relay log found at /var/lib/mysql, up to mysql-mha01-relay-bin.000005
Temporary relay log file is /var/lib/mysql/mysql-mha01-relay-bin.000005
Testing mysql connection and privileges.. done.
Testing mysqlbinlog output.. done.
Cleaning up test file(s).. done.
Thu Jul 23 14:49:03 2015 - [info] Executing command : apply_diff_relay_logs --command=test --slave_user='root' --slave_host=mysql-mha03 --slave_ip=192.168.8.8 --slave_port=3306 --workdir=/var/log/masterha/node --target_version=5.5.43-log --manager_version=0.56 --relay_log_info=/var/lib/mysql/relay-log.info --relay_dir=/var/lib/mysql/ --slave_pass=xxx
Thu Jul 23 14:49:03 2015 - [info] Connecting to root@192.168.8.8(mysql-mha03:22)..
Checking slave recovery environment settings..
Opening /var/lib/mysql/relay-log.info ... ok.
Relay log found at /var/lib/mysql, up to mysql-mha03-relay-bin.000005
Temporary relay log file is /var/lib/mysql/mysql-mha03-relay-bin.000005
Testing mysql connection and privileges.. done.
Testing mysqlbinlog output.. done.
Cleaning up test file(s).. done.
Thu Jul 23 14:49:04 2015 - [info] Slaves settings check done.
Thu Jul 23 14:49:04 2015 - [info]
mysql-mha02(192.168.8.7:3306) (current master)
+--mysql-mha01(192.168.8.6:3306)
+--mysql-mha03(192.168.8.8:3306)

Thu Jul 23 14:49:04 2015 - [warning] master_ip_failover_script is not defined.
Thu Jul 23 14:49:04 2015 - [warning] shutdown_script is not defined.
Thu Jul 23 14:49:04 2015 - [info] Set master ping interval 3 seconds.
Thu Jul 23 14:49:04 2015 - [info] Set secondary check script: masterha_secondary_check -s mysql-mha03 --user=root --master_host=mysql-mha02 --master_ip=172.16.21.13 --master_port=3306
Thu Jul 23 14:49:04 2015 - [info] Starting ping health check on mysql-mha02(192.168.8.7:3306)..
Thu Jul 23 14:49:04 2015 - [info] Ping(SELECT) succeeded, waiting until MySQL doesn't respond..


5. 验证MHA


5.1 故障切换


我们将master节点关闭



[root@mysql-mha02 ~]# service mysql stop
Shutting down MySQL...[ OK ]


查看manager的输出日志


Thu Jul 23 14:59:13 2015 - [warning] Got error on MySQL select ping: 2006 (MySQL server has gone away)
Thu Jul 23 14:59:13 2015 - [info] Executing SSH check script: save_binary_logs --command=test --start_pos=4 --binlog_dir=/var/lib/mysql/logs --output_file=/var/log/masterha/node/save_binary_logs_test --manager_version=0.56 --binlog_prefix=mysql-bin
Thu Jul 23 14:59:13 2015 - [info] Executing secondary network check script: masterha_secondary_check -s mysql-mha03 --user=root --master_host=mysql-mha02 --master_ip=172.16.21.13 --master_port=3306 --user=root --master_host=mysql-mha02 --master_ip=192.168.8.7 --master_port=3306 --master_user=root --master_password=123456 --ping_type=SELECT
Thu Jul 23 14:59:14 2015 - [info] HealthCheck: SSH to mysql-mha02 is reachable.
Monitoring server mysql-mha03 is reachable, Master is not reachable from mysql-mha03. OK.
Thu Jul 23 14:59:14 2015 - [info] Master is not reachable from all other monitoring servers. Failover should start.
Thu Jul 23 14:59:16 2015 - [warning] Got error on MySQL connect: 2013 (Lost connection to MySQL server at 'reading initial communication packet', system error: 111)
Thu Jul 23 14:59:16 2015 - [warning] Connection failed 2 time(s)..
Thu Jul 23 14:59:19 2015 - [warning] Got error on MySQL connect: 2013 (Lost connection to MySQL server at 'reading initial communication packet', system error: 111)
Thu Jul 23 14:59:19 2015 - [warning] Connection failed 3 time(s)..
Thu Jul 23 14:59:22 2015 - [warning] Got error on MySQL connect: 2013 (Lost connection to MySQL server at 'reading initial communication packet', system error: 111)
Thu Jul 23 14:59:22 2015 - [warning] Connection failed 4 time(s)..
Thu Jul 23 14:59:22 2015 - [warning] Master is not reachable from health checker!
Thu Jul 23 14:59:22 2015 - [warning] Master mysql-mha02(192.168.8.7:3306) is not reachable!
Thu Jul 23 14:59:22 2015 - [warning] SSH is reachable.
Thu Jul 23 14:59:22 2015 - [info] Connecting to a master server failed. Reading configuration file /etc/masterha_default.cnf and /etc/MHA/MHAmanager.conf again, and trying to connect to all servers to check server status..
Thu Jul 23 14:59:22 2015 - [warning] Global configuration file /etc/masterha_default.cnf not found. Skipping.
Thu Jul 23 14:59:22 2015 - [info] Reading application default configuration from /etc/MHA/MHAmanager.conf..
Thu Jul 23 14:59:22 2015 - [info] Reading server configuration from /etc/MHA/MHAmanager.conf..
Thu Jul 23 14:59:22 2015 - [info] GTID failover mode = 0
Thu Jul 23 14:59:22 2015 - [info] Dead Servers:
Thu Jul 23 14:59:22 2015 - [info] mysql-mha02(192.168.8.7:3306)
Thu Jul 23 14:59:22 2015 - [info] Alive Servers:
Thu Jul 23 14:59:22 2015 - [info] mysql-mha01(192.168.8.6:3306)
Thu Jul 23 14:59:22 2015 - [info] mysql-mha03(192.168.8.8:3306)
Thu Jul 23 14:59:22 2015 - [info] Alive Slaves:
Thu Jul 23 14:59:22 2015 - [info] mysql-mha01(192.168.8.6:3306) Version=5.5.43-log (oldest major version between slaves) log-bin:enabled
Thu Jul 23 14:59:22 2015 - [info] Replicating from 192.168.8.7(192.168.8.7:3306)
Thu Jul 23 14:59:22 2015 - [info] mysql-mha03(192.168.8.8:3306) Version=5.5.43-log (oldest major version between slaves) log-bin:enabled
Thu Jul 23 14:59:22 2015 - [info] Replicating from 192.168.8.7(192.168.8.7:3306)
Thu Jul 23 14:59:22 2015 - [info] Checking slave configurations..
Thu Jul 23 14:59:22 2015 - [info] read_only=1 is not set on slave mysql-mha01(192.168.8.6:3306).
Thu Jul 23 14:59:22 2015 - [info] read_only=1 is not set on slave mysql-mha03(192.168.8.8:3306).
Thu Jul 23 14:59:22 2015 - [info] Checking replication filtering settings..
Thu Jul 23 14:59:22 2015 - [info] Replication filtering check ok.
Thu Jul 23 14:59:22 2015 - [info] Master is down!
Thu Jul 23 14:59:22 2015 - [info] Terminating monitoring script.
Thu Jul 23 14:59:22 2015 - [info] Got exit code 20 (Master dead).
Thu Jul 23 14:59:22 2015 - [info] MHA::MasterFailover version 0.56.
Thu Jul 23 14:59:22 2015 - [info] Starting master failover.
Thu Jul 23 14:59:22 2015 - [info]
Thu Jul 23 14:59:22 2015 - [info] * Phase 1: Configuration Check Phase..
Thu Jul 23 14:59:22 2015 - [info]
Thu Jul 23 14:59:23 2015 - [info] GTID failover mode = 0
Thu Jul 23 14:59:23 2015 - [info] Dead Servers:
Thu Jul 23 14:59:23 2015 - [info] mysql-mha02(192.168.8.7:3306)
Thu Jul 23 14:59:23 2015 - [info] Checking master reachability via MySQL(double check)...
Thu Jul 23 14:59:23 2015 - [info] ok.
Thu Jul 23 14:59:23 2015 - [info] Alive Servers:
Thu Jul 23 14:59:23 2015 - [info] mysql-mha01(192.168.8.6:3306)
Thu Jul 23 14:59:23 2015 - [info] mysql-mha03(192.168.8.8:3306)
Thu Jul 23 14:59:23 2015 - [info] Alive Slaves:
Thu Jul 23 14:59:23 2015 - [info] mysql-mha01(192.168.8.6:3306) Version=5.5.43-log (oldest major version between slaves) log-bin:enabled
Thu Jul 23 14:59:23 2015 - [info] Replicating from 192.168.8.7(192.168.8.7:3306)
Thu Jul 23 14:59:23 2015 - [info] mysql-mha03(192.168.8.8:3306) Version=5.5.43-log (oldest major version between slaves) log-bin:enabled
Thu Jul 23 14:59:23 2015 - [info] Replicating from 192.168.8.7(192.168.8.7:3306)
Thu Jul 23 14:59:23 2015 - [info] Starting Non-GTID based failover.
Thu Jul 23 14:59:23 2015 - [info]
Thu Jul 23 14:59:23 2015 - [info] ** Phase 1: Configuration Check Phase completed.
Thu Jul 23 14:59:23 2015 - [info]
Thu Jul 23 14:59:23 2015 - [info] * Phase 2: Dead Master Shutdown Phase..
Thu Jul 23 14:59:23 2015 - [info]
Thu Jul 23 14:59:23 2015 - [info] Forcing shutdown so that applications never connect to the current master..
Thu Jul 23 14:59:23 2015 - [warning] master_ip_failover_script is not set. Skipping invalidating dead master IP address.
Thu Jul 23 14:59:23 2015 - [warning] shutdown_script is not set. Skipping explicit shutting down of the dead master.
Thu Jul 23 14:59:23 2015 - [info] * Phase 2: Dead Master Shutdown Phase completed.
Thu Jul 23 14:59:23 2015 - [info]
Thu Jul 23 14:59:23 2015 - [info] * Phase 3: Master Recovery Phase..
Thu Jul 23 14:59:23 2015 - [info]
Thu Jul 23 14:59:23 2015 - [info] * Phase 3.1: Getting Latest Slaves Phase..
Thu Jul 23 14:59:23 2015 - [info]
Thu Jul 23 14:59:23 2015 - [info] The latest binary log file/position on all slaves is mysql-bin.000014:107
Thu Jul 23 14:59:23 2015 - [info] Latest slaves (Slaves that received relay log files to the latest):
Thu Jul 23 14:59:23 2015 - [info] mysql-mha01(192.168.8.6:3306) Version=5.5.43-log (oldest major version between slaves) log-bin:enabled
Thu Jul 23 14:59:23 2015 - [info] Replicating from 192.168.8.7(192.168.8.7:3306)
Thu Jul 23 14:59:23 2015 - [info] mysql-mha03(192.168.8.8:3306) Version=5.5.43-log (oldest major version between slaves) log-bin:enabled
Thu Jul 23 14:59:23 2015 - [info] Replicating from 192.168.8.7(192.168.8.7:3306)
Thu Jul 23 14:59:23 2015 - [info] The oldest binary log file/position on all slaves is mysql-bin.000014:107
Thu Jul 23 14:59:23 2015 - [info] Oldest slaves:
Thu Jul 23 14:59:23 2015 - [info] mysql-mha01(192.168.8.6:3306) Version=5.5.43-log (oldest major version between slaves) log-bin:enabled
Thu Jul 23 14:59:23 2015 - [info] Replicating from 192.168.8.7(192.168.8.7:3306)
Thu Jul 23 14:59:23 2015 - [info] mysql-mha03(192.168.8.8:3306) Version=5.5.43-log (oldest major version between slaves) log-bin:enabled
Thu Jul 23 14:59:23 2015 - [info] Replicating from 192.168.8.7(192.168.8.7:3306)
Thu Jul 23 14:59:23 2015 - [info]
Thu Jul 23 14:59:23 2015 - [info] * Phase 3.2: Saving Dead Master's Binlog Phase..
Thu Jul 23 14:59:23 2015 - [info]
Thu Jul 23 14:59:23 2015 - [info] Fetching dead master's binary logs..
Thu Jul 23 14:59:23 2015 - [info] Executing command on the dead master mysql-mha02(192.168.8.7:3306): save_binary_logs --command=save --start_file=mysql-bin.000014 --start_pos=107 --binlog_dir=/var/lib/mysql/logs --output_file=/var/log/masterha/node/saved_master_binlog_from_mysql-mha02_3306_20150723145922.binlog --handle_raw_binlog=1 --disable_log_bin=0 --manager_version=0.56
Creating /var/log/masterha/node if not exists.. ok.
Concat binary/relay logs from mysql-bin.000014 pos 107 to mysql-bin.000014 EOF into /var/log/masterha/node/saved_master_binlog_from_mysql-mha02_3306_20150723145922.binlog ..
Dumping binlog format description event, from position 0 to 107.. ok.
Dumping effective binlog data from /var/lib/mysql/logs/mysql-bin.000014 position 107 to tail(126).. ok.
Concat succeeded.
Thu Jul 23 14:59:23 2015 - [info] scp from root@192.168.8.7:/var/log/masterha/node/saved_master_binlog_from_mysql-mha02_3306_20150723145922.binlog to local:/var/log/masterha/manager/saved_master_binlog_from_mysql-mha02_3306_20150723145922.binlog succeeded.
Thu Jul 23 14:59:23 2015 - [info] HealthCheck: SSH to mysql-mha01 is reachable.
Thu Jul 23 14:59:24 2015 - [info] HealthCheck: SSH to mysql-mha03 is reachable.
Thu Jul 23 14:59:24 2015 - [info]
Thu Jul 23 14:59:24 2015 - [info] * Phase 3.3: Determining New Master Phase..
Thu Jul 23 14:59:24 2015 - [info]
Thu Jul 23 14:59:24 2015 - [info] Finding the latest slave that has all relay logs for recovering other slaves..
Thu Jul 23 14:59:24 2015 - [info] All slaves received relay logs to the same position. No need to resync each other.
Thu Jul 23 14:59:24 2015 - [info] Searching new master from slaves..
Thu Jul 23 14:59:24 2015 - [info] Candidate masters from the configuration file:
Thu Jul 23 14:59:24 2015 - [info] Non-candidate masters:
Thu Jul 23 14:59:24 2015 - [info] New master is mysql-mha01(192.168.8.6:3306)
Thu Jul 23 14:59:24 2015 - [info] Starting master failover..
Thu Jul 23 14:59:24 2015 - [info]
From:
mysql-mha02(192.168.8.7:3306) (current master)
+--mysql-mha01(192.168.8.6:3306)
+--mysql-mha03(192.168.8.8:3306)

To:
mysql-mha01(192.168.8.6:3306) (new master)
+--mysql-mha03(192.168.8.8:3306)
Thu Jul 23 14:59:24 2015 - [info]
Thu Jul 23 14:59:24 2015 - [info] * Phase 3.3: New Master Diff Log Generation Phase..
Thu Jul 23 14:59:24 2015 - [info]
Thu Jul 23 14:59:24 2015 - [info] This server has all relay logs. No need to generate diff files from the latest slave.
Thu Jul 23 14:59:24 2015 - [info] Sending binlog..
Thu Jul 23 14:59:24 2015 - [info] scp from local:/var/log/masterha/manager/saved_master_binlog_from_mysql-mha02_3306_20150723145922.binlog to root@mysql-mha01:/var/log/masterha/node/saved_master_binlog_from_mysql-mha02_3306_20150723145922.binlog succeeded.
Thu Jul 23 14:59:24 2015 - [info]
Thu Jul 23 14:59:24 2015 - [info] * Phase 3.4: Master Log Apply Phase..
Thu Jul 23 14:59:24 2015 - [info]
Thu Jul 23 14:59:24 2015 - [info] *NOTICE: If any error happens from this phase, manual recovery is needed.
Thu Jul 23 14:59:24 2015 - [info] Starting recovery on mysql-mha01(192.168.8.6:3306)..
Thu Jul 23 14:59:24 2015 - [info] Generating diffs succeeded.
Thu Jul 23 14:59:24 2015 - [info] Waiting until all relay logs are applied.
Thu Jul 23 14:59:24 2015 - [info] done.
Thu Jul 23 14:59:24 2015 - [info] Getting slave status..
Thu Jul 23 14:59:24 2015 - [info] This slave(mysql-mha01)'s Exec_Master_Log_Pos equals to Read_Master_Log_Pos(mysql-bin.000014:107). No need to recover from Exec_Master_Log_Pos.
Thu Jul 23 14:59:24 2015 - [info] Connecting to the target slave host mysql-mha01, running recover script..
Thu Jul 23 14:59:24 2015 - [info] Executing command: apply_diff_relay_logs --command=apply --slave_user='root' --slave_host=mysql-mha01 --slave_ip=192.168.8.6 --slave_port=3306 --apply_files=/var/log/masterha/node/saved_master_binlog_from_mysql-mha02_3306_20150723145922.binlog --workdir=/var/log/masterha/node --target_version=5.5.43-log --timestamp=20150723145922 --handle_raw_binlog=1 --disable_log_bin=0 --manager_version=0.56 --slave_pass=xxx
Thu Jul 23 14:59:24 2015 - [info]
Applying differential binary/relay log files /var/log/masterha/node/saved_master_binlog_from_mysql-mha02_3306_20150723145922.binlog on mysql-mha01:3306. This may take long time...
Applying log files succeeded.
Thu Jul 23 14:59:24 2015 - [info] All relay logs were successfully applied.
Thu Jul 23 14:59:24 2015 - [info] Getting new master's binlog name and position..
Thu Jul 23 14:59:24 2015 - [info] mysql-bin.000014:107
Thu Jul 23 14:59:24 2015 - [info] All other slaves should start replication from here. Statement should be: CHANGE MASTER TO MASTER_HOST='mysql-mha01 or 192.168.8.6', MASTER_PORT=3306, MASTER_LOG_FILE='mysql-bin.000014', MASTER_LOG_POS=107, MASTER_USER='repl', MASTER_PASSWORD='xxx';
Thu Jul 23 14:59:24 2015 - [warning] master_ip_failover_script is not set. Skipping taking over new master IP address.
Thu Jul 23 14:59:24 2015 - [info] ** Finished master recovery successfully.
Thu Jul 23 14:59:24 2015 - [info] * Phase 3: Master Recovery Phase completed.
Thu Jul 23 14:59:24 2015 - [info]
Thu Jul 23 14:59:24 2015 - [info] * Phase 4: Slaves Recovery Phase..
Thu Jul 23 14:59:24 2015 - [info]
Thu Jul 23 14:59:24 2015 - [info] * Phase 4.1: Starting Parallel Slave Diff Log Generation Phase..
Thu Jul 23 14:59:24 2015 - [info]
Thu Jul 23 14:59:24 2015 - [info] -- Slave diff file generation on host mysql-mha03(192.168.8.8:3306) started, pid: 6359. Check tmp log /var/log/masterha/manager/mysql-mha03_3306_20150723145922.log if it takes time..
Thu Jul 23 14:59:24 2015 - [info]
Thu Jul 23 14:59:24 2015 - [info] Log messages from mysql-mha03 ...
Thu Jul 23 14:59:24 2015 - [info]
Thu Jul 23 14:59:24 2015 - [info] This server has all relay logs. No need to generate diff files from the latest slave.
Thu Jul 23 14:59:24 2015 - [info] End of log messages from mysql-mha03.
Thu Jul 23 14:59:24 2015 - [info] -- mysql-mha03(192.168.8.8:3306) has the latest relay log events.
Thu Jul 23 14:59:24 2015 - [info] Generating relay diff files from the latest slave succeeded.
Thu Jul 23 14:59:24 2015 - [info]
Thu Jul 23 14:59:24 2015 - [info] * Phase 4.2: Starting Parallel Slave Log Apply Phase..
Thu Jul 23 14:59:24 2015 - [info]
Thu Jul 23 14:59:24 2015 - [info] -- Slave recovery on host mysql-mha03(192.168.8.8:3306) started, pid: 6361. Check tmp log /var/log/masterha/manager/mysql-mha03_3306_20150723145922.log if it takes time..
Thu Jul 23 14:59:25 2015 - [info]
Thu Jul 23 14:59:25 2015 - [info] Log messages from mysql-mha03 ...
Thu Jul 23 14:59:25 2015 - [info]
Thu Jul 23 14:59:24 2015 - [info] Sending binlog..
Thu Jul 23 14:59:24 2015 - [info] scp from local:/var/log/masterha/manager/saved_master_binlog_from_mysql-mha02_3306_20150723145922.binlog to root@mysql-mha03:/var/log/masterha/node/saved_master_binlog_from_mysql-mha02_3306_20150723145922.binlog succeeded.
Thu Jul 23 14:59:24 2015 - [info] Starting recovery on mysql-mha03(192.168.8.8:3306)..
Thu Jul 23 14:59:24 2015 - [info] Generating diffs succeeded.
Thu Jul 23 14:59:24 2015 - [info] Waiting until all relay logs are applied.
Thu Jul 23 14:59:24 2015 - [info] done.
Thu Jul 23 14:59:24 2015 - [info] Getting slave status..
Thu Jul 23 14:59:24 2015 - [info] This slave(mysql-mha03)'s Exec_Master_Log_Pos equals to Read_Master_Log_Pos(mysql-bin.000014:107). No need to recover from Exec_Master_Log_Pos.
Thu Jul 23 14:59:24 2015 - [info] Connecting to the target slave host mysql-mha03, running recover script..
Thu Jul 23 14:59:24 2015 - [info] Executing command: apply_diff_relay_logs --command=apply --slave_user='root' --slave_host=mysql-mha03 --slave_ip=192.168.8.8 --slave_port=3306 --apply_files=/var/log/masterha/node/saved_master_binlog_from_mysql-mha02_3306_20150723145922.binlog --workdir=/var/log/masterha/node --target_version=5.5.43-log --timestamp=20150723145922 --handle_raw_binlog=1 --disable_log_bin=0 --manager_version=0.56 --slave_pass=xxx
Thu Jul 23 14:59:24 2015 - [info]
Applying differential binary/relay log files /var/log/masterha/node/saved_master_binlog_from_mysql-mha02_3306_20150723145922.binlog on mysql-mha03:3306. This may take long time...
Applying log files succeeded.
Thu Jul 23 14:59:24 2015 - [info] All relay logs were successfully applied.
Thu Jul 23 14:59:24 2015 - [info] Resetting slave mysql-mha03(192.168.8.8:3306) and starting replication from the new master mysql-mha01(192.168.8.6:3306)..
Thu Jul 23 14:59:24 2015 - [info] Executed CHANGE MASTER.
Thu Jul 23 14:59:25 2015 - [info] Slave started.
Thu Jul 23 14:59:25 2015 - [info] End of log messages from mysql-mha03.
Thu Jul 23 14:59:25 2015 - [info] -- Slave recovery on host mysql-mha03(192.168.8.8:3306) succeeded.
Thu Jul 23 14:59:25 2015 - [info] All new slave servers recovered successfully.
Thu Jul 23 14:59:25 2015 - [info]
Thu Jul 23 14:59:25 2015 - [info] * Phase 5: New master cleanup phase..
Thu Jul 23 14:59:25 2015 - [info]
Thu Jul 23 14:59:25 2015 - [info] Resetting slave info on the new master..
Thu Jul 23 14:59:25 2015 - [info] mysql-mha01: Resetting slave info succeeded.
Thu Jul 23 14:59:25 2015 - [info] Master failover to mysql-mha01(192.168.8.6:3306) completed successfully.
Thu Jul 23 14:59:25 2015 - [info]

----- Failover Report -----

MHAmanager: MySQL Master failover mysql-mha02(192.168.8.7:3306) to mysql-mha01(192.168.8.6:3306) succeeded

Master mysql-mha02(192.168.8.7:3306) is down!

Check MHA Manager logs at mysql-mha03:/var/log/masterha/manager.log for details.

Started automated(non-interactive) failover.
The latest slave mysql-mha01(192.168.8.6:3306) has all relay logs for recovery.
Selected mysql-mha01(192.168.8.6:3306) as a new master.
mysql-mha01(192.168.8.6:3306): OK: Applying all logs succeeded.
mysql-mha03(192.168.8.8:3306): This host has the latest relay log events.
Generating relay diff files from the latest slave succeeded.
mysql-mha03(192.168.8.8:3306): OK: Applying all logs succeeded. Slave started, replicating from mysql-mha01(192.168.8.6:3306)
mysql-mha01(192.168.8.6:3306): Resetting slave info succeeded.
Master failover to mysql-mha01(192.168.8.6:3306) completed successfully.


发现MHA已经成功将主节点配置到mysql-mha01上。切换后manager节点自己主动关闭。


[root@mysql-mha03 ~]# ps -ef|grep -i master
root 6414 3295 0 15:00 pts/1 00:00:00 grep -i master

登录到mysql-mha03节点查看


mysql> show slave status\G
*************************** 1. row ***************************
Slave_IO_State: Waiting for master to send event
Master_Host: 192.168.8.6
Master_User: repl
Master_Port: 3306
Connect_Retry: 20
Master_Log_File: mysql-bin.000014
Read_Master_Log_Pos: 107
Relay_Log_File: mysql-mha03-relay-bin.000002
Relay_Log_Pos: 253
Relay_Master_Log_File: mysql-bin.000014
Slave_IO_Running: Yes
Slave_SQL_Running: Yes
Replicate_Do_DB:
Replicate_Ignore_DB:
Replicate_Do_Table:
Replicate_Ignore_Table:
Replicate_Wild_Do_Table:
Replicate_Wild_Ignore_Table:
Last_Errno: 0
Last_Error:
Skip_Counter: 0
Exec_Master_Log_Pos: 107
Relay_Log_Space: 415
Until_Condition: None
Until_Log_File:
Until_Log_Pos: 0
Master_SSL_Allowed: No
Master_SSL_CA_File:
Master_SSL_CA_Path:
Master_SSL_Cert:
Master_SSL_Cipher:
Master_SSL_Key:
Seconds_Behind_Master: 0
Master_SSL_Verify_Server_Cert: No
Last_IO_Errno: 0
Last_IO_Error:
Last_SQL_Errno: 0
Last_SQL_Error:
Replicate_Ignore_Server_Ids:
Master_Server_Id: 1
1 row in set (0.00 sec)

host已经改变成功。

MHA配置成功。



5.2 主动切换


mha能够实现master的主动切换,能够让我们方便的维护集群环境。如我们能够将master切换到别的节点,维护原master节点。


须要将启动的manager节点关闭掉。否则会报错。



[root@mysql-mha03 ~]# masterha_stop --conf=/etc/MHA/MHAmanager.conf 
Stopped MHAmanager successfully.
[1]+ Exit 1 nohup masterha_manager --conf=/etc/MHA/MHAmanager.conf



我们主动切换master节点。原master节点成为slave节点



[root@mysql-mha03 ~]# masterha_master_switch --master_state=alive --conf=/etc/MHA/MHAmanager.conf

查看输出


Thu Jul 23 15:29:38 2015 - [info] MHA::MasterRotate version 0.56.
Thu Jul 23 15:29:38 2015 - [info] Starting online master switch..
Thu Jul 23 15:29:38 2015 - [info]
Thu Jul 23 15:29:38 2015 - [info] * Phase 1: Configuration Check Phase..
Thu Jul 23 15:29:38 2015 - [info]
Thu Jul 23 15:29:38 2015 - [warning] Global configuration file /etc/masterha_default.cnf not found. Skipping.
Thu Jul 23 15:29:38 2015 - [info] Reading application default configuration from /etc/MHA/MHAmanager.conf..
Thu Jul 23 15:29:38 2015 - [info] Reading server configuration from /etc/MHA/MHAmanager.conf..
Thu Jul 23 15:29:39 2015 - [info] GTID failover mode = 0
Thu Jul 23 15:29:39 2015 - [info] Current Alive Master: mysql-mha01(192.168.8.6:3306)
Thu Jul 23 15:29:39 2015 - [info] Alive Slaves:
Thu Jul 23 15:29:39 2015 - [info] mysql-mha02(192.168.8.7:3306) Version=5.5.43-log (oldest major version between slaves) log-bin:enabled
Thu Jul 23 15:29:39 2015 - [info] Replicating from 192.168.8.6(192.168.8.6:3306)
Thu Jul 23 15:29:39 2015 - [info] mysql-mha03(192.168.8.8:3306) Version=5.5.43-log (oldest major version between slaves) log-bin:enabled
Thu Jul 23 15:29:39 2015 - [info] Replicating from 192.168.8.6(192.168.8.6:3306)

It is better to execute FLUSH NO_WRITE_TO_BINLOG TABLES on the master before switching. Is it ok to execute on mysql-mha01(192.168.8.6:3306)? (YES/no): yes
Thu Jul 23 15:29:41 2015 - [info] Executing FLUSH NO_WRITE_TO_BINLOG TABLES. This may take long time..
Thu Jul 23 15:29:41 2015 - [info] ok.
Thu Jul 23 15:29:41 2015 - [info] Checking MHA is not monitoring or doing failover..
Thu Jul 23 15:29:41 2015 - [info] Checking replication health on mysql-mha02..
Thu Jul 23 15:29:41 2015 - [info] ok.
Thu Jul 23 15:29:41 2015 - [info] Checking replication health on mysql-mha03..
Thu Jul 23 15:29:41 2015 - [info] ok.
Thu Jul 23 15:29:41 2015 - [info] Searching new master from slaves..
Thu Jul 23 15:29:41 2015 - [info] Candidate masters from the configuration file:
Thu Jul 23 15:29:41 2015 - [info] Non-candidate masters:
Thu Jul 23 15:29:41 2015 - [info]
From:
mysql-mha01(192.168.8.6:3306) (current master)
+--mysql-mha02(192.168.8.7:3306)
+--mysql-mha03(192.168.8.8:3306)

To:
mysql-mha02(192.168.8.7:3306) (new master)
+--mysql-mha03(192.168.8.8:3306)

Starting master switch from mysql-mha01(192.168.8.6:3306) to mysql-mha02(192.168.8.7:3306)? (yes/NO): yes
Thu Jul 23 15:29:48 2015 - [info] Checking whether mysql-mha02(192.168.8.7:3306) is ok for the new master..
Thu Jul 23 15:29:48 2015 - [info] ok.
Thu Jul 23 15:29:48 2015 - [info] ** Phase 1: Configuration Check Phase completed.
Thu Jul 23 15:29:48 2015 - [info]
Thu Jul 23 15:29:48 2015 - [info] * Phase 2: Rejecting updates Phase..
Thu Jul 23 15:29:48 2015 - [info]
master_ip_online_change_script is not defined. If you do not disable writes on the current master manually, applications keep writing on the current master. Is it ok to proceed? (yes/NO): yes
Thu Jul 23 15:29:59 2015 - [info] Locking all tables on the orig master to reject updates from everybody (including root):
Thu Jul 23 15:29:59 2015 - [info] Executing FLUSH TABLES WITH READ LOCK..
Thu Jul 23 15:29:59 2015 - [info] ok.
Thu Jul 23 15:29:59 2015 - [info] Orig master binlog:pos is mysql-bin.000014:107.
Thu Jul 23 15:29:59 2015 - [info] Waiting to execute all relay logs on mysql-mha02(192.168.8.7:3306)..
Thu Jul 23 15:29:59 2015 - [info] master_pos_wait(mysql-bin.000014:107) completed on mysql-mha02(192.168.8.7:3306). Executed 0 events.
Thu Jul 23 15:29:59 2015 - [info] done.
Thu Jul 23 15:29:59 2015 - [info] Getting new master's binlog name and position..
Thu Jul 23 15:29:59 2015 - [info] mysql-bin.000015:107
Thu Jul 23 15:29:59 2015 - [info] All other slaves should start replication from here. Statement should be: CHANGE MASTER TO MASTER_HOST='mysql-mha02 or 192.168.8.7', MASTER_PORT=3306, MASTER_LOG_FILE='mysql-bin.000015', MASTER_LOG_POS=107, MASTER_USER='repl', MASTER_PASSWORD='xxx';
Thu Jul 23 15:29:59 2015 - [info]
Thu Jul 23 15:29:59 2015 - [info] * Switching slaves in parallel..
Thu Jul 23 15:29:59 2015 - [info]
Thu Jul 23 15:29:59 2015 - [info] -- Slave switch on host mysql-mha03(192.168.8.8:3306) started, pid: 6819
Thu Jul 23 15:29:59 2015 - [info]
Thu Jul 23 15:29:59 2015 - [info] Log messages from mysql-mha03 ...
Thu Jul 23 15:29:59 2015 - [info]
Thu Jul 23 15:29:59 2015 - [info] Waiting to execute all relay logs on mysql-mha03(192.168.8.8:3306)..
Thu Jul 23 15:29:59 2015 - [info] master_pos_wait(mysql-bin.000014:107) completed on mysql-mha03(192.168.8.8:3306). Executed 0 events.
Thu Jul 23 15:29:59 2015 - [info] done.
Thu Jul 23 15:29:59 2015 - [info] Resetting slave mysql-mha03(192.168.8.8:3306) and starting replication from the new master mysql-mha02(192.168.8.7:3306)..
Thu Jul 23 15:29:59 2015 - [info] Executed CHANGE MASTER.
Thu Jul 23 15:29:59 2015 - [info] Slave started.
Thu Jul 23 15:29:59 2015 - [info] End of log messages from mysql-mha03 ...
Thu Jul 23 15:29:59 2015 - [info]
Thu Jul 23 15:29:59 2015 - [info] -- Slave switch on host mysql-mha03(192.168.8.8:3306) succeeded.
Thu Jul 23 15:29:59 2015 - [info] Unlocking all tables on the orig master:
Thu Jul 23 15:29:59 2015 - [info] Executing UNLOCK TABLES..
Thu Jul 23 15:29:59 2015 - [info] ok.
Thu Jul 23 15:29:59 2015 - [info] All new slave servers switched successfully.
Thu Jul 23 15:29:59 2015 - [info]
Thu Jul 23 15:29:59 2015 - [info] * Phase 5: New master cleanup phase..
Thu Jul 23 15:29:59 2015 - [info]
Thu Jul 23 15:29:59 2015 - [info] mysql-mha02: Resetting slave info succeeded.
Thu Jul 23 15:29:59 2015 - [info] Switching master to mysql-mha02(192.168.8.7:3306) completed successfully.

查看mysql-mysql03节点。已经成功切换到mysql-mha02为master

mysql> show slave status\G
*************************** 1. row ***************************
Slave_IO_State: Waiting for master to send event
Master_Host: 192.168.8.7
Master_User: repl
Master_Port: 3306
Connect_Retry: 20
Master_Log_File: mysql-bin.000015
Read_Master_Log_Pos: 107
Relay_Log_File: mysql-mha03-relay-bin.000002
Relay_Log_Pos: 253
Relay_Master_Log_File: mysql-bin.000015
Slave_IO_Running: Yes
Slave_SQL_Running: Yes
Replicate_Do_DB:
Replicate_Ignore_DB:
Replicate_Do_Table:
Replicate_Ignore_Table: