MMM配置官方文档: http://mysql-mmm.org/mmm2:guide

本文环境:    
主库:CentOS6.7 x64 192.168.0.201 mysql-5.6.29    
主库:CentOS6.7 x64 192.168.0.202 mysql-5.6.29    
备库:CentOS6.7 x64 192.168.0.203 mysql-5.6.29    
备库:CentOS6.7 x64 192.168.0.204 mysql-5.6.29    
管理:CentOS6.7 x64 192.168.0.205

MMM服务器角色对应关系:    
node1(主1)     192.168.0.201   db1    1    
node2(主2)     192.168.0.202   db2    2    
node3(从1)     192.168.0.203   db3    11    
node4(从1)     192.168.0.204   db4    12    
node5(监控)    192.168.0.205   mon    -

配置完成后,使用下面的VIP访问MySQL Cluster。    
192.168.0.211  writer 
192.168.0.212  reader 
192.168.0.213  reader    
192.168.0.214  reader    
192.168.0.215  reader

 

一、双主环境,以及从服务器环境准备

1,node1(主1) /etc/my.cnf

[mysqld]    
server-id              = 1    
binlog-format          = ROW    
log-bin                = master-bin    
log-bin-index          = master-bin.index    
log-slave-updates      = true    
auto_increment_offset  = 1    
auto_increment_increment = 2


mysql> show master status;    
+-------------------+----------+--------------+------------------+-------------------+    
| File              | Position | Binlog_Do_DB | Binlog_Ignore_DB | Executed_Gtid_Set |    
+-------------------+----------+--------------+------------------+-------------------+    
| master-bin.000001 |      120 |              |                  |                   |    
+-------------------+----------+--------------+------------------+-------------------+    
1 row in set (0.00 sec)

mysql>

 

2. node1(主2) /etc/my.cnf

[mysqld]    
server-id              = 2    
binlog-format          = ROW    
log-bin                = master-bin    
log-bin-index          = master-bin.index    
log-slave-updates      = true    
auto_increment_offset  = 2    
auto_increment_increment = 2


mysql> show master status;    
+-------------------+----------+--------------+------------------+-------------------+    
| File              | Position | Binlog_Do_DB | Binlog_Ignore_DB | Executed_Gtid_Set |    
+-------------------+----------+--------------+------------------+-------------------+    
| master-bin.000001 |      120 |              |                  |                   |    
+-------------------+----------+--------------+------------------+-------------------+    
1 row in set (0.00 sec)

mysql>

 

3. node1,node2 双主服务器配置同步复制帐号

grant replication slave on *.* to 'repl'@'%' identified by '123456';    
flush privileges;

 

4, node3(从)/etc/my.cnf

[mysqld]    
binlog-format                = ROW    
log-bin                      = mysql-bin    
relay-log                    = slave-relay-bin    
relay-log-index              = slave-relay-bin.index    
log-slave-updates            = true    
server-id                    = 11    
skip-name-resolve

 

5, node4(从)/etc/my.cnf

[mysqld]    
binlog-format                = ROW    
log-bin                      = mysql-bin    
relay-log                    = slave-relay-bin    
relay-log-index              = slave-relay-bin.index    
log-slave-updates            = true    
server-id                    = 12    
skip-name-resolve

 

6. node1从库配置同步过程

(1) 配置同步,手动执行同步参数,该配置会写入master.info文件中。

mysql >

CHANGE MASTER TO    
MASTER_HOST='192.168.0.202',    
MASTER_PORT=3306,    
MASTER_USER='repl',    
MASTER_PASSWORD='123456',    
MASTER_LOG_FILE='master-bin.000001',    
MASTER_LOG_POS=120;

mysql> start slave;

mysql> show slave status\G;    
*************************** 1. row ***************************    
               Slave_IO_State: Waiting for master to send event    
                  Master_Host: 192.168.0.202    
                  Master_User: repl    
                  Master_Port: 3306    
                Connect_Retry: 60    
              Master_Log_File: master-bin.000001    
          Read_Master_Log_Pos: 407    
               Relay_Log_File: node1-relay-bin.000002    
                Relay_Log_Pos: 571    
        Relay_Master_Log_File: master-bin.000001    
             Slave_IO_Running: Yes    
            Slave_SQL_Running: Yes    
              Replicate_Do_DB:

 

7. node1从库配置同步过程

(1) 配置同步,手动执行同步参数,该配置会写入master.info文件中。

mysql >

CHANGE MASTER TO    
MASTER_HOST='192.168.0.201',    
MASTER_PORT=3306,    
MASTER_USER='repl',    
MASTER_PASSWORD='123456',    
MASTER_LOG_FILE='master-bin.000001',    
MASTER_LOG_POS=120;

mysql> start slave;    
mysql> show slave status\G;    
*************************** 1. row ***************************    
               Slave_IO_State: Waiting for master to send event    
                  Master_Host: 192.168.0.201    
                  Master_User: repl    
                  Master_Port: 3306    
                Connect_Retry: 60    
              Master_Log_File: master-bin.000001    
          Read_Master_Log_Pos: 694    
               Relay_Log_File: node2-relay-bin.000002    
                Relay_Log_Pos: 571    
        Relay_Master_Log_File: master-bin.000001    
             Slave_IO_Running: Yes    
            Slave_SQL_Running: Yes    
              Replicate_Do_DB:

 

8. 测试双主相互复制

(1) node1上创建数据库

mysql> create database mydb;    
Query OK, 1 row affected (0.00 sec)

mysql> use mydb;    
Database changed    
mysql> show tables;    
+----------------+    
| Tables_in_mydb |    
+----------------+    
| user           |    
+----------------+    
1 row in set (0.01 sec)

mysql> select * from user;    
+----+----------+----------+    
| id | username | password |    
+----+----------+----------+    
| 1  | koumm    | 123456   |    
+----+----------+----------+    
1 row in set (0.01 sec)

mysql>


(2) node2上的操作如下:    
mysql> show databases;    
+--------------------+    
| Database           |    
+--------------------+    
| information_schema |    
| mydb               |    
| mysql              |    
| performance_schema |    
+--------------------+    
4 rows in set (0.00 sec)

mysql> use mydb;    
Database changed    
mysql> CREATE TABLE `user` (    
    ->   `id` varchar(20) NOT NULL,    
    ->   `username` varchar(20) NOT NULL,    
    ->   `password` char(32) NOT NULL,    
    ->   PRIMARY KEY (`id`)    
    -> ) ;    
Query OK, 0 rows affected (0.14 sec)

mysql> INSERT INTO user  VALUES ('1', 'koumm', '123456');    
Query OK, 1 row affected (0.00 sec)

mysql>

 

二、配置node3, node4同步node1(主)

1,node1主库锁表,导数据库

(1) 主库锁表

mysql> flush tables with read lock;    
Query OK, 0 rows affected (0.00 sec)


mysql> show master status;    
+-------------------+----------+--------------+------------------+-------------------+    
| File              | Position | Binlog_Do_DB | Binlog_Ignore_DB | Executed_Gtid_Set |    
+-------------------+----------+--------------+------------------+-------------------+    
| master-bin.000001 |     1226 |              |                  |                   |    
+-------------------+----------+--------------+------------------+-------------------+    
1 row in set (0.00 sec)


(2) 主库备份

[root@master ~]# mysqldump -uroot -p -B mydb > mydb.sql    
说明:-B参数有建库语句。


(3) 主库解开锁表功能

mysql> unlock tables;    
Query OK, 0 rows affected (0.00 sec)    
mysql>

 

2. node3,node4从库导入数据库

# mysql -uroot -padmin < mydb.sql

 

3. node3,node4 从库配置同步过程

(1) 配置同步,手动执行同步参数,该配置会写入master.info文件中。

mysql >

CHANGE MASTER TO    
MASTER_HOST='192.168.0.201',    
MASTER_PORT=3306,    
MASTER_USER='repl',    
MASTER_PASSWORD='123456',    
MASTER_LOG_FILE='master-bin.000001',    
MASTER_LOG_POS=1226;


mysql> start slave;    
Query OK, 0 rows affected (0.04 sec)

mysql> show slave status\G;    
*************************** 1. row ***************************    
               Slave_IO_State: Waiting for master to send event    
                  Master_Host: 192.168.1.121    
                  Master_User: repl    
                  Master_Port: 3306    
                Connect_Retry: 60    
              Master_Log_File: master-bin.000005    
          Read_Master_Log_Pos: 730    
               Relay_Log_File: slave-relay-bin.000002    
                Relay_Log_Pos: 284    
        Relay_Master_Log_File: master-bin.000005    
             Slave_IO_Running: Yes    
            Slave_SQL_Running: Yes    
              Replicate_Do_DB:    
          Replicate_Ignore_DB:


可以简单测试一下同步数据情况。

 

三、配置MMM过程

1, 安装mmm软件包

(1) node1,node2,node3,node4,node5节点    
# wget http://download.fedoraproject.org/pub/epel/6/x86_64/epel-release-6-8.noarch.rpm    
# rpm -ivh epel-release-6-8.noarch.rpm

(2) node5(监控节点)    
# yum install mysql-mmm mysql-mmm-agent mysql-mmm-tools mysql-mmm-monitor

(3) node1,node2,node3,node4 (mysql节点)    
# yum install mysql-mmm-agent

 

2,所有的MySQL节点添加以下两个用户,这里可以在主库做该操作,会自动同步到其它节点。

mysql> grant replication client on *.* to 'mmm_monitor'@'192.168.0.%' identified by '123456';    
mysql> grant super, replication client, process on *.* to 'mmm_agent'@'192.168.0.%' identified by '123456';    
mysql> flush privileges;

 

3,所有主机node1,node2,node3,node4,node5上创建如下文件

[root@node1 ~]# vi /etc/mysql-mmm/mmm_common.conf    
active_master_role      writer

<host default>    
    cluster_interface       eth0    
    pid_path                /var/run/mysql-mmm/mmm_agentd.pid    
    bin_path                /usr/libexec/mysql-mmm/    
    replication_user        repl    
    replication_password    123456    
    agent_user              mmm_agent    
    agent_password          123456    
</host>

<host db1>    
    ip      192.168.0.201    
    mode    master    
    peer    db2    
</host>

<host db2>    
    ip      192.168.0.202    
    mode    master    
    peer    db1    
</host>

<host db3>    
    ip      192.168.0.203    
    mode    slave    
</host>

<host db4>    
    ip      192.168.0.204    
    mode    slave    
</host>

<role writer>    
    hosts   db1, db2    
    ips     192.168.0.211    
    mode    exclusive    
</role>

<role reader>    
    hosts   db1, db2, db3, db4    
    ips     192.168.0.212,192.168.0.213,192.168.0.214,192.168.0.215    
    mode    balanced    
</role>

复制该配置文件到所有节点上:    
[root@node1 ~]# scp /etc/mysql-mmm/mmm_common.conf node2:/etc/mysql-mmm/    
[root@node1 ~]# scp /etc/mysql-mmm/mmm_common.conf node3:/etc/mysql-mmm/    
[root@node1 ~]# scp /etc/mysql-mmm/mmm_common.conf node4:/etc/mysql-mmm/    
[root@node1 ~]# scp /etc/mysql-mmm/mmm_common.conf node5:/etc/mysql-mmm/

 

4. 配置agent代理节点,所有mysql主从服务器均为代理节点

(1) node1,node2,node3,node4 的MySQL节点配置mmm_agent.conf

# vi /etc/mysql-mmm/mmm_agent.conf    
include mmm_common.conf    
this db1   #在四台mysql节点上设置对应的db,分别为db1、db2、db3、db4


(2) 默认为启用,可以不用修改    
# cat /etc/default/mysql-mmm-agent    
# mysql-mmm-agent defaults    
ENABLED=1

(3) mysql节点启动服务

# chkconfig mysql-mmm-agent on    
# /etc/init.d/mysql-mmm-agent start

 

5,配置MMM管理监控节点node5

(1) 配置监控    
[root@node5 ~]# vi /etc/mysql-mmm/mmm_mon.conf    
include mmm_common.conf

<monitor>    
    ip                  127.0.0.1    
    pid_path            /var/run/mysql-mmm/mmm_mond.pid    
    bin_path            /usr/libexec/mysql-mmm    
    status_path         /var/lib/mysql-mmm/mmm_mond.status    
    ping_ips            192.168.0.201,192.168.0.202,192.168.0.203,192.168.0.204    
    auto_set_online     30

    # The kill_host_bin does not exist by default, though the monitor will    
    # throw a warning about it missing.  See the section 5.10 "Kill Host    
    # Functionality" in the PDF documentation.    
    #    
    # kill_host_bin     /usr/libexec/mysql-mmm/monitor/kill_host    
    #    
</monitor>

<host default>    
    monitor_user        mmm_monitor    
    monitor_password    123456    
</host>

debug 0


(2) mysql节点启动服务

# chkconfig mysql-mmm-monitor on    
# /etc/init.d/mysql-mmm-monitor start

 

6, 查看状态

(1) 管理服务器上查看状态

[root@node1 ~]# mmm_control show    
  db1(192.168.0.201) master/ONLINE. Roles: reader(192.168.0.215), writer(192.168.0.211)    
  db2(192.168.0.202) master/ONLINE. Roles: reader(192.168.0.213)    
  db3(192.168.0.203) slave/ONLINE. Roles: reader(192.168.0.212)    
  db4(192.168.0.204) slave/ONLINE. Roles: reader(192.168.0.214)

[root@node1 ~]# mmm_control checks all    
db4  ping         [last change: 2016/04/27 16:45:49]  OK    
db4  mysql        [last change: 2016/04/27 16:45:49]  OK    
db4  rep_threads  [last change: 2016/04/27 16:45:49]  OK    
db4  rep_backlog  [last change: 2016/04/27 16:45:49]  OK: Backlog is null    
db2  ping         [last change: 2016/04/27 16:45:49]  OK    
db2  mysql        [last change: 2016/04/27 16:45:49]  OK    
db2  rep_threads  [last change: 2016/04/27 16:45:49]  OK    
db2  rep_backlog  [last change: 2016/04/27 16:45:49]  OK: Backlog is null    
db3  ping         [last change: 2016/04/27 16:45:49]  OK    
db3  mysql        [last change: 2016/04/27 16:45:49]  OK    
db3  rep_threads  [last change: 2016/04/27 16:45:49]  OK    
db3  rep_backlog  [last change: 2016/04/27 16:45:49]  OK: Backlog is null    
db1  ping         [last change: 2016/04/27 16:45:49]  OK    
db1  mysql        [last change: 2016/04/27 16:45:49]  OK    
db1  rep_threads  [last change: 2016/04/27 16:45:49]  OK    
db1  rep_backlog  [last change: 2016/04/27 16:45:49]  OK: Backlog is null    
[root@node1 ~]#


(2) 服务器的启动VIP地址

[root@node1 ~]# ip a    
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN    
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00    
    inet 127.0.0.1/8 scope host lo    
    inet6 ::1/128 scope host    
       valid_lft forever preferred_lft forever    
2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000    
    link/ether 00:0c:29:ff:33:6a brd ff:ff:ff:ff:ff:ff    
    inet 192.168.0.201/24 brd 192.168.0.255 scope global eth0    
    inet 192.168.0.215/32 scope global eth0    
    inet 192.168.0.211/32 scope global eth0    
    inet6 fe80::20c:29ff:feff:336a/64 scope link    
       valid_lft forever preferred_lft forever    
[root@node1 ~]#

[root@node2 ~]# ip a    
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN    
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00    
    inet 127.0.0.1/8 scope host lo    
    inet6 ::1/128 scope host    
       valid_lft forever preferred_lft forever    
2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000    
    link/ether 00:0c:29:8c:60:58 brd ff:ff:ff:ff:ff:ff    
    inet 192.168.0.202/24 brd 192.168.0.255 scope global eth0    
    inet 192.168.0.213/32 scope global eth0    
    inet6 fe80::20c:29ff:fe8c:6058/64 scope link    
       valid_lft forever preferred_lft forever    
[root@node2 ~]#

服务器读写采有VIP地址进行读写,出现故障时VIP会漂移到其它节点,由其它节点提供服务。


四、MMM高可用测试

首先查看整个集群的状态,可以看到整个集群状态正常。
[root@node5 ~]# mmm_control show     
  db1(192.168.0.201) master/ONLINE. Roles: reader(192.168.0.212), writer(192.168.0.211)
  db2(192.168.0.202) master/ONLINE. Roles: reader(192.168.0.213)
  db3(192.168.0.203) slave/ONLINE. Roles: reader(192.168.0.215)
  db4(192.168.0.204) slave/ONLINE. Roles: reader(192.168.0.214)
[root@192.168.0.30 ~]#

1. 模拟node2宕机,手动停止mysql服务,观察monitor日志node2由关闭到启动的日志如下:

[root@192.168.0.30 ~]# tail -f /var/log/mysql-mmm/mmm_mond.log
2016/04/29 09:58:33 FATAL State of host 'db2' changed from ONLINE to HARD_OFFLINE (ping: OK, mysql: not OK)
2016/04/29 10:00:54 FATAL State of host 'db2' changed from HARD_OFFLINE to AWAITING_RECOVERY
2016/04/29 10:01:24 FATAL State of host 'db2' changed from AWAITING_RECOVERY to ONLINE because of auto_set_online(30 seconds). It was in state AWAITING_RECOVERY for 30

seconds

重新查看集群的最新状态:

[root@node5 ~]# mmm_control show
  db1(192.168.0.201) master/ONLINE. Roles: reader(192.168.0.212), writer(192.168.0.211)
  db2(192.168.0.202) master/HARD_OFFLINE. Roles:   #从日志发现db2的状态有ONLINE转换为HARD_OFFLINE
  db3(192.168.0.203) slave/ONLINE. Roles: reader(192.168.0.213), reader(192.168.0.215)
  db4(192.168.0.204) slave/ONLINE. Roles: reader(192.168.0.214)

[root@node5 ~]# mmm_control checks all
db4  ping         [last change: 2016/04/29 09:01:08]  OK
db4  mysql        [last change: 2016/04/29 09:01:08]  OK
db4  rep_threads  [last change: 2016/04/29 09:01:08]  OK
db4  rep_backlog  [last change: 2016/04/29 09:01:08]  OK: Backlog is null
db2  ping         [last change: 2016/04/29 09:01:08]  OK
db2  mysql        [last change: 2016/04/29 09:58:33]  ERROR: Connect error (host = 192.168.0.202:3306, user = mmm_monitor)! Lost connection to MySQL server at 'reading

initial communication packet', system error: 111   能ping通,说明只是服务死掉了。
db2  rep_threads  [last change: 2016/04/29 09:01:08]  OK
db2  rep_backlog  [last change: 2016/04/29 09:01:08]  OK: Backlog is null
db3  ping         [last change: 2016/04/29 09:01:08]  OK
db3  mysql        [last change: 2016/04/29 09:01:08]  OK
db3  rep_threads  [last change: 2016/04/29 09:01:08]  OK
db3  rep_backlog  [last change: 2016/04/29 09:01:08]  OK: Backlog is null
db1  ping         [last change: 2016/04/29 09:01:08]  OK
db1  mysql        [last change: 2016/04/29 09:56:33]  OK
db1  rep_threads  [last change: 2016/04/29 09:01:08]  OK
db1  rep_backlog  [last change: 2016/04/29 09:01:08]  OK: Backlog is null

[root@node5 ~]#

启动node2,node2由HARD_OFFLINE转到AWAITING_RECOVERY状态。这里db2再次接管读请求。

[root@node5 ~]# mmm_control show     
  db1(192.168.0.201) master/ONLINE. Roles: reader(192.168.0.212), writer(192.168.0.211)
  db2(192.168.0.202) master/ONLINE. Roles: reader(192.168.0.213)
  db3(192.168.0.203) slave/ONLINE. Roles: reader(192.168.0.215)
  db4(192.168.0.204) slave/ONLINE. Roles: reader(192.168.0.214)
[root@node5 ~]#


2. 模拟node1主库宕机,查看集群状态。

[root@node5 ~]# mmm_control show
  db1(192.168.0.201) master/HARD_OFFLINE. Roles:
  db2(192.168.0.202) master/ONLINE. Roles: reader(192.168.0.213), writer(192.168.0.211)
  db3(192.168.0.203) slave/ONLINE. Roles: reader(192.168.0.212), reader(192.168.0.215)
  db4(192.168.0.204) slave/ONLINE. Roles: reader(192.168.0.214)

node1主库再启动:

[root@node5 ~]# mmm_control show
  db1(192.168.0.201) master/ONLINE. Roles: reader(192.168.0.212)
  db2(192.168.0.202) master/ONLINE. Roles: reader(192.168.0.213), writer(192.168.0.211)
  db3(192.168.0.203) slave/ONLINE. Roles: reader(192.168.0.215)
  db4(192.168.0.204) slave/ONLINE. Roles: reader(192.168.0.214)

可以看到主库启动用会自动转到读的角色,不会接管主,只到现有的主再次宕机。

[root@192.168.0.30 ~]# tail -f /var/log/mysql-mmm/mmm_mond.log
2016/04/29 10:03:25 FATAL State of host 'db1' changed from ONLINE to HARD_OFFLINE (ping: OK, mysql: not OK)
2016/04/29 10:06:20 FATAL State of host 'db1' changed from HARD_OFFLINE to AWAITING_RECOVERY
2016/04/29 10:06:51 FATAL State of host 'db1' changed from AWAITING_RECOVERY to ONLINE because of auto_set_online(30 seconds). It was in state AWAITING_RECOVERY for 31

seconds

3. 测试小结:
(1)node2备选主节点宕机不影响集群的状态,就是移除了node2备选节点的读状态。
(2)node1主节点宕机,由node2备选主节点接管写角色,node3,node4指向新node2主库进行复制,node3,node4会自动change master到node2.
(3)问题来了:
如果node1主库宕机,node2复制应用又落后于node1时就变成了主可写状态,这进的数据主无法保证一致性。
如果node2,node3,node4延迟于node1主,这个时node1宕机,node3,node4将会等待数据追上db1后,再重新指向新的主node2进行复制操作,这时的数据也无法保证同步的一致性。
(4)如查采用MMM高可用架构,主,主备选节点机器配置一样,而且开启半同步进一步提高安全性或采用MariaDB进行多线程从复制,提高复制的性能。

   
说明:MMM(Master-Master Replication Manager for MySQL)主主复制管理器, MMM集群套件具有良好的稳定性、高可用性和可扩展性。当活动的Master节点出现故障时,备用Master节点可以立即接管,而其他的Slave节点也能自动切换到备用Master节点继续进行同步复制,而无需人为干涉;MMM架构需要多个节点、多个IP,对服务器数量有要求,在读写非常繁忙的业务系统下表现不是很稳定,可能会出现复制延时、切换失效等问题。MMM方案并不太适应于对数据安全性要求很高,并且读、写繁忙的环境中。