一,基本概念:

NDB是一种“内存中”存储引擎,它具有可用性高和数据一致性好的特点。能够使用多种故障切换和负载平衡选项配置NDB存储引擎,但以簇层面上的存储引擎开始最简单。MySQL簇的NDB存储引擎包含完整的数据集,仅取决于簇本身内的其他数据。

下面,我们介绍了设置由NDB存储引擎和一些MySQL服务器构成的MySQL簇的设置方法。

目前,MySQL簇的簇部分可独立于MySQL服务器进行配置。在MySQL簇中,簇的每个部分被视为1个节点。

 

  

有三类簇节点,在最低的MySQL簇配置中,至少有三个节点,这三类节点分别是:

管理(MGM)节点:这类节点的作用是管理MySQL簇内的其他节点,如提供配置数据、启动并停止节点、运行备份等。由于这类节点负责管理其他节点的配置,应在启动其他节点之前首先启动这类节点。MGM节点是用命令ndb_mgmd启动的。

 

数据节点:这类节点用于保存簇的数据。数据节点的数目与副本的数目相关,是片段的倍数。例如,对于两个副本,每个副本有两个片段,那么就有4个数据节点。没有必要有一个以上的副本。数据节点是用命令ndbd启动的。

 

SQL节点:这是用来访问簇数据的节点。对于MySQL簇,客户端节点是使用NDB簇存储引擎的传统MySQL服务器。典型情况下,SQL节点是使用命令mysqld –ndbcluster启动的,或将ndbcluster添加到my.cnf后使用mysqld启动。

 

簇配置包括对簇中单独节点的配置,以及设置节点之间的单独通信链路。对于目前设计的MySQL簇,其意图在于,从处理器的能力、内存空间和带宽来讲,存储节点是同质的,此外,为了提供单一的配置点,作为整体,簇的所有配置数据均位于1个配置文件中。

 

管理服务器(MGM节点)负责管理簇配置文件和簇日志。簇中的每个节点从管理服务器检索配置数据,并请求确定管理服务器所在位置的方式。当数据节点内出现有趣的事件时,节点将关于这类事件的信息传输到管理服务器,然后,将这类信息写入簇日志。

 

 

二,配置实例:

    服务器使用情况:

    Node

    IP Address

    Management (MGMD) node

    192.168.0.10

    MySQL server (SQL) node

    192.168.0.20

    Data (NDBD) node "A"

    192.168.0.30

    Data (NDBD) node "B"

    192.168.0.40

     

     1,安装ManagementSql node Data node 重复以下步骤:

    [root@www local]# tar -zxvf mysql-cluster-gpl-7.1.3-linux-i686-glibc23.tar.gz

    [root@www local]# mv mysql-cluster-gpl-7.1.3-linux-i686-glibc23 mysql

    [root@www local]# groupadd mysql

    [root@www local]# useradd -g mysql mysql

    [root@www local]# chown -R mysql.mysql mysql

    [root@www local]# /usr/local/mysql/scripts/mysql_install_db  --user=mysql

    [root@www mysql]# cp support-files/mysql.server /etc/rc.d/init.d/

    [root@www mysql]# chmod +x /etc/rc.d/init.d/mysql.server

    [root@www mysql]# chkconfig --add mysql.server

      

    2,配置Management节点:

    [root@www mysql]# mkdir /var/lib/mysql-cluster

    [root@www mysql]# vi /var/lib/mysql-cluster/config.ini

    [ndbd default]

    NoOfReplicas=2    # Number of replicas

    DataMemory=80M    # How much memory to allocate for data storage

    IndexMemory=18M   # How much memory to allocate for index storage

                      # For DataMemory and IndexMemory, we have used the

                      # default values. Since the "world" database takes up

                      # only about 500KB, this should be more than enough for

                      # this example Cluster setup.

     

    # TCP/IP options:

    [tcp default]

    portnumber=2202   # This the default; however, you can use any port that is free

                      # for all the hosts in the cluster

                      # Note: It is recommended that you do not specify the port

                      # number at all and allow the default value to be used instead

     

    # Management process options:

    [ndb_mgmd]

    hostname=192.168.0.10           # Hostname or IP address of management node

    datadir=/var/lib/mysql-cluster  # Directory for management node log files

     

    # Options for data node "A":

    [ndbd]

                                    # (one [ndbd] section per data node)

    hostname=192.168.0.30           # Hostname or IP address

    datadir=/usr/local/mysql/data   # Directory for this data node's data files

     

    # Options for data node "B":

    [ndbd]

    hostname=192.168.0.40           # Hostname or IP address

    datadir=/usr/local/mysql/data   # Directory for this data node's data files

     

    # SQL node options:

    [mysqld]

    hostname=192.168.0.20           # Hostname or IP address

                                    # (additional mysqld connections can be

                                    # specified for this node for various

                                    # purposes such as running ndb_restore)

     

    [root@www mysql-cluster]# cp bin/ndb_mgm* /usr/local/bin

     

    3,配置MySQL server (SQL) node

    创建  /etc/my.cnf并添加下面的内容

    #ptions for mysqld process:

    [mysqld]

    ndbcluster                      # run NDB storage engine

    ndb-connectstring=192.168.0.10  # location of management server

     

    # Options for ndbd process:

    [mysql_cluster]

    ndb-connectstring=192.168.0.10  # location of management server

     

    4,配置 Data (NDBD) node "A"

    创建/etc/my.cnf文件,并添加下面的内容

    #ptions for mysqld process:

    [mysqld]

    ndbcluster                      # run NDB storage engine

    ndb-connectstring=192.168.0.10  # location of management server

     

    # Options for ndbd process:

    [mysql_cluster]

    ndb-connectstring=192.168.0.10  # location of management server

     

    5,配置 Data (NDBD) node "B"

    创建/etc/my.cnf文件,并添加下面的内容

    #ptions for mysqld process:

    [mysqld]

    ndbcluster                      # run NDB storage engine

    ndb-connectstring=192.168.0.10  # location of management server

     

    # Options for ndbd process:

    [mysql_cluster]

    ndb-connectstring=192.168.0.10  # location of management server

     

    6,启动Management节点:

    [root@www local]# ndb_mgmd -f /var/lib/mysql-cluster/config.ini

    2010-05-05 13:20:19 [MgmtSrvr] INFO     -- NDB Cluster Management Server. mysql-5.1.44 ndb-7.1.3

    2010-05-05 13:20:19 [MgmtSrvr] INFO     -- Loaded config from '/usr/local/mysql/mysql-cluster/ndb_1_config.bin.1'

     

    7,起动  Data (NDBD) node "A" Data (NDBD) node "B"

    [root@www local]# /usr/local/mysql/bin/ndbd

    2010-05-05 13:20:30 [ndbd] INFO     -- Configuration fetched from '192.168.0.10:1186', generation: 1

     

    8,起动  MySQL server (SQL) node

    [root@www local]# /etc/init.d/mysql.server start

     

     

    9,如果一切顺利,执行ndb_mgm命令之后,其输出如下:

    [root@www local]# ndb_mgm
    -- NDB Cluster -- Management Client --
    ndb_mgm> show
    Connected to Management Server at: localhost:1186
    Cluster Configuration
    ---------------------
    [ndbd(NDB)]     2 node(s)
    id=2    @192.168.0.30  (mysql-5.1.44 ndb-7.1.3, Nodegroup: 0, Master)
    id=3    @192.168.0.40  (mysql-5.1.44 ndb-7.1.3, Nodegroup: 0)
    [ndb_mgmd(MGM)] 1 node(s)
    id=1    @192.168.0.10  (mysql-5.1.44 ndb-7.1.3)
    [mysqld(API)]   1 node(s)
    id=4    @192.168.0.20  (mysql-5.1.44 ndb-7.1.3)
     

     

    测试单点故障:
    1,模拟NDB节点崩溃:
    Data (NDBD) node "A"
    [root@www ~]# netstat -tunlp
    Active Internet connections (only servers)
    Proto Recv-Q Send-Q Local Address               Foreign Address             State       PID/Program name  
    tcp        0      0 127.0.0.1:2208              0.0.0.0:*                   LISTEN      4456/hpiod         
    tcp        0      0 0.0.0.0:111                 0.0.0.0:*                   LISTEN      4240/portmap       
    tcp        0      0 0.0.0.0:627                 0.0.0.0:*                   LISTEN      4261/rpc.statd     
    tcp        0      0 127.0.0.1:631               0.0.0.0:*                   LISTEN      4487/cupsd         
    tcp        0      0 127.0.0.1:25                0.0.0.0:*                   LISTEN      4507/sendmail: acce
    tcp        0      0 192.168.0.30:2202           0.0.0.0:*                   LISTEN      6205/ndbd          
    tcp        0      0 127.0.0.1:2207              0.0.0.0:*                   LISTEN      4461/python        
    tcp        0      0 :::3306                     :::*                        LISTEN      11766/mysqld       
    tcp        0      0 :::22                       :::*                        LISTEN      4476/sshd          
    udp        0      0 0.0.0.0:32768               0.0.0.0:*                               4617/avahi-daemon: 
    udp        0      0 0.0.0.0:5353                0.0.0.0:*                               4617/avahi-daemon: 
    udp        0      0 0.0.0.0:621                 0.0.0.0:*                               4261/rpc.statd     
    udp        0      0 0.0.0.0:111                 0.0.0.0:*                               4240/portmap       
    udp        0      0 0.0.0.0:624                 0.0.0.0:*                               4261/rpc.statd     
    udp        0      0 0.0.0.0:631                 0.0.0.0:*                               4487/cupsd         
    udp        0      0 :::32769                    :::*                                    4617/avahi-daemon: 
    udp        0      0 :::5353                     :::*                                    4617/avahi-daemon: 
    [root@www ~]# kill -9 6205
    [root@www ~]# netstat -tunlp
    Active Internet connections (only servers)
    Proto Recv-Q Send-Q Local Address               Foreign Address             State       PID/Program name  
    tcp        0      0 127.0.0.1:2208              0.0.0.0:*                   LISTEN      4456/hpiod         
    tcp        0      0 0.0.0.0:111                 0.0.0.0:*                   LISTEN      4240/portmap       
    tcp        0      0 0.0.0.0:627                 0.0.0.0:*                   LISTEN      4261/rpc.statd     
    tcp        0      0 127.0.0.1:631               0.0.0.0:*                   LISTEN      4487/cupsd         
    tcp        0      0 127.0.0.1:25                0.0.0.0:*                   LISTEN      4507/sendmail: acce
    tcp        0      0 127.0.0.1:2207              0.0.0.0:*                   LISTEN      4461/python        
    tcp        0      0 :::3306                     :::*                        LISTEN      11766/mysqld       
    tcp        0      0 :::22                       :::*                        LISTEN      4476/sshd          
    udp        0      0 0.0.0.0:32768               0.0.0.0:*                               4617/avahi-daemon: 
    udp        0      0 0.0.0.0:5353                0.0.0.0:*                               4617/avahi-daemon: 
    udp        0      0 0.0.0.0:621                 0.0.0.0:*                               4261/rpc.statd     
    udp        0      0 0.0.0.0:111                 0.0.0.0:*                               4240/portmap       
    udp        0      0 0.0.0.0:624                 0.0.0.0:*                               4261/rpc.statd     
    udp        0      0 0.0.0.0:631                 0.0.0.0:*                               4487/cupsd         
    udp        0      0 :::32769                    :::*                                    4617/avahi-daemon: 
    udp        0      0 :::5353                     :::*                                    4617/avahi-daemon: 
    [root@www ~]# kill -9

     

    mysql> use test;
    Reading table information for completion of table and column names
    You can turn off this feature to get a quicker startup with -A

    Database changed
    mysql> select * from youshengtao;
    +------+
    | a    |
    +------+
    |  400 |
    |  300 |
    |  200 |
    |  500 |
    +------+
    4 rows in set (0.60 sec)


     

    2, 在 MySQL server (SQL) node执行查询,可见down掉其中的一台data node,对cluster没有任何影响
    mysql> use test;
    Reading table information for completion of table and column names
    You can turn off this feature to get a quicker startup with -A

    Database changed
    mysql> select * from youshengtao;
    +------+
    | a    |
    +------+
    |  400 |
    |  300 |
    |  200 |
    |  500 |
    +------+
    4 rows in set (0.60 sec)


    3,模拟NDB节点崩溃:
    Data (NDBD) node "B"
    [root@www ~]# netstat -tunlp
    Active Internet connections (only servers)
    Proto Recv-Q Send-Q Local Address               Foreign Address             State       PID/Program name  
    tcp        0      0 127.0.0.1:2208              0.0.0.0:*                   LISTEN      4453/hpiod         
    tcp        0      0 0.0.0.0:622                 0.0.0.0:*                   LISTEN      4256/rpc.statd     
    tcp        0      0 0.0.0.0:111                 0.0.0.0:*                   LISTEN      4235/portmap       
    tcp        0      0 127.0.0.1:631               0.0.0.0:*                   LISTEN      4484/cupsd         
    tcp        0      0 127.0.0.1:25                0.0.0.0:*                   LISTEN      4504/sendmail: acce
    tcp        0      0 192.168.0.40:2202           0.0.0.0:*                   LISTEN      6105/ndbd          
    tcp        0      0 127.0.0.1:2207              0.0.0.0:*                   LISTEN      4458/python        
    tcp        0      0 :::3306                     :::*                        LISTEN      11940/mysqld       
    tcp        0      0 :::22                       :::*                        LISTEN      4473/sshd          
    udp        0      0 0.0.0.0:32768               0.0.0.0:*                               4614/avahi-daemon: 
    udp        0      0 0.0.0.0:616                 0.0.0.0:*                               4256/rpc.statd     
    udp        0      0 0.0.0.0:5353                0.0.0.0:*                               4614/avahi-daemon: 
    udp        0      0 0.0.0.0:619                 0.0.0.0:*                               4256/rpc.statd     
    udp        0      0 0.0.0.0:111                 0.0.0.0:*                               4235/portmap       
    udp        0      0 0.0.0.0:631                 0.0.0.0:*                               4484/cupsd         
    udp        0      0 :::32769                    :::*                                    4614/avahi-daemon: 
    udp        0      0 :::5353                     :::*                                    4614/avahi-daemon: 
    [root@www ~]# kill -9 6105
    [root@www ~]# netstat -tunlp
    Active Internet connections (only servers)
    Proto Recv-Q Send-Q Local Address               Foreign Address             State       PID/Program name  
    tcp        0      0 127.0.0.1:2208              0.0.0.0:*                   LISTEN      4453/hpiod         
    tcp        0      0 0.0.0.0:622                 0.0.0.0:*                   LISTEN      4256/rpc.statd     
    tcp        0      0 0.0.0.0:111                 0.0.0.0:*                   LISTEN      4235/portmap       
    tcp        0      0 127.0.0.1:631               0.0.0.0:*                   LISTEN      4484/cupsd         
    tcp        0      0 127.0.0.1:25                0.0.0.0:*                   LISTEN      4504/sendmail: acce
    tcp        0      0 127.0.0.1:2207              0.0.0.0:*                   LISTEN      4458/python        
    tcp        0      0 :::3306                     :::*                        LISTEN      11940/mysqld       
    tcp        0      0 :::22                       :::*                        LISTEN      4473/sshd          
    udp        0      0 0.0.0.0:32768               0.0.0.0:*                               4614/avahi-daemon: 
    udp        0      0 0.0.0.0:616                 0.0.0.0:*                               4256/rpc.statd     
    udp        0      0 0.0.0.0:5353                0.0.0.0:*                               4614/avahi-daemon: 
    udp        0      0 0.0.0.0:619                 0.0.0.0:*                               4256/rpc.statd     
    udp        0      0 0.0.0.0:111                 0.0.0.0:*                               4235/portmap       
    udp        0      0 0.0.0.0:631                 0.0.0.0:*                               4484/cupsd         
    udp        0      0 :::32769                    :::*                                    4614/avahi-daemon: 
    udp        0      0 :::5353                     :::*                                    4614/avahi-daemon: 
    [root@www ~]#



    4,在 MySQL server (SQL) node执行查询,data node节点全down掉之后无法进行查询了:
    mysql> select * from youshengtao;
    ERROR 1296 (HY000): Got error 157 'Unknown error code' from NDBCLUSTER
    mysql> select * from youshengtao;
    ERROR 1296 (HY000): Got error 157 'Unknown error code' from NDBCLUSTER
    mysql> show tables;
    +----------------+
    | Tables_in_test |
    +----------------+
    | t1             |
    | youshengtao    |
    +----------------+
    2 rows in set (0.00 sec)
    mysql> desc youshengtao;
    ERROR 1296 (HY000): Got error 157 'Unknown error code' from NDBCLUSTER
    mysql> desc t1;
    ERROR 1296 (HY000): Got error 157 'Unknown error code' from NDBCLUSTER