HeartBeat+DRBD+NFS 高可用

原创

it1693 2015-07-30 10:10:32 博主文章分类：Linux ©著作权

文章标签 客户端 文章分类 运维

©著作权归作者所有：来自51CTO博客作者it1693的原创作品，请联系作者获取转载授权，否则将追究法律责任

heartbeat、drbd、nfs全部使用yum安装。

NFS客户端使用的是centos 7 可以实现自动挂载。

此文是测试成功后的总结

主节点：

Last login: Thu Jul 30 09:12:52 2015 from 10.0.0.1
[root@heartbeat1 ~]#
[root@heartbeat1 ~]# cat /etc/redhat-release
CentOS release 6.6 (Final)
[root@heartbeat1 ~]# uname -ra
Linux heartbeat1 2.6.32-504.el6.x86_64 #1 SMP Wed Oct 15 04:27:16 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
[root@heartbeat1 ~]# tail -3 /etc/hosts
::1         localhost localhost.localdomain localhost6 localhost6.localdomain6
10.0.0.7    heartbeat1
10.0.0.8    heartbeat2
[root@heartbeat1 ~]# tail -3 /etc/rc.local
/sbin/route add -host 10.0.0.26 dev eth1
modprobe drbd
drbdadm up data
[root@heartbeat1 ~]# ifconfig
eth0      Link encap:Ethernet HWaddr 00:0C:29:51:A3:1E
          inet addr:10.0.0.7 Bcast:10.0.0.255 Mask:255.255.255.0     #备节点ip：10.0.0.8
eth1      Link encap:Ethernet HWaddr 00:0C:29:51:A3:28
          inet addr:10.0.10.7 Bcast:10.0.10.255 Mask:255.255.255.0    #备节点心跳ip10.0.10.8

heartbeat 配置文件：（主节点备节点配置相同）
[root@heartbeat1 ha.d]# grep -v ^# /etc/ha.d/authkeys
auth 3
3 md5 Hello!
[root@heartbeat1 ha.d]# grep -v ^# /etc/ha.d/ha.cf
debugfile /var/log/ha-debug
logfile    /var/log/ha-log
logfacility    local0
keepalive 2
deadtime 5
warntime 10
initdead 60
udpport 694
mcast eth1 225.0.0.1 694 1 0
auto_failback on
node heartbeat1
node heartbeat2
[root@heartbeat1 ha.d]# grep -v ^# /etc/ha.d/haresources
heartbeat1      IPaddr::10.0.0.50/24/eth0 drbddisk::data Filesystem::/dev/drbd0::/md1::ext4
[root@heartbeat1 ha.d]#

共享磁盘分区：（主备节点相同）
parted /dev/sdc mklabel gpt
parted /dev/sdc mkpart primary 0 1024
parted /dev/sdc mkpart primary 1025 2146
parted /dev/sdc p 查看分区表

共享磁盘分区信息：（主备节点配置相同）
[root@heartbeat1 ~]# parted /dev/sdc p
Model: VMware, VMware Virtual S (scsi)
Disk /dev/sdc: 2147MB
Sector size (logical/physical): 512B/512B
Partition Table: gpt
Number Start   End     Size    File system Name     Flags
1      17.4kB 1024MB 1024MB ext4         primary
2      1025MB 2146MB 1121MB               primary

DRBD配置文件：（主备节点配置相同）
[root@heartbeat1 ~]# cat /etc/drbd.d/global_common.conf
global    {
    usage-count no;
}
common    {
    syncer    {
        rate 1M;
        verify-alg crc32c;
        al-extents 517;
    }
}
#primary for drbd1
resource data {
    protocol C;
    disk    {
        on-io-error detach;
        }
    on heartbeat1 {
        device    /dev/drbd0;
        disk    /dev/sdc1;
        address    10.0.0.7:7788;
        meta-disk    /dev/sdc2 [0];
    }
    on heartbeat2    {
        device /dev/drbd0;
        disk    /dev/sdc1;
        address    10.0.0.8:7788;
        meta-disk    /dev/sdc2 [0];
    }
}
[root@heartbeat1 ~]#

DRBD启动：
第一步初始化：
[root@heartbeat1 ~]# drbdadm create-md data 初始化data磁盘
[root@heartbeat1 ~]# drbdadm up data
[root@heartbeat1 ~]# cat /proc/drbd 如果是Secondary 说明启动正常
启动后进行格式化磁盘：（只是主节点上配置，备节点会自动同步配置）
[root@heartbeat1 drbd.d]# mkfs.ext4 -b 4096 /dev/drbd0 (对应的是/dev/sdc1 ,drbd 主上配置)
[root@heartbeat1 drbd.d]# tune2fs -c -1 /dev/drbd0
tune2fs 1.41.12 (17-May-2010)
Setting maximal mount count to -1

测试磁盘：
mkdir /md1    创建挂载目录
mount /dev/drbd0 /md1    挂载drbd0设备测试
[root@heartbeat1 ~]# df -h
Filesystem            Size Used Avail Use% Mounted on
/dev/mapper/VolGroup-lv_root
                       35G 4.0G   30G 13% /
tmpfs                 491M     0 491M   0% /dev/shm
/dev/sda1             477M   34M 418M   8% /boot
/dev/drbd0            749M 3.5M 707M   1% /md1    自动挂载点

NFS配置文件：（主备节点相同）
[root@heartbeat1 ~]# cat /etc/exports
/md1 10.0.0.*(rw,sync) 目录允许ip段权限

NFS启动：主从同时配置：先启动rpcbind 服务
[root@heartbeat2 ~]# /etc/init.d/rpcbind restart
Stopping rpcbind:                                          [ OK ]
Starting rpcbind:                                          [ OK ]
[root@heartbeat2 ~]# /etc/init.d/nfs restart
Shutting down NFS daemon:                                  [FAILED]
Shutting down NFS mountd:                                  [FAILED]
Shutting down NFS quotas:                                  [FAILED]
Starting NFS services:                                     [ OK ]
Starting NFS quotas:                                       [ OK ]
Starting NFS mountd:                                       [ OK ]
Starting NFS daemon:                                       [ OK ]
Starting RPC idmapd:                                       [ OK ]
[root@heartbeat2 ~]# chkconfig rpcbind on
[root@heartbeat2 ~]# chkconfig nfs on
[root@heartbeat1 ~]# rpcinfo -p localhost      检查启动状态

[root@heartbeat1 ~]# showmount -e 10.0.0.50
Export list for 10.0.0.50:
/md1 10.0.0.*

客户端挂载：
[root@localhost ~]# mount -t nfs 10.0.0.50:/md1/ /md1/
[root@localhost ~]#
[root@localhost ~]# ls /md1/
lafjd oldboy14 oldboy21
[root@localhost ~]# echo "mount -t nfs 10.0.0.50:/md1/ /md1/" >> /etc/rc.local 添加开机启动
[root@localhost ~]# tail -1 /etc/rc.local
mount -t nfs 10.0.0.50:/md1/ /md1/

也可以使用exportfs 配置NFS（详情 man exportfs）
[root@heartbeat1 ha.d]#exportfs -o rw,sync,all_squash,anonuid=65534,anongid=65534,mp,fsid=2

10.0.0.0/24:/md1
[root@heartbeat1 ha.d]# /etc/init.d/nfs restart
[root@heartbeat1 ha.d]# showmount -e 10.0.0.7
Export list for 10.0.0.7:
/md1 10.0.0.*

----------------------------------------------------------------------------
查看drbd运行状态：
[root@heartbeat1 ~]# cat /proc/drbd
version: 8.4.4 (api:1/proto:86-101)
GIT-hash: 74402fecf24da8e5438171ee8c19e28627e1c98a build by root@heartbeat1, 2015-07-21 14:36:54
0: cs:Connected ro:Primary/Secondary ds:UpToDate/UpToDate C r-----
ns:8 nr:0 dw:4 dr:776 al:1 bm:1 lo:0 pe:0 ua:0 ap:0 ep:1 wo:f oos:0
[root@heartbeat1 ~]#

[root@heartbeat1 ~]# lsmod |grep drbd
drbd 327515 4
libcrc32c 1246 1 drbd

drbdadm 命令说明：
[root@heartbeat1 ~]# drbdadm -h

USAGE: drbdadm COMMAND [OPTION...] {all|RESOURCE...}

GENERAL OPTIONS:
--stacked, -S
--dry-run, -d
--verbose, -v
--config-file=..., -c ...
--config-to-test=..., -t ...
--drbdsetup=..., -s ...
--drbdmeta=..., -m ...
--drbd-proxy-ctl=..., -p ...
--sh-varname=..., -n ...
--peer=..., -P ...
--version, -V
--setup-option=..., -W ...
--help, -h

COMMANDS:
attach                             disk-options
detach                             connect
net-options                        disconnect
up                                 resource-options
down                               primary
secondary                          invalidate
invalidate-remote                  outdate
resize                             verify
pause-sync                         resume-sync
adjust                             adjust-with-progress
wait-connect                       wait-con-int
role                               cstate
dstate                             dump
dump-xml                           create-md
show-gi                            get-gi
dump-md                            wipe-md
apply-al                           hidden-commands

Version: 8.4.4 (api:1)
GIT-hash: 74402fecf24da8e5438171ee8c19e28627e1c98a build by root@heartbeat1, 2015-07-21 14:36:42

[root@heartbeat1 ~]# drbdadm create-md data    初始化data磁盘
[root@heartbeat1 ~]# drbdadm disconnect data    断开data磁盘连接
[root@heartbeat1 ~]# drbdadm connect data    连接data磁盘

注意事项：
启动heartbeat：(重启先stop在start)
/etc/init.d/heartbeat start
关掉开机启动（heartbeat开机启动容易发生裂脑问题）
chkconfig heartbeat off
chkconfig drbd off

解决drbd Unknown 问题：
cs:StandAlone ro:Secondary/Unknown ds:UpToDate/DUnknown r-----
方法一：
在选定的非主节点上执行
drbdadm secondary data
drbdadm -- --discard-my-data connect mysql
主节点上执行
drbdadm connect data

如果以上方法不行：
重新初始化
drbdadm detach data
###dd if=/dev/zero bs=1M count=100 of=/dev/sdb1
drbdadm down data
drbdadm create-md data
在主节点上执行
drbdadm connect data