文档课题：向现有集群环境添加新磁盘组——ASMLIB场景.
系统：centos 7.9 64位
数据库：oracle 11.2.0.4 64位
环境：rac(双节点) + dg + asmlib创建磁盘组
1、新增磁盘
节点1添加1块10g磁盘，节点2添加节点1新增的磁盘.
1.1、磁盘分区
节点1对新增磁盘分区.
[root@hisdb1 ~]# lsblk -p
NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINT
/dev/sda 8:0 0 100G 0 disk
├─/dev/sda1 8:1 0 1G 0 part /boot
└─/dev/sda2 8:2 0 99G 0 part
├─/dev/mapper/centos-root 253:0 0 95G 0 lvm /
└─/dev/mapper/centos-swap 253:1 0 4G 0 lvm [SWAP]
/dev/sdb 8:16 0 10G 0 disk
├─/dev/sdb1 8:17 0 10G 0 part
└─/dev/mapper/ocr 253:2 0 10G 0 mpath
└─/dev/mapper/ocr1 253:8 0 10G 0 part
/dev/sdc 8:32 0 10G 0 disk
├─/dev/sdc1 8:33 0 10G 0 part
└─/dev/mapper/fra 253:3 0 10G 0 mpath
└─/dev/mapper/fra1 253:6 0 10G 0 part
/dev/sdd 8:48 0 20G 0 disk
├─/dev/sdd1 8:49 0 20G 0 part
└─/dev/mapper/data 253:5 0 20G 0 mpath
└─/dev/mapper/data1 253:7 0 20G 0 part
/dev/sde 8:64 0 10G 0 disk
└─/dev/mapper/mpatha 253:4 0 10G 0 mpath
/dev/sr0 11:0 1 4.4G 0 rom /run/media/grid/CentOS 7 x86_64
说明：/dev/sde为新增磁盘.
[root@hisdb1 ~]# fdisk /dev/sde
Welcome to fdisk (util-linux 2.23.2).
Changes will remain in memory only, until you decide to write them.
Be careful before using the write command.
Device does not contain a recognized partition table
Building a new DOS disklabel with disk identifier 0x1b499d05.
Command (m for help): n
Partition type:
p primary (0 primary, 0 extended, 4 free)
e extended
Select (default p):
Using default response p
Partition number (1-4, default 1):
First sector (2048-20971519, default 2048):
Using default value 2048
Last sector, +sectors or +size{K,M,G} (2048-20971519, default 20971519):
Using default value 20971519
Partition 1 of type Linux and of size 10 GiB is set
Command (m for help): w
The partition table has been altered!
Calling ioctl() to re-read partition table.
Syncing disks.
[root@hisdb1 ~]# partprobe
[root@hisdb2 ~]# partprobe
[root@hisdb1 ~]# lsblk -p
NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINT
/dev/sda 8:0 0 100G 0 disk
├─/dev/sda1 8:1 0 1G 0 part /boot
└─/dev/sda2 8:2 0 99G 0 part
├─/dev/mapper/centos-root 253:0 0 95G 0 lvm /
└─/dev/mapper/centos-swap 253:1 0 4G 0 lvm [SWAP]
/dev/sdb 8:16 0 10G 0 disk
├─/dev/sdb1 8:17 0 10G 0 part
└─/dev/mapper/ocr 253:2 0 10G 0 mpath
└─/dev/mapper/ocr1 253:8 0 10G 0 part
/dev/sdc 8:32 0 10G 0 disk
├─/dev/sdc1 8:33 0 10G 0 part
└─/dev/mapper/fra 253:3 0 10G 0 mpath
└─/dev/mapper/fra1 253:6 0 10G 0 part
/dev/sdd 8:48 0 20G 0 disk
├─/dev/sdd1 8:49 0 20G 0 part
└─/dev/mapper/data 253:5 0 20G 0 mpath
└─/dev/mapper/data1 253:7 0 20G 0 part
/dev/sde 8:64 0 10G 0 disk
├─/dev/sde1 8:65 0 10G 0 part
└─/dev/mapper/mpatha 253:4 0 10G 0 mpath
└─/dev/mapper/mpatha1 253:9 0 10G 0 part
/dev/sr0 11:0 1 4.4G 0 rom /run/media/grid/CentOS 7 x86_64
1.2、asmlib判断
判断磁盘组配置方式.
[grid@hisdb1 ~]$ ls -ltr /dev/oracleasm/disks
total 0
brw-rw---- 1 grid asmadmin 8, 17 Nov 5 16:11 DATA01
brw-rw---- 1 grid asmadmin 8, 49 Nov 5 16:11 DATA03
brw-rw---- 1 grid asmadmin 8, 33 Nov 5 16:11 DATA02
说明：磁盘组配置方式为asmlib.
1.3、多路径
1.3.1、多路径显示
[root@hisdb1 ~]# multipath -ll
mpatha (36000c29087f5e11b96a53b025d0e8d60) dm-4 VMware, ,VMware Virtual S
size=10G features='0' hwhandler='0' wp=rw
`-+- policy='service-time 0' prio=1 status=active
`- 0:0:4:0 sde 8:64 active ready running
fra (36000c2935008c75902f7cbbe7ee6cb1a) dm-3 VMware, ,VMware Virtual S
size=10G features='0' hwhandler='0' wp=rw
`-+- policy='service-time 0' prio=1 status=active
`- 0:0:2:0 sdc 8:32 active ready running
data (36000c294842ccadd084847980127e55a) dm-5 VMware, ,VMware Virtual S
size=20G features='0' hwhandler='0' wp=rw
`-+- policy='service-time 0' prio=1 status=active
`- 0:0:3:0 sdd 8:48 active ready running
ocr (36000c29d0b8efae055435249dec3dfad) dm-2 VMware, ,VMware Virtual S
size=10G features='0' hwhandler='0' wp=rw
`-+- policy='service-time 0' prio=1 status=active
`- 0:0:1:0 sdb 8:16 active ready running
1.3.2、获取wwid
获取新磁盘wwid.
[root@hisdb1 disks]# /usr/lib/udev/scsi_id -g -u /dev/sde
36000c29087f5e11b96a53b025d0e8d60
1.3.3、配置multipath
双节点向/etc/multipath.conf文件添加以下内容.
multipath {
wwid "36000c29087f5e11b96a53b025d0e8d60"
alias ocrbk
}
1.3.4、激活multipath
说明：双节点激活multipath多路径.
[root@hisdb1 ~]# multipath -F
[root@hisdb1 ~]# multipath -v2
create: ocr (36000c29d0b8efae055435249dec3dfad) undef VMware, ,VMware Virtual S
size=10G features='0' hwhandler='0' wp=undef
`-+- policy='service-time 0' prio=1 status=undef
`- 0:0:1:0 sdb 8:16 undef ready running
create: fra (36000c2935008c75902f7cbbe7ee6cb1a) undef VMware, ,VMware Virtual S
size=10G features='0' hwhandler='0' wp=undef
`-+- policy='service-time 0' prio=1 status=undef
`- 0:0:2:0 sdc 8:32 undef ready running
create: data (36000c294842ccadd084847980127e55a) undef VMware, ,VMware Virtual S
size=20G features='0' hwhandler='0' wp=undef
`-+- policy='service-time 0' prio=1 status=undef
`- 0:0:3:0 sdd 8:48 undef ready running
create: orcbk (36000c29087f5e11b96a53b025d0e8d60) undef VMware, ,VMware Virtual S
size=10G features='0' hwhandler='0' wp=undef
`-+- policy='service-time 0' prio=1 status=undef
`- 0:0:4:0 sde 8:64 undef ready running
[root@hisdb1 ~]# multipath -ll
fra (36000c2935008c75902f7cbbe7ee6cb1a) dm-4 VMware, ,VMware Virtual S
size=10G features='0' hwhandler='0' wp=rw
`-+- policy='service-time 0' prio=1 status=active
`- 0:0:2:0 sdc 8:32 active ready running
data (36000c294842ccadd084847980127e55a) dm-6 VMware, ,VMware Virtual S
size=20G features='0' hwhandler='0' wp=rw
`-+- policy='service-time 0' prio=1 status=active
`- 0:0:3:0 sdd 8:48 active ready running
ocrbk (36000c29087f5e11b96a53b025d0e8d60) dm-8 VMware, ,VMware Virtual S
size=10G features='0' hwhandler='0' wp=rw
`-+- policy='service-time 0' prio=1 status=active
`- 0:0:4:0 sde 8:64 active ready running
ocr (36000c29d0b8efae055435249dec3dfad) dm-2 VMware, ,VMware Virtual S
size=10G features='0' hwhandler='0' wp=rw
`-+- policy='service-time 0' prio=1 status=active
`- 0:0:1:0 sdb 8:16 active ready running
[root@hisdb1 ~]# lsblk -p
NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINT
/dev/sda 8:0 0 100G 0 disk
├─/dev/sda1 8:1 0 1G 0 part /boot
└─/dev/sda2 8:2 0 99G 0 part
├─/dev/mapper/centos-root 253:0 0 95G 0 lvm /
└─/dev/mapper/centos-swap 253:1 0 4G 0 lvm [SWAP]
/dev/sdb 8:16 0 10G 0 disk
├─/dev/sdb1 8:17 0 10G 0 part
└─/dev/mapper/ocr 253:2 0 10G 0 mpath
└─/dev/mapper/ocr1 253:3 0 10G 0 part
/dev/sdc 8:32 0 10G 0 disk
├─/dev/sdc1 8:33 0 10G 0 part
└─/dev/mapper/fra 253:4 0 10G 0 mpath
└─/dev/mapper/fra1 253:5 0 10G 0 part
/dev/sdd 8:48 0 20G 0 disk
├─/dev/sdd1 8:49 0 20G 0 part
└─/dev/mapper/data 253:6 0 20G 0 mpath
└─/dev/mapper/data1 253:7 0 20G 0 part
/dev/sde 8:64 0 10G 0 disk
├─/dev/sde1 8:65 0 10G 0 part
└─/dev/mapper/orcbk 253:8 0 10G 0 mpath
└─/dev/mapper/ocrbk1 253:9 0 10G 0 part
/dev/sr0 11:0 1 4.4G 0 rom /run/media/grid/CentOS 7 x86_64
2、创建新磁盘组
2.1、新建asm磁盘
节点1创建新asm磁盘。
[root@hisdb1 ~]# oracleasm createdisk data04 /dev/mapper/ocbk1
Writing disk header: done
Instantiating disk: done
双节点验证
[root@hisdb1 ~]# oracleasm scandisks
Reloading disk partitions: done
Cleaning any stale ASM disks...
Scanning system for ASM disks...
[root@hisdb1 ~]# oracleasm listdisks
DATA01
DATA02
DATA03
DATA04
[root@hisdb2 ~]# oracleasm scandisks
Reloading disk partitions: done
Cleaning any stale ASM disks...
Scanning system for ASM disks...
Instantiating disk "DATA04"
[root@hisdb2 ~]# oracleasm listdisks
DATA01
DATA02
DATA03
DATA04
2.2、创建新磁盘组
开启xmanager，节点1创建新磁盘组ocrbk.
[grid@hisdb1 ~]$ export DISPLAY=192.168.133.1:0.0
[grid@hisdb1 ~]$ asmca
