运用Starwind配置ISCSI共享存储

文档课题：运用Starwind配置ISCSI共享存储.
1、存储配置
备注：1.1-1.2在windows执行，1.3双节点执行
共享存储可以使用第三方软件提供的方式来共享，也可以使用 WMware Workstation软件进行存储共享，或者使用ISCSI网络存储服务来配置共享存储.
本次使用ISCSI网络存储服务来模拟共享存储，常用ISCSI配置共享存储的软件：Openfiler 和 StarWind，本次讲解StarWind 软件.
1.1、安装StarWind
安装包下载地址.
链接：https://pan.baidu.com/s/1eeIr2qkH2VjQ0G8B05ZoIw 
提取码：d50d
说明：StarWind软件安装在windows 8.
解压starwind6_jb51安装包.

双击starwind.6.0.5713

starwind安装完毕，注意安装时需将杀毒软件关闭.
1.2、配置starwind iscsi
通过软件配置共享存储，打开starwind软件.

新建starwind server

cmd命令查看本机ip

填写本机ip，点击ok

新建成功后，选择server，双击connect连接：

填写target别名

注意：此处一定要勾选“Allow multiple concurrent iSCSI….”,否则客户端只能连接一个.

添加device存储盘

选择虚拟硬盘

选择镜像文件

创建新的虚拟盘

选择镜像文件路径和大小

选择已有target

starwind共享存储配置完成，按此方法再添加2块5g ocr磁盘，5块data磁盘.
备注：第一次只配置了3块data磁盘，后面安装gi时发现磁盘组+DATA空间不够后额外添加2块磁盘给磁盘组扩容.

1.3、连接共享存储
linux通过iscsi连接共享存储
1.3.1、linux客户端安装iscsi
# yum install -y iscsi-initiator-utils*
1.3.2、搜索服务端iscsi target
# iscsiadm -m discovery -t st -p 192.168.133.159
192.168.133.159:3260,-1 iqn.2008-08.com.starwindsoftware:192.168.133.159-liujun
1.3.3、连接服务端iscsi共享存储
# iscsiadm -m node -T iqn.2008-08.com.starwindsoftware:192.168.133.159-liujun -p 192.168.133.159 -l

Logging in to [iface: default, target: iqn.2008-08.com.starwindsoftware:192.168.133.159-liujun, portal: 192.168.133.159,3260] (multiple)
Login to [iface: default, target: iqn.2008-08.com.starwindsoftware:192.168.133.159-liujun, portal: 192.168.133.159,3260] successful.
1.3.4、linux客户端查看共享存储
[root@hisdb1 ~]# lsblk
NAME            MAJ:MIN RM  SIZE RO TYPE MOUNTPOINT
sda               8:0    0  100G  0 disk 
├─sda1            8:1    0    2G  0 part /boot
└─sda2            8:2    0   98G  0 part 
  ├─centos-root 253:0    0   90G  0 lvm  /
  └─centos-swap 253:1    0    8G  0 lvm  [SWAP]
sdb               8:16   0    5G  0 disk 
sdc               8:32   0    5G  0 disk 
sdd               8:48   0    5G  0 disk 
sde               8:64   0   10G  0 disk 
sdf               8:80   0   10G  0 disk 
sdg               8:96   0   10G  0 disk 
sdh               8:80   0   10G  0 disk 
sdi               8:96   0   10G  0 disk 
sr0              11:0    1  4.4G  0 rom  /mnt   
如上所示，sdb至sdi为挂载磁盘
2、multipath
备注：5.1-5.5两个节点执行
2.1、安装multipath
[root@hisdb1 ~]# yum -y install device-mapper*
[root@hisdb1 ~]# mpathconf --enable --with_multipathd y
2.2、查看共享盘的scsi_id
[root@hisdb1 ~]# /usr/lib/udev/scsi_id -g -u /dev/sdb
2949b33226e13a67e
[root@hisdb1 ~]# /usr/lib/udev/scsi_id -g -u /dev/sdc
2c8b83cb31aac6f39
[root@hisdb1 ~]# /usr/lib/udev/scsi_id -g -u /dev/sdd
2db2a03bea26bf86f
[root@hisdb1 ~]# /usr/lib/udev/scsi_id -g -u /dev/sde
28b5c47889192de60
[root@hisdb1 ~]# /usr/lib/udev/scsi_id -g -u /dev/sdf
259760f4c88f48992
[root@hisdb1 ~]# /usr/lib/udev/scsi_id -g -u /dev/sdg
2507b8b00f05efdda
[root@hisdb1 ~]# /usr/lib/udev/scsi_id -g -u /dev/sdh
27328bdb3122301cd
[root@hisdb1 ~]# /usr/lib/udev/scsi_id -g -u /dev/sdi
23f9852c09f79cbd8

2.3、配置multipath
wwid值为上面获取的scsi_id,alias可自定义，这里配置3块OCR盘，5块DATA盘
[root@hisdb1 ~]# cat <<EOF>/etc/multipath.conf
defaults {
    user_friendly_names yes
}
 
blacklist {
  devnode "^sda"
}

multipaths {
  multipath {
  wwid "2949b33226e13a67e"
  alias ocr_1
  }
  multipath {
  wwid "2c8b83cb31aac6f39"
  alias ocr_2
  }
  multipath {
  wwid "2db2a03bea26bf86f"
  alias ocr_3
  }
  multipath {
  wwid "28b5c47889192de60"
  alias data_1
  }
  multipath {
  wwid "259760f4c88f48992"
  alias data_2
  }
  multipath {
  wwid "2507b8b00f05efdda"
  alias data_3
  }
multipath {
  wwid "27328bdb3122301cd"
  alias data_4
  }
multipath {
  wwid "23f9852c09f79cbd8"
  alias data_5
  }
}
EOF
2.4、激活multipath多路径
[root@hisdb1 ~]# multipath -F
[root@hisdb1 ~]# multipath -v2
create: ocr_1 (2949b33226e13a67e) undef ROCKET  ,IMAGEFILE       
size=5.0G features='0' hwhandler='0' wp=undef
`-+- policy='service-time 0' prio=1 status=undef
  `- 3:0:0:0 sdb 8:16  undef ready running
create: ocr_2 (2c8b83cb31aac6f39) undef ROCKET  ,IMAGEFILE       
size=5.0G features='0' hwhandler='0' wp=undef
`-+- policy='service-time 0' prio=1 status=undef
  `- 3:0:0:1 sdc 8:32  undef ready running
create: ocr_3 (2db2a03bea26bf86f) undef ROCKET  ,IMAGEFILE       
size=5.0G features='0' hwhandler='0' wp=undef
`-+- policy='service-time 0' prio=1 status=undef
  `- 3:0:0:2 sdd 8:48  undef ready running
create: data_1 (28b5c47889192de60) undef ROCKET  ,IMAGEFILE       
size=10G features='0' hwhandler='0' wp=undef
`-+- policy='service-time 0' prio=1 status=undef
  `- 3:0:0:3 sde 8:64  undef ready running
create: data_2 (259760f4c88f48992) undef ROCKET  ,IMAGEFILE       
size=10G features='0' hwhandler='0' wp=undef
`-+- policy='service-time 0' prio=1 status=undef
  `- 3:0:0:4 sdf 8:80  undef ready running
create: data_3 (2507b8b00f05efdda) undef ROCKET  ,IMAGEFILE       
size=10G features='0' hwhandler='0' wp=undef
`-+- policy='service-time 0' prio=1 status=undef
  `- 3:0:0:5 sdg 8:96  undef ready running
create: data_4 (27328bdb3122301cd) undef ROCKET  ,IMAGEFILE       
size=10G features='0' hwhandler='0' wp=undef
`-+- policy='service-time 0' prio=1 status=undef
  `- 3:0:0:6 sdh 8:112 undef ready running
create: data_5 (23f9852c09f79cbd8) undef ROCKET  ,IMAGEFILE       
size=10G features='0' hwhandler='0' wp=undef
`-+- policy='service-time 0' prio=1 status=undef
  `- 3:0:0:7 sdi 8:128 undef ready running
[root@hisdb1 ~]# multipath -ll
data_5 (23f9852c09f79cbd8) dm-9 ROCKET  ,IMAGEFILE       
size=10G features='0' hwhandler='0' wp=rw
`-+- policy='service-time 0' prio=1 status=active
  `- 3:0:0:7 sdi 8:128 active ready running
data_4 (27328bdb3122301cd) dm-8 ROCKET  ,IMAGEFILE       
size=10G features='0' hwhandler='0' wp=rw
`-+- policy='service-time 0' prio=1 status=active
  `- 3:0:0:6 sdh 8:112 active ready running
data_3 (2507b8b00f05efdda) dm-7 ROCKET  ,IMAGEFILE       
size=10G features='0' hwhandler='0' wp=rw
`-+- policy='service-time 0' prio=1 status=active
  `- 3:0:0:5 sdg 8:96  active ready running
data_2 (259760f4c88f48992) dm-6 ROCKET  ,IMAGEFILE       
size=10G features='0' hwhandler='0' wp=rw
`-+- policy='service-time 0' prio=1 status=active
  `- 3:0:0:4 sdf 8:80  active ready running
ocr_3 (2db2a03bea26bf86f) dm-4 ROCKET  ,IMAGEFILE       
size=5.0G features='0' hwhandler='0' wp=rw
`-+- policy='service-time 0' prio=1 status=active
  `- 3:0:0:2 sdd 8:48  active ready running
data_1 (28b5c47889192de60) dm-5 ROCKET  ,IMAGEFILE       
size=10G features='0' hwhandler='0' wp=rw
`-+- policy='service-time 0' prio=1 status=active
  `- 3:0:0:3 sde 8:64  active ready running
ocr_2 (2c8b83cb31aac6f39) dm-3 ROCKET  ,IMAGEFILE       
size=5.0G features='0' hwhandler='0' wp=rw
`-+- policy='service-time 0' prio=1 status=active
  `- 3:0:0:1 sdc 8:32  active ready running
ocr_1 (2949b33226e13a67e) dm-2 ROCKET  ,IMAGEFILE       
size=5.0G features='0' hwhandler='0' wp=rw
`-+- policy='service-time 0' prio=1 status=active
  `- 3:0:0:0 sdb 8:16  active ready running
2.5、绑盘
[root@hisdb1 rules.d]# for i in b c d e f g h i; do 
echo "KERNEL==\"sd*\",ENV{DEVTYPE}==\"disk\",SUBSYSTEM==\"block\",PROGRAM==\"/usr/lib/udev/scsi_id -g -u -d \$devnode\",RESULT==\"`/usr/lib/udev/scsi_id -g -u /dev/sd$i`\", RUN+=\"/bin/sh -c 'mknod /dev/asm-disk$i b  \$major \$minor; chown grid:asmadmin /dev/asm-disk$i; chmod 0660 /dev/asm-disk$i'\"" >> /etc/udev/rules.d/99-oracle-asmdevices.rules
done
[root@hisdb1 rules.d]# cat 99-oracle-asmdevices.rules
KERNEL=="sd*",ENV{DEVTYPE}=="disk",SUBSYSTEM=="block",PROGRAM=="/usr/lib/udev/scsi_id -g -u -d $devnode",RESULT=="2949b33226e13a67e", RUN+="/bin/sh -c 'mknod /dev/asm-diskb b  $major $minor; chown grid:asmadmin /dev/asm-diskb; chmod 0660 /dev/asm-diskb'"
KERNEL=="sd*",ENV{DEVTYPE}=="disk",SUBSYSTEM=="block",PROGRAM=="/usr/lib/udev/scsi_id -g -u -d $devnode",RESULT=="2c8b83cb31aac6f39", RUN+="/bin/sh -c 'mknod /dev/asm-diskc b  $major $minor; chown grid:asmadmin /dev/asm-diskc; chmod 0660 /dev/asm-diskc'"
KERNEL=="sd*",ENV{DEVTYPE}=="disk",SUBSYSTEM=="block",PROGRAM=="/usr/lib/udev/scsi_id -g -u -d $devnode",RESULT=="2db2a03bea26bf86f", RUN+="/bin/sh -c 'mknod /dev/asm-diskd b  $major $minor; chown grid:asmadmin /dev/asm-diskd; chmod 0660 /dev/asm-diskd'"
KERNEL=="sd*",ENV{DEVTYPE}=="disk",SUBSYSTEM=="block",PROGRAM=="/usr/lib/udev/scsi_id -g -u -d $devnode",RESULT=="28b5c47889192de60", RUN+="/bin/sh -c 'mknod /dev/asm-diske b  $major $minor; chown grid:asmadmin /dev/asm-diske; chmod 0660 /dev/asm-diske'"
KERNEL=="sd*",ENV{DEVTYPE}=="disk",SUBSYSTEM=="block",PROGRAM=="/usr/lib/udev/scsi_id -g -u -d $devnode",RESULT=="259760f4c88f48992", RUN+="/bin/sh -c 'mknod /dev/asm-diskf b  $major $minor; chown grid:asmadmin /dev/asm-diskf; chmod 0660 /dev/asm-diskf'"
KERNEL=="sd*",ENV{DEVTYPE}=="disk",SUBSYSTEM=="block",PROGRAM=="/usr/lib/udev/scsi_id -g -u -d $devnode",RESULT=="2507b8b00f05efdda", RUN+="/bin/sh -c 'mknod /dev/asm-diskg b  $major $minor; chown grid:asmadmin /dev/asm-diskg; chmod 0660 /dev/asm-diskg'"
KERNEL=="sd*",ENV{DEVTYPE}=="disk",SUBSYSTEM=="block",PROGRAM=="/usr/lib/udev/scsi_id -g -u -d $devnode",RESULT=="27328bdb3122301cd", RUN+="/bin/sh -c 'mknod /dev/asm-diskh b  $major $minor; chown grid:asmadmin /dev/asm-diskh; chmod 0660 /dev/asm-diskh'"
KERNEL=="sd*",ENV{DEVTYPE}=="disk",SUBSYSTEM=="block",PROGRAM=="/usr/lib/udev/scsi_id -g -u -d $devnode",RESULT=="23f9852c09f79cbd8", RUN+="/bin/sh -c 'mknod /dev/asm-diski b  $major $minor; chown grid:asmadmin /dev/asm-diski; chmod 0660 /dev/asm-diski'"

参考网址：https://mp.weixin.qq.com/s/xjYZmGcwshbJBICcr0xvBw