使用openfiler测试rhel中的device-mapper-multipath多路径软件

一台机器装openfiler充当存储，另外一台当作客户端,主机名为server.xzxj.edu.cn.

openfile的安装这里不做具体介绍，完成多路径在openfiler端必须存在两块网卡，这里如下：

eth0:192.168.1.5

eth1:192.168.1.6

openfiler中的iscsi target配置如图所示：

 

server.xzxj.edu.cn端的配置如下:

启动iscsi进程：service iscsi start;chkconfig iscsi on

首先发现openfiler端的iscsi目标，其次是登录到iscsi目标上，如图所示：

因为要测试linux自带的多路径软件，所以这里要登录两次。两次的IP是不同的,但iqn名称是一致的。这里是我写的一个脚本，所以显示是这样子的。

现在已经登录上去了，可以使用fdisk -l命令查看是否已经识别到磁盘了：

[root@server ~]# fdisk -l

Disk /dev/sda: 21.4 GB, 21474836480 bytes
255 heads, 63 sectors/track, 2610 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes

   Device Boot      Start         End      Blocks   Id  System
/dev/sda1   *           1          13      104391   83  Linux
/dev/sda2              14        2349    18763920   8e  Linux LVM
/dev/sda3            2350        2610     2096482+  82  Linux swap / Solaris

Disk /dev/sdj: 10.7 GB, 10737418240 bytes
64 heads, 32 sectors/track, 10240 cylinders
Units = cylinders of 2048 * 512 = 1048576 bytes

Disk /dev/sdj doesn't contain a valid partition table

Disk /dev/sdk: 10.7 GB, 10737418240 bytes
64 heads, 32 sectors/track, 10240 cylinders
Units = cylinders of 2048 * 512 = 1048576 bytes

Disk /dev/sdk doesn't contain a valid partition table

Disk /dev/sdl: 10.7 GB, 10737418240 bytes
64 heads, 32 sectors/track, 10240 cylinders
Units = cylinders of 2048 * 512 = 1048576 bytes

Disk /dev/sdl doesn't contain a valid partition table

Disk /dev/sdm: 10.7 GB, 10737418240 bytes
64 heads, 32 sectors/track, 10240 cylinders
Units = cylinders of 2048 * 512 = 1048576 bytes

Disk /dev/sdm doesn't contain a valid partition table

Disk /dev/sdn: 10.7 GB, 10737418240 bytes
64 heads, 32 sectors/track, 10240 cylinders
Units = cylinders of 2048 * 512 = 1048576 bytes

Disk /dev/sdn doesn't contain a valid partition table

Disk /dev/sdo: 10.7 GB, 10737418240 bytes
64 heads, 32 sectors/track, 10240 cylinders
Units = cylinders of 2048 * 512 = 1048576 bytes

Disk /dev/sdo doesn't contain a valid partition table

Disk /dev/sdp: 10.7 GB, 10737418240 bytes
64 heads, 32 sectors/track, 10240 cylinders
Units = cylinders of 2048 * 512 = 1048576 bytes

Disk /dev/sdp doesn't contain a valid partition table

Disk /dev/sdq: 10.7 GB, 10737418240 bytes
64 heads, 32 sectors/track, 10240 cylinders
Units = cylinders of 2048 * 512 = 1048576 bytes

Disk /dev/sdq doesn't contain a valid partition table

看到了吧，这里多出了8块盘。在openfiler中只map了四次，为什么这里是8块而不是4块呢？因为在上面提过，使用两个IP登录两次iscsi target，所以这里有两块是重复的。

下面是多路径软件device-mapper-multipath的配置文件具体配置，如下所示：

完了后，启动multipathd进程：

service multipathd start

chkconfig multipathd on

启动之后，使用multipath -ll可以查看是否成功：

看到了吧，重复的磁盘现在绑定到一起了。

给这几块盘分区，创建文件系统，使用fdisk /dev/dm-1或者fdisk /dev/mapper/openfiler_vol01即可。如下所示：

[root@server45 ~]# fdisk /dev/dm-3
Device contains neither a valid DOS partition table, nor Sun, SGI or OSF disklabel
Building a new DOS disklabel. Changes will remain in memory only,
until you decide to write them. After that, of course, the previous
content won't be recoverable.

The number of cylinders for this disk is set to 1305.
There is nothing wrong with that, but this is larger than 1024,
and could in certain setups cause problems with:
1) software that runs at boot time (e.g., old versions of LILO)
2) booting and partitioning software from other OSs
   (e.g., DOS FDISK, OS/2 FDISK)
Warning: invalid flag 0x0000 of partition table 4 will be corrected by w(rite)

Command (m for help): p

Disk /dev/dm-3: 10.7 GB, 10737418240 bytes
255 heads, 63 sectors/track, 1305 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes

     Device Boot      Start         End      Blocks   Id  System

Command (m for help): n
Command action
   e   extended
   p   primary partition (1-4)
p
Partition number (1-4): 1
First cylinder (1-1305, default 1):
Using default value 1
Last cylinder or +size or +sizeM or +sizeK (1-1305, default 1305): +5120M

Command (m for help): t
Selected partition 1
Hex code (type L to list codes): fd
Changed system type of partition 1 to fd (Linux raid autodetect)

Command (m for help): n
Command action
   e   extended
   p   primary partition (1-4)
p
Partition number (1-4): 2
First cylinder (624-1305, default 624):
Using default value 624
Last cylinder or +size or +sizeM or +sizeK (624-1305, default 1305):
Using default value 1305

Command (m for help): t
Partition number (1-4): 2
Hex code (type L to list codes): fd
Changed system type of partition 2 to fd (Linux raid autodetect)

Command (m for help): w
The partition table has been altered!

Calling ioctl() to re-read partition table.

WARNING: Re-reading the partition table failed with error 22: Invalid argument.
The kernel still uses the old table.
The new table will be used at the next reboot.
Syncing disks.

分区完结果如下：

[root@server45 ~]# fdisk -l

Disk /dev/sda: 21.4 GB, 21474836480 bytes
255 heads, 63 sectors/track, 2610 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes

   Device Boot      Start         End      Blocks   Id  System
/dev/sda1   *           1          13      104391   83  Linux
/dev/sda2              14        2349    18763920   8e  Linux LVM
/dev/sda3            2350        2610     2096482+  82  Linux swap / Solaris

Disk /dev/sdb: 10.7 GB, 10737418240 bytes
255 heads, 63 sectors/track, 1305 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes

   Device Boot      Start         End      Blocks   Id  System
/dev/sdb1               1         623     5004216   fd  Linux raid autodetect
/dev/sdb2             624        1305     5478165   fd  Linux raid autodetect

Disk /dev/sdc: 10.7 GB, 10737418240 bytes
255 heads, 63 sectors/track, 1305 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes

   Device Boot      Start         End      Blocks   Id  System
/dev/sdc1               1         623     5004216   fd  Linux raid autodetect
/dev/sdc2             624        1305     5478165   fd  Linux raid autodetect

Disk /dev/sdd: 10.7 GB, 10737418240 bytes
255 heads, 63 sectors/track, 1305 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes

   Device Boot      Start         End      Blocks   Id  System
/dev/sdd1               1         623     5004216   fd  Linux raid autodetect
/dev/sdd2             624        1305     5478165   fd  Linux raid autodetect

Disk /dev/sde: 10.7 GB, 10737418240 bytes
255 heads, 63 sectors/track, 1305 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes

   Device Boot      Start         End      Blocks   Id  System
/dev/sde1               1         623     5004216   fd  Linux raid autodetect
/dev/sde2             624        1305     5478165   fd  Linux raid autodetect

Disk /dev/sdf: 10.7 GB, 10737418240 bytes
255 heads, 63 sectors/track, 1305 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes

   Device Boot      Start         End      Blocks   Id  System
/dev/sdf1               1         623     5004216   fd  Linux raid autodetect
/dev/sdf2             624        1305     5478165   fd  Linux raid autodetect

Disk /dev/sdg: 10.7 GB, 10737418240 bytes
255 heads, 63 sectors/track, 1305 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes

   Device Boot      Start         End      Blocks   Id  System
/dev/sdg1               1         623     5004216   fd  Linux raid autodetect
/dev/sdg2             624        1305     5478165   fd  Linux raid autodetect

Disk /dev/sdh: 10.7 GB, 10737418240 bytes
255 heads, 63 sectors/track, 1305 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes

   Device Boot      Start         End      Blocks   Id  System
/dev/sdh1               1         623     5004216   fd  Linux raid autodetect
/dev/sdh2             624        1305     5478165   fd  Linux raid autodetect

Disk /dev/sdi: 10.7 GB, 10737418240 bytes
255 heads, 63 sectors/track, 1305 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes

   Device Boot      Start         End      Blocks   Id  System
/dev/sdi1               1         623     5004216   fd  Linux raid autodetect
/dev/sdi2             624        1305     5478165   fd  Linux raid autodetect

Disk /dev/dm-1: 10.7 GB, 10737418240 bytes
255 heads, 63 sectors/track, 1305 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes

     Device Boot      Start         End      Blocks   Id  System
/dev/dm-1p1               1         623     5004216   fd  Linux raid autodetect
/dev/dm-1p2             624        1305     5478165   fd  Linux raid autodetect

Disk /dev/dm-2: 10.7 GB, 10737418240 bytes
255 heads, 63 sectors/track, 1305 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes

     Device Boot      Start         End      Blocks   Id  System
/dev/dm-2p1               1         623     5004216   fd  Linux raid autodetect
/dev/dm-2p2             624        1305     5478165   fd  Linux raid autodetect

Disk /dev/dm-3: 10.7 GB, 10737418240 bytes
255 heads, 63 sectors/track, 1305 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes

     Device Boot      Start         End      Blocks   Id  System
/dev/dm-3p1               1         623     5004216   fd  Linux raid autodetect
/dev/dm-3p2             624        1305     5478165   fd  Linux raid autodetect

Disk /dev/dm-4: 10.7 GB, 10737418240 bytes
255 heads, 63 sectors/track, 1305 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes

     Device Boot      Start         End      Blocks   Id  System
/dev/dm-4p1               1         623     5004216   fd  Linux raid autodetect
/dev/dm-4p2             624        1305     5478165   fd  Linux raid autodetect

这里的/dev/dm-[1,2,3,4]p[1,2]就是分区1和2。

查看下/dev/mapper目录下有何不同：

[root@server45 ~]# ll /dev/mapper/
total 0
crw------- 1 root root  10, 63 Apr 17  2010 control
brw-rw---- 1 root disk 253,  1 Apr 17 10:05 openfiler_vol01
brw-rw---- 1 root disk 253,  2 Apr 17 09:58 openfiler_vol02
brw-rw---- 1 root disk 253,  3 Apr 17 09:58 openfiler_vol03
brw-rw---- 1 root disk 253,  4 Apr 17 09:58 openfiler_vol04
brw-rw---- 1 root disk 253,  0 Apr 17 09:43 vg-lv01
[root@server45 ~]# fdisk -l /dev/mapper/openfiler_vol01

Disk /dev/mapper/openfiler_vol01: 10.7 GB, 10737418240 bytes
255 heads, 63 sectors/track, 1305 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes

                       Device Boot      Start         End      Blocks   Id  System
/dev/mapper/openfiler_vol01p1               1         623     5004216   fd  Linux raid autodetect
/dev/mapper/openfiler_vol01p2             624        1305     5478165   fd  Linux raid autodetect

[root@server45 ~]# ll /dev/dm-*
brw-rw---- 1 root root 253, 1 Apr 17 09:58 /dev/dm-1
brw-rw---- 1 root root 253, 2 Apr 17 10:06 /dev/dm-2
brw-rw---- 1 root root 253, 3 Apr 17 10:07 /dev/dm-3
brw-rw---- 1 root root 253, 4 Apr 17 10:07 /dev/dm-4

如果要了解iscsi多路径的访问机制，可以参考我的另外一篇文章：深入了解iSCSI的2种多路径访问机制