一.vif接口概念
netapp把端口聚合叫做vif(与术语port trunk/etherchanell/link aggregate是同一个意思),支持以下三中聚合方式:
a.single-mode,端口工作在active和standby状态,只有一个端口处于actvie状况,当active端口失效,系统自动从standby中选择一个端口成为active口;不需要交换机支持aggre,也不需要在交换机上做任何相关配置.所有端口具有相同的ＭＡＣ地址(如下所示)
b.multi-mode(static),与IEEE 802.3ad(static)兼容,所有端口处于active状态,具有相同的mac地址;能够检测端口的链路状态,但不能数据的丢失;交换机需要支持LACP;在multimode(包括static和dynamic)下,存储系统对outbound traffic支持三种load-balance算法:IP-based/MAC-based/Round robin;
c.multi-mode(dynamic),与IEEE 802.3ad(dynamic)兼容,即lLACP协议(cisco叫做PAgP),所有端口均处于active状态,具有相同的mac地址;交换机需要支持LACP;在lacp下,vif接口必须配置为first-level,并且LB应该配置为IP-based(这也是默认方式);在lacp下,存储系统不仅能检测链路的丢失,还能检测数据流的丢失,因此该模式与HA环境相兼容;
二.vif接口配置
使用vif命令创建删除vif接口,添加或删除物理网口到vif中,查看vif接口状态;
最多支持16个物理口聚合成1个vif接口;所有物理口需具有相同的MTU;不能将vlan接口添加到vif中;支持10/100/1000m网卡;
对于10G TOE卡有以下限制:multimode支持2个物理口,LACP不支持,TOE功能会被禁用;
nasim*> vif
Usage:
vif create [single|multi|lacp] <vif_name> -b [rr|mac|ip] [<interface_list>] 创建vif接口,指定模式/接口名称/LB算法/相应的物理接口;
vif add <vif_name> <interface_list> 将物理口添加到以存在的vif接口中;
vif delete <vif_name> <interface_name> 从vif接口中删除物理接口;
vif destroy <vif_name> 删除vif逻辑接口;
vif {favor|nofavor} <interface> 在接口组中,选择active接口;
vif status [<vif_name>] 查看vif接口状态;
vif stat <vif_name> [interval] 查看vif流量统计信息;
1.single vif配置
step1:使用vif create命令创建single vif逻辑接口:
nasim*> vif create single singletrunk1 ns0 ns1
nasim*>
查看系统接口信息,
nasim*> ifconfig -a
ns0: flags=808042<BROADCAST,RUNNING,MULTICAST> mtu 1500
ether 02:50:56:1d:ff:f7 (Linux AF_PACKET socket)
trunked singletrunk1
ns1: flags=808042<BROADCAST,RUNNING,MULTICAST> mtu 1500
ether 02:50:56:1d:ff:f7 (Linux AF_PACKET socket)
trunked singletrunk1
lo: flags=1948049<UP,LOOPBACK,RUNNING,MULTICAST,TCPCKSUM> mtu 9188
inet 127.0.0.1 netmask 0xff000000 broadcast 127.0.0.1
singletrunk1: flags=8042<BROADCAST,RUNNING,MULTICAST> mtu 1500
ether 02:50:56:1d:ff:f7 (Disabled virtual interface)
nasim*>
step2:给虚拟接口配置IP地址:
nasim*>
nasim*> ifconfig singletrunk1 192.168.1.20 netmask 255.255.255.0
nasim*> Mon Jan 18 04:35:58 GMT [pvif.switchLink:warning]: singletrunk1: switching to ns1 (系统随机选择ns1口为active接口)
Mon Jan 18 04:35:58 GMT [iscsi.notice:notice]: ISCSI: New session from initiator iqn.2005-03.com.redhat:01.74394c92c697 at IP addr 192.168.1.60
nasim*> ifconfig -a
ns0: flags=848043<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500
ether 02:50:56:1d:ff:f7 (Linux AF_PACKET socket)
trunked singletrunk1
ns1: flags=848043<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500
ether 02:50:56:1d:ff:f7 (Linux AF_PACKET socket)
trunked singletrunk1
lo: flags=1948049<UP,LOOPBACK,RUNNING,MULTICAST,TCPCKSUM> mtu 9188
inet 127.0.0.1 netmask 0xff000000 broadcast 127.0.0.1
singletrunk1: flags=848043<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500
inet 192.168.1.20 netmask 0xffffff00 broadcast 192.168.1.255
ether 02:50:56:1d:ff:f7 (Enabled virtual interface)
nasim*>
step3:手动选择active接口,查看vif接口状态:
nasim*>
nasim*> vif favor ns0
nasim*>
nasim*> vif status singletrunk1 (在singlemode下,只有一个物理口为up,即active,其他为down,即standby)
default: transmit 'IP Load balancing', VIF Type 'multi_mode', fail 'log'
singletrunk1: 1 link, transmit 'none', VIF Type 'single_mode' fail 'default'
VIF Status Up Addr_set
up:
ns1: state up, since 18Jan2010 05:19:38 (00:55:06)
mediatype: Linux AF_PACKET socket
flags: enabled
input packets 0, input bytes 0
output packets 562, output bytes 534476
output probe packets 0, input probe packets 0
up indications 1, broken indications 0
drops (if) 0, drops (link) 0
indication: up at 18Jan2010 05:19:38
consecutive 3304, transitions 1
down:
ns0: state down, since 18Jan2010 05:19:38 (00:55:06)
mediatype: Linux AF_PACKET socket
flags: enabled
input packets 0, input bytes 0
output packets 0, output bytes 0
output probe packets 0, input probe packets 0
up indications 1, broken indications 0
drops (if) 0, drops (link) 0
indication: up at 18Jan2010 05:19:38
consecutive 3304, transitions 1
nasim*> Mon Jan 18 04:44:02 GMT [pvif.switchLink:warning]: singletrunk1: switching to ns0
step4:验证linux iscsi initiator连接
nasim*> iscsi session show
No active sessions
没有活动iscsi会话,查看主机端启动日志:
[root@rhe5 log]# grep iscsi /var/log/messages
Jan 17 03:42:20 rhe5 iscsid: connect failed (113)
Jan 17 04:36:52 rhe5 iscsid: Portal group tag mismatch, expected 1000, received 2000
Jan 17 04:36:57 rhe5 iscsid: connect failed (113)
Jan 17 05:04:11 rhe5 kernel: iscsi: cmd 0x28 is not queued (8)
Jan 17 05:04:11 rhe5 kernel: iscsi: cmd 0x28 is not queued (8)
Jan 17 05:04:14 rhe5 iscsid: connect failed (113)
Jan 17 05:04:41 rhe5 iscsid: connect failed (113)
Jan 17 05:04:47 rhe5 iscsid: connect failed (113)
Jan 17 05:06:08 rhe5 kernel: Loading iSCSI transport class v2.0-685.<5>iscsi: registered transport (tcp)
Jan 17 05:06:08 rhe5 kernel: iscsi: registered transport (iser)
Jan 17 05:06:08 rhe5 iscsid: connection0:0 is operational now
Jan 17 05:06:08 rhe5 iscsid: picking unique OUI for the same target node name iqn.1992-08.com.netapp:sn.99900641
Jan 17 05:06:09 rhe5 iscsid: connection1:0 is operational now
Jan 17 05:12:01 rhe5 iscsid: connect failed (113)
Jan 17 05:12:01 rhe5 iscsid: connect failed (113)
Jan 17 05:12:04 rhe5 iscsid: Portal group tag mismatch, expected 1000, received 2000
Jan 17 05:12:06 rhe5 iscsid: connect failed (113)
Jan 17 05:13:39 rhe5 kernel: session0: iscsi: session recovery timed out after 120 secs
Jan 17 05:13:39 rhe5 iscsid: connect failed (113)
Jan 17 05:21:23 rhe5 iscsid: connect failed (113)
Jan 17 21:23:19 rhe5 iscsid: connect failed (113)
Jan 17 21:23:25 rhe5 iscsid: connect failed (113)
Jan 17 21:25:10 rhe5 kernel: Loading iSCSI transport class v2.0-685.<5>iscsi: registered transport (tcp)
Jan 17 21:25:10 rhe5 kernel: iscsi: registered transport (iser)
Jan 17 21:25:14 rhe5 iscsid: connect failed (113)
Jan 17 21:25:15 rhe5 iscsid: Portal group tag mismatch, expected 1000, received 2000
上述错误信息表明,portal group tag不匹配导致iscsi login 失败;查看存储设备和主机上的配置:
[root@rhe5 log]# cd /var/lib
[root@rhe5 lib]# cd iscsi
[root@rhe5 iscsi]# ls
nodes send_targets
[root@rhe5 iscsi]# cd nodes
[root@rhe5 nodes]# ls
iqn.1992-08.com.netapp:sn.99900641
[root@rhe5 nodes]# cd iqn.1992-08.com.netapp\:sn.99900641/
[root@rhe5 iqn.1992-08.com.netapp:sn.99900641]# ls
192.168.1.20,3260 192.168.1.21,3260
[root@rhe5 iqn.1992-08.com.netapp:sn.99900641]# more 192.168.1.20,3260
node.name = iqn.1992-08.com.netapp:sn.99900641
node.transport_name = tcp
node.tpgt = 1000
node.active_conn = 1
node.startup = automatic
node.session.initial_cmdsn = 0
node.session.auth.authmethod = None
node.session.timeo.replacement_timeout = 120
……
nasim*> iscsi portal show
Network portals:
IP address TCP Port TPGroup Interface
192.168.1.20 3260 2000 singletrunk1
修改主机上portal tag与存储设备上一致,重启主机;
[root@rhe5 send_targets]# fdisk -l
Disk /dev/sda: 23.6 GB, 23622320128 bytes
255 heads, 63 sectors/track, 2871 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Device Boot Start End Blocks Id System
/dev/sda1 * 1 6 48163+ 83 Linux
/dev/sda2 7 515 4088542+ 83 Linux
/dev/sda3 516 2741 17880345 83 Linux
/dev/sda4 2742 2871 1044225 5 Extended
/dev/sda5 2742 2871 1044193+ 82 Linux swap / Solaris
Disk /dev/sdb: 36 MB, 36700160 bytes
2 heads, 35 sectors/track, 1024 cylinders
Units = cylinders of 70 * 512 = 35840 bytes
Device Boot Start End Blocks Id System
/dev/sdb1 1 1024 35822+ 83 Linux
[root@rhe5 send_targets]# cd /
[root@rhe5 /]# mkdir /mnt/nalun
[root@rhe5 /]# mount /dev/sdb1 /mnt/nalun
nasim*>
nasim*> iscsi session show
Session 5
Initiator Information
Initiator Name: iqn.2005-03.com.redhat:01.74394c92c697
ISID: 00:02:3d:00:00:00
Initiator Alias: rhe5
2.multimode vif配置
nasim*>
nasim*> vif destroy singletrunk1
vif: cannot destroy a pvif that is up
nasim*>
nasim*>
nasim*> ifconfig singletrunk1 down
nasim*> Mon Jan 18 06:17:29 GMT [pvif.vifConfigDown:info]: singletrunk1: Configured down
nasim*> vif destroy singletrunk1
nasim*>
nasim*>
nasim*> ifconfig -a
ns0: flags=808042<BROADCAST,RUNNING,MULTICAST> mtu 1500
ether 00:50:56:1d:ff:f7 (Linux AF_PACKET socket)
ns1: flags=808042<BROADCAST,RUNNING,MULTICAST> mtu 1500
ether 00:50:56:1e:ff:f7 (Linux AF_PACKET socket)
lo: flags=1948049<UP,LOOPBACK,RUNNING,MULTICAST,TCPCKSUM> mtu 9188
inet 127.0.0.1 netmask 0xff000000 broadcast 127.0.0.1
nasim*>
nasim*>
nasim*> vif create multi multitrunk1 ns0 ns1
nasim*>
nasim*> ifconfig multitrunk1 192.168.1.20 netmask 255.255.255.0
nasim*> Mon Jan 18 06:18:45 GMT [pvif.switchLink:warning]: multitrunk1: switching to ns1
Mon Jan 18 06:18:48 GMT [iscsi.notice:notice]: ISCSI: New session from initiator iqn.2005-03.com.redhat:01.74394c92c697 at IP addr 192.168.1.60
nasim*>
nasim*> vif status multitrunk1 (在multimode下,vif接口组中所有物理接口都为up状态,即active状况)
default: transmit 'IP Load balancing', VIF Type 'multi_mode', fail 'log'
multitrunk1: 2 links, transmit 'IP Load balancing', VIF Type 'multi_mode' fail 'default'
VIF Status Up Addr_set
up:
ns1: state up, since 18Jan2010 06:18:45 (00:03:01)
mediatype: Linux AF_PACKET socket
flags: enabled
input packets 0, input bytes 0
output packets 37, output bytes 2938
up indications 1, broken indications 0
drops (if) 0, drops (link) 0
indication: up at 18Jan2010 06:18:45
consecutive 180, transitions 1
ns0: state up, since 18Jan2010 06:18:45 (00:03:01)
mediatype: Linux AF_PACKET socket
flags: enabled
input packets 0, input bytes 0
output packets 119, output bytes 63710
up indications 1, broken indications 0
drops (if) 0, drops (link) 0
indication: up at 18Jan2010 06:18:45
consecutive 180, transitions 1
nasim*> iscsi session show
Session 6
Initiator Information
Initiator Name: iqn.2005-03.com.redhat:01.74394c92c697
ISID: 00:02:3d:00:00:00
Initiator Alias: rhe5
3.lacp vif配置
nasim*>
nasim*> ifconfig multitrunk1 down
nasim*> Mon Jan 18 06:31:28 GMT [pvif.vifConfigDown:info]: multitrunk1: Configured down
nasim*>
nasim*> vif destroy multitrunk1
nasim*>
nasim*> vif create lacp lacptrunk1 ns0 ns1
nasim*>
nasim*>
nasim*> ifconfig lacptrunk1 192.168.1.20 netmask 255.255.255.0 up
nasim*>
nasim*> ifconfig -a
ns0: flags=848043<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500
ether 02:50:56:1d:ff:f7 (Linux AF_PACKET socket)
trunked lacptrunk1
ns1: flags=848043<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500
ether 02:50:56:1d:ff:f7 (Linux AF_PACKET socket)
trunked lacptrunk1
lo: flags=1948049<UP,LOOPBACK,RUNNING,MULTICAST,TCPCKSUM> mtu 9188
inet 127.0.0.1 netmask 0xff000000 broadcast 127.0.0.1
lacptrunk1: flags=48043<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500
inet 192.168.1.20 netmask 0xffffff00 broadcast 192.168.1.255
ether 02:50:56:1d:ff:f7 (Disabled virtual interface)
nasim*>
nasim*>
nasim*>
nasim*> ifconfig lacptrunk1 up
nasim*>
nasim*>
nasim*> vif status lacptrunk1 ()
default: transmit 'IP Load balancing', VIF Type 'multi_mode', fail 'log'
lacptrunk1: 0 links, transmit 'IP Load balancing', VIF Type 'lacp' fail 'default'
VIF Status Addr_set
lag_inactive:
ns1: state lag_inactive, since 18Jan2010 06:32:21 (00:00:50)
mediatype: Linux AF_PACKET socket
flags: lacp enabled
input packets 0, input bytes 0
input lacp packets 0, output lacp packets 2
output packets 2, output bytes 248
up indications 1, broken indications 0
drops (if) 0, drops (link) 0
indication: lag_inactive at 18Jan2010 06:32:21
consecutive 0, transitions 1
ns0: state lag_inactive, since 18Jan2010 06:32:21 (00:00:50)
mediatype: Linux AF_PACKET socket
flags: lacp enabled
input packets 0, input bytes 0
input lacp packets 0, output lacp packets 2
output packets 2, output bytes 248
up indications 1, broken indications 0
drops (if) 0, drops (link) 0
indication: lag_inactive at 18Jan2010 06:32:21
consecutive 0, transitions 1
nasim*>
nasim*> ping 192.168.1.20
ping: wrote 192.168.1.20 64 chars, error=No route to host
ping: wrote 192.168.1.20 64 chars, error=No route to host
nasim*>
nasim*> route -s
Routing tables
Internet:
Destination Gateway Flags Refs Use Interface
localhost localhost UH 0 0 lo
由于交换机不支持lacp,lacp vif配置不成功
