Cisco 交换机端口故障分析（一）
两台Catalyst 6000 之间互联的GE２／１出现了errdisable故障，what's up告警，链路中断。
查看log记录：
Sep 4 19:18:50 UTC: %PM-SP-4-ERR_DISABLE: link-flap error detected on Gi2/1, putting Gi2/1 in err-disable state
Sep 4 19:19:09 UTC: %OSPF-5-ADJCHG: Process 108, Nbr **** on Vlan12 from FULL to DOWN, Neighbor Down: Dead timer expired
Sep 5 00:03:34 UTC: %SYS-5-CONFIG_I: Configured from console by gcc&et on vty0
Sep 5 00:07:10 UTC: %SYS-5-CONFIG_I: Configured from console by gcc&et on vty0
Sep 5 00:08:13 UTC: %PM-SP-4-ERR_RECOVER: Attempting to recover from link-flap err-disable state on Gi2/1
Sep 5 00:08:58 UTC: %OSPF-5-ADJCHG: Process 108, Nbr **** on Vlan12 from LOADING to FULL, Loading Done
Sep 5 05:14:07 UTC: %OSPF-5-ADJCHG: Process 108, Nbr **** on Vlan12 from LOADING to FULL, Loading Done
Sep 5 06:06:17 UTC: %OSPF-5-ADJCHG: Process 108, Nbr **** on Vlan12 from FULL to DOWN, Neighbor Down: Dead timer expired
Sep 5 06:06:47 UTC: %OSPF-5-ADJCHG: Process 108, Nbr **** on Vlan12 from LOADING to FULL, Loading Done
Sep 5 06:14:34 UTC: %OSPF-5-ADJCHG: Process 108, Nbr **** on Vlan12 from FULL to DOWN, Neighbor Down: Dead timer expired
Sep 5 06:14:37 UTC: %OSPF-5-ADJCHG: Process 108, Nbr **** on Vlan12 from LOADING to FULL, Loading Done
Sep 5 06:26:58 UTC: %OSPF-5-ADJCHG: Process 108, Nbr **** on Vlan12 from LOADING to FULL, Loading Done
注： 由于为真实的网络环境，为了不必要的安全隐患，以＊＊＊＊代替IP地址，请各位见谅！
故障处理：
进入configure终端模式，具体如下：
SW1>en
Password:
SW1#conf t
Enter configuration commands, one per line. End with CNTL/Z.
SW1(config)#err?
errdisable
SW1#errdi
SW1#errdisable ?
detect Error disable detection
recovery Error disable recovery
SW1(config)#errdisable re
SW1(config)#errdisable recovery ?
cause Enable error disable recovery for application
interval Error disable recovery timer value
SW1(config)#errdisable recovery cau
SW1(config)#errdisable recovery cause ?
all Enable timer to recover from all causes
bpduguard Enable timer to recover from BPDU Guard error disable state
channel-misconfig Enable timer to recover from channel misconfig disable state
dtp-flap Enable timer to recover from dtp-flap error disable state
l2ptguard Enable timer to recover from l2protocol-tunnel error disable state
link-flap Enable timer to recover from link-flap error disable state
pagp-flap Enable timer to recover from pagp-flap error disable state
psecure-violation Enable timer to recover from psecure violation disable state
udld Enable timer to recover from udld error disable state
SW1(config)#errdisable recovery cause all ?
SW1(config)#errdisable recovery cause all
执行此命令后 不久端口就显示connected状态 What's up 告警消失。
Log Buffer (4096 bytes):
: Process 100, Nbr 172.19.253.2 on GigabitEthernet1/1 from LOADING to FULL, Loading Done
Jul 2 13:04:47.533 BJT: %OSPF-5-ADJCHG: Process 100, Nbr 172.19.253.2 on GigabitEthernet1/1 from FULL to DOWN, Neighbor Down: Interface down or detached
Jul 2 13:05:30.128 BJT: %OSPF-5-ADJCHG: Process 100, Nbr 172.19.253.2 on GigabitEthernet1/1 from LOADING to FULL, Loading Done
Jul 2 13:06:50.126 BJT: %OSPF-5-ADJCHG: Process 100, Nbr 172.19.253.2 on GigabitEthernet1/1 from INIT to DOWN, Neighbor Down: Interface down or detached
Jul 2 13:07:40.137 BJT: %OSPF-5-ADJCHG: Process 100, Nbr 172.19.253.2 on GigabitEthernet1/1 from LOADING to FULL, Loading Done
Jul 2 13:17:49.948 BJT: %OSPF-5-ADJCHG: Process 100, Nbr 172.19.253.2 on GigabitEthernet1/1 from FULL to DOWN, Neighbor Down: Interface down or detached
Jul 2 13:18:33.135 BJT: %OSPF-5-ADJCHG: Process 100, Nbr 172.19.253.2 on GigabitEthernet1/1 from LOADING to FULL, Loading Done
Jul 2 13:18:46.175 BJT: %OSPF-5-ADJCHG: Process 100, Nbr 172.19.253.2 on GigabitEthernet1/1 from FULL to DOWN, Neighbor Down: Interface down or detached
Jul 2 13:19:29.134 BJT: %OSPF-5-ADJCHG: Process 100, Nbr 172.19.253.2 on GigabitEthernet1/1 from LOADING to FULL, Loading Done
Jul 2 13:21:13.507 BJT: %OSPF-5-ADJCHG: Process 100, Nbr 172.19.253.2 on GigabitEthernet1/1 from FULL to DOWN, Neighbor Down: Interface down or detached
Jul 2 13:21:56.106 BJT: %OSPF-5-ADJCHG: Process 100, Nbr 172.19.253.2 on GigabitEthernet1/1 from LOADING to FULL, Loading Done
Jul 2 13:30:22.296 BJT: %OSPF-5-ADJCHG: Process 100, Nbr 172.19.253.2 on GigabitEthernet1/1 from FULL to DOWN, Neighbor Down: Interface down or detached
Jul 2 13:31:05.091 BJT: %OSPF-5-ADJCHG: Process 100, Nbr 172.19.253.2 on GigabitEthernet1/1 from LOADING to FULL, Loading Done
Jul 2 13:31:56.185 BJT: %OSPF-5-ADJCHG: Process 100, Nbr 172.19.253.2 on GigabitEthernet1/1 from FULL to DOWN, Neighbor Down: Interface down or detached
Jul 2 13:32:31.120 BJT: %OSPF-5-ADJCHG: Process 100, Nbr 172.19.253.2 on GigabitEthernet1/1 from 2WAY to DOWN, Neighbor Down: Interface down or detached
Jul 2 13:33:14.083 BJT: %OSPF-5-ADJCHG: Process 100, Nbr 172.19.253.2 on GigabitEthernet1/1 from LOADING to FULL, Loading Done
Jul 2 13:34:09.958 BJT: %OSPF-5-ADJCHG: Process 100, Nbr 172.19.253.2 on GigabitEthernet1/1 from FULL to DOWN, Neighbor Down: Interface down or detached
Jul 2 13:34:52.113 BJT: %OSPF-5-ADJCHG: Process 100, Nbr 172.19.253.2 on GigabitEthernet1/1 from LOADING to FULL, Loading Done
Jul 2 13:37:54.624 BJT: %OSPF-5-ADJCHG: Process 100, Nbr 172.19.253.2 on GigabitEthernet1/1 from FULL to DOWN, Neighbor Down: Interface down or detached
Jul 2 13:38:07.063 BJT: %PM-4-ERR_DISABLE: link-flap error detected on Gi1/1, putting Gi1/1 in err-disable state
Jul 2 13:38:35.347 BJT: %PM-4-ERR_RECOVER: Attempting to recover from link-flap err-disable state on Gi1/1
Jul 2 13:39:18.085 BJT: %OSPF-5-ADJCHG: Process 100, Nbr 172.19.253.2 on GigabitEthernet1/1 from LOADING to FULL, Loading Done
Jul 2 13:39:55.432 BJT: %OSPF-5-ADJCHG: Process 100, Nbr 172.19.253.2 on GigabitEthernet1/1 from FULL to DOWN, Neighbor Down: Interface down or detached
Jul 2 13:40:13.488 BJT: %OSPF-5-ADJCHG: Process 100, Nbr 172.19.253.2 on GigabitEthernet1/1 from 2WAY to DOWN, Neighbor Down: Interface down or detached
Jul 2 13:40:56.075 BJT: %OSPF-5-ADJCHG: Process 100, Nbr 172.19.253.2 on GigabitEthernet1/1 from LOADING to FULL, Loading Done
Jul 2 13:44:03.098 BJT: %OSPF-5-ADJCHG: Process 100, Nbr 172.19.253.2 on GigabitEthernet1/1 from FULL to DOWN, Neighbor Down: Interface down or detached
Jul 2 13:44:42.121 BJT: %OSPF-5-ADJCHG: Process 100, Nbr 172.19.253.2 on GigabitEthernet1/1 from 2WAY to DOWN, Neighbor Down: Interface down or detached
Jul 2 13:45:06.340 BJT: %PM-4-ERR_DISABLE: link-flap error detected on Gi1/1, putting Gi1/1 in err-disable state
Jul 2 13:45:32.083 BJT: %PM-4-ERR_RECOVER: Attempting to recover from link-flap err-disable state on Gi1/1
Jul 2 13:46:26.066 BJT: %OSPF-5-ADJCHG: Process 100, Nbr 172.19.253.2 on GigabitEthernet1/1 from LOADING to FULL, Loading Done
//
no file verify auto
errdisable recovery cause udld
errdisable recovery cause bpduguard
errdisable recovery cause security-violation
errdisable recovery cause channel-misconfig
errdisable recovery cause pagp-flap
errdisable recovery cause dtp-flap
errdisable recovery cause link-flap
errdisable recovery cause gbic-invalid
errdisable recovery cause l2ptguard
errdisable recovery cause psecure-violation
errdisable recovery cause dhcp-rate-limit
errdisable recovery cause unicast-flood
errdisable recovery cause vmps
errdisable recovery cause storm-control
errdisable recovery cause arp-inspection
errdisable recovery interval 30
spanning-tree mode pvst
spanning-tree extend system-id
power redundancy-mode redundant
!
!
!
**********************************
防止err-disable,使用以下命令：errdisable recovery cause udld
errdisable recovery cause bpduguard
errdisable recovery cause security-violation
errdisable recovery cause channel-misconfig
errdisable recovery cause pagp-flap
errdisable recovery cause dtp-flap
errdisable recovery cause link-flap
errdisable recovery cause psecure-violation
errdisable recovery cause vmps
errdisable recovery cause gbic-invalid
errdisable recovery cause loopback
errdisable recovery interval 30
查看log记录：
Sep 4 19:18:50 UTC: %PM-SP-4-ERR_DISABLE: link-flap error detected on Gi2/1, putting Gi2/1 in err-disable state
Sep 4 19:19:09 UTC: %OSPF-5-ADJCHG: Process 108, Nbr **** on Vlan12 from FULL to DOWN, Neighbor Down: Dead timer expired
Sep 5 00:03:34 UTC: %SYS-5-CONFIG_I: Configured from console by gcc&et on vty0
Sep 5 00:07:10 UTC: %SYS-5-CONFIG_I: Configured from console by gcc&et on vty0
Sep 5 00:08:13 UTC: %PM-SP-4-ERR_RECOVER: Attempting to recover from link-flap err-disable state on Gi2/1
Sep 5 00:08:58 UTC: %OSPF-5-ADJCHG: Process 108, Nbr **** on Vlan12 from LOADING to FULL, Loading Done
Sep 5 05:14:07 UTC: %OSPF-5-ADJCHG: Process 108, Nbr **** on Vlan12 from LOADING to FULL, Loading Done
Sep 5 06:06:17 UTC: %OSPF-5-ADJCHG: Process 108, Nbr **** on Vlan12 from FULL to DOWN, Neighbor Down: Dead timer expired
Sep 5 06:06:47 UTC: %OSPF-5-ADJCHG: Process 108, Nbr **** on Vlan12 from LOADING to FULL, Loading Done
Sep 5 06:14:34 UTC: %OSPF-5-ADJCHG: Process 108, Nbr **** on Vlan12 from FULL to DOWN, Neighbor Down: Dead timer expired
Sep 5 06:14:37 UTC: %OSPF-5-ADJCHG: Process 108, Nbr **** on Vlan12 from LOADING to FULL, Loading Done
Sep 5 06:26:58 UTC: %OSPF-5-ADJCHG: Process 108, Nbr **** on Vlan12 from LOADING to FULL, Loading Done
注： 由于为真实的网络环境，为了不必要的安全隐患，以＊＊＊＊代替IP地址，请各位见谅！
故障处理：
进入configure终端模式，具体如下：
SW1>en
Password:
SW1#conf t
Enter configuration commands, one per line. End with CNTL/Z.
SW1(config)#err?
errdisable
SW1#errdi
SW1#errdisable ?
detect Error disable detection
recovery Error disable recovery
SW1(config)#errdisable re
SW1(config)#errdisable recovery ?
cause Enable error disable recovery for application
interval Error disable recovery timer value
SW1(config)#errdisable recovery cau
SW1(config)#errdisable recovery cause ?
all Enable timer to recover from all causes
bpduguard Enable timer to recover from BPDU Guard error disable state
channel-misconfig Enable timer to recover from channel misconfig disable state
dtp-flap Enable timer to recover from dtp-flap error disable state
l2ptguard Enable timer to recover from l2protocol-tunnel error disable state
link-flap Enable timer to recover from link-flap error disable state
pagp-flap Enable timer to recover from pagp-flap error disable state
psecure-violation Enable timer to recover from psecure violation disable state
udld Enable timer to recover from udld error disable state
SW1(config)#errdisable recovery cause all ?
SW1(config)#errdisable recovery cause all
执行此命令后 不久端口就显示connected状态 What's up 告警消失。
Log Buffer (4096 bytes):
: Process 100, Nbr 172.19.253.2 on GigabitEthernet1/1 from LOADING to FULL, Loading Done
Jul 2 13:04:47.533 BJT: %OSPF-5-ADJCHG: Process 100, Nbr 172.19.253.2 on GigabitEthernet1/1 from FULL to DOWN, Neighbor Down: Interface down or detached
Jul 2 13:05:30.128 BJT: %OSPF-5-ADJCHG: Process 100, Nbr 172.19.253.2 on GigabitEthernet1/1 from LOADING to FULL, Loading Done
Jul 2 13:06:50.126 BJT: %OSPF-5-ADJCHG: Process 100, Nbr 172.19.253.2 on GigabitEthernet1/1 from INIT to DOWN, Neighbor Down: Interface down or detached
Jul 2 13:07:40.137 BJT: %OSPF-5-ADJCHG: Process 100, Nbr 172.19.253.2 on GigabitEthernet1/1 from LOADING to FULL, Loading Done
Jul 2 13:17:49.948 BJT: %OSPF-5-ADJCHG: Process 100, Nbr 172.19.253.2 on GigabitEthernet1/1 from FULL to DOWN, Neighbor Down: Interface down or detached
Jul 2 13:18:33.135 BJT: %OSPF-5-ADJCHG: Process 100, Nbr 172.19.253.2 on GigabitEthernet1/1 from LOADING to FULL, Loading Done
Jul 2 13:18:46.175 BJT: %OSPF-5-ADJCHG: Process 100, Nbr 172.19.253.2 on GigabitEthernet1/1 from FULL to DOWN, Neighbor Down: Interface down or detached
Jul 2 13:19:29.134 BJT: %OSPF-5-ADJCHG: Process 100, Nbr 172.19.253.2 on GigabitEthernet1/1 from LOADING to FULL, Loading Done
Jul 2 13:21:13.507 BJT: %OSPF-5-ADJCHG: Process 100, Nbr 172.19.253.2 on GigabitEthernet1/1 from FULL to DOWN, Neighbor Down: Interface down or detached
Jul 2 13:21:56.106 BJT: %OSPF-5-ADJCHG: Process 100, Nbr 172.19.253.2 on GigabitEthernet1/1 from LOADING to FULL, Loading Done
Jul 2 13:30:22.296 BJT: %OSPF-5-ADJCHG: Process 100, Nbr 172.19.253.2 on GigabitEthernet1/1 from FULL to DOWN, Neighbor Down: Interface down or detached
Jul 2 13:31:05.091 BJT: %OSPF-5-ADJCHG: Process 100, Nbr 172.19.253.2 on GigabitEthernet1/1 from LOADING to FULL, Loading Done
Jul 2 13:31:56.185 BJT: %OSPF-5-ADJCHG: Process 100, Nbr 172.19.253.2 on GigabitEthernet1/1 from FULL to DOWN, Neighbor Down: Interface down or detached
Jul 2 13:32:31.120 BJT: %OSPF-5-ADJCHG: Process 100, Nbr 172.19.253.2 on GigabitEthernet1/1 from 2WAY to DOWN, Neighbor Down: Interface down or detached
Jul 2 13:33:14.083 BJT: %OSPF-5-ADJCHG: Process 100, Nbr 172.19.253.2 on GigabitEthernet1/1 from LOADING to FULL, Loading Done
Jul 2 13:34:09.958 BJT: %OSPF-5-ADJCHG: Process 100, Nbr 172.19.253.2 on GigabitEthernet1/1 from FULL to DOWN, Neighbor Down: Interface down or detached
Jul 2 13:34:52.113 BJT: %OSPF-5-ADJCHG: Process 100, Nbr 172.19.253.2 on GigabitEthernet1/1 from LOADING to FULL, Loading Done
Jul 2 13:37:54.624 BJT: %OSPF-5-ADJCHG: Process 100, Nbr 172.19.253.2 on GigabitEthernet1/1 from FULL to DOWN, Neighbor Down: Interface down or detached
Jul 2 13:38:07.063 BJT: %PM-4-ERR_DISABLE: link-flap error detected on Gi1/1, putting Gi1/1 in err-disable state
Jul 2 13:38:35.347 BJT: %PM-4-ERR_RECOVER: Attempting to recover from link-flap err-disable state on Gi1/1
Jul 2 13:39:18.085 BJT: %OSPF-5-ADJCHG: Process 100, Nbr 172.19.253.2 on GigabitEthernet1/1 from LOADING to FULL, Loading Done
Jul 2 13:39:55.432 BJT: %OSPF-5-ADJCHG: Process 100, Nbr 172.19.253.2 on GigabitEthernet1/1 from FULL to DOWN, Neighbor Down: Interface down or detached
Jul 2 13:40:13.488 BJT: %OSPF-5-ADJCHG: Process 100, Nbr 172.19.253.2 on GigabitEthernet1/1 from 2WAY to DOWN, Neighbor Down: Interface down or detached
Jul 2 13:40:56.075 BJT: %OSPF-5-ADJCHG: Process 100, Nbr 172.19.253.2 on GigabitEthernet1/1 from LOADING to FULL, Loading Done
Jul 2 13:44:03.098 BJT: %OSPF-5-ADJCHG: Process 100, Nbr 172.19.253.2 on GigabitEthernet1/1 from FULL to DOWN, Neighbor Down: Interface down or detached
Jul 2 13:44:42.121 BJT: %OSPF-5-ADJCHG: Process 100, Nbr 172.19.253.2 on GigabitEthernet1/1 from 2WAY to DOWN, Neighbor Down: Interface down or detached
Jul 2 13:45:06.340 BJT: %PM-4-ERR_DISABLE: link-flap error detected on Gi1/1, putting Gi1/1 in err-disable state
Jul 2 13:45:32.083 BJT: %PM-4-ERR_RECOVER: Attempting to recover from link-flap err-disable state on Gi1/1
Jul 2 13:46:26.066 BJT: %OSPF-5-ADJCHG: Process 100, Nbr 172.19.253.2 on GigabitEthernet1/1 from LOADING to FULL, Loading Done
//
no file verify auto
errdisable recovery cause udld
errdisable recovery cause bpduguard
errdisable recovery cause security-violation
errdisable recovery cause channel-misconfig
errdisable recovery cause pagp-flap
errdisable recovery cause dtp-flap
errdisable recovery cause link-flap
errdisable recovery cause gbic-invalid
errdisable recovery cause l2ptguard
errdisable recovery cause psecure-violation
errdisable recovery cause dhcp-rate-limit
errdisable recovery cause unicast-flood
errdisable recovery cause vmps
errdisable recovery cause storm-control
errdisable recovery cause arp-inspection
errdisable recovery interval 30
spanning-tree mode pvst
spanning-tree extend system-id
power redundancy-mode redundant
!
!
!
**********************************
防止err-disable,使用以下命令：errdisable recovery cause udld
errdisable recovery cause bpduguard
errdisable recovery cause security-violation
errdisable recovery cause channel-misconfig
errdisable recovery cause pagp-flap
errdisable recovery cause dtp-flap
errdisable recovery cause link-flap
errdisable recovery cause psecure-violation
errdisable recovery cause vmps
errdisable recovery cause gbic-invalid
errdisable recovery cause loopback
errdisable recovery interval 30
