07-01-2015 09:03 AM
Hi community,
a customer of us just removed one of his two ports used to link a FEX to it Nexus5K. Because he wanted to change the cabling a bit.
His idea was, since he has two "uplinks" to the bigger switch, all should be fine.
For sure, a FEX is no switch, and the uplink was no PortChannel, like he has this in his other switches configured on his campus.
But what made me wonder too...the FEX completely went offline, and sort of rebooted.
2015 Jul 1 14:27:17 NEXUS1 %ETHPORT-5-IF_DOWN_LINK_FAILURE: Interface Ethernet1/29 is down (Link failure)
2015 Jul 1 14:27:17 NEXUS1 last message repeated 1 time
2015 Jul 1 14:27:17 NEXUS1 %FEX-5-PINNING_CHANGED: Fex 101 pinning information is changed
2015 Jul 1 14:27:17 NEXUS1 %FEX-5-FEX_PORT_STATUS_NOTI: Uplink-ID 7 of Fex 101 that is connected with Ethernet1/29 changed its status from Active to Disconnected
2015 Jul 1 14:27:17 NEXUS1 %ETHPORT-5-IF_DOWN_NONE: Interface Ethernet101/1/9 is down (None)
2015 Jul 1 14:27:17 NEXUS1 last message repeated 1 time
2015 Jul 1 14:27:17 NEXUS1 %ETHPORT-5-IF_DOWN_NONE: Interface Ethernet101/1/10 is down (None)
2015 Jul 1 14:27:17 NEXUS1 last message repeated 1 time
2015 Jul 1 14:27:17 NEXUS1 %ETHPORT-5-IF_DOWN_NONE: Interface Ethernet101/1/1 is down (None)
2015 Jul 1 14:27:17 NEXUS1 last message repeated 1 time
2015 Jul 1 14:27:17 NEXUS1 %ETHPORT-5-IF_DOWN_NONE: Interface Ethernet101/1/2 is down (None)
2015 Jul 1 14:27:17 NEXUS1 last message repeated 1 time
2015 Jul 1 14:27:17 NEXUS1 %ETH_PORT_CHANNEL-5-PORT_DOWN: port-channel16: Ethernet101/1/11 is down
2015 Jul 1 14:27:17 NEXUS1 %ETH_PORT_CHANNEL-5-PORT_DOWN: port-channel16: port-channel16 is down
2015 Jul 1 14:27:17 NEXUS1 %ETH_PORT_CHANNEL-5-PORT_DOWN: port-channel17: Ethernet101/1/12 is down
2015 Jul 1 14:27:17 NEXUS1 %ETH_PORT_CHANNEL-5-PORT_DOWN: port-channel17: port-channel17 is down
2015 Jul 1 14:27:17 NEXUS1 %ETH_PORT_CHANNEL-5-FOP_CHANGED: port-channel16: first operational port changed from Ethernet101/1/11 to none
2015 Jul 1 14:27:17 NEXUS1 %ETH_PORT_CHANNEL-5-FOP_CHANGED: port-channel17: first operational port changed from Ethernet101/1/12 to none
2015 Jul 1 14:27:17 NEXUS1 %ETHPORT-5-IF_DOWN_PORT_CHANNEL_MEMBERS_DOWN: Interface port-channel16 is down (No operational members)
2015 Jul 1 14:27:17 NEXUS1 %ETHPORT-5-IF_DOWN_PORT_CHANNEL_MEMBERS_DOWN: Interface port-channel17 is down (No operational members)
2015 Jul 1 14:27:18 NEXUS1 %ETHPORT-5-IF_DOWN_NONE: Interface Ethernet101/1/11 is down (None)
2015 Jul 1 14:27:18 NEXUS1 %ETHPORT-5-IF_DOWN_PORT_CHANNEL_MEMBERS_DOWN: Interface port-channel16 is down (No operational members)
2015 Jul 1 14:27:18 NEXUS1 %ETHPORT-5-IF_DOWN_NONE: Interface Ethernet101/1/12 is down (None)
2015 Jul 1 14:27:18 NEXUS1 %ETHPORT-5-IF_DOWN_PORT_CHANNEL_MEMBERS_DOWN: Interface port-channel17 is down (No operational members)
2015 Jul 1 14:27:18 NEXUS1 %ETHPORT-5-IF_DOWN_NONE: Interface Ethernet101/1/32 is down (None)
2015 Jul 1 14:27:18 NEXUS1 %ETHPORT-5-IF_DOWN_NONE: Interface Ethernet101/1/31 is down (None)
2015 Jul 1 14:27:18 NEXUS1 %ETHPORT-5-IF_DOWN_NONE: Interface Ethernet101/1/30 is down (None)
2015 Jul 1 14:27:18 NEXUS1 %ETHPORT-5-IF_DOWN_NONE: Interface Ethernet101/1/29 is down (None)
2015 Jul 1 14:27:18 NEXUS1 %ETHPORT-5-IF_DOWN_NONE: Interface Ethernet101/1/28 is down (None)
2015 Jul 1 14:27:18 NEXUS1 %ETHPORT-5-IF_DOWN_NONE: Interface Ethernet101/1/27 is down (None)
2015 Jul 1 14:27:18 NEXUS1 %ETHPORT-5-IF_DOWN_NONE: Interface Ethernet101/1/26 is down (None)
2015 Jul 1 14:27:18 NEXUS1 %ETHPORT-5-IF_DOWN_NONE: Interface Ethernet101/1/25 is down (None)
2015 Jul 1 14:27:18 NEXUS1 %ETHPORT-5-IF_DOWN_NONE: Interface Ethernet101/1/24 is down (None)
2015 Jul 1 14:27:18 NEXUS1 %ETHPORT-5-IF_DOWN_NONE: Interface Ethernet101/1/23 is down (None)
2015 Jul 1 14:27:18 NEXUS1 %ETHPORT-5-IF_DOWN_NONE: Interface Ethernet101/1/22 is down (None)
2015 Jul 1 14:27:18 NEXUS1 %ETHPORT-5-IF_DOWN_NONE: Interface Ethernet101/1/21 is down (None)
2015 Jul 1 14:27:18 NEXUS1 %ETHPORT-5-IF_DOWN_NONE: Interface Ethernet101/1/20 is down (None)
2015 Jul 1 14:27:18 NEXUS1 %ETHPORT-5-IF_DOWN_NONE: Interface Ethernet101/1/19 is down (None)
2015 Jul 1 14:27:18 NEXUS1 %ETHPORT-5-IF_DOWN_NONE: Interface Ethernet101/1/18 is down (None)
2015 Jul 1 14:27:18 NEXUS1 %ETHPORT-5-IF_DOWN_NONE: Interface Ethernet101/1/17 is down (None)
2015 Jul 1 14:27:18 NEXUS1 %ETHPORT-5-IF_DOWN_NONE: Interface Ethernet101/1/9 is down (None)
2015 Jul 1 14:27:18 NEXUS1 %ETHPORT-5-IF_DOWN_NONE: Interface Ethernet101/1/10 is down (None)
2015 Jul 1 14:27:18 NEXUS1 %ETHPORT-5-IF_DOWN_NONE: Interface Ethernet101/1/1 is down (None)
2015 Jul 1 14:27:18 NEXUS1 %ETHPORT-5-IF_DOWN_NONE: Interface Ethernet101/1/2 is down (None)
2015 Jul 1 14:27:18 NEXUS1 %ETHPORT-5-IF_DOWN_NONE: Interface Ethernet101/1/3 is down (None)
2015 Jul 1 14:27:18 NEXUS1 %ETHPORT-5-IF_DOWN_NONE: Interface Ethernet101/1/4 is down (None)
2015 Jul 1 14:27:18 NEXUS1 %ETHPORT-5-IF_DOWN_NONE: Interface Ethernet101/1/5 is down (None)
2015 Jul 1 14:27:18 NEXUS1 %ETHPORT-5-IF_DOWN_NONE: Interface Ethernet101/1/6 is down (None)
2015 Jul 1 14:27:18 NEXUS1 %ETHPORT-5-IF_DOWN_NONE: Interface Ethernet101/1/7 is down (None)
2015 Jul 1 14:27:18 NEXUS1 %ETHPORT-5-IF_DOWN_NONE: Interface Ethernet101/1/8 is down (None)
2015 Jul 1 14:27:18 NEXUS1 %ETHPORT-5-IF_DOWN_NONE: Interface Ethernet101/1/16 is down (None)
2015 Jul 1 14:27:18 NEXUS1 %ETHPORT-5-IF_DOWN_NONE: Interface Ethernet101/1/15 is down (None)
2015 Jul 1 14:27:18 NEXUS1 %ETHPORT-5-IF_DOWN_NONE: Interface Ethernet101/1/14 is down (None)
2015 Jul 1 14:27:18 NEXUS1 %ETHPORT-5-IF_DOWN_NONE: Interface Ethernet101/1/13 is down (None)
2015 Jul 1 14:27:18 NEXUS1 %ETHPORT-5-IF_DOWN_NONE: Interface Ethernet101/1/11 is down (None)
2015 Jul 1 14:27:18 NEXUS1 %ETHPORT-5-IF_DOWN_NONE: Interface Ethernet101/1/12 is down (None)
2015 Jul 1 14:27:19 NEXUS1 %FEX-5-FEX_PORT_STATUS_NOTI: Uplink-ID 7 of Fex 101 that is connected with Ethernet1/29 changed its status from Disconnected to Configured
2015 Jul 1 14:27:51 NEXUS1 %ETHPORT-5-SPEED: Interface Ethernet1/29, operational speed changed to 10 Gbps
2015 Jul 1 14:27:51 NEXUS1 %ETHPORT-5-IF_DUPLEX: Interface Ethernet1/29, operational duplex mode changed to Full
2015 Jul 1 14:27:51 NEXUS1 %ETHPORT-5-IF_RX_FLOW_CONTROL: Interface Ethernet1/29, operational Receive Flow Control state changed to off
2015 Jul 1 14:27:51 NEXUS1 %ETHPORT-5-IF_TX_FLOW_CONTROL: Interface Ethernet1/29, operational Transmit Flow Control state changed to off
2015 Jul 1 14:27:51 NEXUS1 %FEX-5-FEX_PORT_STATUS_NOTI: Uplink-ID 0 of Fex 101 that is connected with Ethernet1/29 changed its status from Configured to Fabric Up
2015 Jul 1 14:27:51 NEXUS1 %FEX-5-FEX_PORT_STATUS_NOTI: Uplink-ID 7 of Fex 101 that is connected with Ethernet1/29 changed its status from Fabric Up to Connecting
2015 Jul 1 14:27:51 NEXUS1 %ETHPORT-5-IF_UP: Interface Ethernet1/29 is up in mode Fex Fabric
2015 Jul 1 14:27:51 NEXUS1 %FEX-5-PINNING_CHANGED: Fex 101 pinning information is changed
2015 Jul 1 14:27:51 NEXUS1 %FEX-5-FEX_PORT_STATUS_NOTI: Uplink-ID 7 of Fex 101 that is connected with Ethernet1/29 changed its status from Connecting to Active
2015 Jul 1 14:27:58 NEXUS1 %ETHPORT-5-SPEED: Interface Ethernet101/1/2, operational speed changed to 1 Gbps
2015 Jul 1 14:27:58 NEXUS1 %ETHPORT-5-IF_DUPLEX: Interface Ethernet101/1/2, operational duplex mode changed to Full
2015 Jul 1 14:27:58 NEXUS1 %ETHPORT-5-IF_RX_FLOW_CONTROL: Interface Ethernet101/1/2, operational Receive Flow Control state changed to off
2015 Jul 1 14:27:58 NEXUS1 %ETHPORT-5-IF_TX_FLOW_CONTROL: Interface Ethernet101/1/2, operational Transmit Flow Control state changed to on
2015 Jul 1 14:27:58 NEXUS1 %ETHPORT-5-IF_UP: Interface Ethernet101/1/2 is up in mode access
2015 Jul 1 14:27:58 NEXUS1 %ETHPORT-5-SPEED: Interface Ethernet101/1/12, operational speed changed to 10 Gbps
2015 Jul 1 14:27:58 NEXUS1 %ETHPORT-5-IF_DUPLEX: Interface Ethernet101/1/12, operational duplex mode changed to Full
2015 Jul 1 14:27:58 NEXUS1 %ETHPORT-5-IF_RX_FLOW_CONTROL: Interface Ethernet101/1/12, operational Receive Flow Control state changed to off
2015 Jul 1 14:27:58 NEXUS1 %ETHPORT-5-IF_TX_FLOW_CONTROL: Interface Ethernet101/1/12, operational Transmit Flow Control state changed to on
2015 Jul 1 14:27:58 NEXUS1 %ETHPORT-5-SPEED: Interface port-channel17, operational speed changed to 10 Gbps
2015 Jul 1 14:27:58 NEXUS1 %ETHPORT-5-IF_DUPLEX: Interface port-channel17, operational duplex mode changed to Full
2015 Jul 1 14:27:58 NEXUS1 %ETHPORT-5-IF_RX_FLOW_CONTROL: Interface port-channel17, operational Receive Flow Control state changed to off
2015 Jul 1 14:27:58 NEXUS1 %ETHPORT-5-IF_TX_FLOW_CONTROL: Interface port-channel17, operational Transmit Flow Control state changed to on
2015 Jul 1 14:27:58 NEXUS1 %ETHPORT-5-SPEED: Interface Ethernet101/1/11, operational speed changed to 10 Gbps
2015 Jul 1 14:27:58 NEXUS1 %ETHPORT-5-IF_DUPLEX: Interface Ethernet101/1/11, operational duplex mode changed to Full
2015 Jul 1 14:27:58 NEXUS1 %ETHPORT-5-IF_RX_FLOW_CONTROL: Interface Ethernet101/1/11, operational Receive Flow Control state changed to off
2015 Jul 1 14:27:58 NEXUS1 %ETHPORT-5-IF_TX_FLOW_CONTROL: Interface Ethernet101/1/11, operational Transmit Flow Control state changed to on
2015 Jul 1 14:27:58 NEXUS1 %ETHPORT-5-SPEED: Interface port-channel16, operational speed changed to 10 Gbps
2015 Jul 1 14:27:58 NEXUS1 %ETHPORT-5-IF_DUPLEX: Interface port-channel16, operational duplex mode changed to Full
2015 Jul 1 14:27:58 NEXUS1 %ETHPORT-5-IF_RX_FLOW_CONTROL: Interface port-channel16, operational Receive Flow Control state changed to off
2015 Jul 1 14:27:58 NEXUS1 %ETHPORT-5-IF_TX_FLOW_CONTROL: Interface port-channel16, operational Transmit Flow Control state changed to on
2015 Jul 1 14:27:59 NEXUS1 %ETHPORT-5-SPEED: Interface Ethernet101/1/10, operational speed changed to 10 Gbps
2015 Jul 1 14:27:59 NEXUS1 %ETHPORT-5-IF_DUPLEX: Interface Ethernet101/1/10, operational duplex mode changed to Full
2015 Jul 1 14:27:59 NEXUS1 %ETHPORT-5-IF_RX_FLOW_CONTROL: Interface Ethernet101/1/10, operational Receive Flow Control state changed to off
2015 Jul 1 14:27:59 NEXUS1 %ETHPORT-5-IF_TX_FLOW_CONTROL: Interface Ethernet101/1/10, operational Transmit Flow Control state changed to on
2015 Jul 1 14:27:59 NEXUS1 %ETHPORT-5-IF_UP: Interface Ethernet101/1/10 is up in mode access
2015 Jul 1 14:27:59 NEXUS1 %ETHPORT-5-IF_DOWN_LINK_FAILURE: Interface Ethernet101/1/10 is down (Link failure)
2015 Jul 1 14:27:59 NEXUS1 last message repeated 1 time
2015 Jul 1 14:27:59 NEXUS1 %ETHPORT-5-SPEED: Interface Ethernet101/1/9, operational speed changed to 10 Gbps
2015 Jul 1 14:27:59 NEXUS1 %ETHPORT-5-IF_DUPLEX: Interface Ethernet101/1/9, operational duplex mode changed to Full
2015 Jul 1 14:27:59 NEXUS1 %ETHPORT-5-IF_RX_FLOW_CONTROL: Interface Ethernet101/1/9, operational Receive Flow Control state changed to off
2015 Jul 1 14:27:59 NEXUS1 %ETHPORT-5-IF_TX_FLOW_CONTROL: Interface Ethernet101/1/9, operational Transmit Flow Control state changed to on
2015 Jul 1 14:27:59 NEXUS1 %ETHPORT-5-IF_UP: Interface Ethernet101/1/9 is up in mode access
2015 Jul 1 14:27:59 NEXUS1 %ETHPORT-5-SPEED: Interface Ethernet101/1/1, operational speed changed to 1 Gbps
2015 Jul 1 14:27:59 NEXUS1 %ETHPORT-5-IF_DUPLEX: Interface Ethernet101/1/1, operational duplex mode changed to Full
2015 Jul 1 14:27:59 NEXUS1 %ETHPORT-5-IF_RX_FLOW_CONTROL: Interface Ethernet101/1/1, operational Receive Flow Control state changed to off
2015 Jul 1 14:27:59 NEXUS1 %ETHPORT-5-IF_TX_FLOW_CONTROL: Interface Ethernet101/1/1, operational Transmit Flow Control state changed to on
2015 Jul 1 14:27:59 NEXUS1 %ETHPORT-5-IF_UP: Interface Ethernet101/1/1 is up in mode access
2015 Jul 1 14:28:02 NEXUS1 %ETH_PORT_CHANNEL-5-PORT_UP: port-channel16: Ethernet101/1/11 is up
2015 Jul 1 14:28:02 NEXUS1 %ETH_PORT_CHANNEL-5-FOP_CHANGED: port-channel16: first operational port changed from none to Ethernet101/1/11
2015 Jul 1 14:28:02 NEXUS1 %ETHPORT-5-IF_UP: Interface Ethernet101/1/11 is up in mode trunk
2015 Jul 1 14:28:02 NEXUS1 %ETHPORT-5-IF_UP: Interface port-channel16 is up in mode trunk
2015 Jul 1 14:28:03 NEXUS1 %ETH_PORT_CHANNEL-5-PORT_UP: port-channel17: Ethernet101/1/12 is up
2015 Jul 1 14:28:03 NEXUS1 %ETH_PORT_CHANNEL-5-FOP_CHANGED: port-channel17: first operational port changed from none to Ethernet101/1/12
2015 Jul 1 14:28:03 NEXUS1 %ETHPORT-5-IF_UP: Interface Ethernet101/1/12 is up in mode trunk
2015 Jul 1 14:28:03 NEXUS1 %ETHPORT-5-IF_UP: Interface port-channel17 is up in mode trunk
2015 Jul 1 14:28:04 NEXUS1 %ETHPORT-5-SPEED: Interface Ethernet101/1/10, operational speed changed to 10 Gbps
2015 Jul 1 14:28:04 NEXUS1 %ETHPORT-5-IF_DUPLEX: Interface Ethernet101/1/10, operational duplex mode changed to Full
2015 Jul 1 14:28:04 NEXUS1 %ETHPORT-5-IF_RX_FLOW_CONTROL: Interface Ethernet101/1/10, operational Receive Flow Control state changed to off
2015 Jul 1 14:28:04 NEXUS1 %ETHPORT-5-IF_TX_FLOW_CONTROL: Interface Ethernet101/1/10, operational Transmit Flow Control state changed to on
2015 Jul 1 14:28:04 NEXUS1 %ETHPORT-5-IF_UP: Interface Ethernet101/1/10 is up in mode access
# # # # # # # # # # # # # # #
Configuration of the N5K:
interface Ethernet1/29
description UPLINK 1 FEX 101
switchport mode fex-fabric
fex associate 101
interface Ethernet1/30
description UPLINK 2 FEX 101
switchport mode fex-fabric
fex associate 101
NEXUS1# sh fex 101
FEX: 101 Description: FEX0101 state: Online
FEX version: 7.0(5)N1(1) [Switch version: 7.0(5)N1(1)]
Extender Serial: xxxxxxxxxxxx
Extender Model: N2K-C2232TM-E-10GE, Part No: 73-14645-04
Pinning-mode: static Max-links: 1
Fabric port for control traffic: Eth1/30
FCoE Admin: false
FCoE Oper: true
FCoE FEX AA Configured: false
Fabric interface state:
Eth1/29 - Interface Up. State: Active
Eth1/30 - Interface Up. State: Active
NEXUS1# sh fex
FEX FEX FEX FEX Fex
Number Description State Model Serial
------------------------------------------------------------------------
101 FEX0101 Online N2K-C2232TM-E-10GE xxxxxxxxxxx
Any ideas how i could achieve redundancy between the two uplinks to the FEX so that such things can happen without this issue?
Many thanks in advance,
Andreas
Discover and save your favorite ideas. Come back to expert answers, step-by-step guides, recent topics, and more.
New here? Get started with these tips. How to use Community New member guide