Buy or Renew
Buy or Renew
cancel
Turn on suggestions
Auto-suggest helps you quickly narrow down your search results by suggesting possible matches as you type.
Showing results for 
Search instead for 
Did you mean: 
cancel
Start a conversation
2887
Views
0
Helpful
2
Replies

ASR1002 - taken out of port-channel1

manuel.piacenti1
Level 1
Level 1

‎05-28-2021 06:01 AM

Hello,

 

we have a asr1002 connected with a lacp (mode active) with its 2 tengiga ports on a virtual port-channel between 2 nexus3k.

The asr have started randomly throwing out these errors:

*May 26 04:44:41: TenGigabitEthernet0/3/0 taken out of port-channel1
*May 26 04:45:58: %EC-5-MINLINKS_NOTMET: Port-channel Port-channel1 is down bundled ports (0) doesn't meet min-links
*May 26 04:45:58: TenGigabitEthernet0/1/0 taken out of port-channel1
*May 26 04:46:00: %LINK-3-UPDOWN: Interface Port-channel1, changed state to down
[ ... bunch of bgp and ospf notifications ...]

The traffic stops completely and in this situation both the nexus3k reports both the tengiga interfaces and port-channel as down, on the ASR instead the port-channel is down but both the tengiga are up.

 

The problem is that those ports aren't automatically added back to the port-channel on the ASR until we manually enter in config mode and do a shut/no shut of both the interfaces.

*May 26 05:06:25: TenGigabitEthernet0/3/0 added as member-1 to port-channel1
*May 26 05:06:25: %IOSXE_SPA-6-UPDOWN: Interface TenGigabitEthernet0/1/0, link down due to local fault
*May 26 05:06:25: %IOSXE_SPA-6-UPDOWN: Interface TenGigabitEthernet0/1/0, link down due to remote fault
*May 26 05:06:25: %IOSXE_SPA-6-UPDOWN: Interface TenGigabitEthernet0/1/0, link down due to remote fault
*May 26 05:06:25: %IOSXE_SPA-6-UPDOWN: Interface TenGigabitEthernet0/1/0, link down due to remote fault
*May 26 05:06:25: %IOSXE_SPA-6-UPDOWN: Interface TenGigabitEthernet0/1/0, link down due to remote fault
*May 26 05:06:27: %LINK-3-UPDOWN: Interface Port-channel1, changed state to up
*May 26 05:06:28: %LINEPROTO-5-UPDOWN: Line protocol on Interface Port-channel1, changed state to up
*May 26 05:06:29: TenGigabitEthernet0/1/0 added as member-2 to port-channel1
[ ... bunch of bgp and ospf notifications ...]
*May 26 05:07:54: %IOSXE_SPA-6-UPDOWN: Interface TenGigabitEthernet0/1/0, link down due to remote fault
*May 26 05:07:54: %IOSXE_SPA-6-UPDOWN: Interface TenGigabitEthernet0/1/0, link down due to remote fault
*May 26 05:07:57: TenGigabitEthernet0/1/0 taken out of port-channel1
*May 26 05:07:59: TenGigabitEthernet0/1/0 added as member-2 to port-channel1

 Problem started recently, never had this issue before, no recent configuration changes, no CPU overload.

 

Have you experienced something similar?

Thanks

Labels:
0 Helpful
2 Replies 2

balaji.bandi
Hall of Fame
Hall of Fame

‎05-28-2021 06:06 AM - edited ‎05-28-2021 06:06 AM

is the Physical cables checked all ok ?  what is the nexus side log ? can you post both the side config ?

 

%EC-5-MINLINKS_NOTMET: Port-channel Port-channel1 is down bundled ports (0) doesn't meet min-links

not sure what is the code running here - some bug reported - worth checking :

 

https://bst.cloudapps.cisco.com/bugsearch/bug/CSCuz52041/?rfs=iqvred

BB

***** Rate All Helpful Responses *****

How to Ask The Cisco Community for Help

0 Helpful

manuel.piacenti1
Level 1
Level 1
In response to balaji.bandi

‎05-28-2021 06:59 AM - edited ‎06-10-2021 02:33 AM

Hi,

 

thanks for your help.

 

The two physical links are ok, they follow 2 completely different paths so there's no chance both have the same problem in the same timeframe.

These are the nexus logs (don't mind the timestamp, they are not synchronized and circa half an hour off):

 

nx1

 

2021 May 26 05:14:26 SWITCH1 %ETH_PORT_CHANNEL-5-PORT_DOWN: port-channel14: Ethernet1/6 is down
2021 May 26 05:14:26 SWITCH1 %ETH_PORT_CHANNEL-5-PORT_DOWN: port-channel14: port-channel14 is down
2021 May 26 05:14:26 SWITCH1 %ETH_PORT_CHANNEL-5-FOP_CHANGED: port-channel14: first operational port changed from Ethernet1/6 to none
2021 May 26 05:14:27 SWITCH1 %ETHPORT-5-IF_DOWN_PORT_CHANNEL_MEMBERS_DOWN: Interface port-channel14 is down (No operational members)
2021 May 26 05:14:27 SWITCH1 %ETHPORT-5-IF_DOWN_INITIALIZING: Interface Ethernet1/6 is down (Initializing)
2021 May 26 05:14:27 SWITCH1 %ETHPORT-5-IF_DOWN_PORT_CHANNEL_MEMBERS_DOWN: Interface port-channel14 is down (No operational members)
2021 May 26 05:14:27 SWITCH1 %ETHPORT-5-SPEED: Interface port-channel14, operational speed changed to 10 Gbps
2021 May 26 05:14:27 SWITCH1 %ETHPORT-5-IF_DUPLEX: Interface port-channel14, operational duplex mode changed to Full
2021 May 26 05:14:27 SWITCH1 %ETHPORT-5-IF_RX_FLOW_CONTROL: Interface port-channel14, operational Receive Flow Control state changed to off
2021 May 26 05:14:27 SWITCH1 %ETHPORT-5-IF_TX_FLOW_CONTROL: Interface port-channel14, operational Transmit Flow Control state changed to off
2021 May 26 05:14:52 SWITCH1 %ETH_PORT_CHANNEL-5-PORT_HOT_STANDBY: port-channel14: Ethernet1/6 goes to hot-standby
2021 May 26 05:34:54 SWITCH1 %ETHPORT-5-IF_DOWN_INITIALIZING: Interface port-channel14 is down (Initializing)
2021 May 26 05:34:54 SWITCH1 %ETH_PORT_CHANNEL-5-PORT_HOT_STANDBY_DOWN: port-channel14: hot-standby port Ethernet1/6 is down
2021 May 26 05:34:54 SWITCH1 %ETHPORT-5-IF_DOWN_PORT_CHANNEL_MEMBERS_DOWN: Interface port-channel14 is down (No operational members)
2021 May 26 05:34:56 SWITCH1 %ETHPORT-5-SPEED: Interface Ethernet1/6, operational speed changed to 10 Gbps
2021 May 26 05:34:56 SWITCH1 %ETHPORT-5-IF_DUPLEX: Interface Ethernet1/6, operational duplex mode changed to Full
2021 May 26 05:34:56 SWITCH1 %ETHPORT-5-IF_RX_FLOW_CONTROL: Interface Ethernet1/6, operational Receive Flow Control state changed to off
2021 May 26 05:34:56 SWITCH1 %ETHPORT-5-IF_TX_FLOW_CONTROL: Interface Ethernet1/6, operational Transmit Flow Control state changed to off
2021 May 26 05:34:56 SWITCH1 %ETHPORT-5-SPEED: Interface port-channel14, operational speed changed to 10 Gbps
2021 May 26 05:34:56 SWITCH1 %ETHPORT-5-IF_DUPLEX: Interface port-channel14, operational duplex mode changed to Full
2021 May 26 05:34:56 SWITCH1 %ETHPORT-5-IF_RX_FLOW_CONTROL: Interface port-channel14, operational Receive Flow Control state changed to off
2021 May 26 05:34:56 SWITCH1 %ETHPORT-5-IF_TX_FLOW_CONTROL: Interface port-channel14, operational Transmit Flow Control state changed to off
2021 May 26 05:34:58 SWITCH1 %ETH_PORT_CHANNEL-5-PORT_UP: port-channel14: Ethernet1/6 is up
2021 May 26 05:34:58 SWITCH1 %ETH_PORT_CHANNEL-5-FOP_CHANGED: port-channel14: first operational port changed from none to Ethernet1/6
2021 May 26 05:34:58 SWITCH1 %ETHPORT-5-IF_UP: Interface Ethernet1/6 is up in mode trunk
2021 May 26 05:34:58 SWITCH1 %ETHPORT-5-IF_UP: Interface port-channel14 is up in mode trunk
2021 May 26 05:36:22 SWITCH1 %ETH_PORT_CHANNEL-5-FOP_CHANGED: port-channel14: first operational port changed from Ethernet1/6 to none
2021 May 26 05:36:22 SWITCH1 %ETH_PORT_CHANNEL-5-PORT_DOWN: port-channel14: Ethernet1/6 is down
2021 May 26 05:36:22 SWITCH1 %ETH_PORT_CHANNEL-5-PORT_DOWN: port-channel14: port-channel14 is down
2021 May 26 05:36:22 SWITCH1 %ETHPORT-5-IF_DOWN_PORT_CHANNEL_MEMBERS_DOWN: Interface port-channel14 is down (No operational members)
2021 May 26 05:36:22 SWITCH1 %ETHPORT-5-IF_DOWN_LINK_FAILURE: Interface Ethernet1/6 is down (Link failure)
2021 May 26 05:36:22 SWITCH1 %ETHPORT-5-IF_DOWN_PORT_CHANNEL_MEMBERS_DOWN: Interface port-channel14 is down (No operational members)
2021 May 26 05:36:24 SWITCH1 %ETHPORT-5-SPEED: Interface Ethernet1/6, operational speed changed to 10 Gbps
2021 May 26 05:36:24 SWITCH1 %ETHPORT-5-IF_DUPLEX: Interface Ethernet1/6, operational duplex mode changed to Full
2021 May 26 05:36:24 SWITCH1 %ETHPORT-5-IF_RX_FLOW_CONTROL: Interface Ethernet1/6, operational Receive Flow Control state changed to off
2021 May 26 05:36:24 SWITCH1 %ETHPORT-5-IF_TX_FLOW_CONTROL: Interface Ethernet1/6, operational Transmit Flow Control state changed to off
2021 May 26 05:36:24 SWITCH1 %ETHPORT-5-SPEED: Interface port-channel14, operational speed changed to 10 Gbps
2021 May 26 05:36:24 SWITCH1 %ETHPORT-5-IF_DUPLEX: Interface port-channel14, operational duplex mode changed to Full
2021 May 26 05:36:24 SWITCH1 %ETHPORT-5-IF_RX_FLOW_CONTROL: Interface port-channel14, operational Receive Flow Control state changed to off
2021 May 26 05:36:24 SWITCH1 %ETHPORT-5-IF_TX_FLOW_CONTROL: Interface port-channel14, operational Transmit Flow Control state changed to off
2021 May 26 05:36:28 SWITCH1 %ETH_PORT_CHANNEL-5-PORT_UP: port-channel14: Ethernet1/6 is up
2021 May 26 05:36:28 SWITCH1 %ETH_PORT_CHANNEL-5-FOP_CHANGED: port-channel14: first operational port changed from none to Ethernet1/6
2021 May 26 05:36:28 SWITCH1 %ETHPORT-5-IF_UP: Interface Ethernet1/6 is up in mode trunk
2021 May 26 05:36:28 SWITCH1 %ETHPORT-5-IF_UP: Interface port-channel14 is up in mode trunk

nx2

 

2021 May 26 05:10:50 SWITCH2 %ETH_PORT_CHANNEL-5-PORT_DOWN: port-channel14: Ethernet1/6 is down
2021 May 26 05:10:50 SWITCH2 %ETH_PORT_CHANNEL-5-PORT_DOWN: port-channel14: port-channel14 is down
2021 May 26 05:10:50 SWITCH2 %ETH_PORT_CHANNEL-5-FOP_CHANGED: port-channel14: first operational port changed from Ethernet1/6 to none
2021 May 26 05:10:50 SWITCH2 %ETHPORT-5-IF_DOWN_PORT_CHANNEL_MEMBERS_DOWN: Interface port-channel14 is down (No operational members)
2021 May 26 05:10:51 SWITCH2 %ETHPORT-5-IF_DOWN_INITIALIZING: Interface Ethernet1/6 is down (Initializing)
2021 May 26 05:10:51 SWITCH2 %ETHPORT-5-IF_DOWN_PORT_CHANNEL_MEMBERS_DOWN: Interface port-channel14 is down (No operational members)
2021 May 26 05:10:51 SWITCH2 %ETHPORT-5-SPEED: Interface port-channel14, operational speed changed to 10 Gbps
2021 May 26 05:10:51 SWITCH2 %ETHPORT-5-IF_DUPLEX: Interface port-channel14, operational duplex mode changed to Full
2021 May 26 05:10:51 SWITCH2 %ETHPORT-5-IF_RX_FLOW_CONTROL: Interface port-channel14, operational Receive Flow Control state changed to off
2021 May 26 05:10:51 SWITCH2 %ETHPORT-5-IF_TX_FLOW_CONTROL: Interface port-channel14, operational Transmit Flow Control state changed to off
2021 May 26 05:11:16 SWITCH2 %ETH_PORT_CHANNEL-5-PORT_HOT_STANDBY: port-channel14: Ethernet1/6 goes to hot-standby
2021 May 26 05:32:29 SWITCH2 %ETH_PORT_CHANNEL-5-PORT_HOT_STANDBY_DOWN: port-channel14: hot-standby port Ethernet1/6 is down
2021 May 26 05:32:29 SWITCH2 %ETHPORT-5-SPEED: Interface port-channel14, operational speed changed to 10 Gbps
2021 May 26 05:32:29 SWITCH2 %ETHPORT-5-IF_DUPLEX: Interface port-channel14, operational duplex mode changed to Full
2021 May 26 05:32:29 SWITCH2 %ETHPORT-5-IF_RX_FLOW_CONTROL: Interface port-channel14, operational Receive Flow Control state changed to off
2021 May 26 05:32:29 SWITCH2 %ETHPORT-5-IF_TX_FLOW_CONTROL: Interface port-channel14, operational Transmit Flow Control state changed to off
2021 May 26 05:32:35 SWITCH2 %ETH_PORT_CHANNEL-5-PORT_UP: port-channel14: Ethernet1/6 is up
2021 May 26 05:32:35 SWITCH2 %ETH_PORT_CHANNEL-5-FOP_CHANGED: port-channel14: first operational port changed from none to Ethernet1/6
2021 May 26 05:32:35 SWITCH2 %ETHPORT-5-IF_UP: Interface Ethernet1/6 is up in mode trunk
2021 May 26 05:32:35 SWITCH2 %ETHPORT-5-IF_UP: Interface port-channel14 is up in mode trunk
0 Helpful
Customers Also Viewed These Support Documents