port-channel on n2k fex interfaces

[Matt Craig]

Hi,

I saw the documentation on Single Homed Fabric Extender vPC Topology:

http://www.cisco.com/en/US/docs/switches/datacenter/nexus5000/sw/layer2/502_n1_1/Cisco_n5k_layer2_config_gd_rel_502_N1_1_chapter8.html#concept_6126947023044217647

I was wondering if I could connect a switch to the FEXs with a vPC.  Just replace the server in the diagram with a 3750.

I have tried this with the following configuration:

on switch NX-5000-1:

interface port-channel10
  vpc 10

interface Ethernet100/1/2

  desc Feed to 3750 - GigabitEthernet1/0/1
  channel-group 10 mode active

on switch NX-5000-2:

interface port-channel10
   vpc 10

interface Ethernet101/1/2

  desc Feed to 3750 - GigabitEthernet1/0/2
   channel-group 10 mode active

on 3750:

interface Port-channel10
spanning-tree bpdufilter enable
!
interface GigabitEthernet1/0/1
desc Feed from NX-2000-100 - Ethernet100/1/2
channel-group 10 mode active
spanning-tree bpdufilter enable
!
interface GigabitEthernet1/0/2

desc Feed from NX-2000-101 - Ethernet101/1/2

channel-group 10 mode active
spanning-tree bpdufilter enable
!

and I get the following error on the N5ks:

%ETHPORT-2-IF_DOWN_ERROR_DISABLED: Interface Ethernet100/1/2 is down (Error disabled. Reason:Object doesn't exist)

%ETHPORT-2-IF_DOWN_ERROR_DISABLED: Interface Ethernet101/1/2 is down (Error disabled. Reason:Object doesn't exist)

NX-5000-1# sh interface Ethernet100/1/2
Ethernet100/1/2 is down (Internal-Fail errDisable)

NX-5000-2# sh interface Ethernet101/1/2
Ethernet101/1/2 is down (Internal-Fail errDisable)

Any ideas?

[Jerry Ye]

Can you provide the NXOS version running on your Nexus 5000?

Regards,

jerry

[Matt Craig]

The version is 5.0(2)N1(1).

However, I have this figured out.  I got with TAC, and did more testing...

The errdisable happened because of a bug:

CSCth81348 - ports failed into Internal-Fail errDisable when configuring vpc

Per the bug notes, I just re-booted both the 5ks, and the port channel came up no problem... however if anything happens to any of the interfaces that make them go down, they go right back into errdisable when they try to come back up and you have to re-boot the switches again, so its not feasible under this version of code.  TAC tells me there is new version of code comming out very soon: 5.0(2)N2(1), and that this bug will be fixed under it.

A few other bugs are supposed to be fixed with this version as well.  All in all, I think 5.0(2)N1(1) is buggy and the nexus line in general seems to be learning how to walk still.  Hopefully 5.0(2)N2(1) will be better and the product line will mature.

[Jerry Ye]

Thanks for providing more info.

Regards,

jerry

[Mathias Rufer]

thanks for posting this, I had the same problem on a N5K-C5548UP with N2K-C2232PP-10GE and NXOS 5.2(1)N1(6)

A reboot helps. Hopefully it doesn't happen in production

Greetings

Rufer

[nasser joolfun]

Hi

Same issue with version 6.2(12) N7K-S2.

N@ss

[dudarra16]

hi guys,

since yesterday we have the same issue on productive enviroment...we changed a active active scenarion form the nexus ports to the FEX...

kickstart: version 7.0(8)N1(1)
system:    version 7.0(8)N1(1)

2017 Mar 14 13:21:36.441 %ETHPORT-5-IF_DOWN_ERROR_DISABLED: Interface Ethernet104/1/9 is down (Error disabled. Reason:Object doesn't exist) 2017 Mar 14 13:23:36.450 %ETHPORT-5-IF_ERRDIS_RECOVERY: Interface Ethernet104/1/9 is being recovered from error disabled state (Last Reason:Object doesn't exist) 2017 Mar 14 13:23:39.144 %ETHPORT-5-SPEED: Interface Ethernet104/1/9, operational speed changed to 1 Gbps 2017 Mar 14 13:23:39.144 %ETHPORT-5-IF_DUPLEX: Interface Ethernet104/1/9, operational duplex mode changed to Full 2017 Mar 14 13:23:39.144 %ETHPORT-5-IF_RX_FLOW_CONTROL: Interface Ethernet104/1/9, operational Receive Flow Control state changed to off 2017 Mar 14 13:23:39.144 %ETHPORT-5-IF_TX_FLOW_CONTROL: Interface Ethernet104/1/9, operational Transmit Flow Control state changed to on 2017 Mar 14 13:23:42.420 %ETHPORT-5-IF_SEQ_ERROR: Error ("Object doesn't exist") communicating with MTS_SAP_IPQOS_MGR for opcode MTS_OPC_ETHPM_BUNDLE_MEMBER_BRINGUP (RID_PORT: Ethernet104/1/9) - Ethpm didn't receive a Link UP from the driver on time or had an internal error, check Port Client and Ethpm lo 2017 Mar 14 13:23:42.731 %ETHPORT-5-IF_DOWN_ERROR_DISABLED: Interface Ethernet104/1/9 is down (Error disabled. Reason:Object doesn't exist) 2017 Mar 14 13:25:42.730 %ETHPORT-5-IF_ERRDIS_RECOVERY: Interface Ethernet104/1/9 is being recovered from error disabled state (Last Reason:Object doesn't exist) 2017 Mar 14 13:25:45.379 %ETHPORT-5-SPEED: Interface Ethernet104/1/9, operational speed changed to 1 Gbps 2017 Mar 14 13:25:45.379 %ETHPORT-5-IF_DUPLEX: Interface Ethernet104/1/9, operational duplex mode changed to Full 2017 Mar 14 13:25:45.380 %ETHPORT-5-IF_RX_FLOW_CONTROL: Interface Ethernet104/1/9, operational Receive Flow Control state changed to off 2017 Mar 14 13:25:45.380 %ETHPORT-5-IF_TX_FLOW_CONTROL: Interface Ethernet104/1/9, operational Transmit Flow Control state changed to on 2017 Mar 14 13:25:48.818 %ETHPORT-5-IF_SEQ_ERROR: Error ("Object doesn't exist") communicating with MTS_SAP_IPQOS_MGR for opcode MTS_OPC_ETHPM_BUNDLE_MEMBER_BRINGUP (RID_PORT: Ethernet104/1/9) - Ethpm didn't receive a Link UP from the driver on time or had an internal error, check Port Client and Ethpm lo 2017 Mar 14 13:25:49.094 %ETHPORT-5-IF_DOWN_ERROR_DISABLED: Interface Ethernet104/1/9 is down (Error disabled. Reason:Object doesn't exist) 2017 Mar 14 13:27:49.110 %ETHPORT-5-IF_ERRDIS_RECOVERY: Interface Ethernet104/1/9 is being recovered from error disabled state (Last Reason:Object doesn't exist) 2017 Mar 14 13:27:51.803 %ETHPORT-5-SPEED: Interface Ethernet104/1/9, operational speed changed to 1 Gbps 2017 Mar 14 13:27:51.803 %ETHPORT-5-IF_DUPLEX: Interface Ethernet104/1/9, operational duplex mode changed to Full 2017 Mar 14 13:27:51.803 %ETHPORT-5-IF_RX_FLOW_CONTROL: Interface Ethernet104/1/9, operational Receive Flow Control state changed to off 2017 Mar 14 13:27:51.803 %ETHPORT-5-IF_TX_FLOW_CONTROL: Interface Ethernet104/1/9, operational Transmit Flow Control state changed to on 2017 Mar 14 13:27:54.165 %ETHPORT-5-IF_SEQ_ERROR: Error ("Object doesn't exist") communicating with MTS_SAP_IPQOS_MGR for opcode MTS_OPC_ETHPM_BUNDLE_MEMBER_BRINGUP (RID_PORT: Ethernet104/1/9) - Ethpm didn't receive a Link UP from the driver on time or had an internal error, check Port Client and Ethpm lo

will try to find a solutions --> REBOOT is not an option!

cheers

[pratheesh.venu]

I had a similar issue issue while working on one of the partner lab- Lab 13: Implementing vPC with Cisco Nexus 7000 - NX-OS-2

When I try to bring the N7k interface to port-channel - it goes to error-disable mode. If I remove from PoC and bounce the interface - it comes backonline. however port-cahnneling is not working. The NXOS was one of the older 5.x series....

2014 Jun 13 08:41:50 N7K11-pod2 %ETHPORT-5-IF_DOWN_PORT_CHANNEL_MEMBERS_DOWN: Interface port-channel2 is down (No operational members)

2014 Jun 13 08:41:50 N7K11-pod2 %ETHPORT-5-IF_DOWN_ERROR_DISABLED: Interface Ethernet2/10 is down (Error disabled. Reason:Port structure doesn't exist.)

[Fernando Ibarra Hernandez]

Hello I the same problem in LAB vPC in PEC.

My configuration is same in both N7K, but active interface the logs is only:

Logs:

N7K11-pod1(config-if-range)# 2014 Jun 15 19:52:00 N7K11-pod1 %ETHPORT-5-IF_ADMIN_UP: Interface Ethernet2/2 is admin up .
2014 Jun 15 19:52:00 N7K11-pod1 %ETHPORT-5-IF_ADMIN_UP: Interface Ethernet2/1 is admin up .
2014 Jun 15 19:52:00 N7K11-pod1 %ETHPORT-5-IF_SEQ_ERROR: Error ("Port structure doesn't exist.") communicating with MTS_SAP_ELTM for opcode MTS_OPC_ETHPM_PORT_PRE_CFG (RID_PORT: Ethernet2/2)
2014 Jun 15 19:52:00 N7K11-pod1 %ETHPORT-5-IF_DOWN_ERROR_DISABLED: Interface Ethernet2/2 is down (Error disabled. Reason:Port structure doesn't exist.)
2014 Jun 15 19:52:00 N7K11-pod1 %ETHPORT-5-IF_SEQ_ERROR: Error ("Port structure doesn't exist.") communicating with MTS_SAP_ELTM for opcode MTS_OPC_ETHPM_PORT_PRE_CFG (RID_PORT: Ethernet2/1)
2014 Jun 15 19:52:00 N7K11-pod1 %ETHPORT-5-IF_DOWN_ERROR_DISABLED: Interface Ethernet2/1 is down (Error disabled. Reason:Port structure doesn't exist.)

N7K12-pod1(config-if)# sh int br | grep Error
Eth2/1        1       eth  trunk  down    Error disabled             auto(S) 1
Eth2/2        1       eth  trunk  down    Error disabled             auto(S) 1

Configs:

vpc domain 1
  peer-keepalive destination 10.11.4.2 source 10.11.4.1 vrf PKAL

interface port-channel1
  vpc peer-link

N7K11-pod1(config-if-range)# sh run int port 1

!Command: show running-config interface port-channel1
!Time: Sun Jun 15 19:55:32 2014

version 5.2(4)

interface port-channel1
  switchport
  switchport mode trunk
  switchport trunk allowed vlan 1-100
  spanning-tree port type network
  vpc peer-link

N7K11-pod1(config-if-range)# sh run int e 2/1

!Command: show running-config interface Ethernet2/1
!Time: Sun Jun 15 19:56:38 2014

version 5.2(4)

interface Ethernet2/1
  switchport
  switchport mode trunk
  switchport trunk allowed vlan 1-100
  channel-group 1 mode active
  no shutdown

In theory, the config is fine, but here I dont know what happen. Only interface work with Trunk.

Regards.

[pratheesh.venu]

I guess it is same the lab, where we got the PoC issue reported. Google  search alos couldn't get much details on the error

2014 Jun 15 19:52:00 N7K11-pod1 %ETHPORT-5-IF_DOWN_ERROR_DISABLED: Interface Ethernet2/1 is down (Error disabled. Reason:Port structure doesn't exist.)

one option would be trying to reload and try.....

I also submitted a ticket to LAB support team... no positive response yet....

Regards

Pratheesh