Nexus vPC DAC cable connection issue

BorislavPenchev0962 · ‎01-17-2022

Hello all,

we have a 100G vPC Peer-Link between two Nexus N9k-C93240XC-FX2.

tried to connect Eth1/59 and Eth1/60 per QSFP-100G-CU1M DAC cable, but only Port 60 get link.

cable has been correct recognized by both nexuses (show interface status/ show int transceiver detail)

when we use instead of 2x DACs , a 1x DAC and 2x 100G-LR4 SFPs at Ports 59/60 - we get link

when using Ports 58 + 60 - we get link
NX-OS 9.3(8)
tried to Factory Default both switches - no go

swapped one of the DACs with the QSFP+ Setup ( which is ok) and crossed the DACs with each other to exclude a defect.

Any suggestions on how to resolve this or is this in general accepted by specs ?

Regards

Boris

balaji.bandi · ‎01-17-2022

what is the status if you only connect port 59 ?

can you post snippet config ?

BB

***** Rate All Helpful Responses *****

How to Ask The Cisco Community for Help

BorislavPenchev0962 · ‎01-26-2022

l do believe this is port-channel issue , but not sure where the problem is:

- If only port 60 is connected -> works

- If only port 59 is connected -> no link

switch1

`show port-channel summary`--------------------------------------------------------------------------------Group Port- Type Protocol Member Ports Channel--------------------------------------------------------------------------------123 Po123(SU) Eth LACP Eth1/59(D) Eth1/60(P)

switch2

`show port-channel summary`

--------------------------------------------------------------------------------

Group Port- Type Protocol Member Ports

Channel

--------------------------------------------------------------------------------

123 Po123(SU) Eth LACP Eth1/59(D) Eth1/60(P)

snippet config of ports eth1/59 and 1/60 on switch 1:

Ethernet1/59
transceiver is present
type is QSFP-100G-CR4
revision is A
nominal bitrate is 25500 MBit/sec
Link length supported for copper is 1 m
cisco product id is QSFP-100G-CU1M


Ethernet1/60
transceiver is present
type is QSFP-100G-CR4
nominal bitrate is 25500 MBit/sec
Link length supported for copper is 1 m
cisco product id is QSFP-100G-CU1M


interface Ethernet1/59
switchport
switchport mode trunk
channel-group 123 mode active
no sh


interface Ethernet1/60
switchport
switchport mode trunk
channel-group 123 mode active
no sh

snippet config of ports eth1/59 and 1/60 on switch 2:

Ethernet1/59
transceiver is present
type is QSFP-100G-CR4
nominal bitrate is 25500 MBit/sec
Link length supported for copper is 1 m
cisco product id is QSFP-100G-CU1M


Ethernet1/60
transceiver is present
type is QSFP-100G-CR4
nominal bitrate is 25500 MBit/sec
Link length supported for copper is 1 m
cisco product id is QSFP-100G-CU1M \


interface Ethernet1/59
description vPC
switchport
switchport mode trunk
channel-group 123 mode active
no shutdown


interface Ethernet1/60
description vPC
switchport
switchport mode trunk
channel-group 123 mode active
no shutdown

BorislavPenchev0962 · ‎01-31-2022

just noticed alert healthcheck for the below config, what would be the error here ?

- If only port 60 is connected -> works

- If only port 59 is connected -> no link

Georg Pauwen · ‎02-01-2022

Hello,

post the output of:

show port-channel compatibility-parameters

debug port-channel error

The command:

channel-group 123 force

used to be around, not sure if that is available on the N9K, but if it is, give that a try:

interface Ethernet1/59

--> channel-group 123 force

BorislavPenchev0962 · ‎02-02-2022

Hi,

see attached output of show port-channel compatibility-parameters for sw1 and sw2

the channel is a full trunk with all VLANs allowed on it.
sh int po123 trunk

Port Native Status Port
Vlan Channel
--------------------------------------------------------------------------------
Po123 1 trunking --
--------------------------------------------------------------------------------
Port Vlans Allowed on Trunk
--------------------------------------------------------------------------------
Po123 1-4094
-------------------------------------------------------------------------------
Port Vlans Err-disabled on Trunk
--------------------------------------------------------------------------------
Po123 none
--------------------------------------------------------------------------------
Port STP Forwarding
--------------------------------------------------------------------------------
Po123 1
--------------------------------------------------------------------------------
Port Vlans in spanning tree forwarding state and not pruned
--------------------------------------------------------------------------------
Po123 Feature VTP is not enabled

debug is enabled when I try the „force“ command on Port Eth1/59 I get the following output:

Switch 1:

 eth_port_channel: eth_pcm_objstore_process_tx_begin(3367): MTS_OPC_DME_TX_BEGIN for txid=1190199
 eth_port_channel: pcm_get_txn_info_entry(3802): Unable to find TXN INfo for TXN ID 1190199
 eth_port_channel: pcm_demux(1066): (1066): (Warning) unexpected mts msg (opcode - 409604)
eth_port_channel: pcm_dme_membership_update_handler(2778): Incremented dme_gwrap_ref_count to 1 for txid:1190199, opc:MTS_OPC_MODIFY_ImPcRsMbrIfsPolicyelem
 eth_port_channel: pcm_pc_ac_membership_change(1648): found the mbr 0x1a007600
eth_port_channel: pcm_pc_ac_membership_change(1648): found the mbr 0x1a007400
 eth_port_channel: pcm_perform_pre_drop_action_per_lock(1668): No pre-drop needed for non-PC rid, type 6
eth_port_channel: pcm_pss_save_data(926): unknown type being stored to pss ffffffff
 eth_port_channel: pcm_wrap_send_response(2013): Decremented dme_gwrap_ref_count to 0 for txid:1190199, opc:MTS_OPC_MODIFY_ImPcRsMbrIfsPolicyelem
 eth_port_channel: pcm_wrap_send_response(2083): TXN END NOT rxed for 1190199 updating evseq ended
 eth_port_channel: pcm_wrap_send_response(2091): setting evseq_end to true in pcm_wrap_send_response for opc MTS_OPC_MODIFY_ImPcRsMbrIfsPolicyelem
 eth_port_channel: eth_pcm_objstore_process_tx_end(3431): eth_pcm_objstore_process_tx_end: sending response from tx_end for txid 1190199

Switch 2:

eth_port_channel: eth_pcm_objstore_process_tx_begin(3367): MTS_OPC_DME_TX_BEGIN for txid=1192193
 eth_port_channel: pcm_get_txn_info_entry(3802): Unable to find TXN INfo for TXN ID 1192193
 eth_port_channel: pcm_demux(1066): (1066): (Warning) unexpected mts msg (opcode - 409604)
 eth_port_channel: pcm_dme_membership_update_handler(2778): Incremented dme_gwrap_ref_count to 1 for txid:1192193, opc:MTS_OPC_MODIFY_ImPcRsMbrIfsPolicyelem
 eth_port_channel: pcm_pc_ac_membership_change(1648): found the mbr 0x1a007600
 eth_port_channel: pcm_pc_ac_membership_change(1648): found the mbr 0x1a007400
 eth_port_channel: pcm_perform_pre_drop_action_per_lock(1668): No pre-drop needed for non-PC rid, type 6
eth_port_channel: pcm_pss_save_data(926): unknown type being stored to pss ffffffff
eth_port_channel: pcm_wrap_send_response(2013): Decremented dme_gwrap_ref_count to 0 for txid:1192193, opc:MTS_OPC_MODIFY_ImPcRsMbrIfsPolicyelem
eth_port_channel: pcm_wrap_send_response(2083): TXN END NOT rxed for 1192193 updating evseq ended
eth_port_channel: pcm_wrap_send_response(2091): setting evseq_end to true in pcm_wrap_send_response for opc MTS_OPC_MODIFY_ImPcRsMbrIfsPolicyelem
 eth_port_channel: eth_pcm_objstore_process_tx_end(3431): eth_pcm_objstore_process_tx_end: sending response from tx_end for txid 1192193

But still:
Port Eth1/59 stays down and does not join the Channel.

# sh port-c sum
Flags: D - Down P - Up in port-channel (members) I - Individual H - Hot-standby (LACP only)
s - Suspended r - Module-removed b - BFD Session Wait S - Switched R - Routed U - Up (port-channel)
p - Up in delay-lacp mode (member) M - Not in use. Min-links not met
--------------------------------------------------------------------------------
Group Port- Type Protocol Member Ports
Channel
--------------------------------------------------------------------------------
123 Po123(SU) Eth LACP Eth1/59(D) Eth1/60(P)

Regards

Andrew White · ‎05-13-2022

Every time I have an issue with a DAC cable on a Nexus device manually setting the speed parameter on the port seems to resolve the issue for me. So for your 100G DAC it would be adding the line "speed 1000000" for port eth 1/59.

BorislavPenchev0962 · ‎05-16-2022

thanks for reply, we did replace hardware!