N5K err-disabled a port due to DCX-No ACK in 100 PDUs

[jneiberger]

I'm super new to Nexus, so I'm not really sure how to troubleshoot this. I did a quick search and found that this is related to DCBX TLVs in LLDP, which we apparently shouldn't be getting on a regular ethernet port. I'm pretty sure this is just a regular ethernet port. (Like I said, I'm pretty new to nexus. lol)  I wonder if the following output indicates that we are receiving and sending DCBX TLVs in LLDP. If so, it sounds like the interface will go into err-disable state if the server stops sending ACK frames.

# show lldp dcbx int e1/17

Local DCBXP Control information:

Operation version: 00  Max version: 00  Seq no: 1  Ack no: 1 

Type/

Subtype    Version    En/Will/Adv Config

004/000     000        Y/N/Y      8906001b21 08

002/000     000        Y/N/Y      0000000064 00000000 00000001

Peer's DCBXP Control information:

Operation version: 00  Max version: 00  Seq no: 1  Ack no: 1 

Type/      Max/Oper

Subtype    Version    En/Will/Err Config

004/000     000/000    Y/Y/N      8906001b21 08

003/000     000/000    Y/Y/Y      ff08

002/000     000/000    Y/Y/N      ffffffff00 00000000 00000008

This is the first time we've run into this. Any idea what might really be going on?

Thanks!

[calopez2 · Accepted solution]

Hi,

Please investigate/verify the Server CNA, i.e. drivers, Compatibility with Server OS version.

If this is recurring, as a temporary workaround you can disable LLDP on interface that connects to Server.

no lldp transmit

no lldp receive

[jneiberger]

Isn't a CNA for FC or FCoE? Can they do regular Ethernet, too? I thought that a CNA was only for FC or FCoE. I verified that we're not doing FC or FCoE on these Nexus switches. I'll have the server owners check into it.

Thanks!

[calopez2]

Hi,

Per the display it  the Server Adapter is doing LLDP DCBX negotiation.

If not needed, then you might want to check the Server adapter settings.

DCBX is an extension of LLDP link layer discovery protocol; not restricted for FCOE usage.

Further informatin on LLDP for Nexus:

http://www.cisco.com/en/US/docs/switches/datacenter/nexus5500/sw/layer2/602_N1_1/b_5500_Layer2_Config_602N11_chapter_01010.html#task_1152779

Thanks!

Regards,

Carlos

[jneiberger]

Thanks! That's very helpful information. I'll have the server owners check into that setting. I suspect we'll probably just disable LLDP since we don't really need it on any of these interfaces.

I appreciate the help!

[fracjackmac]

Rather than disable LLDP on an interface, I found that it was much more productive to disable the DCBXP TLVs being generated by the N5K switch.

configure
no lldp tlv-select dcbxp
end
copy run start

Problem solved.

This snippet provides additional information: About DCBXP

[Christoph Viethen]

Hi,

actually I think this is the best solution. You still have the benefits of LLDP but eliminate the problem with the, in my very humble opinion, somewhat obscure DCBXP feature.

"no lldp tlv-select dcbxp" has been in our best-practices standard config for our Nexus switches for quite some time now.

Your mileage may vary, obviously.

Cheers

  Christoph

[fracjackmac]

Hello @Christoph Viethen,

Is there a link available to take me to the best-practices standard config for Nexus switches?

Thanks,

fjm / vrs

@ittybittypacket