Can LLDP frames disrupt FCoE in converged switches?
I had an interesting - and terrible - experience with DELL-Force10 S5000 switches, where hosts FCoE paths were simultaneously going down, disrupting storage connectivity. This was happening on a VMware cluster with 10 DELL R620 hosts, mounting QLogic QLE8262 converged adapters.
These CNAs have 2 ports, each connected to a S5000 switch. Each switch also connects a number of storage devices with native FC ports.
What happened is that, randomly, hosts were losing storage paths on both FCoE adapters, at least for a few minutes. After a while, this issue could disappear on one host and start on another one - in practice bringing the whole cluster down.
The diagnosis made by DELL technical support identified the cause in LLDP frames received by their switches, from VMware host ports, which were conflicting with DCBx frames - used by DCB to negotiate FCoE attributes, therefore bringing down DCB and FCoE paths on the port.
The diagnosis was mainly driven by messages like the following, on the switches log:
%STKUNIT0-M:CP %LLDP-5-LLDP_MULTIPLE_PEER_DETECTED: DCBX operationally disabled due to more than one PEER being present on interface Te 0/52
According to DELL, such LLDP frames could come from VMware dVS, where we had enabled LLDP, but also from VM - they mentioned that Windows 2012 and later by default enables LLDP on network interfaces.
I was really astonished! I could not imagine that enabling LLDP on a dVS could bring a virtual infrastructure to its knees, and even more to hear that this could be caused by a VM - breaking all the assumptions of isolation of a VM in a virtual environment. On the other hand, DELL says that this is the DCBx behavior by design - LLDP frames from more than one source would block it.
I therefore submit this important question to the community. Is anyone aware of this issue? How is DCBx implemented on Nexus switches, and is it also potentially vulnerable to such problems? Could it ever react to regular LLDP frames from a dVS or VM (if LLDP frames generated by VMs can go through a VS - actually I wonder about that...)?
Thanks you very much for your replies and insight!
In the previous article, we have seen, how to assign IP Address to Cisco UCS Management Interface. This way we gain access to the on-board management software called Cisco Integrated Management Controller (CIMC).
The next step of building UCS Server ...
How does everyone feel about service graphs in a brownfield network centric cutover? maybe I'm an old school engineer but I was taught that if you have to do a Pbr then your design is wrong. they are a pain to troubleshoot because you always f...
To participate in this event, please use the button to ask your questions
This special event - formerly known as Ask the Expert- is open only to Cisco Customers and Partners.
Many pages in the Cisco Community are acce...
Going to Cisco LIVE EMEAR 2020? Give your Data Center agenda a boost with these top recommended sessions. Register and join us!
Data Center Networking:
From Reactive to Proactive NetOps With Cisco Network Assurance and Insights- The Power of Data...