bridge loop from ESX hosts

[tanzo]

I'm scratching my head at this one, hoping someone out there may have seen this.

Have a standard ESX host to NXOS 9K VPC build. Four links from each ESX host (we have 4 total ESX hosts) distributed across our two 9Ks. About a dozen VLANS configured on the port-channels. This has been in production w/o changes (at least on the network) for years.

About 24 hours ago we lost connectivity to VMs on one VLAN on one of the ESX hosts. Troubleshooting the 9Ks identified the VLAN was in a STP altn blk role/state on the port-channel connected to that ESX host. All other VLANs were forwarding as expected. After a while the symptoms, connectivity loss on the VLAN and altn/blk, moved to another ESX host, and then again to a third ESX host.

Applying bpdufilter to the port-channels connected to the ESX hosts resulted in intermittent connectivity loss to hosts across the vlan, so a bridge loop.

It certainly seems like the ESX distributed switches are bridging this one vlan, which happens to be used for systems management, but from my VMWare experience, that shouldn't happen. Our ESX guys are telling me the hosts don't have physical connections to the network other than the 4 uplinks to the 9Ks. They are also looking into their LACP config and firmware.

Has anyone seen anything like this in their environment and have recommendations?

Thanks

[marce1000]

  - You should start (or  try to) collect logs from all involved devices through a central syslog server ;
     That makes it easier to  get insights on events and correlate  them

     For ESX, for  instance have a look at : https://knowledge.broadcom.com/external/article/318939/configuring-syslog-on-esxi.html
                                                               https://www.youtube.com/watch?v=blerJksFeMk

   M.