I ran into an issue at a customer's site and has me a little stumped. The customer has 4 6500 switches. 2 Physical locations, and 2 switches at each locations. The locations are connected via 1 200mpbs metro ethernet(layer 2). We ran into an issue during a DR exercise. We had created a VLAN for DR testing purposes, that did not have any SVI configured, but the VLAN did exist on one of the 6500s. When the customer restored a VM to the DR VLAN, we lost connectivity to the production server. After some troubleshooting, we found the cause to be the same mac address showing up, but in a different vlan. Once we removed the DR vlan from the 6500 all was well. It seems like switches are ignoring the VLAN in the mac address table, and forwarding to the incorrect ports. The switch is not reporting any mac flapping in the logs. No spanning tree topology changes are occurring. Code version is 12.2(33)SXJ2 on all switches. Any insight would be appreciated.
I think we found the cause of this issue today. After some more thorough troubleshooting I had an aha moment and bummed that it took me a few days to find this. The problem occurs when the duplicate MACs exist on different sides of the metro ethernet connection. It looks like the carrier is ignoring the dot1q tag, and just looking at the source and destination MAC address. This appears to be confusing the carriers equipment. Not sure if this will ever be helpful to any other users, but thought I would share.