mac-flap; limit or tuning?

[brian.kennedy]

Any know how many times w/in a time frame a mac address needs to show up on two ports before a mac-flap error is logged?  And is there a way to tune that threshold?

We're running some ap's in flex mode, and a few handheld clients are roaming between access point a little too often and we're seeing mac-flap errors from these devices.  I know that's going to be somewhat normal as devices roam from one ap to another, but we obviously don't see this on every other device as it roams, so there must be a number of instances that need to happen before the alert is triggered.

We're also looking at the client to see why they're trying to roam so much.

thanks

brian

[Enes Simnica]

@Sha123 Ur VM and host are probably sharing the same MAC address because of how SR-IOV is set up. When both try to talk on the network at the same time, switches freak out seeing the MAC "jump" between ports  that’s the flapping. Thats why STP "fixes" it by blocking one of the ports (like a parent separating fighting kids). Shutting down either the server or VM port also stops the fight because only one MAC remains.

also if u want to check if the VM stealing the host’s MAC, all u have to do is run ip a on both the VM and host if the MACs match, that’s ur problem. and G, about SR-IOV config: U should know that some NICs lazily reuse the host MAC for VMs unless u force a unique one. Check ur hypervisor’s SR-IOV settings.

for VMware's implementation guide: https://www.vmware.com/docs/vsphere-esxi-vcenter-server-80-performance-best-practices

and for For Intel NICs: file:///C:/Users/enes.simnica/Downloads/630211_E810%20SR-IOV%20RHEL8%20ConfigGuide_Rev1.2.pdf

-Enes

[Sha123]

what is the typical connection layout between the physical switch, the host’s Physical Function (PF), and the Virtual Functions (VFs) used by the VMs?

[Sha123]

@Enes Simnica Sir, do u know the answer pls share

[Enes Simnica]

@Sha123 yessssirrrrrr. In ur scenario, the 9 VMs using SR-IOV Virtual Functions (VFs) are not physically connected to separate switch ports. All traffic from those VFs goes through the same physical NIC port (the Physical Function or PF), and therefore through a single switch port. So physically, the switch only sees one uplink port carrying all traffic from the VMs.

 MAC flapping in this setup can still happen because the physical switch may see the same MAC address appearing on different interfaces, which means it is not due to actual physical movement, but can happen  due to:

• VLAN misconfigurations or dynamic changes in the VF mapping
• VFs being reassigned to different VMs or migrated between PFs on different hosts
• MAC spoofing (two VMs using the same MAC)
• A virtual switch (like OVS or a Linux bridge) involved in the data path, especially if it's misconfigured
• Multicast/broadcast storms, or traffic loops without proper loop prevention

also from my experiences a common cause is live migration or reconfiguration without informing the switch properly, especially if the switch has MAC learning enabled on trunk ports. and while searching I found this; https://bugzilla.redhat.com/show_bug.cgi?id=1871032 

just ping me if u have more questions......

-Enes

[Sha123]

Even though all VMs and the host connect through a single physical switch port, MAC flapping?

[Enes Simnica]

The switch detects MAC flapping by monitoring MAC address movements across ports. In your SR-IOV setup, even though there's only one physical connection, the switch sees the same MAC address appearing to come from multiple logical sources (the PF and VFs) within that single port. When multiple functions transmit using the same MAC, the switch interprets this as the MAC rapidly moving between sources, triggering flapping alerts. The root cause is MAC address duplication - either between the host's PF and a VF, or between multiple VFs. This creates a logical conflict where the switch sees one MAC being claimed by multiple entities on what appears to be the same port.

Enabling STP fixes this by artificially blocking one of the conflicting paths, while shutting down either the server port or VM port removes the duplicate MAC sources. But listen man, I'm telling you from experience - the real fix isn't STP. You gotta dig deeper. Check those MAC assignments in your SR-IOV config, make sure your VFs aren't stepping on each other's toes with duplicate addresses. And if you're running a virtual switch, double-check those settings too. I've seen this exact scenario burn hours of debugging time before we caught the MAC conflict.

This isn't some physical loop messing with you - it's the switch doing its job right, catching what looks like an unstable MAC. STP might quiet the alerts, but proper MAC management is the real solution here. Trust me, I've been down this road before and that's the truth coming from hands-on experience.

[MHM Cisco World]

Please make new post 

Thanks 

MHM

[Sha123]

What are the port states in STP?

[Enes Simnica]

@Sha123 MST has three port states, which are simplified from the classic STP, and it uses faster convergence..

Port State	Time spent (default)	Notes
Discarding	immediate	Doesnt forward traffic or learn MACs... includes blocking, listening and disables..
Learning	15 seconds (if not fast-converging)	learns Mac addresses, but it doesnt forward traffic
Forwarding	after learning	learns MACs and forwards traffic

and also in practice, with MSTP's fast convergence (using proposal/agreement), ports often transition from Discarding to Forwarding in under a second, bypassing timers,  unless legacy behavior is enforced...: https://www.cisco.com/c/en/us/support/docs/lan-switching/spanning-tree-protocol/24248-147.html 

2. As for multicast packet amplification in SR-IOV, it’s a known bug where multicast packets get duplicated across multiple VFs. Even if MACs are unique, the switch sees the same traffic from different interfaces, which causes MAC flapping. This isn't a physical loop, but it looks like one to the switch. The real fix is checking your NIC driver, VF config, and any virtual switches involvednot relying on STP to mask the issue....

hope it helps

-Enes