Invalid MAC address endlessly flapping between two ports

[Razvan Craciun]

Hi guys,

I've hit an issue that I can't get to the end of, and thought that you might be able to help...

I have an invalid MAC address that is flapping and continously looping between in aVLAN of one of our remote sites LAN. The mac address is

6000.86dd.6000 and I can't find it on any access port of any switch in my network, with no idea of what to generate. I mention that I have rapid-pvst enabled on all switches and that the STP topology is stable, with the proper switches blocking the proper ports.

The core stack CPU is at 90% CPU with the below process being the main resource drain:

69    290298741597995435         18 13.25% 10.89% 10.59%   0 HLFM address lea

The log buffer is full of these messages, but only on the core switch, as the other access switches see that mac on the uplink ports only, and no flapping is detected.

Dec 20 19:38:39 PHT: %SW_MATM-4-MACFLAP_NOTIF: Host 6000.86dd.6000 in vlan 75 is flapping between port Gi1/0/25 and port Gi2/0/26

Dec 20 19:38:54 PHT: %SW_MATM-4-MACFLAP_NOTIF: Host 6000.86dd.6000 in vlan 75 is flapping between port Gi1/0/25 and port Gi2/0/26

Dec 20 19:39:09 PHT: %SW_MATM-4-MACFLAP_NOTIF: Host 6000.86dd.6000 in vlan 75 is flapping between port Gi1/0/25 and port Gi2/0/26

Dec 20 19:39:24 PHT: %SW_MATM-4-MACFLAP_NOTIF: Host 6000.86dd.6000 in vlan 75 is flapping between port Gi2/0/26 and port Gi1/0/25

The core switch is a 2 3750 stack and all the access switches are 2960S stacks (3-4 switches per stack).

For a better understanding of this issue, I have attached a network diagram and some command outputs.

I already tried clearing the cam tables simlutaneously, but with no effect (the siwtches forward frames a lot faster than me sending the clear commands from the ssh sessions).

I would appreciate any idea for solving this issue.

[fbean]

I'm a coworker of Razvan's.

the log entries for those two swtiches are pretty clean..  For example, today, there are only entries regarding power to IP phones and logins. 

I'd post them but i've got cut and paste issues at the moment.

I did find that one of the trunk/uplink ports was configured with portfast which I removed.  Not that it mattered because it was only portfast, not trunk portfast.

I also removed vlan 75 from the trunk on both sides of one of the connections but the errors continue.

I think the key is if we can identify the where/why on the weird mac address it is seeing coming through the two links.

Another thought I had was that  somehow our wireless access points were involved.  They are connected to these two switches.  but I removed vlan 75 from their uplink config, also with no impact.

[paul driver]

Hello

The mac flap is between switches 29stk1 and 30stk2 so looking at you topology they are not connected and shouldnt be either, but obviously the core is reporting seeing it from either switch,

As I stated in a previous post look for stp bpdu-filter applied to a port or a monitor port that is not being used and now is an access port
also sh int trunk check the inferaces that are forwarding on this vlan 75 - between your cloest switches

Lastly if this issue isnt occurring all the time, beginning at the core you can trace the existing port this mac is located on by
sh ip arp | in 6000.86dd.6000 
sh mac-address address 6000.86dd.6000  -

res
Paul

[Tom Vanhout]

Hi,

we have had the same issue on 3 separate occasions on our network.

CPU of the router (in our case a 4900M) would go to 100%

debugging on the switch would show packet like fe.

Index 15:

194 days 1:16:48:536508 - RxVlan: 7, RxPort: Te1/8

Priority: Normal, Tag: Dot1Q Tag, Event: Input Acl Fwd, Flags: 0x40, Size: 516

Eth: Src 60:36:86:DD:60:36 Dst 86:DD:60:00:86:DD Type/Len 0x86DD

Remaining data:

 0: 0x60 0x36 0x86 0x0  0x0  0x36 0x0  0x0  0x0  0x36

10: 0x0  0x1  0xFE 0x80 0x0  0x1  0xFE 0x80 0x0  0x1

20: 0xFE 0x80 0x0  0x1  0xFE 0x80 0x11 0x1  0xFE 0x0

30: 0x11 0x0  0x0  0x0  0x0  0x0  0x0  0x0  0x0  0x0

40: 0x0  0x0  0x0  0x0  0x0  0x0  0x0  0x0  0x2  0x60

As you can see, both source and desination are somewhat peculiar and something seems to point at ipv6 due to the 86dd  (even though there is no ipv6 config on that routers L3 interface)

The destination addresses seem to be 86:DD:60:00:86:DD or 86:DD:60:84:86:DD

We place a "mac address-table static 86dd.6000.86dd vlan x drop" on the router and then the packets went away

not a problem of loops, but something specifically tied to these strange mac-adresses.

We suspect either a funny application that messes things up, or something happening on the router.

It would bounce even on a single link (no loop environment), looking at the mac-tables you would conclude they send out the packet back on the interface on which they received it, which normally can't be the case.

In our last case, this caused a problem with hardphones (internal switch) and authentication.

As somehow it would trigger packets responses or something with mac-adres sources like 6084.86dd.xxxx