802.11 probe requests with 00:00:00:00:00:00 source mac

[N3t W0rK3r]

Hello,

We're having a weird issue that I hope someone here can shed light on.

On several of our 4500 VSS pairs of distribution switches we've observed this type of log message every 6 hours:

Oct 28 11:15:06.463 EDT: %C4K_L2MAN-6-INVALIDSOURCEADDRESSPACKET: (Suppressed 20025 times)Packet received with invalid source MAC address (00:00:00:00:00:00) on port Po40 in vlan 508

We have traced the source of these logs to one of our edge 3850 switch stacks that are running 16.3.6 code.  There are no such logs appearing on the 3850 stack however.

After placing a sniffer on the the 3850 stacks' portchannel uplink to the distribution switch 4500 vss pair, We discovered that the this odd traffic is coming from ALL of the Cisco APs connected to this stack.  The traffic in question is predominantly probe requests to other APs.

Any idea why just these 20 APs out of the hundreds that we have across our enterpirse, are performing this way, generating these malformed frames?

Thanks in advance.

John

[Rasika Nayanajith]

What model of APs are they ? What version of AireOS running on these AP/WLC ? All APs connected to that 3850 behaving in same manner ?

Do you have another 3850 running on 16.3.6 & not having this issue ? 

If a given AP directly connect to that 3850, pls take a capture from AP connected switchport to confirm its coming from a given AP. 

HTH

Rasika

[N3t W0rK3r]

Thank you very much for your reply Rasika.

To answer your questions...

AP's are AIR-AP3802I-A-K9 running this code 8.2.166.0.

All AP's on this stack are exhibiting this behavior.

We do not have any other 3850 stack running this code.

In response to your request, I took captures on a few of the AP ports and quickly realized something I had not realized before... the destination MAC addresses of these frames are the base radio MACs of the APs themselves.  The traffic is not originating on the AP switch ports, its destined there... but just to the APs of this stack, for some reason.

I will keep digging around to see I can locate the source of the traffic.

[Rasika Nayanajith]

I did not have good experience with IOS-XE 16.x.x codes yet :) Running couple of stacks 16.6.4a at the moment (3802 had trouble registering after power outage with 16.6.4). Most of our 3850 still running 3.7.5E

If you could not figure it out source of these traffic, I would suggest to try different software version on 3850

HTH

Rasika

[N3t W0rK3r]

What's even more confusing to me is that these 802.11 frames going TO the base radio MAC of the AP, are encapped in an ipv4 packet going from the AP to the WLC (which is in the opposite direction to the 802.11 frames). 

[Rasika Nayanajith]

Yes, this is strange. Probe requests should not go to wire. So it could be AP side of issue.

Pls check it on a working 3850 connected AP & ensure this behavior is not there.

Rasika

[Rich R]

Something in that location bridging wireless traffic onto the LAN?

I can't see how the switch or its OS could make any difference to the behaviour of the APs.