12-06-2019 12:12 PM
So I had this dropped into my lap and I can't really make any headway on it. We started noticing some MAB failures and started looking for the devices. Although being reported on a certain interface there is nothing but what is supposed to be there; an ip phone and a pc behind it. Logs will even show the same MAC on multiple interfaces across different switches and vlans. It's like "phantom" devices are trying to access an interface, but get dropped at the MAB. Here's a snipit of one of our logs:
16694398: Dec 6 14:55:48.880 EST: %MAB-5-FAIL: Authentication failed for client (b4b6.8690.6566) on Interface Gi2/16 AuditSessionID 0A4B266E00033041A38758D8
16694399: Dec 6 14:55:51.957 EST: %MAB-5-FAIL: Authentication failed for client (54bf.6495.e670) on Interface Gi5/25 AuditSessionID 0A4B266E0003AB3E8C35253C
16694400: Dec 6 14:55:55.024 EST: %MAB-5-FAIL: Authentication failed for client (10e7.c6a7.83e3) on Interface Gi1/38 AuditSessionID 0A4B266E000391DF1E06D7B4
16694401: Dec 6 14:55:56.057 EST: %MAB-5-FAIL: Authentication failed for client (b4b6.8646.cafa) on Interface Gi5/7 AuditSessionID 0A4B266E00032D0598BC6DAC
16694402: Dec 6 14:55:57.087 EST: %MAB-5-FAIL: Authentication failed for client (b4b6.8690.6545) on Interface Gi2/20 AuditSessionID 0A4B266E0003426FE0374E68
16694403: Dec 6 14:55:57.087 EST: %MAB-5-FAIL: Authentication failed for client (b4b6.8690.6db0) on Interface Gi1/6 AuditSessionID 0A4B266E00033D48CF503284
16694404: Dec 6 14:55:57.087 EST: %MAB-5-FAIL: Authentication failed for client (b4b6.8648.415c) on Interface Gi1/31 AuditSessionID 0A4B266E00034071D9992C4C
16694405: Dec 6 14:55:58.104 EST: %MAB-5-FAIL: Authentication failed for client (80ce.621e.6f0a) on Interface Gi5/17 AuditSessionID 0A4B266E0003ECD9E88B7958
We're getting upwards of 1300 failures on some interfaces daily. If I pull the mac list for the interface only the two that should be on there are listed. Where could these other MACs be coming from? My research hasn't produced much for me as of yet.
12-09-2019 05:45 AM
"Logs will even show the same MAC on multiple interfaces across different switches and vlans"
mac-vendor lookup associates the given MACs with valid vendors like DELL HP etc.
first check ip these PC's do not have a management interface shared with the data interface (iDRAC, CIMC, ILO)
if not -> looks like packets come from a software modified mac address -->> do a serious malware scan !
if yes these management interfaces may be the source (but guess not with same mac on multiple switchports)
other option is to create a SPAN port and do a packet capture to see what data is sent in the packets with these MAC addresses.
12-10-2019 06:46 AM
I majority of the MACs involved in the failed authentication have an OUI from HP.
12-09-2019 06:50 AM
Hello,
could be the bug below. Known fixed releases are:
15.2(2)E
15.0(2)SE8
15.2(6.3.0i)E
15.2(5.0)ST
15.2(4.0)ST
15.2(4.0.64a)E
15.2(2.2.32)EA
3.6(0)E
abnormal dot1x authentication failure msg from some specific mac address
CSCum75962
Description
Symptom:
Frequent dot1x authentication failure msg came out at many different port, all of them has same source mac address, such as:
053909: Jan 23 02:56:39.597 BJ: %AUTHMGR-5-START: Starting 'dot1x' for client (0025.4619.7214) on Interface Gi1/0/28 AuditSessionID AC014579000001B524053E46
053910: Jan 23 02:56:55.073 BJ: %DOT1X-5-FAIL: Authentication failed for client (0025.4619.7214) on Interface Gi1/0/28 AuditSessionID AC014579000001B524053E46
053911: Jan 23 02:56:55.073 BJ: %AUTHMGR-7-STOPPING: Stopping 'dot1x' for client 0025.4619.7214 on Interface Gi1/0/28 AuditSessionID AC014579000001B524053E46
053912: Jan 23 02:56:55.073 BJ: %AUTHMGR-5-START: Starting 'mab' for client (0025.4619.7214) on Interface Gi1/0/28 AuditSessionID AC014579000001B524053E46
053913: Jan 23 02:56:55.099 BJ: %MAB-5-FAIL: Authentication failed for client (0025.4619.7214) on Interface Gi1/0/28 AuditSessionID AC014579000001B524053E46
053914: Jan 23 02:56:55.099 BJ: %AUTHMGR-7-STOPPING: Stopping 'mab' for client 0025.4619.7214 on Interface Gi1/0/28 AuditSessionID AC014579000001B524053E46
053915: Jan 23 02:56:55.099 BJ: %AUTHMGR-5-FAIL: Authorization failed or unapplied for client (0025.4619.7214) on Interface Gi1/0/28 AuditSessionID AC014579000001B524053E46
056306: Jan 23 12:23:24.894 BJ: %AUTHMGR-5-START: Starting 'dot1x' for client (0025.4619.7214) on Interface Gi1/0/23 AuditSessionID AC014579000001D42811BF05
056307: Jan 23 12:23:40.370 BJ: %DOT1X-5-FAIL: Authentication failed for client (0025.4619.7214) on Interface Gi1/0/23 AuditSessionID AC014579000001D42811BF05
056308: Jan 23 12:23:40.370 BJ: %AUTHMGR-7-STOPPING: Stopping 'dot1x' for client 0025.4619.7214 on Interface Gi1/0/23 AuditSessionID AC014579000001D42811BF05
056309: Jan 23 12:23:40.370 BJ: %AUTHMGR-5-START: Starting 'mab' for client (0025.4619.7214) on Interface Gi1/0/23 AuditSessionID AC014579000001D42811BF05
056310: Jan 23 12:23:40.395 BJ: %MAB-5-FAIL: Authentication failed for client (0025.4619.7214) on Interface Gi1/0/23 AuditSessionID AC014579000001D42811BF05
056311: Jan 23 12:23:40.412 BJ: %AUTHMGR-7-STOPPING: Stopping 'mab' for client 0025.4619.7214 on Interface Gi1/0/23 AuditSessionID AC014579000001D42811BF05
056312: Jan 23 12:23:40.412 BJ: %AUTHMGR-5-FAIL: Authorization failed or unapplied for client (0025.4619.7214) on Interface Gi1/0/23 AuditSessionID AC014579000001D42811BF05
this happened at every customer switch stack and the source mac has regular pattern:
SO-OAW3750-18F-C1 0064.4001.949c
SO-OAW3750-19F-C1 0064.4001.949d
SO-OAW3750-20F-C1 0025.4619.7213
SO-OAW3750-21F-C1 0025.4619.7214
SO-OAW3750-22F-C1 0025.4619.7215
SO-OAW3750-23F-C1 0025.4619.7216
SO-OAW3750-25F-C1 0025.4619.7217
Conditions:
enable dot1x authentication
Workaround:
none
Further Problem Description:
this problem happened randomly at many ports of every switch stack and the mac address OUI is Cisco. we used monitor session to try to capture those frames but can't capture.
Discover and save your favorite ideas. Come back to expert answers, step-by-step guides, recent topics, and more.
New here? Get started with these tips. How to use Community New member guide