Phantom Authentication Sessions Appearing on a Switchport

[Matthew Martin]

Hello All,

Not sure when this started as I noticed it today while looking at our ISE LiveLogs. I started seeing a whole bunch of failed auths coming through and noticed they were all coming in on the same switchport.

The switch is a WS-C4510R+E, running version 3.6.3.E... I know its an old version and we are completely replacing this device in the next few months anyway.

When I hopped on the switch to see the output of show auth sess. There was just over 200 auth sessions showing on port Gi10/4. I cleared them all out except for the ONE laptop that is plugged into that port. I just checked again about 20 min later and there was about 10 sessions showing. Checked again 10-15 min later and there's just over 30 now.

Here's the Switchport config:

4510R-HQ#show run int Gi10/4
Building configuration...

Current configuration : 715 bytes
!
interface GigabitEthernet10/4
 switchport access vlan 114
 switchport mode access
 switchport voice vlan 124
 speed 100
 authentication event fail action next-method
 authentication event server dead action authorize vlan 114
 authentication event server dead action authorize voice
 authentication event server alive action reinitialize
 authentication host-mode multi-auth
 authentication order dot1x mab
 authentication priority dot1x mab
 authentication port-control auto
 authentication violation restrict
 mab
 snmp trap mac-notification change added
 snmp trap mac-notification change removed
 dot1x pae authenticator
 dot1x timeout tx-period 10
 macro description enable-ise
 spanning-tree portfast
end

Show auth sessions output:

4510R-HQ#show auth sess int Gi10/4

Interface    MAC Address    Method  Domain  Status Fg Session ID
----------------------------------------------------------------------
Gi10/4       61a0.c176.d1a5 N/A     UNKNOWN Unauth    C0A80201000176C04BFB3580
Gi10/4       1f8e.3d00.0000 N/A     UNKNOWN Unauth    C0A80201000176CA4BFB3FAC
Gi10/4       4537.da2b.77b5 N/A     UNKNOWN Unauth    C0A80201000176B94BFB3578
Gi10/4       0000.9f58.0000 N/A     UNKNOWN Unauth    C0A80201000176C74BFB3584
Gi10/4       d2b1.d282.ecb1 N/A     UNKNOWN Unauth    C0A80201000176B34BFA1D9C
Gi10/4       45c4.200d.557a N/A     UNKNOWN Unauth    C0A80201000176AD4BFA1D98
Gi10/4       12c9.a691.f0ec N/A     UNKNOWN Unauth    C0A80201000176C44BFB3584
Gi10/4       077e.f6bd.a821 N/A     UNKNOWN Unauth    C0A80201000176BE4BFB357C
Gi10/4       4d59.23a1.f473 N/A     UNKNOWN Unauth    C0A80201000176B14BFA1D9C
Gi10/4       69d3.ed10.a18c N/A     UNKNOWN Unauth    C0A80201000176AF4BFA1D98
Gi10/4       d046.0cb7.XXXX dot1x   DATA    Auth      C0A802010001732445709CE4 ----> Only device actually plugged in
Gi10/4       c167.1947.532f N/A     UNKNOWN Unauth    C0A80201000176C34BFB3580
Gi10/4       cd6d.21f4.a7dd N/A     UNKNOWN Unauth    C0A80201000176C94BFB3FAC
Gi10/4       e374.c3f6.382f N/A     UNKNOWN Unauth    C0A80201000176C54BFB3584
Gi10/4       0000.a12f.6393 N/A     UNKNOWN Unauth    C0A80201000176CC4BFB3FB0
Gi10/4       0000.3d00.0000 N/A     UNKNOWN Unauth    C0A80201000176CB4BFB3FB0
Gi10/4       52d9.a5bf.fdb6 N/A     UNKNOWN Unauth    C0A80201000176BF4BFB3580
Gi10/4       2b3e.a557.c2ba N/A     UNKNOWN Unauth    C0A80201000176B64BFB3578
Gi10/4       491c.9f58.0000 N/A     UNKNOWN Unauth    C0A80201000176C84BFB3584
Gi10/4       1346.35ec.87c3 N/A     UNKNOWN Unauth    C0A80201000176AC4BFA1D98
Gi10/4       3d59.6306.96a2 N/A     UNKNOWN Unauth    C0A80201000176C24BFB3580
Gi10/4       b628.f702.23df N/A     UNKNOWN Unauth    C0A80201000176BB4BFB357C
Gi10/4       aaea.613a.4fe5 N/A     UNKNOWN Unauth    C0A80201000176B74BFB3578
Gi10/4       0000.5145.28ba N/A     UNKNOWN Unauth    C0A80201000176C64BFB3584
Gi10/4       0000.d282.ecb1 N/A     UNKNOWN Unauth    C0A80201000176B44BFA1D9C
Gi10/4       a02d.fa3f.4439 N/A     UNKNOWN Unauth    C0A80201000176BC4BFB357C
Gi10/4       241c.1243.f7a7 N/A     UNKNOWN Unauth    C0A80201000176AE4BFA1D98
Gi10/4       9083.e11c.4c97 N/A     UNKNOWN Unauth    C0A80201000176B24BFA1D9C
Gi10/4       28bd.1fd0.3b23 N/A     UNKNOWN Unauth    C0A80201000176B84BFB3578
Gi10/4       6dbb.8184.77ed N/A     UNKNOWN Unauth    C0A80201000176BD4BFB357C
Gi10/4       11c9.05b7.9d78 N/A     UNKNOWN Unauth    C0A80201000176C14BFB3580
Gi10/4       74a4.d6fa.e6c0 N/A     UNKNOWN Unauth    C0A80201000176BA4BFB357C
Gi10/4       a760.2457.35d7 N/A     UNKNOWN Unauth    C0A80201000176B04BFA1D98


Key to Session Events Blocked Status Flags:

  A - Applying Policy (multi-line status for details)
  D - Awaiting Deletion
  F - Final Removal in progress
  I - Awaiting IIF ID allocation
  N - Waiting for AAA to come up
  P - Pushed Session
  R - Removing User Profile (multi-line status for details)
  U - Applying User Profile (multi-line status for details)
  X - Unknown Blocker

Runnable methods list:
  Handle  Priority  Name
    17       5      dot1x
    18       10     mab
    21       15     webauth

And here is a small snippet from the logging:

869198: Aug 27 11:48:34 EDT: %DOT1X-5-FAIL: Authentication failed for client (2894.c7c3.342e) on Interface Gi10/4 AuditSessionID C0A80201000175854A8FF1F8
869199: Aug 27 11:48:34 EDT: %DOT1X-5-FAIL: Authentication failed for client (8dd9.8513.1c48) on Interface Gi10/4 AuditSessionID C0A80201000175EA4AFADB00
869200: Aug 27 11:48:34 EDT: %DOT1X-5-FAIL: Authentication failed for client (728e.7ce3.37b5) on Interface Gi10/4 AuditSessionID C0A80201000175844A8FF1F8
869201: Aug 27 11:48:34 EDT: %DOT1X-5-FAIL: Authentication failed for client (c1e7.78de.85e0) on Interface Gi10/4 AuditSessionID C0A80201000175EB4AFADB04
869202: Aug 27 11:48:34 EDT: %DOT1X-5-FAIL: Authentication failed for client (d40a.5f5c.04b4) on Interface Gi10/4 AuditSessionID C0A80201000175824A8FF1F8
869203: Aug 27 11:48:34 EDT: %DOT1X-5-FAIL: Authentication failed for client (0a7b.9d97.dfd8) on Interface Gi10/4 AuditSessionID C0A802010001758E4A8FF200
869204: Aug 27 11:48:34 EDT: %DOT1X-5-FAIL: Authentication failed for client (0000.814e.8837) on Interface Gi10/4 AuditSessionID C0A80201000175ED4AFADB04
869205: Aug 27 11:48:34 EDT: %DOT1X-5-FAIL: Authentication failed for client (ccc0.d8cc.99d1) on Interface Gi10/4 AuditSessionID C0A80201000175E54AFADAFC
869206: Aug 27 11:48:34 EDT: %DOT1X-5-FAIL: Authentication failed for client (eb58.61c2.aeb6) on Interface Gi10/4 AuditSessionID C0A80201000175E64AFADAFC
869207: Aug 27 11:48:34 EDT: %DOT1X-5-FAIL: Authentication failed for client (e92d.0ce6.9fb4) on Interface Gi10/4 AuditSessionID C0A80201000175834A8FF1F8
869208: Aug 27 11:48:34 EDT: %DOT1X-5-FAIL: Authentication failed for client (4957.838c.fd0c) on Interface Gi10/4 AuditSessionID C0A802010001758B4A8FF1FC
869209: Aug 27 11:48:34 EDT: %DOT1X-5-FAIL: Authentication failed for client (0000.7f77.0000) on Interface Gi10/4 AuditSessionID C0A802010001758F4A8FF200
869210: Aug 27 11:48:34 EDT: %DOT1X-5-FAIL: Authentication failed for client (131d.ec18.c03e) on Interface Gi10/4 AuditSessionID C0A80201000175814A8FF1F4
869211: Aug 27 11:48:34 EDT: %DOT1X-5-FAIL: Authentication failed for client (d6f8.9a86.7959) on Interface Gi10/4 AuditSessionID C0A80201000175884A8FF1FC
869212: Aug 27 11:48:34 EDT: %MAB-5-FAIL: Authentication failed for client (728e.7ce3.37b5) on Interface Gi10/4 AuditSessionID C0A80201000175844A8FF1F8
869213: Aug 27 11:48:34 EDT: %MAB-5-FAIL: Authentication failed for client (ccc0.d8cc.99d1) on Interface Gi10/4 AuditSessionID C0A80201000175E54AFADAFC
869214: Aug 27 11:48:34 EDT: %MAB-5-FAIL: Authentication failed for client (2894.c7c3.342e) on Interface Gi10/4 AuditSessionID C0A80201000175854A8FF1F8
869215: Aug 27 11:48:34 EDT: %MAB-5-FAIL: Authentication failed for client (4957.838c.fd0c) on Interface Gi10/4 AuditSessionID C0A802010001758B4A8FF1FC
869216: Aug 27 11:48:34 EDT: %MAB-5-FAIL: Authentication failed for client (8dd9.8513.1c48) on Interface Gi10/4 AuditSessionID C0A80201000175EA4AFADB00
869217: Aug 27 11:48:34 EDT: %MAB-5-FAIL: Authentication failed for client (e92d.0ce6.9fb4) on Interface Gi10/4 AuditSessionID C0A80201000175834A8FF1F8
869218: Aug 27 11:48:34 EDT: %MAB-5-FAIL: Authentication failed for client (0000.814e.8837) on Interface Gi10/4 AuditSessionID C0A80201000175ED4AFADB04
869219: Aug 27 11:48:34 EDT: %MAB-5-FAIL: Authentication failed for client (d40a.5f5c.04b4) on Interface Gi10/4 AuditSessionID C0A80201000175824A8FF1F8
869220: Aug 27 11:48:34 EDT: %MAB-5-FAIL: Authentication failed for client (eb58.61c2.aeb6) on Interface Gi10/4 AuditSessionID C0A80201000175E64AFADAFC
869221: Aug 27 11:48:34 EDT: %MAB-5-FAIL: Authentication failed for client (0000.7f77.0000) on Interface Gi10/4 AuditSessionID C0A802010001758F4A8FF200
869222: Aug 27 11:48:34 EDT: %MAB-5-FAIL: Authentication failed for client (0a7b.9d97.dfd8) on Interface Gi10/4 AuditSessionID C0A802010001758E4A8FF200
869223: Aug 27 11:48:34 EDT: %MAB-5-FAIL: Authentication failed for client (131d.ec18.c03e) on Interface Gi10/4 AuditSessionID C0A80201000175814A8FF1F4
869224: Aug 27 11:48:34 EDT: %MAB-5-FAIL: Authentication failed for client (c1e7.78de.85e0) on Interface Gi10/4 AuditSessionID C0A80201000175EB4AFADB04
869225: Aug 27 11:48:34 EDT: %MAB-5-FAIL: Authentication failed for client (d6f8.9a86.7959) on Interface Gi10/4 AuditSessionID C0A80201000175884A8FF1FC

 Any ideas what could be going on here?

The port plugs into a Dell docking station which has a single Dell laptop connected to it.

Thanks in Advance,
Matt

[Arne Bier]

Unsure - I don't suppose you have some sort of traffic generator running on your attached PC?  I have not heard of docking stations going mad like that either - and the fact that it's 802.1X ... almost as if the attached devices are generating random MAC addresses to obfuscate their identity and then perform 802.1X.  In the wireless world I would see that as normal (randomise MAC address when probing for networks).

I suspect it's a faulty network device driver generating spurious MAC addresses.

What happens when you clear access-sessions on that interface? Do they all come back?

What does the mac address table look like?  show mac address int x/y/z ?

The IOS version is a bit old - but if you have other devices that are not exhibiting the same behaviour, then it looks more likely to be the device driver for the Ethernet chip inside the dock.

[Matthew Martin]

Thanks for the reply.

Yes, VERY ancient...!!! We are actually scrapping the 4510 before the end of the year.

Strange indeed... If I clear all the sessions on the port, they don't all come back immediately. When I saw it initially, I cleared it out and after about an hour there was about 30 Unath'ed sessions showing. In that office, where this port is, the cable is going directly from the wall into a Dell docking station as mentioned.

I do understand what you're saying about the randomized Mac... As if it were creating a new sessions each time it reconnected. I'm pretty sure we disabled Randomized Macs for the Wireless adapters. Don't know if that's an option for wired. But, the Mac for the laptop itself doesn't seem to ever change.

Now that the person is gone for the day I'm going to do a Shut, no shut on the port and see what it looks like in the morning.

Will comment back tomorrow when I know more.

Thanks Again,
Matt

[Matthew Martin · Accepted solution]

Well it's been about 18 hours or so since I did the shut/no shut on the port and so far it's just the Dell laptop appearing on that port...

No idea what caused this... But I guess it's cleared up for now.

Thanks for the assistance!