Successful AP 2702i authentications "spam" the ISE2.6 livelog

[Worko]

Have a strange low-priority problem. We currently build a new network for a customer with following Cisco products:

• Access = Catalyst 3650-48FD (XE 16.9.4)
• WLC = 5520 (8.5.161.0)
• APs = 2702i
• ISE = 2.6 (patch 4)

We use MAB authentication for the APs with pre-defined OUI authorization policies and also MAC address groups defined on the ISE. Pretty easy and straight forward scenario. The problem is, that the presence of two APs we currently use for testing is reoccurring exactly every 5 minutes in the live log of the ISE. I assume that once we go live with all cca. 700 APs, the live-log will be full of these reoccurring positive actions.

See below in the image. The APs starting with 78:72:5D and 70:6D:15 are the ones we've been testing with:

I checked the interface configuration and removed the authentication timer reauthenticate server which wasn't used anyway, since the session-timeout was not applied in the authz profile. After deletion the authentication timer is locally set to 3600 seconds as per default. Now the port configuration looks following:

interface GigabitEthernet1/0/5
description DEFAULT
switchport access vlan XXX
switchport mode access
switchport nonegotiate
switchport voice vlan YYY
device-tracking
authentication event server dead action authorize vlan XXX
authentication event no-response action authorize vlan XXX
authentication host-mode multi-auth
authentication order mab dot1x
authentication priority mab dot1x
authentication port-control auto
authentication periodic
authentication timer restart 30
authentication timer inactivity 600
authentication violation restrict
mab
dot1x pae authenticator
spanning-tree portfast

I did also try to debug (everything related to AAA and RADIUS) on the switch and the CAPWAP AP, but nothing showed up. The RADIUS protocol configuration is set to defaults: Suppress Successful Reports = YES .

Does anybody know a quick fix? It will definitely not influence operations and productions in any ways. But for troubleshooting it will be important to have the live-log as clean as possible.

Thanx

[Worko]

Just a not related side note: I was trying to deploy 802.1x authentication for the APs but after a day of troubleshooting I gave up. Everything was set up pretty simple and the APs were not able to get a successful authorization result neither with a:

• correctly configured supplicant credentials in WLC GUI, nor
• with a correctly set supplicant credentials on the AP via console

That's why we decided to go with the OUI based condition and after deployment put the MACs into a pre-defined MAC group.

[Cristian Matei]

Hi,

   I never ran into issues with having the AP act as 802.1x supplicant in order to be authenticated on the wired side.This document is good enough, what exactly was not working?

https://www.cisco.com/c/en/us/support/docs/wireless-mobility/wireless-fixed/107946-LAP-802-1x.html

Regards,

Cristian Matei.

[Worko]

Regarding the 802.1x AP authentication. I did follow exactly the link you posted Cristian and always get a successful authentication in the ISE looking like this:

As you can see, that AUTH process is successful, but for some reason ISE doesn't give the Result (and smartport) back. It works great with MAB btw. The last line of this successful session in the details are:

12104 Extracted EAP-Response containing EAP-FAST challenge-response
11401 Prepared RADIUS Access-Reject after the successful in-band PAC provisioning
61025 Open secure connection with TLS peer
11504 Prepared EAP-Failure
11003 Returned RADIUS Access-Reject

The switch'es output in this case is:

001578: Mar 5 13:02:40.439: %DOT1X-5-FAIL: Switch 1 R0/0: sessmgrd: Authentication failed for client (7872.5dXX.XXXX) with reason (Cred Fail) on Interface Gi6/0/47 AuditSessionID 0A7A2010000000C5AAC245ED
001579: Mar 5 13:02:40.440: %SESSION_MGR-5-FAIL: Switch 1 R0/0: sessmgrd: Authorization failed or unapplied for client (7872.5dXX.XXXX) on Interface GigabitEthernet6/0/47 AuditSessionID 0A7A2010000000C5AAC245ED. Failure reason: Authc fail. Authc failure reason: Cred Fail.

I have tested several usernames and it simply doesn't work with any of them. Funny part is that the second authentication method is MAB and it even doesn't go there. It constantly just circles around (every 30 seconds)  dot1x.

[Colby LeMaire · Accepted solution]

In your switch global configuration, use the command "aaa accounting update newinfo" to send accounting records only when there is new information to report.  It sounds like these records are being sent every 5 minutes which updates the session information in ISE.

[Worko]

That helped! I was analyzing this feature yesterday but obviously misunderstood its purpose.

Thanx a lot!

[Cristian Matei]

Hi,

   I see the two lines are for different endpoints, and the "Repeat" counter increased, so you only got one log per device, so log suppression seems to be working. What is your exact problem?

Regards,

Cristian Matei.

[Worko]

That's true that the repeat counter slowly increases which would not be the problem. In contrary - this is desired. 

The problem is that these two APs show exactly every 5 minutes up with the blue "i" Status sign on the top of the live log. Now imagine having hundreds of devices coming on top every 5 minutes. Live log would be litteraly stormed by these positive session updates.

Is this behavior the normal one?

[Greg Gibbs]

Yes, this is the intended behaviour. The Live Logs are intended to be used for live troubleshooting or basic analysis. You can use the filters to limit what information is seen (for the past 24 hours), but this is not useful for 'big picture' analysis.

For more historical analysis, you can use the built-in ISE reports. The reports are not highly customizable, however, so most customers send auth events via syslog to a more purpose-built logging solution (like Splunk) and build dashboards and search queries there.