ISE 1.2.1.198 patch 5 - Operations Authentications not loading or displaying

[bberry]

Is anyone else having an issue with getting Authentications to display under operations? We were running 1.2.0.899 and started to run into a couple bugs so we upgraded to 1.2.1.198. Ever since then the Operations - Authentications have not been working right. I may occasionally see and actual authentication but not as many as I should. Most of the messages I saw yesterday pertained to radius processes already in progress from endpoint which was my wireless controller. Today I just get a loading data message at the bottom of the screen. It does not seem to be affecting system operation as users are still properly authenticating but I am unable to monitor the process or troubleshoot a users if they were to have an issue. We are on the edge of moving this into full production but really cannot until I get this resolved.

I have a case open with tac and their comment was that the issue of authentications not displaying was fixed in 1.2.1 and not sure what may be happening. We went ahead and applied patch 5 just in case there was something else going on. That did not fix things and it now seens to be getting worse.

I just wanted to see if anyone else had seen this and could possible shed some light on a resolution.

I am running a cluster containing the following. Primary admin on a VM - two policy Services servers both on VMs - secondary admin on retired ACS 2111 appliance. All three VMs are on the same physical server. Memory utilization on the admin server is just under 50% with the Policy servers both in the 30% range. I do have one policy server that is showing authentications in the 10-12ms latency but do not think that should affect anything. The ISE cluster is also tied into our 5508 wireless controller for support of the wireless networks. I have two SSIDs in production here at corporate and trying to figure out FlexConnect for the remote locations so we can centralize everything.

Brent

[Charles Hill]

Not sure if this is the same issue we ran into or not. 

Our issue, Authentications were not seen under operations, because they were being supressed.

To disable the suppression, go to:

-> Administration

-> Logging

-> Collection filters

-> Click Add

-> Attribute will be the MAC Address

-> Enter the mac address in the xx:xx:xx:xx:xx:xx format

-> Filter type: disable suppression.

Go back to Operation -> Authentications and this mac address should start popping up.

Hope this helps.

[bberry]

I have nothing displaying not just missing a single device.

My collection filters list is completely blank.

[Charles Hill]

I would run a report on current active sessions, just verify the authentications are there.

-> Operations

-> Reports

-> Endpoints and User

-> Current Acitve Sessions

If they are there,

you could disable suppression on one of the mac addresses, and then reauthenticate and verify if it is displayed under operations -> Authentications.

Sounds like a bug or should I say a caveat.  :)

[bberry]

Started the report - went to lunch with retrieving data and preparing report - came back and the screen still says retrieving data and preparing report.

[Charles Hill]

Just ran the same report and it took a couple of seconds.

Server may need to be rebooted.

If you log in to a switch and do a "show authe sess int g0/1 d"

Does it show authenticated?  If so, do a "dot1x re-authenticate interface g0/1", and see if it reauthenticates.

I would probably restart ISE.

[bberry]

Would you reboot the hardware (VM) or just the ISE services?

I currently do not have any wired clients I will connect and see. All my clients are currently coming across wireless.

[Charles Hill]

What is Cisco Tac recommending?

I do see a bug that describes this scenario, but it was resolved in patch 4.

Have you tried wired authentications and are they showing up?

[bberry]

TAC recommendation was to install patch 5 which should include patch 4 plus other things. They took logs from my servers and asked to give them a day or so to look at the issue. Today is day three with no update.

I am going to reboot all the servers in the cluster tonight. I do not have console access to the VMs so am hoping that I can reload from the CLI and accomplish the same thing rather than just reload the services.

I tried a wired connection this morning and it popped into the authentications report but will have to test to make sure it repeats.

What is mostly in the log is simply the reports of the supplicant stopped responding to ISE. I know thought that I have at least 5 people that are connected via wireless. Here is a sample of what is in the log.

[Charles Hill]

If you run a Health report, does it hang up as well.

Do all of the processes show green within the health report?

I wonder if its only having difficulty retrieving and displaying the authentications.

[bberry]

Regardless of which of the 5 servers in the cluster I pick today comes back in just a few seconds and reports no data available. I picked yesterday and there is a bit longer pause but still comes back no data available.