ISE 2.4 P8 Health Status Unavailable : Server=ise1

[cmendezm]

Hello Team, 

We are constantly getting alarms for the Health status but the local store is showing that the notifications are generated every five minutes. This is a two ISE node deployment and ISE 1 is the Primary MNT which is generating the alarm for him self

Where can I see if ISE is not parsing or if the alert arrived a little later in the logs and for this reason the alarm is triggered?

Alarm Health Status Unavailable     warn     Health Status Unavailable : Server=ise1     18-JUL-19 06.15.03.606549 AM -03:00

2019-07-18 06:00:17.253 -03:00 0001044262 70000 NOTICE System-Stats: ISE Utilization
2019-07-18 06:05:17.280 -03:00 0001044685 70000 NOTICE System-Stats: ISE Utilization
2019-07-18 06:10:17.312 -03:00 0001045285 70000 NOTICE System-Stats: ISE Utilization
2019-07-18 06:15:17.340 -03:00 0001045706 70000 NOTICE System-Stats: ISE Utilization
2019-07-18 06:20:17.358 -03:00 0001046305 70000 NOTICE System-Stats: ISE Utilization
2019-07-18 06:25:17.390 -03:00 0001046725 70000 NOTICE System-Stats: ISE Utilization
2019-07-18 06:30:17.404 -03:00 0001047324 70000 NOTICE System-Stats: ISE Utilization
2019-07-18 06:35:17.435 -03:00 0001047747 70000 NOTICE System-Stats: ISE Utilization
2019-07-18 06:40:17.454 -03:00 0001048352 70000 NOTICE System-Stats: ISE Utilization
2019-07-18 06:45:17.474 -03:00 0001048773 70000 NOTICE System-Stats: ISE Utilization
2019-07-18 06:50:17.488 -03:00 0001049372 70000 NOTICE System-Stats: ISE Utilization
2019-07-18 06:55:17.510 -03:00 0001049792 70000 NOTICE System-Stats: ISE Utilization

[Damien Miller]

There are a few issues that can cause this but you will have to work with TAC to diagnose. Your thought process is correct here, the most common cause is if the syslog buffers do not get ingested by the collector process quickly enough, then the health status alarm will be triggered since ISE think's the node is missing.

I am working through this on 2.4 p8/9 right now, and it was also a frequent issue for a number of my customers on earlier patches. I was not expecting to face this issue again since most of those were corrected in earlier patches.

If by chance you are leveraging TACACS on this deployment, then we may be facing the same issue and I could share the case number for TAC to check on.

[navtom]

Hello Damien,

were you able to resolve this situation? It seems we have encountered it too. Strangely enough during weekends, when there is no heavy load.

[Damien Miller · Accepted solution]

I would recommend any individual facing health status unavailable alarms within their ISE deployment to work with TAC. There are many reason that this can happen and it's quite hard to say.

At least in my last case, the hotfix issued is supposed to have the bug fixed in patch 10, but I have yet to try it.
https://bst.cloudapps.cisco.com/bugsearch/bug/CSCvq73457

[mpofut0901]

This is what Cisco TAC said about this issue

"Kindly note that this behavior is due to bug CSCvo87602 as I also could see high memory utilization in the show tech. The workaround for this bug is to reload the ISE and it is fixed in ISE 2.4 patch 11 and ISE 2.6 patch 5.

Start with the Primary Admin node as it is also the Primary MnT, wait for 5-10 minutes after the node is up, if the issue was not solved please proceed with reloading the secondary node"

After reloading the Primary Node. The issue did not go away. The Nodes were then synchronised: Administration>Deployment>Select Secondary Node>Syncup. The health status unavailable alarm stopped and on the Home Screen>System Summary, the CPU status of the secondary Node is no longer grayed out - Both Nodes now have Green Ticks.