ISE v2.3 p6 alerts: Health Status Unavailable : Server=SERVER_NAME

[bortnichie]

We upgraded ISE servers from v.2.3 patch 1 to v.2.3 patch 6.  These are physical servers (not VM)

Ever since, we started getting email alerts around the same time every morning for each node (admin, monitoring, policy)

Health Status Unavailable : Server=SERVER_NAME

Has anyone encountered this issue before?

Here is one of the alerts:

Alarm Name : Health Status Unavailable

Details : Health Status Unavailable : Server=SERVER_NAME

Description : The M&T node has not received the health status from the ISE node

Severity : Warning

Suggested Actions :

Please ensure if ISE nodes are up and running, Please ensure if ISE nodes are able to communicate to M&T nodes

*** This message is generated by Cisco Identity Services Engine (ISE) ***

Sent By Host : ISE_MONITORING_SERVER

I read this blog (link), but I dont see "logs similar to “NOTICE System-Stats: ISE Process Health"

[Damien Miller]

Are you using TACACS on this deployment? I have an issue we identified on 2.4 that causes this under load. I haven't used 2.3 with any customers deployments but possible the same issues exist.

All of the ISE nodes send health status syslogs to the MNT's. The typical cause for this issues exists not because the nodes stop sending the syslogs (as indicated by Craigs old post), but the MNT collector cannot ingest them quickly enough. Log ingestion lags behind to the point where the nodes appear to be down because the syslogs are still stuck in buffer files. There are a number of reasons why the logs don't get ingested quickly enough, most of them have been fixed.

Is there a reason you are on patch 6 and not patch 7? The following two issues are related to causing these messages and are fixed in 2.3 patch 7.
https://bst.cloudapps.cisco.com/bugsearch/bug/CSCvn12442
https://bst.cloudapps.cisco.com/bugsearch/bug/CSCvq73457

If you move to patch 7 and this is still and issue, you will need to work with TAC.