cancel
Showing results for 
Search instead for 
Did you mean: 
cancel
Announcements
Learn how to use ISE with the latest Cisco Wireless Controller in our ISE and Catalyst 9800 Series Integration Guide!
The ISE 2.5 Beta software is now available! Join the ISE Beta Community to try it in your lab!

Choose one of the topics below for ISE Resources to help you on your journey with ISE

This community is for technical, feature, configuration and deployment questions.
For production deployment issues, please contact the TAC!
We will not comment or assist with your TAC case in these forums.

94
Views
0
Helpful
4
Replies
Beginner

Cisco ISE 2.3 Patch 5 - Alarms: Health Status Unavailable

Hello everyone ,

 

We have an ISE implementation with 2 PAN nodes that are also and PSN and MnT nodes .

The last 2 days we receive alerts for one of the nodes with message "Alarms: Health Status Unavailable" .

 

The issue is that we are not losing the node and all the services seems to work/running fine .

The alert comes every hour but we never lost the node .

I want to refer that the specific node is a VM not physical appliance .

 

Is there any way to find the reason of these alerts and fix the issue .

 

Thank You ,

Palaiologos .

3 ACCEPTED SOLUTIONS

Accepted Solutions
Cisco Employee

Re: Cisco ISE 2.3 Patch 5 - Alarms: Health Status Unavailable

Check iseLocalStore.log from Operations > Troubleshoot > Download Logs > [Choose the node for which the alarms are thrown for] > Debug Logs.

Search for logs similar to “NOTICE System-Stats: ISE Process Health” and see the time difference between these logs. They are supposed to be generated every 5 minutes. If the delay is more than say 10 minutes, your MnT node will throw the error that it did not receive the health status. There is a bug CSCvh17334 for this issue as well which is fixed in 2.4.

Another reason why this could happen is if there is high load on the MnT nodes and that the logs are not processed enough. If you see any latency in the live logs or anything then this could be it, if not then it is most likely not a problem with MnT node.
Beginner

Re: Cisco ISE 2.3 Patch 5 - Alarms: Health Status Unavailable

Hello Everyone ,

 

We performed a reload on the virtual machine and the issue and alarms stopped .

 

Thank You,

Palaiologos

Contributor

Re: Cisco ISE 2.3 Patch 5 - Alarms: Health Status Unavailable

If the issue comes up again, keep in mind that the health status is sent from each node to the MnT nodes. If you're not receiving a health status summary from an ISE node to the active MnT, that could many a few things. 

 

1) It could mean that your resources are very high on either MnT or the syslog client, although that would be noticeable in other ways such as show commands and VM summary within your hypervisor.

2) Maybe you're using Secure Syslog between the two, and have CRL's enabled for their trusted certificate, that one of the nodes can't download the CRL and therefore the Secure Syslog isn't trusted after initial setup. This can be seen within the localstore.log file on both MnT and syslog client. 

3) Maybe you have a firewall between the two nodes and due to the long-lived TCP connection, the connection is being dropped ungracefully by the firewall so that the flow is dropped. 

 

 

4 REPLIES
Cisco Employee

Re: Cisco ISE 2.3 Patch 5 - Alarms: Health Status Unavailable

Check iseLocalStore.log from Operations > Troubleshoot > Download Logs > [Choose the node for which the alarms are thrown for] > Debug Logs.

Search for logs similar to “NOTICE System-Stats: ISE Process Health” and see the time difference between these logs. They are supposed to be generated every 5 minutes. If the delay is more than say 10 minutes, your MnT node will throw the error that it did not receive the health status. There is a bug CSCvh17334 for this issue as well which is fixed in 2.4.

Another reason why this could happen is if there is high load on the MnT nodes and that the logs are not processed enough. If you see any latency in the live logs or anything then this could be it, if not then it is most likely not a problem with MnT node.
Highlighted
Beginner

Re: Cisco ISE 2.3 Patch 5 - Alarms: Health Status Unavailable

Its seems the last 2 days i have the problem i did not receice 

 

NOTICE System-Stats: ISE Process Health messages but i get 

NOTICE System-Stats: ISE Process Health Unavailable messages .

 

I get them every 5 minutes .

 

Also i noticed 

 

2018-12-07 08:24:25.787 +02:00 0066061587 34140 WARN  System-Management: ISE failed secure syslog connection because of unknown certificate in syslog server certificate chain, ConfigVersionId=78, DestinationPort=6514, LoggerName=SecureSyslogCollector, 

 

These messages but i get the same messages when the alarm was not triggered and all works normally without alarms .

 

So any idea ? 

 

Thank You,

Palaiologos

Beginner

Re: Cisco ISE 2.3 Patch 5 - Alarms: Health Status Unavailable

Hello Everyone ,

 

We performed a reload on the virtual machine and the issue and alarms stopped .

 

Thank You,

Palaiologos

Contributor

Re: Cisco ISE 2.3 Patch 5 - Alarms: Health Status Unavailable

If the issue comes up again, keep in mind that the health status is sent from each node to the MnT nodes. If you're not receiving a health status summary from an ISE node to the active MnT, that could many a few things. 

 

1) It could mean that your resources are very high on either MnT or the syslog client, although that would be noticeable in other ways such as show commands and VM summary within your hypervisor.

2) Maybe you're using Secure Syslog between the two, and have CRL's enabled for their trusted certificate, that one of the nodes can't download the CRL and therefore the Secure Syslog isn't trusted after initial setup. This can be seen within the localstore.log file on both MnT and syslog client. 

3) Maybe you have a firewall between the two nodes and due to the long-lived TCP connection, the connection is being dropped ungracefully by the firewall so that the flow is dropped. 

 

 

CreatePlease to create content
Ask the Expert- DMVPN on Cisco routers