topic Re: ISE 2.4 Patch 0 - CPU and Memory problems in Network Access Control

ISE 2.4 Patch 9 - CPU and Memory problems

OrkhanRustamli — Mon, 25 Nov 2019 16:10:54 GMT

Hey all,

I have 2 ISE nodes (Active/Standby PAN) and both are PSN. Month ago I had a weird problem: Memory in my first node was in 90 percentages although 802.1x were not deployed on network, I mean it was idle, only pxGrid was used. I opened a TAC case for that and after one day of opening the case, I noticed that Memory downgraded to 30s% that was awesome 🙂

Today I got notified that users are claiming that they cannot authenticate and access Wifi for 10 mins and then everything turned to be okay. I checked ISE which showed very high "Authentication Latency" for that period. While I was investigating the problem, ISE stopped to show system summary for that node:

I connected to node via SSH and checked system utilization there and noticed one process is taking huge amount of CPU:

I do not know what can be cause of the problem? Am i hitting any bug? I think I have to do reboot but I wanted to make sure before doing it.

Thanks in advance!

Re: ISE 2.4 Patch 0 - CPU and Memory problems

Jason Kunst — Mon, 25 Nov 2019 16:05:37 GMT

All troubleshooting break fix priority needs to go through TAC

Why are you running patch 0!? You should be on latest patch unless tac says otherwise

Re: ISE 2.4 Patch 0 - CPU and Memory problems

Damien Miller — Mon, 25 Nov 2019 16:07:13 GMT

Are you actually running 2.4 with no patches? Or was that a typo and you meant patch 10? It would be highly recommended that you patch the deployment to patch 10 if you are not there already.

There are hundreds of fixed bugs, of which, quite a few present symptoms like you are seeing. If you are running patch 10, there is an unpatched memory leak that can also cause similar features.

Re: ISE 2.4 Patch 0 - CPU and Memory problems

OrkhanRustamli — Mon, 25 Nov 2019 16:11:37 GMT

@Jason Kunst sorry, my bad. It was typing error. I am.using patch 9

Re: ISE 2.4 Patch 0 - CPU and Memory problems

OrkhanRustamli — Mon, 25 Nov 2019 16:12:33 GMT

@Damien Miller sorry, as you said mis typo. It is patch 9

Re: ISE 2.4 Patch 0 - CPU and Memory problems

Damien Miller — Mon, 25 Nov 2019 16:36:01 GMT

You may be impacted by this memory leak https://bst.cloudapps.cisco.com/bugsearch/bug/CSCvo87602
You would have to open a tac case to confirm, a fix is coming in a later patch and the temporary solution is to reload affected nodes.

Patch 10 resolved 101 bugs that patch 9 did not. It would still be beneficial to move to it to rule them out.
https://www.cisco.com/c/en/us/td/docs/security/ise/2-4/release_notes/b_ise_24_rn.html#id_122459