10-17-2023 04:44 AM
Hi Team,
Can you help me to verify what process is making memory overutilization in my ISE server.
ise-mgt01/admin# show memory
total memory: 98833168 kB
free memory: 1853356 kB
cached: 30809084 kB
swap-cached: 13292 kB
output of free command:
total used free shared buff/cache available
Mem: 98833168 31477140 1852612 17325252 65503416 49447604
Swap: 8191996 74880 8117116
ise-mgt01/admin
Solved! Go to Solution.
12-26-2023 04:09 PM
@rakesh nair If not already done, please continue working with Cisco TAC to track down the issue with the show cpu usage command on the two ISE admin nodes. The admin nodes usually have more CPU cores than PSN nodes do and that might have triggered this.
01-02-2024 07:10 AM
HI,
I checked with Cisco and they said its another bug.
From Cisco ISE CLI, read-only users can not run a show CPU usage command. |
10-17-2023 06:09 AM
Not possible unless you have "root" access to the ISE.
10-17-2023 06:21 AM
How to go to root profile
10-17-2023 07:11 AM
Hi @rakesh nair ,
To review the process that might be causing you problem in your ISE , review the command "tech top" from the command line the utilization of ISE resources among the processes that are running.
You should see a column indicating the %MEM used by the process, you should get an idea from there.
Let me know if that helped you.
10-17-2023 08:10 AM
Thanks Rodigro, this is the output we see
Tasks: 561 total, 1 running, 560 sleeping, 0 stopped, 0 zombie
%Cpu(s): 0.7 us, 0.2 sy, 0.0 ni, 99.1 id, 0.0 wa, 0.0 hi, 0.0 si, 0.0 st
KiB Mem : 98833168 total, 1847680 free, 31471720 used, 65513768 buff/cache
KiB Swap: 8191996 total, 8116860 free, 75136 used. 49455832 avail Mem
PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
14076 iseadmi+ 20 0 26.4g 5.2g 61432 S 9.3 5.5 167179:42 jsvc
19038 iseelas+ 20 0 14.3g 12.7g 180032 S 7.9 13.5 13788:07 java
16595 iserabb+ 20 0 1944172 170280 3264 S 1.0 0.2 8849:45 beam.smp
9 root 20 0 0 0 0 S 0.3 0.0 208:17.08 rcu_sched
1949 iseprof+ 20 0 702624 553588 1388 S 0.3 0.6 1797:35 redis-serv+
6701 root 20 0 0 0 0 S 0.3 0.0 58:02.07 jbd2/sda7-8
9968 100 20 0 433108 82380 11020 S 0.3 0.1 110:57.44 nginx
15869 root 20 0 164516 2804 1632 R 0.3 0.0 0:00.46 top
10-17-2023 04:11 PM
High memory utilisation is not an indication that there is a problem. Unless you're constantly swapping RAM pages to/from disk, the operating system is making good use of the RAM it has at its disposal. A good operating system will put any additional RAM to good use (e.g. caching). ISE will allocate however much RAM it needs to run the apps. The only time we should be concerned is if the ISE applications memory requirements are steadily increasing over time ... memory leaks. Those situations could lead to memory exhaustion. But in my opinion, if ISE is sitting at 90% memory util all day long, I don't worry.
10-18-2023 05:03 AM
@Arne Bier: Agreed. But you can get TAC to root into the box and check to make sure there are no potential memory leak. Not that hard to do when you have "root" access :-).
The reason ISE always operates at 90% memory utilization is because the Oracle database backend. Oracle will grab as much memory as it can, depending on the platform of 32GB, 64GB, 98GB or 128GB of RAM. This parameter can be configured in Oracle database, if you have "root" access and also access to the Oracle backend.
10-18-2023 06:51 AM
I am working with TAC to see if this is concerning or expected behaviour , also we are planning to upgrade the ISE server too
12-19-2023 11:14 PM
Has TAC come to a useful conclusion? I'm asking because we do have the same issues after the 3.2 upgrade, high RAM usage for two nodes, yet we do not have troubles with the overal situation, except that at one moment for no explainable reason the second node went unresponsive to web gui access and cli access, which forced us to a total reinstall.
12-20-2023 07:23 AM
WE have upgraded ISE server to version 3.2 and update patch 2 , i dont see memory issue currently.
12-20-2023 02:36 PM
Hi @rakesh nair ,
please take a look at Field Notice: FN74005 - Identity Services Engine: Java Heap Size May Significantly Impact System Performance - Software Upgrade Recommended, an issue till ISE 3.0 Patch 6, but fixed on ISE 3.2 Patch 2.
Hope this helps !!!
12-21-2023 11:21 AM
Currently im seeing this on 3.2 for tech top command, but when i do sh cpu usage command , its not showing cpu output now, but old version, it was giving output.
se-mgt01/admin#show cpu usage
ise-mgt01/admin#
12-21-2023 11:53 AM
What patch? I have some systems on 3.2 patch 4 working properly with "tech top", "show cpu", "show cpu statistics", "show cpu usage", and "show memory". BTW, does anyone else love "tech top" then hitting 1 to show you each individual CPU's status?
12-21-2023 12:03 PM
3.2 patch 4 we are using . the show cpu usage output is not showing only on PAN and SAN nodes , but its giving output on monitoring and PSN nodes
12-26-2023 04:09 PM
@rakesh nair If not already done, please continue working with Cisco TAC to track down the issue with the show cpu usage command on the two ISE admin nodes. The admin nodes usually have more CPU cores than PSN nodes do and that might have triggered this.
Discover and save your favorite ideas. Come back to expert answers, step-by-step guides, recent topics, and more.
New here? Get started with these tips. How to use Community New member guide