IDS 4215 CPU 100 % high utilization issue

bastien.migette — Sun, 10 Mar 2019 11:55:26 GMT

Hi there, I upgraded my IDS 4215 to the 6.0.1 version, I got 512 Mb of ram, but the system is very slow, so I created a service user, then I ran top, and I saw that near 100% of CPU is used (see code below) by the sensor app, and then it crashes.
The CPU values below are after a fresh recovery of the IPS, and there is no traffic sent to the sensor, only management interface is connected.

I checked some logs:
ids4215# show clock
*11:47:09 UTC Sat Mar 13 2010

top - 11:24:47 up 0 min,  1 user,  load average: 0.71, 0.20, 0.06
Tasks:  48 total,   5 running,  43 sleeping,   0 stopped,   0 zombie
Cpu(s😞   0.3% user,  49.3% system,  50.0% nice,   0.3% idle
Mem:    490260k total,   485500k used,     4760k free,     1608k buffers
Swap:        0k total,        0k used,        0k free,    91600k cached

  PID USER      PR  NI  VIRT  RES  SHR S %CPU %MEM    TIME+  COMMAND            
  388 cids      18   5 10584 398m 394m R 81.0 83.2   0:17.22 sensorApp          
  368 cids      17  15  9348 9284 6936 R  2.6  1.9   0:00.13 mainApp            
  398 admin     14   0   988  988  800 R  1.7  0.2   0:00.16 top                
    4 root       9   0     0    0    0 S  1.3  0.0   0:00.10 kswapd             
   47 root       9   0     0    0    0 S  0.3  0.0   0:00.08 kjournald          
  305 cids      13   5  9348 9284 6936 S  0.3  1.9   0:00.71 mainApp            
  306 cids       9   0  9348 9284 6936 S  0.3  1.9   0:00.03 mainApp            
  339 cids      13   5  9348 9284 6936 S  0.3  1.9   0:00.11 mainApp            
  366 cids      17  15  9348 9284 6936 S  0.3  1.9   0:00.02 mainApp            
    1 root       8   0   576  576  492 S  0.0  0.1   0:09.91 init               
    2 root       9   0     0    0    0 S  0.0  0.0   0:00.00 keventd            
    3 root      18  19     0    0    0 S  0.0  0.0   0:00.00 ksoftirqd_CPU0     
    5 root       9   0     0    0    0 S  0.0  0.0   0:00.00 bdflush            
    6 root       9   0     0    0    0 S  0.0  0.0   0:00.00 kupdated           
   72 root       9   0     0    0    0 S  0.0  0.0   0:00.06 kjournald          
  104 root       9   0   584  584  504 S  0.0  0.1   0:00.01 syslogd            
  107 root       9   0   572  572  496 S  0.0  0.1   0:00.00 klogd

//or :

sensor# sh stat host | begin CPU
CPU Statistics
   Usage over last 5 seconds = 100
   Usage over last minute = 98
   Usage over last 5 minutes = 98

//on the sh ver:
Using 405282816 out of 502026240 bytes of available memory (80% usage)
system is using 17.8M out of 29.0M bytes of available disk space (61% usage)
application-data is using 35.0M out of 166.8M bytes of available disk space (22% usage)
boot is using 37.7M out of 68.6M bytes of available disk space (58% usage)

I reimaged with a 6.04 image, and I got the same error.

everytime, I use a .img file, booting on rommon, so I assume it format the flash.

I checked the log and it shows me this:

-bash-2.05b$ tail -n 50 /var/log/messages
Mar 13 12:21:32 sensor user.notice /etc/init.d/S80cids: Clean up:
Mar 13 12:21:34 sensor user.notice /etc/init.d/S80cids: mainApp -d startup
Mar 13 12:21:35 sensor user.notice root: mainApp (cids) started
Mar 13 12:21:35 sensor user.notice /etc/init.d/S80cids: Starting CIDS:
Mar 13 12:21:36 sensor user.notice kernel: e100: fe0_0 NIC Link is Up 100 Mbps Full duplex
Mar 13 12:21:38 sensor daemon.info init: ^MStarting pid 385, console /dev/tty1: '/sbin/getty'
Mar 13 12:21:38 sensor daemon.info init: ^MStarting pid 386, console /dev/tty2: '/sbin/getty'
Mar 13 12:21:38 sensor daemon.info init: ^MStarting pid 387, console /dev/ttyS0: '/sbin/getty'
Mar 13 12:21:38 sensor daemon.info init: ^MStarting pid 388, console /dev/ttyS1: '/sbin/getty'
Mar 13 12:22:58 sensor daemon.info init: ^MStarting pid 416, console /dev/null: '/etc/init.d/rc.down'
Mar 13 12:22:58 sensor user.err kernel: Out of Memory: Killed process 389 (sensorApp).
Mar 13 12:22:58 sensor user.err kernel: Out of Memory: Killed process 389 (sensorApp).
Mar 13 12:22:58 sensor user.err kernel: Out of Memory: Killed process 411 (sensorApp).
Mar 13 12:22:58 sensor user.err kernel: Out of Memory: Killed process 412 (sensorApp).
Mar 13 12:22:58 sensor user.err kernel: Out of Memory: Killed process 413 (sensorApp).
Mar 13 12:22:58 sensor user.err kernel: Out of Memory: Killed process 414 (sensorApp).
Mar 13 12:22:58 sensor user.err kernel: Out of Memory: Killed process 346 (mainApp).
Mar 13 12:22:58 sensor user.err kernel: Out of Memory: Killed process 328 (mainApp).
Mar 13 12:22:58 sensor user.err kernel: Out of Memory: Killed process 331 (mainApp).
Mar 13 12:22:58 sensor user.err kernel: Out of Memory: Killed process 332 (mainApp).
Mar 13 12:22:58 sensor user.err kernel: Out of Memory: Killed process 333 (mainApp).
Mar 13 12:22:58 sensor user.err kernel: Out of Memory: Killed process 346 (mainApp).
Mar 13 12:22:58 sensor user.err kernel: Out of Memory: Killed process 347 (mainApp).
Mar 13 12:22:58 sensor user.err kernel: Out of Memory: Killed process 348 (mainApp).
Mar 13 12:22:58 sensor user.err kernel: Out of Memory: Killed process 363 (mainApp).
Mar 13 12:22:58 sensor user.err kernel: Out of Memory: Killed process 364 (mainApp).
Mar 13 12:22:58 sensor user.err kernel: Out of Memory: Killed process 365 (mainApp).
Mar 13 12:22:58 sensor user.err kernel: Out of Memory: Killed process 366 (mainApp).
Mar 13 12:22:58 sensor user.err kernel: Out of Memory: Killed process 367 (mainApp).
Mar 13 12:22:58 sensor user.err kernel: Out of Memory: Killed process 368 (mainApp).
Mar 13 12:22:58 sensor user.err kernel: Out of Memory: Killed process 369 (mainApp).
Mar 13 12:22:58 sensor user.err kernel: Out of Memory: Killed process 370 (mainApp).
Mar 13 12:22:58 sensor user.err kernel: Out of Memory: Killed process 371 (mainApp).
Mar 13 12:22:58 sensor user.err kernel: Out of Memory: Killed process 372 (mainApp).
Mar 13 12:22:58 sensor user.err kernel: Out of Memory: Killed process 373 (mainApp).
Mar 13 12:22:58 sensor user.err kernel: Out of Memory: Killed process 374 (mainApp).
Mar 13 12:22:58 sensor user.err kernel: Out of Memory: Killed process 375 (mainApp).
Mar 13 12:22:58 sensor user.err kernel: Out of Memory: Killed process 376 (mainApp).
Mar 13 12:22:58 sensor user.err kernel: Out of Memory: Killed process 377 (mainApp).
Mar 13 12:22:58 sensor user.err kernel: Out of Memory: Killed process 390 (mainApp).
Mar 13 12:22:58 sensor user.err kernel: Out of Memory: Killed process 392 (mainApp).
Mar 13 12:22:58 sensor user.err kernel: Out of Memory: Killed process 121 (inetd).
Mar 13 12:22:58 sensor user.err kernel: Out of Memory: Killed process 121 (inetd).
Mar 13 12:22:58 sensor user.err kernel: Out of Memory: Killed process 1 (init).
Mar 13 12:22:58 sensor user.err kernel: Out of Memory: Killed process 1 (init).
Mar 13 12:22:59 sensor auth.info sshd[125]: Received signal 15; terminating.
Mar 13 12:22:59 sensor user.notice /etc/init.d/S60ssh: sshd -TERM
Mar 13 12:23:00 sensor auth.info login(pam_unix)[387]: session opened for user admin by (uid=0)
Mar 13 12:23:00 sensor auth.info -- admin[387]: DIALUP AT ttyS0 BY admin
Mar 13 12:23:00 sensor auth.info -- admin[387]: LOGIN ON ttyS0 BY admin

I have just upgraded to 512M to upgrade to 6.0x version, So i guess the sensorapp is doing some nasty things that hangs up all the CPU and then it crashes... I supect a signature error, wouldn't it be erased after a reimaging or a recovery ?

Thanks for your help.

Re: IDS 4215 CPU 100 % high utilization issue

bastien.migette — Sat, 13 Mar 2010 18:37:36 GMT

can someone try these commands and see that I really got a problem : ?

sensor(config)# username admin passw password priv service
sensor(config)# exit
sensor# exi
login: admin
Password:
bash-2.05b$su
Password: !same pass as admin
bash-2.05b# cd /usr/cids/idsRoot/etc/config/analysisEngine/
bash-2.05b# ls
current.xml default.xml typedefs.xml
default.sig.xml typedefs.sig.xml

bash-2.05b# cat current.xml
£ }Ecurrent.xml5ÍM@0@á½S4Ý3õ³’_
˜“ÐŠ)·Äî[½g1†‘&%×æyð;S
®Œi_Sç¦Ôê\—ÀžE¶ ¥T 1Æã
HŽÜ“Ñ}¦”eÙ”—ðÛÂ·é³‡spbash-2.05b#
bash-2.05b#

topic IDS 4215 CPU 100 % high utilization issue in Network Security

IDS 4215 CPU 100 % high utilization issue

Re: IDS 4215 CPU 100 % high utilization issue