Hi All,
We are using RHEL 5 64 bit on blade server CISCO UCS B230-M2.
Below is the memory details on the server. I could see from the sar -r command it did not used the swap memory at all but it calls the OOM killer everyday.
su@ /root-> free -g
total used free shared buffers cached
Mem: 125 119 5 0 2 81
-/+ buffers/cache: 35 89
Swap: 251 0 251
The issue is occuring every day and doesnt allow any user to connect to the server in that perticular time, After some time we could see the below messages in /var/log/messages and once the java processes are getting killed it allow the new telnet connections.
Any help on this is much appreciated.
Oct 18 01:47:57 usdc01qd51 kernel: cpu 28 cold: high 62, batch 15 used:10
Oct 18 01:47:57 usdc01qd51 kernel: cpu 29 hot: high 186, batch 31 used:20
Oct 18 01:47:57 usdc01qd51 kernel: cpu 29 cold: high 62, batch 15 used:6
Oct 18 01:47:57 usdc01qd51 kernel: cpu 30 hot: high 186, batch 31 used:93
Oct 18 01:47:57 usdc01qd51 kernel: cpu 30 cold: high 62, batch 15 used:10
Oct 18 01:47:57 usdc01qd51 kernel: cpu 31 hot: high 186, batch 31 used:137
Oct 18 01:47:57 usdc01qd51 kernel: cpu 31 cold: high 62, batch 15 used:6
Oct 18 01:47:57 usdc01qd51 kernel: HighMem per-cpu: empty
Oct 18 01:47:57 usdc01qd51 kernel: Free pages: 37295696kB (0kB HighMem)
Oct 18 01:47:57 usdc01qd51 kernel: Active:15955130 inactive:6377121 dirty:1470607 writeback:0 unstable:0 free:9323924 slab:301569 mapped-file:1506506 mapped-anon:7952475 pagetables:78246
Oct 18 01:47:57 usdc01qd51 kernel: DMA free:0kB min:0kB low:0kB high:0kB active:0kB inactive:0kB present:9076kB pages_scanned:0 all_unreclaimable? yes
Oct 18 01:47:58 usdc01qd51 kernel: lowmem_reserve[]: 0 4024 126242 126242
Oct 18 01:47:58 usdc01qd51 kernel: DMA32 free:1202200kB min:1448kB low:1808kB high:2172kB active:275860kB inactive:55140kB present:4120800kB pages_scanned:0 all_unreclaimable? no
Oct 18 01:47:58 usdc01qd51 kernel: lowmem_reserve[]: 0 0 122217 122217
Oct 18 01:47:58 usdc01qd51 kernel: Normal free:36093496kB min:44024kB low:55028kB high:66036kB active:63544660kB inactive:25453344kB present:125151120kB pages_scanned:0 all_unreclaimable? no
Oct 18 01:47:58 usdc01qd51 kernel: lowmem_reserve[]: 0 0 0 0
Oct 18 01:47:58 usdc01qd51 kernel: HighMem free:0kB min:128kB low:128kB high:128kB active:0kB inactive:0kB present:0kB pages_scanned:0 all_unreclaimable? no
Oct 18 01:47:58 usdc01qd51 kernel: lowmem_reserve[]: 0 0 0 0
Oct 18 01:47:58 usdc01qd51 kernel: DMA: 0*4kB 0*8kB 0*16kB 0*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 0*2048kB 0*4096kB = 0kB
Oct 18 01:47:58 usdc01qd51 kernel: DMA32: 21352*4kB 18549*8kB 13617*16kB 5772*32kB 1097*64kB 52*128kB 8*256kB 1*512kB 1*1024kB 1*2048kB 118*4096kB = 1202200kB
Oct 18 01:47:58 usdc01qd51 kernel: Normal: 144164*4kB 1746735*8kB 1130515*16kB 102420*32kB 1956*64kB 83*128kB 42*256kB 2*512kB 3*1024kB 1*2048kB 6*4096kB = 36093496kB
Oct 18 01:47:58 usdc01qd51 kernel: HighMem: empty
Oct 18 01:47:58 usdc01qd51 kernel: 14377659 pagecache pages
Oct 18 01:47:59 usdc01qd51 kernel: Swap cache: add 507, delete 507, find 163/244, race 0+0
Oct 18 01:47:59 usdc01qd51 kernel: Free swap = 263740076kB
Oct 18 01:47:59 usdc01qd51 kernel: Total swap = 263740648kB
Oct 18 01:47:59 usdc01qd51 kernel: Free swap: 263740076kB
Oct 18 01:47:59 usdc01qd51 kernel: 32770048 pages of RAM
Oct 18 01:47:59 usdc01qd51 kernel: 648871 reserved pages
Oct 18 01:47:59 usdc01qd51 kernel: 10183423 pages shared
Oct 18 01:47:59 usdc01qd51 kernel: 0 pages swap cached
Oct 18 01:47:59 usdc01qd51 kernel: Out of memory: Killed process 9529, UID 502, (java).
--
Thanks,
Sachin.
