lsmpi_io pool low memory Cisco ISR 4321

[Hasan AlNahawy]

Dears,

as showing below the memory free of lsmpi_io is very low :

Router_X#show memory 
                Head    Total(b)     Used(b)     Free(b)   Lowest(b)  Largest(b)
Processor  7F33741EA010   1728098704   232211688   1495887016   679445864   1048575908
 lsmpi_io  7F3373AE81A8     6295128     6294304         824         824         412

and from this command : show platform software status control-processor brief , it is showing that 98% usage of memory .
Load Average
 Slot  Status  1-Min  5-Min 15-Min
  RP0 Healthy   1.00   1.00   1.00

Memory (kB)
 Slot  Status    Total     Used (Pct)     Free (Pct) Committed (Pct)
  RP0 Healthy  3950588  3853448 (98%)    97140 ( 2%)   2259984 (57%)

CPU Utilization
 Slot  CPU   User System   Nice   Idle    IRQ   SIRQ IOwait
  RP0    0   2.90   9.30   0.00  87.78   0.00   0.00   0.00
         1   1.19   1.99   0.00  96.80   0.00   0.00   0.00
         2   0.00   0.00   0.00 100.00   0.00   0.00   0.00
         3  23.22  76.77   0.00   0.00   0.00   0.00   0.00

there is no any high traffic on this router , 

 5 minute input rate 1000 bits/sec, 1 packets/sec
  5 minute output rate 1000 bits/sec, 1 packets/se

this router look like sleeping , but why the memory usage is high like this !!!!!!!

[Vinit Jain]

Could you please check if you have redundancy mode SSO configured on the router. You have a single RP so that config can still cause the system memory to be used up.

[Hasan AlNahawy]

Hi Vinit,

Thanks for your reply , there is no redundancy configured on this router as shown below:

Router_X#show redundancy states 
       my state = 13 -ACTIVE 
     peer state = 1  -DISABLED 
           Mode = Simplex
           Unit = Primary
        Unit ID = 48

Redundancy Mode (Operational) = Non-redundant
Redundancy Mode (Configured)  = Non-redundant
Redundancy State              = Non Redundant
     Maintenance Mode = Disabled
    Manual Swact = disabled (system is simplex (no peer unit))
 Communications = Down      Reason: Simplex mode

   client count = 77
 client_notification_TMR = 30000 milliseconds
           RF debug mask = 0x0   .

[Vinit Jain]

Could you please share the show version output from the router. Hopefully i can figure out if this is due to a software defect.

I would also like to know, is there any alarm getting generated or if there is any service impact.

The committed percent on the memory is 57%, which doesnt look like much of a concern to me.

Will share further inputs once i have a reply on the above requested.

Thanks

Vinit

[Hasan AlNahawy]

please find attached result for show log and show version command , the router as i said look like sleepy , there is no any load traffic exist , the purpose of this router is for third party connection so just small traffic going though . i observed this memory utilization from my solarwinds tool , the software on this router was released in may 2015 so it is still new .

[Vinit Jain]

Hello Hasan

This issue seems to be due to the software defect# CSCuc40262

If your network management tool is raising an alarm for the high memory due to this issue, then you can use the below OID to monitor the memory:

http://tools.cisco.com/Support/SNMP/do/BrowseOID.do?local=en&translate=Translate&objectInput=1.3.6.1.4.1.9.9.109.1.1.1.1.27

Hope this answers your question.

Regards

Vinit

PS: Please mark the question as answered if your query has been answered.

[Hasan AlNahawy]

Dear Vinit,

actually i upgrade the router to safeharbor image (isr4300-universalk9.03.13.03.S.154-3.S3-ext.SPA) and still the memory showing utilized , then i follow your comment and i was changed the OID from my monitoring tool with objectInput=1.3.6.1.4.1.9.9.109.1.1.1.1.27 and i start receiving data for the memory as shown in attached file . as per my research I got this conclusion : lsmpi_io , Processor are a two memories in ISR routers, the Router will always show high level for lsmpi_io because this is used by Linux Kernal for control plane while Processor is the standard memory to take care of data traffic. 

but the question now : why we need to treat with 98% memory usage as a normal memory usage !!!!!:

show platform software status control-processor brief
Load Average
 Slot  Status  1-Min  5-Min 15-Min
  RP0 Healthy   1.00   1.00   1.00

Memory (kB)
 Slot  Status    Total     Used (Pct)     Free (Pct) Committed (Pct)
  RP0 Healthy  3950588  3853564 (98%)    97024 ( 2%)   2260344 (57%)

CPU Utilization
 Slot  CPU   User System   Nice   Idle    IRQ   SIRQ IOwait
  RP0    0   3.30  10.70   0.00  86.00   0.00   0.00   0.00
         1   0.50   1.10   0.00  98.39   0.00   0.00   0.00
         2   0.00   0.00   0.00 100.00   0.00   0.00   0.00
         3  22.90  77.10   0.00   0.00   0.00   0.00   0.00

[Vinit Jain · Accepted solution]

Yes, on IOS XE platforms like ISR 4421, ASR1k and others, there are two memories. One is for linux kernel and the other is for IOS which runs on top of it. 

i dont think the software defect that i mentioned is fixed in 15.4(3)S3 release. It is fixed in 15.5. and teh reason you are seeing the utilization of 98% is because of the software defect that i mentioned before. I dont really think you need to worry about it much but again if you have some doubts you can open a TAC case. I am pretty confident that you are hitting that software defect.

Please let me know if you have any further queries.

Regards

Vinit

[Hasan AlNahawy]

thanks for your kind response and support :)

[Vysakh Sundar]

Dear Vinit

We have the same issue with ISR 4431 running on Version 15.5(3)S4a

[chathurangaj@kbsl.lk]

I have the cisco ISR 4321/K9 router.

the software version on it is Version 15.5(3)S4b. i have this poblem too. what colud be the issue on it ?

Here i attached the show version details and memory details.

show redundancy states
my state = 13 -ACTIVE
peer state = 1 -DISABLED
Mode = Simplex
Unit = Primary
Unit ID = 48

Redundancy Mode (Operational) = Non-redundant
Redundancy Mode (Configured) = Non-redundant
Redundancy State = Non Redundant
Maintenance Mode = Disabled
Manual Swact = disabled (system is simplex (no peer unit))
Communications = Down Reason: Simplex mode

client count = 81
client_notification_TMR = 30000 milliseconds
RF debug mask = 0x0

show memory
Head Total(b) Used(b) Free(b) Lowest(b) Largest(b)
Processor 7F82E8166010 1688360784 276622532 1411738252 639707944 1048575908
lsmpi_io 7F82E79631A8 6295128 6294304 824 824 412
Dynamic heap limit(MB) 1000 Use(MB) 0

show platform software status control-processor brief
Load Average
Slot Status 1-Min 5-Min 15-Min
RP0 Healthy 1.01 1.03 1.00

Memory (kB)
Slot Status Total Used (Pct) Free (Pct) Committed (Pct)
RP0 Healthy 3950540 3918376 (99%) 32164 ( 1%) 2512796 (64%)

CPU Utilization
Slot CPU User System Nice Idle IRQ SIRQ IOwait
RP0 0 3.30 8.40 0.00 87.68 0.60 0.00 0.00
1 4.30 4.80 0.00 90.80 0.10 0.00 0.00
2 1.10 0.20 0.00 98.70 0.00 0.00 0.00
3 24.40 75.60 0.00 0.00 0.00 0.00 0.00