ASR 1000 memory issue

[mze]

An issue on one of our ASR 1000  where it seemed to have run out of memory

[MHM Cisco World]

 ...

[balaji.bandi]

what Code running adding to other post look below bug also :

https://bst.cloudapps.cisco.com/bugsearch/bug/CSCvg65632

[mze]

thanks

[Leo Laohoo]

What firmware is the router running on? 

What is the uptime of this router?

Post the complete output to the following commands: 

sh platform resource
sh platform software status control-processor brief

---

@mze wrote:

Mar 11 12:10:11.501 GMT: %PMAN-3-PROCHOLDDOWN: F0: pman.sh:  The process cpp_cdm_svr has been helddown (rc 137)
Mar 11 12:10:11.923 GMT: %PMAN-3-PROCHOLDDOWN: F0: pman.sh:  The process cpp_ha_top_level_server has been helddown (rc 69)
Mar 11 12:10:12.348 GMT: %PMAN-0-PROCFAILCRIT: F0: pvp.sh:  A critical process cpp_cdm_svr has failed (rc 137)

---

I think you are hitting multiple bugs. 

The ones above look like CSCun09640 or CSCvn97744. 

---

@mze wrote:

Mar 11 12:10:12.516 GMT: %CPPDRV-3-LOCKDOWN: F0: cpp_cp:  QFP0.0 CPP Driver LOCKDOWN encountered due to previous fatal error (SW: Userspace fault).
Mar 11 12:10:12.559 GMT: %CPPOSLIB-3-ERROR_NOTIFY: F0: cpp_cp:  cpp_cp encountered an error -Traceback= 1#2f78e9d02b9eba8cf2488d63bb15a429   errmsg:C23B000+2230 cpp_common_os:C90B000+C670 cpp_common_os:C90B000+C470 cpp_common_os:C90B000+19DAC cpp_rrm_svr_lib:E790000+8BE8 cpp_rrm_svr_lib:E790000+81EC cpp_rrm_svr_lib:E790000+5240 cpp_rrm_svr_lib:E790000+4200 cpp_common_os:C90B000+115E4 cpp_common_os:C90B000+11C1C evlib:C613000+E1FC evlib:C613000+105E4 cpp_common_os:C90B000+1338C :10000000+4AB0 c:943C000+1E938 c:94
Mar 11 12:10:13.283 GMT: %CPPDRV-3-LOCKDOWN: F0: fman_fp_image:  QFP0.0 CPP Driver LOCKDOWN encountered due to previous fatal error (SW: Userspace fault).

---

These, however, look more like CSCvf80163 than CSCvt68268. 

[mze]

thanks for reply 

[Leo Laohoo]

---

@mze wrote:

Memory (kB)
 Slot  Status    Total     Used (Pct)     Free (Pct) Committed (Pct)
  RP0 Healthy  3874484  3342964 (86%)   531520 (14%)   2915096 (75%)
 ESP0 Healthy   969068   962520 (99%)     6548 ( 1%)    875892 (90%)
 SIP0 Healthy   471812   378464 (80%)    93348 (20%)    355000 (75%)

---

Whao!  Those do not look good.  

The way I see it the ESP0 is dragging everyone down.  

Can I see the "first page" of the output to the following commands: 

• sh process memory platform sort location F0
• sh process memory platform sort location R0
• sh process memory platform sort location RP
• sh process memory platform sort location FP

[mze]

thanks

[Leo Laohoo]

Contact Cisco TAC.  I get them to verify those Bug IDs we've highlighted above.

[mze]

thanks will do 

[MHM Cisco World]

https://www.cisco.com/c/en/us/support/docs/routers/asr-1000-series-aggregation-services-routers/109723-asr-crash.html#xe_crash