CISCO ASR1002-x: Critical software exception

[thestudent101]

Hi,

What does critical software exception means? Thank you

Router01 uptime is 6 days, 1 hour, 15 minutes
Uptime for this control processor is 6 days, 1 hour, 18 minutes
System returned to ROM by Critical software exception, check bootflash:crashinfo_RP_00_00_20190705-153903-PHILIPP
System restarted at 15:45:55 PHILIPP Fri Jul 5 2019
System image file is "bootflash:/asr1002x-universalk9.16.07.01.SPA.bin"
Last reload reason: Critical software exception, check bootflash:crashinfo_RP_00_00_20190705-153903-PHILIPP

[Leo Laohoo]

Update the firmware.

Alternatively, raise a TAC Case and give them the crashinfo file for analysis.

[premspandey1995]

critical software exception means software crash. You may find out the crash file in your bootflash, you just need to open a TAC case with Cisco & provide the crash file in order to find out the reason for the crash. 

For your understanding, I am enclosing the link  

https://www.cisco.com/c/en/us/support/docs/routers/7500-series-routers/26145-crashes-swforced-troubleshoot.html

Thank you & have a great time!!

[wag-web]

I'm facing the same issue right now.
Have you found a solution?

[Leo Laohoo]

Each crash is unique.  

Please provide more information.  Start with "sh version".  

[wag-web]

Hello Leo. Thanks for your response...
I found in my log the following lines:

*Dec 11 17:13:39.909: %IOSXE-5-PLATFORM: R0/0: cpp_cp: QFP:0.0 Thread:105 TS:00000049356367826532 %PUNT_INJECT-5-DROP_PUNT_CAUSE: punt policer drops packets, cause: for-us-data (0xb) from Port-channel1.10
CMD: 'show plat hard qfp act infra exmem stat diram' 15:13:40 BRV Wed Dec 11 2024
CMD: 'show plat hard qfp act infra exmem stat diram' 15:13:46 BRV Wed Dec 11 2024
*Dec 11 17:13:48.439: %AAA-3-ACCT_LOW_MEM_UID_FAIL: AAA unable to create UID for incoming calls due to insufficient processor memory
*Dec 11 17:13:50.629: %AAA-3-ACCT_LOW_PROC_MEM_TRASH: AAA unable to handle accounting requests due to insufficient processor memory and could be trashing the queued accounting records
CMD: 'show plat hard qfp act infra exmem stat diram' 15:13:51 BRV Wed Dec 11 2024
*Dec 11 17:13:52.309: %SYS-2-MALLOCFAIL: Memory allocation of 2756 bytes failed from 0x80005620B4ED8A99, alignment 8
Pool: Processor Free: 609440 Cause: Memory fragmentation
Alternate Pool: None Free: 0 Cause: No Alternate pool
-Process= "Pool Manager", ipl= 0, pid= 10

Is there any kind of solution or at least some workaround for that?

[Leo Laohoo]

Post the complete output to the following commands: 

sh version
sh platform resources

[wag-web]

sh version

Cisco IOS XE Software, Version 16.09.07
Cisco IOS Software [Fuji], ASR1000 Software (X86_64_LINUX_IOSD-UNIVERSALK9-M), Version 16.9.7, RELEASE SOFTWARE (fc1)
Technical Support: http://www.cisco.com/techsupport
Copyright (c) 1986-2021 by Cisco Systems, Inc.
Compiled Wed 10-Feb-21 09:19 by mcpre

platform resources

**State Acronym: H - Healthy, W - Warning, C - Critical
Resource Usage Max Warning Critical State
----------------------------------------------------------------------------------------------------
RP0 (ok, active) H
Control Processor 9.90% 100% 80% 90% H
DRAM 6752MB(42%) 15919MB 88% 93% H
ESP0(ok, active) H
QFP H
TCAM 66cells(0%) 524288cells 65% 85% H
DRAM 443568KB(42%) 1048576KB 80% 90% H
IRAM 12245KB(9%) 131072KB 80% 90% H
CPU Utilization 10.00% 100% 90% 95% H

[Leo Laohoo]

Memory utilization for both the control-plane and the QFP are abnormally high (both at 42%).  It should not be >25%.

[wag-web]

Ok.. Got it!
I'll check that...
This router is running BGP and some services.
I'll try to "clean" at maximum. Thanks for your help!