Switch rebooted with error Critical software exception.

[bibin.chaithanya1]

we have WS-C4507R+E at our client place. and today morning its goes rebooted. we can see that the last reload reason is critical software exception. please advice me what could be the reason for that.

isco IOS Software, IOS-XE Software, Catalyst 4500 L3 Switch Software (cat4500e-UNIVERSALK9-M), Version 03.05.02.E RELEASE SOFTWARE (fc1)
Technical Support: http://www.cisco.com/techsupport
Copyright (c) 1986-2014 by Cisco Systems, Inc.
Compiled Tue 11-Mar-14 16:27 by prod_rel_team

Cisco IOS-XE software, Copyright (c) 2005-2013 by cisco Systems, Inc.
All rights reserved. Certain components of Cisco IOS-XE software are
licensed under the GNU General Public License ("GPL") Version 2.0. The
software code licensed under GPL Version 2.0 is free software that comes
with ABSOLUTELY NO WARRANTY. You can redistribute and/or modify such
GPL code under the terms of GPL Version 2.0. For more details, see the
documentation or "License Notice" file accompanying the IOS-XE software,
or the applicable URL provided on the flyer accompanying the IOS-XE
software.

ROM: 15.0(1r)SG1
S4507 uptime is 1 hour, 43 minutes
Uptime for this control processor is 1 hour, 45 minutes
System returned to ROM by reload
System restarted at 20:38:30 UTC Wed Jul 6 2016
System image file is "bootflash:cat4500e-universalk9.SPA.03.05.02.E.152-1.E2.bin"
Jawa Revision 7, Snowtrooper Revision 0x0.0x1C

Last reload reason: Critical software exception

[Leo Laohoo]

Post the complete output to the command "dir". 

The IOS train 3.5.X hasn't been updated since the last & final version of 3.5.3E (09 May 2014).

[bibin.chaithanya1]

Thanks Leo,

we have crash info file also generated. please see that attached crashinfo file.

[Leo Laohoo]

Jul  6 20:29:52.263: %SYS-2-MALLOCFAIL: Memory allocation of 16556 bytes failed from 0x20FA2E14, alignment 8 
Pool: Processor  Free: 63180184  Cause: Memory fragmentation 
Alternate Pool: None  Free: 0  Cause: No Alternate pool 
 -Process= "Pool Manager", ipl= 0, pid= 8
-Traceback= 1#9351078809671a64511152dddc798b9b  :1F8B1000+2CFF560 :1F8B1000+2CFF924 :1F8B1000+416315C :1F8B1000+416A334 :1F8B1000+16F1E18 :1F8B1000+4185184 :1F8B1000+418543C
Jul  6 20:30:03.745: %CRYPTO-0-SELF_TEST_FAILURE: Encryption self-test failed (RBG: instantiate did not give catastrophic failure when given insufficient entropy)

The crash is caused by memory allocation error which exhausted available memory.  

Recommendation?  Think about upgrading the IOS.  Read the Release Notes carefully, there are some IOS which states that the ROMmon needs upgrade too.

[n.ghafarzadeh]

Hi,

Our network contains two 4507+E Cisco switches and another two SUP 7L-E.Their IOS was cat4500e-universalk9.SPA.03.07.00.E.152-3.E.bin.
MACsec technology on a communication port channel having two 3850 Stack Cisco switches was used.
Afterwards, one of the switches started experiencing random reboots every couple of days while displaying in show version reload reason
"software critical exception" after each restart.

Checking the logs,this error was being seen: "%C4K_SWITCHINGENGINEMAN-4-VFEIMINTERRUPT: Q.8�)L�d"
Supervisors were upgraded to 3.8.6 and 3.8.1 versions but it did not address the problem.
Rommon was upgraded to cat4500-e-ios-promupgrade-150-1r-SG14 version and it did not solved the issue as well.
VSL Links expansion from 4 GB to 20 GB tried and the restarts were still happening.

Finally, MACsec was removed from its communication link with the Cisco 3850 Stack and it did the trick!

I am still wondering whether Cisco does have any solution for this problem?

regards.