Cisco 4900 Switch Rebooted

[Abdul Farooq]

Hi Guys,

One of my customer routers is having strange issue, it reboots itself after the clock updates.but no one is taken console that switch. Below is the log

*Feb 16 07:37:55.831: %SPANTREE-5-EXTENDED_SYSID: Extended SysId enabled for type vlan
*Feb 16 07:37:57.367: %C4K_IOSMODPORTMAN-6-MODULEONLINE: Module 1 (WS-C4900M S/N: JAE15330A9C Hw: 1.8) is online
*Feb 16 07:37:57.367: %C4K_IOSMODPORTMAN-6-MODULEONLINE: Module 2 (WS-X4920-GB-RJ45 S/N: JAE15320HFH Hw: 1.2) is online
*Feb 16 07:37:57.367: %C4K_IOSMODPORTMAN-6-MODULEONLINE: Module 3 (WS-X4920-GB-RJ45 S/N: JAE15320HGN Hw: 1.2) is online
*Feb 16 13:07:57.979 IST: %SYS-6-CLOCKUPDATE: System clock has been updated from 07:37:57 UTC Fri Feb 16 2018 to 13:07:57 IST Fri Feb 16 2018, configured from console by console.
*Feb 16 13:07:58.919 IST: %SYS-5-CONFIG_I: Configured from memory by console
*Feb 16 13:07:59.072 IST: %SYS-5-RESTART: System restarted --
Cisco IOS Software, Catalyst 4500 L3 Switch Software (cat4500e-ENTSERVICESK9-M), Version 15.0(2)SG9, RELEASE SOFTWARE (fc2)
Technical Support: http://www.cisco.com/techsupport
Copyright (c) 1986-2014 by Cisco Systems, Inc.
Compiled Thu 25-Sep-14 04:35 by prod_rel_team
*Feb 16 13:07:59.132 IST: %SSH-5-ENABLED: SSH 1.99 has been enabled
*Feb 16 13:07:59.468 IST: %SNMP-5-COLDSTART: SNMP agent on host ABM-HO-CoreSw1 is undergoing a cold start
*Feb 16 13:07:59.504 IST: %SYS-6-LOGGINGHOST_STARTSTOP: Logging to host 192.168.245.45 Port 514 started - CLI initiated
*Feb 16 13:07:59.504 IST: %SYS-6-LOGGINGHOST_STARTSTOP: Logging to host 192.168.243.150 Port 514 started - CLI initiated
*Feb 16 13:08:08.973 IST: %EC-5-BUNDLE: Interface Gi3/2 joined port-channel Po6
*Feb 16 13:08:09.017 IST: %EC-5-BUNDLE: Interface Gi2/20 joined port-channel Po6
*Feb 16 13:08:11.073 IST: %C4K_IOSINTF-5-TRANSCEIVERINSERTED: Slot=1 Port=1: Transceiver has been inserted
*Feb 16 13:08:12.061 IST: %C4K_IOSINTF-5-TRANSCEIVERINSERTED: Slot=1 Port=2: Transceiver has been inserted
*Feb 16 13:08:15.061 IST: %EC-5-BUNDLE: Interface TenGigabitEthernet1/1 joined port-channel Port-channel5
*Feb 16 13:08:15.077 IST: %EC-5-BUNDLE: Interface TenGigabitEthernet1/2 joined port-channel Port-channel5
*Feb 16 13:08:16.713 IST: %PKI-4-NOAUTOSAVE: Configuration was modified.  Issue "write memory" to save new certificate
*Feb 16 13:08:19.134 IST: %TRACKING-5-STATE: 10 ip sla 10 reachability Up->Down
*Feb 16 13:08:34.135 IST: %TRACKING-5-STATE: 1 ip sla 1 reachability Up->Down
*Feb 16 13:08:36.444 IST: %PIM-5-DRCHG: DR change from neighbor 0.0.0.0 to 172.20.0.5 on interface Vlan41
*Feb 16 13:08:36.444 IST: %PIM-5-DRCHG: DR change from neighbor 0.0.0.0 to 192.168.60.5 on interface Vlan45
*Feb 16 13:08:36.444 IST: %PIM-5-DRCHG: DR change from neighbor 0.0.0.0 to 172.20.80.5 on interface Vlan46
*Feb 16 13:08:36.444 IST: %PIM-5-DRCHG: DR change from neighbor 0.0.0.0 to 142.1.0.48 on interface Vlan100
*Feb 16 13:08:36.444 IST: %PIM-5-DRCHG: DR change from neighbor 0.0.0.0 to 192.168.247.7 on interface Vlan106
*Feb 16 13:08:36.444 IST: %PIM-5-DRCHG: DR change from neighbor 0.0.0.0 to 192.168.245.7 on interface Vlan215
*Feb 16 13:08:36.448 IST: %PIM-5-DRCHG: DR change from neighbor 0.0.0.0 to 192.168.45.9 on interface Vlan320
*Feb 16 13:08:37.568 IST: %PIM-5-NBRCHG: neighbor 192.168.60.13 UP on interface Vlan45
*Feb 16 13:08:37.572 IST: %PIM-5-DRCHG: DR change from neighbor 192.168.60.5 to 192.168.60.13 on interface Vlan45
*Feb 16 13:08:45.221 IST: %HSRP-5-STATECHANGE: Vlan41 Grp 10 state Listen -> Active
*Feb 16 13:08:45.249 IST: %HSRP-5-STATECHANGE: Vlan200 Grp 10 state Listen -> Active
*Feb 16 13:08:45.297 IST: %HSRP-5-STATECHANGE: Vlan47 Grp 20 state Listen -> Active
*Feb 16 13:08:45.317 IST: %HSRP-5-STATECHANGE: Vlan48 Grp 20 state Listen -> Active
*Feb 16 13:08:45.329 IST: %HSRP-5-STATECHANGE: Vlan46 Grp 10 state Listen -> Active
*Feb 16 13:08:45.329 IST: %HSRP-5-STATECHANGE: Vlan100 Grp 10 state Listen -> Active
*Feb 16 13:08:45.457 IST: %HSRP-5-STATECHANGE: Vlan216 Grp 20 state Listen -> Active
*Feb 16 13:08:45.477 IST: %HSRP-5-STATECHANGE: Vlan54 Grp 10 state Listen -> Active
 --More--         *Feb 16 13:08:45.709 IST: %HSRP-5-STATECHANGE: Vlan45 Grp 10 state Listen -> Active
*Feb 16 13:08:45.737 IST: %HSRP-5-STATECHANGE: Vlan215 Grp 10 state Listen -> Active
*Feb 16 13:08:45.753 IST: %HSRP-5-STATECHANGE: Vlan310 Grp 10 state Listen -> Active
*Feb 16 13:08:45.825 IST: %HSRP-5-STATECHANGE: Vlan31 Grp 20 state Listen -> Active
*Feb 16 13:08:45.869 IST: %HSRP-5-STATECHANGE: Vlan106 Grp 20 state Listen -> Active
*Feb 16 13:08:45.905 IST: %HSRP-5-STATECHANGE: Vlan6 Grp 10 state Listen -> Active
*Feb 16 13:08:45.917 IST: %HSRP-5-STATECHANGE: Vlan43 Grp 20 state Listen -> Active
*Feb 16 13:08:46.013 IST: %HSRP-5-STATECHANGE: Vlan44 Grp 20 state Listen -> Active
*Feb 16 13:08:48.789 IST: %PIM-5-NBRCHG: neighbor 192.168.60.14 UP on interface Vlan45
*Feb 16 13:08:48.789 IST: %PIM-5-DRCHG: DR change from neighbor 192.168.60.13 to 192.168.60.14 on interface Vlan45
*Feb 16 13:08:56.986 IST: %PIM-5-NBRCHG: neighbor 172.20.0.6 UP on interface Vlan41
*Feb 16 13:08:56.990 IST: %PIM-5-DRCHG: DR change from neighbor 172.20.0.5 to 172.20.0.6 on interface Vlan41
*Feb 16 13:09:05.459 IST: %PIM-5-NBRCHG: neighbor 192.168.245.8 UP on interface Vlan215
*Feb 16 13:09:05.463 IST: %PIM-5-DRCHG: DR change from neighbor 192.168.245.7 to 192.168.245.8 on interface Vlan215
*Feb 16 13:09:07.215 IST: %PIM-5-NBRCHG: neighbor 172.20.80.6 UP on interface Vlan46
*Feb 16 13:09:07.215 IST: %PIM-5-DRCHG: DR change from neighbor 172.20.80.5 to 172.20.80.6 on interface Vlan46

[Mark Malone]

whats the show version show in this section below
System returned to ROM by xxxxxxxxxxx

looks like something is happening before that as the moduyles are only coming online ?
*Feb 16 07:37:55.831: %SPANTREE-5-EXTENDED_SYSID: Extended SysId enabled for type vlan
*Feb 16 07:37:57.367: %C4K_IOSMODPORTMAN-6-MODULEONLINE: Module 1 (WS-C4900M S/N: JAE15330A9C Hw: 1.8) is online

[Abdul Farooq]

Show Version

Cisco IOS Software, Catalyst 4500 L3 Switch Software (cat4500e-ENTSERVICESK9-M), Version 15.0(2)SG9, RELEASE SOFTWARE (fc2)
Technical Support: http://www.cisco.com/techsupport
Copyright (c) 1986-2014 by Cisco Systems, Inc.
Compiled Thu 25-Sep-14 04:35 by prod_rel_team
Image text-base: 0x10000000, data-base: 0x12FE0F00

ROM: 12.2(44r)SG10
Sushi Revision 11, Tatooine Revision 141, Forerunner Revision 1.79

ABM-HO-CoreSw1 uptime is 3 days, 19 hours, 28 minutes
System returned to ROM by abort at PC 0x0
System restarted at 20:47:55 IST Fri Feb 16 2018
System image file is "bootflash:cat4500e-entservicesk9-mz.150-2.SG9.bin"


This product contains cryptographic features and is subject to United
States and local country laws governing import, export, transfer and
use. Delivery of Cisco cryptographic products does not imply
third-party authority to import, export, distribute or use encryption.
Importers, exporters, distributors and users are responsible for
compliance with U.S. and local country laws. By using this product you
agree to comply with applicable laws and regulations. If you are unable
to comply with U.S. and local laws, return this product immediately.

A summary of U.S. laws governing Cisco cryptographic products may be found at:
http://www.cisco.com/wwl/export/crypto/tool/stqrg.html

If you require further assistance please contact us by sending email to
export@cisco.com.

cisco WS-C4900M (MPC8548) processor (revision 2) with 1048576K bytes of memory.
Processor board ID JAE153306DJ
MPC8548 CPU at 1.33GHz, Cisco Catalyst 4900M
Last reset from System Crash
23 Virtual Ethernet interfaces
40 Gigabit Ethernet interfaces
8 Ten Gigabit Ethernet interfaces
511K bytes of non-volatile configuration memory.

Configuration register is 0x2102

[Leo Laohoo]

Post the complete output to the command "dir crashinfo:".

[Abdul Farooq]

Already tried to get crashinfo but user not enabled crashinfo

[Mark Malone · Accepted solution]

Could be this bug same IOS same sh ver restart output but you should TAC it

https://bst.cloudapps.cisco.com/bugsearch/bug/CSCue11730/?referring_site=bugquickviewredir

ABM-HO-CoreSw1 uptime is 3 days, 19 hours, 28 minutes
System returned to ROM by abort at PC 0x0


4900M crash 'IPP PRM pktParityInt interrupt' and redzone overwritten
CSCue11730
Description
Symptom:
Certain Catalyst 4500 switch crashes with 'System returned to ROM by abort at PC 0x0' in show version output. Crashdump reports "IPP PRM pktParityInt interrupt" error followed by a crash. Crash may indicate corrupted block redzone.

Conditions:
Possibly affected: 4900M, 4948E, Sup6E, Sup6L-E
High amounts of multicast traffic is the one trigger.
Another trigger is high amounts of replicated SPAN traffic. In order to see the crash due to SPAN, the SPAN destination ports must receive pauses frames from the remote side of the span destination port.
In most of the cases it's observed that any changes to the TenGigE interfaces (interface flapping, manual shut/no shut) preceded the crash. Parity error and Corrupted redzone blk is common across all such cases

[Abdul Farooq]

Hi Malone,

kindly provide the solution of this case. if IOS crash mean can i update the IOS ?

[Mark Malone · Accepted solution]

yes you can upgrade you must chose from one of the fixed versions in the bug link in the previous post , first though i would definitely confirm with TAC is it that bug exactly if the 4900 is under support