10-07-2014 01:12 PM - edited 03-07-2019 09:01 PM
I had a unscheduled service interuption that caused my 6509 to lose power. I have the 6509 plugged into an UPS. When power was restored it immediately booted into rommod. I figured it may be a corrupt ios image so I went to a working 6509 and copied the ios to a flash card. I then booted the inoperable 6509 from the flash card. It boots all the way up to the login screen. Shortly after it gives me a %Software-forced reload and crashes into rommod.
I was able to log in briefly and found none of the cards are initiated which leads me to believe that initiating the cards is what is crashing the switch. I have redundant supervisor cards which makes me think its a chasis issue as the likelyhood of both cards going bad at the same time is very unlikely.
Anyone have any thoughts? I pasted my output below
If you require further assistance please contact us by sending email to
export@cisco.com.
cisco WS-C6509-E (R7000) processor (revision 1.5) with 458720K/65536K bytes of memory.
Processor board ID SMG1416N02C
SR71000 CPU at 600Mhz, Implementation 0x504, Rev 1.2, 512KB L2 Cache
Last reset from s/w reset
1 Virtual Ethernet interface
52 Gigabit Ethernet interfaces
8 Ten Gigabit Ethernet interfaces
1917K bytes of non-volatile configuration memory.
8192K bytes of packs of Flash internal SIMM (Sector size 512K).
Press RETURN to get started!
*Oct 7 18:18:56.099: RP: Currently running ROMMON from S (Gold) region
*Oct 7 18:18:56.843: %SPANTREE-5-EXTENDED_SYSID: Extended SysId enabled for type vlan. The Bridge IDs of all active STP instances have been updated, which might change the spanning tree topology
*Oct 7 18:19:02d from memory by console
*Oct 7 18:19:06.227: %SYS-5-RESTART: System restarted --
Cisco IOS Software, s72033_rp Software (s72033_rp-IPSERVICESK9-M), Version 12.2(33)SXI2a, RELEASE SOFTWARE (fc2)
Technical Support: http://www.cisco.com/techsupport
Copyright (c) 1986-2009 by Cisco Systems, Inc.
Compiled Wed 02-Sep-09 01:00 by prod_rel_team
*Oct: SSH 2.0 has been enabled
*Oct 7 18:19:09.463: %SNMP-5-COLDSTART: SNMP agent on host B641_WMEZZ_6509 is undergoing a cold start
*Oct 7 18:36:43.663: %SYS-SP-3-LOGGER_FLUSHED: System was paused for 00:00:00 to ensure console debugging output.
*Oct 7 18:18:51.719: %SPANTREE-SP-5-EXTENDED_SYSID: Extended SysId enabled for type vlan. The Bridge IDs of all active STP instances have been updated, which might change the spanning tree topology
*Oct 7 18:18:51.727: SP: SP: Currently running ROMMON from S (Gold) region
*Oct 7 18:19:01.936: %SW_VLAN-SP-6-VTP_DOMAIN_NAME_CHG: VTP domain name changed to 133AW.
*Oct 7 18:19:05.453: %SYS-SP-5-RESTART: System restarted --
Cisco IOS Software, s72033_sp Software (s72033_sp-IPSERVICESK9-M), Version 12.2(33)SXI2a, RELEASE SOFTWARE (fc2)
Technical Support: http://www.cisco.c
/techsupport
Copyright (c) 1986-2009 by Cisco Systems, Inc.
Compiled Wed 02-Sep-09 01:25 by prod_rel_team
*Oct 7 18:19:06.445: %OIR-SP-6-INSPS: Power supply inserted in slot 1
*Oct 7 18:19:06.545: %C6KPWR-SP-4-PSOK: power supply 1 turned on.
*Oct 7 18:19:06.889: %OIR-SP-6-INSPS: Power supply inserted in slot 2
*Oct 7 18:19:11.507: %SNMP-5-CHASSISALARM: Chassis Alarm Trap: tmpAlarm(OFF), minorAlarm(ON), and/or majorAlarm(OFF).
-,
LE or CI inves
*Oct 7 18:19:13.144: %FABRIC-SP-5-CLEAR_BLOCK: Clear block option is off for the fabric in slot 6.
*Oct 7 18:19:13.244: %FABRIC-SP-5-FABRIC_MODULE_ACTIVE: The Switch Fabric Module in slot 6 became active.tigative searching or
User Access Verification
Username:
*Oct 7 18:19:14.793: %DIAG-SP-6-RUN_MINIMUM: Module 6: Running Minimal Diagnostics...
Username:
*********************************************************************
* *
* ALL CONFIGURATION CHANGES AND COMMANDS ARE LOGGED *
* *
*********************************************************************
xxx_6509>en
Password:
xxx_6509#sh int status
Port Name Status Vlan Duplex Speed Type
Po1 notconnect unass 2 notconnect unassigne auto auto
Po3 disabled unassigne auto auto
Po4 notconnect unassigne auto auto
Po5 notconnect unassigne auto auto
Po6 notconnect7 notconnect unassigne auto auto
Po8 notconnect unassigne auto auto
xxxx_6509#
%Software-forced reload
18:19:36 UTC Tue Oct 7 2014: Breakpoint exception, CPU signal 23, PC = 0x42735338
-------------------------------------
Possible software fault. Upon reccurence, please collect
crashinfo, "show tech" and contact Cisco Technical Support.
--------------------------------------------------------------------
-Traceback= 42735338 42732E8C 423A6220 423A624C 421C1388 4223F614 4223F670 40962164 40963034 40962F54 40963D28 423259D0 42319604 42319820 42727FEC
$0 : 00000000, AT : 43FE0000, v0 : 45870000, v1 : 00000200
a0 : 50EA96B0, a1 : 44E02078, a2 : 00000000, a3 : 00000040
t0 : 00000010, t1 : 3400F101, t2 : 3400F100, t3 : FFFF00FF
t4 : 42728608, t5 : 00000000, t6 : 00000000, t7 : 00000000
s0 : 00000000, s1 : 43ED0000, s2 : 503AD42C, s3 : 0000001F
s4 : 503AD42C, s5 : 511937A4, s6 : 00000000, s7 : 08AF24C8
t8 : 08028FEC, t9 : 00000000, k0 : 00000000, k1 : 00000000
gp : 43FDA7AC, sp : 5000DBB0, s8 : 00000000, ra : 42732E8C
EPC : 42735338, ErrorEPC : F0FFFD77, SREG : 3400F103
MDLO : 00000000, MDHI : 00000000, BadVaddr : 00000000
DATA_START : 0x43A4E790
Cause 00000024 (Code 0x9): Breakpoint exception
Writing crashinfo to bootflash:crashinfo_20141007-181936
=== Flushing messages (18:19:36 UTC Tue Oct 7 2014) ===
Buffered messages:
*Oct 7 18:18:56.099: RP: Currently running ROMMON from S (Gold) region
*Oct 7 18:18:56.843: %SPANTREE-5-EXTENDED_SYSID: Extended SysId enabled for type vlan. The Bridge IDs of all active STP instances have been updated, which might change the spanning tree topology
*Oct 7 18:19:02.811: %SYS-5-CONFIG_I: Configured from memory by console
*Oct 7 18:19:06.227: %SYS-5-RESTART: System restarted --
Cisco IOS Software, s72033_rp Software (s72033_rp-IPSERVICESK9-M), Version 12.2(33)SXI2a, RELEASE SOFTWARE (fc2)
Technical Support: http://www.cisco.com/techsupport
Copyright (c) 1986-2009 by Cisco Systems, Inc.
Compiled Wed 02-Sep-09 01:00 by prod_rel_team
*Oct 7 18:19:06.235: %SSH-5-ENABLED: SSH 2.0 has been enabled
*Oct 7 18:19:09.463: %SNMP-5-COLDSTART: SNMP agent on host B641_WMEZZ_6509 is undergoing a cold start
*Oct 7 18:36:43.663: %SYS-SP-3-LOGGER_FLUSHED: System was paused for 00:00:00 to ensure console debugging output.
*Oct 7 18:18:51.719: %SPANTREE-SP-5-EXTENDED_SYSID: Extended SysId enabled for type vlan. The Bridge IDs of all active STP instances have been updated, which might change the spanning tree topology
*Oct 7 18:18:51.727: SP: SP: Currently running ROMMON from S (Gold) region
*Oct 7 18:19:01.936: %SW_VLAN-SP-6-VTP_DOMAIN_NAME_CHG: VTP domain name changed to 133AW.
*Oct 7 18:19:05.453: %SYS-SP-5-RESTART: System restarted --
Cisco IOS Software, s72033_sp Software (s72033_sp-IPSERVICESK9-M), Version 12.2(33)SXI2a, RELEASE SOFTWARE (fc2)
Technical Support: http://www.cisco.com/techsupport
Copyright (c) 1986-2009 by Cisco Systems, Inc.
Compiled Wed 02-Sep-09 01:25 by prod_rel_team
*Oct 7 18:19:06.445: %OIR-SP-6-INSPS: Power supply inserted in slot 1
*Oct 7 18:19:06.545: %C6KPWR-SP-4-PSOK: power supply 1 turned on.
*Oct 7 18:19:06.889: %OIR-SP-6-INSPS: Power supply inserted in slot 2
*Oct 7 18:19:11.507: %SNMP-5-CHASSISALARM: Chassis Alarm Trap: tmpAlarm(OFF), minorAlarm(ON), and/or majorAlarm(OFF)
*Oct 7 18:19:13.144: %FABRIC-SP-5-CLEAR_BLOCK: Clear block option is off for the fabric in slot 6.
*Oct 7 18:19:13.244: %FABRIC-SP-5-FABRIC_MODULE_ACTIVE: The Switch Fabric Module in slot 6 became active.
*Oct 7 18:19:14.793: %DIAG-SP-6-RUN_MINIMUM: Module 6: Running Minimal Diagnostics...
*Oct 7 18:19:35.927: %DIAG-SP-3-MAJOR: Module 6: Online Diagnostics detected a Major Error. Please use 'show diagnostic result <target>' to see test results.
*Oct 7 18:19:35.931: %CONST_DIAG-SP-3-BOOTUP_TEST_FAIL: Module 6: TestFibDevices failed
*Oct 7 18:19:35.931: %CONST_DIAG-SP-3-BOOTUP_TEST_FAIL: Module 6: TestIPv4FibShortcut failed
*Oct 7 18:19:35.931: %CONST_DIAG-SP-3-BOOTUP_TEST_FAIL: Module 6: TestL3Capture2 failed
*Oct 7 18:19:35.931: %CONST_DIAG-SP-3-BOOTUP_TEST_FAIL: Module 6: TestIPv6FibShortcut failed
*Oct 7 18:19:35.931: %CONST_DIAG-SP-3-BOOTUP_TEST_FAIL: Module 6: TestMPLSFibShortcut failed
*Oct 7 18:19:35.931: %CONST_DIAG-SP-3-BOOTUP_TEST_FAIL: Module 6: TestNATFibShortcut failed
Queued messages:
*** System received a Software forced crash ***
signal= 0x17, code= 0x24, context= 0x45873874
PC = 0x42728664, SP = 0x43ed65f8, RA = 0x413ad2f4
Cause Reg = 0x00003820, Status Reg = 0xrommon 1 >
10-07-2014 01:23 PM
Hi,
My recommendation is to open a case with TAC to have them decode the crash, since most of the folks over here will not have the proper tools to give you an answer.
Hope this helps.
10-07-2014 01:46 PM
*Oct 7 18:19:14.793: %DIAG-SP-6-RUN_MINIMUM: Module 6: Running Minimal Diagnostics... *Oct 7 18:19:35.927: %DIAG-SP-3-MAJOR: Module 6: Online Diagnostics detected a Major Error. Please use 'show diagnostic result' to see test results. *Oct 7 18:19:35.931: %CONST_DIAG-SP-3-BOOTUP_TEST_FAIL: Module 6: TestFibDevices failed *Oct 7 18:19:35.931: %CONST_DIAG-SP-3-BOOTUP_TEST_FAIL: Module 6: TestIPv4FibShortcut failed *Oct 7 18:19:35.931: %CONST_DIAG-SP-3-BOOTUP_TEST_FAIL: Module 6: TestL3Capture2 failed *Oct 7 18:19:35.931: %CONST_DIAG-SP-3-BOOTUP_TEST_FAIL: Module 6: TestIPv6FibShortcut failed *Oct 7 18:19:35.931: %CONST_DIAG-SP-3-BOOTUP_TEST_FAIL: Module 6: TestMPLSFibShortcut failed *Oct 7 18:19:35.931: %CONST_DIAG-SP-3-BOOTUP_TEST_FAIL: Module 6: TestNATFibShortcut failed
Exactly what's in slot 6?
Discover and save your favorite ideas. Come back to expert answers, step-by-step guides, recent topics, and more.
New here? Get started with these tips. How to use Community New member guide