cancel
Showing results for 
Search instead for 
Did you mean: 
cancel
2041
Views
0
Helpful
1
Replies

6500 hang

WEERAKOO69BA
Level 1
Level 1

dear friends,

I have WS-C6509-E ,whci has two Supervisor Engine 32 10GE .Suddenly it hung and customers could not do anything & they restarted by powering off  and after that Iit is working  fine up to now & never know whether it will happen again or not.

I managed to get  following logs.Base on following logs(full log  mentioned here) can anybody tell me what  was the problem which  caused to hang the switch?

thanks

*Jun 12 05:02:23.119: RP: Currently running ROMMON from S (Gold) region
*Jun 12 05:02:23.971: %SPANTREE-5-EXTENDED_SYSID: Extended SysId enabled for type vlan. The Bridge IDs of all active STP instances have been updated, which might change the spanning tree topology
*Jun 12 05:02:28.775: %SYS-5-CONFIG_I: Configured from memory by console
*Jun 12 05:02:33.371: %SYS-5-RESTART: System restarted --
Cisco IOS Software, s3223_rp Software (s3223_rp-IPBASEK9-M), Version 12.2(33)SXI1, RELEASE SOFTWARE (fc3)
Technical Support: http://www.cisco.com/techsupport
Copyright (c) 1986-2009 by Cisco Systems, Inc.
Compiled Fri 27-Mar-09 11:40 by prod_rel_team
*Jun 12 04:52:21.691: %SYS-SP-3-LOGGER_FLUSHED: System was paused for 00:00:00 to ensure console debugging output.
*Jun 12 05:02:19.327: %SPANTREE-SP-5-EXTENDED_SYSID: Extended SysId enabled for type vlan. The Bridge IDs of all active STP instances have been updated, which might change the spanning tree topology
*Jun 12 05:02:32.620: %SYS-SP-5-RESTART: System restarted --
Cisco IOS Software, s3223_sp Software (s3223_sp-IPBASEK9-M), Version 12.2(33)SXI1, RELEASE SOFTWARE (fc3)
Technical Support: http://www.cisco.com/techsupport
--More--                           Copyright (c) 1986-2009 by Cisco Systems, Inc.
Compiled Fri 27-Mar-09 12:03 by prod_rel_team
*Jun 12 05:02:32.640: SP: Currently running ROMMON from S (Gold) region
*Jun 12 05:02:33.584: %OIR-SP-6-INSPS: Power supply inserted in slot 1
*Jun 12 05:02:33.588: %C6KPWR-SP-4-PSOK: power supply 1 turned on.
*Jun 12 05:02:33.652: %OIR-SP-6-INSPS: Power supply inserted in slot 2
*Jun 12 05:02:33.652: %C6KPWR-SP-4-PSOK: power supply 2 turned on.
*Jun 12 05:02:33.656: %C6KPWR-SP-4-PSREDUNDANTBOTHSUPPLY: in power-redundancy mode, system is operating on both power supplies.
*Jun 12 05:02:35.135: %SNMP-5-COLDSTART: SNMP agent on host QURUM_Core is undergoing a cold start
*Jun 12 05:02:40.951: %DIAG-SP-6-RUN_MINIMUM: Module 5: Running Minimal Diagnostics...
*Jun 12 05:03:06.267: %DIAG-SP-6-DIAG_OK: Module 5: Passed Online Diagnostics
Jun 12 05:03:07.571: %MFIB_CONST_RP-6-REPLICATION_MODE_CHANGE: Replication Mode Change Detected. Current system replication mode is Ingress
*Jun 12 05:03:07.859: %OIR-SP-6-INSCARD: Card inserted in slot 5, interfaces are now online
*Jun 12 05:03:07.919: %DIAG-SP-6-RUN_MINIMUM: Module 1: Running Minimal Diagnostics...
*Jun 12 05:03:08.959: %DIAG-SP-6-RUN_MINIMUM: Module 4: Running Minimal Diagnostics...
Jun 12 05:03:18.563: %CAPI_EC-4-RATE_LIMITED: Adding interfaces on WS-X6148-GE-TX to an etherchannel will limit channel throughput to 1 Gbps!
--More--                           *Jun 12 05:03:17.468: %DIAG-SP-6-DIAG_OK: Module 1: Passed Online Diagnostics
Jun 12 05:03:18.571: %CAPI_EC-4-RATE_LIMITED: Adding interfaces on WS-X6148-GE-TX to an etherchannel will limit channel throughput to 1 Gbps!
Jun 12 05:03:18.579: %CAPI_EC-4-RATE_LIMITED: Adding interfaces on WS-X6148-GE-TX to an etherchannel will limit channel throughput to 1 Gbps!
Jun 12 05:03:18.583: %CAPI_EC-4-RATE_LIMITED: Adding interfaces on WS-X6148-GE-TX to an etherchannel will limit channel throughput to 1 Gbps!
*Jun 12 05:03:19.790: %OIR-SP-6-INSCARD: Card inserted in slot 1, interfaces are now online
*Jun 12 05:03:20.658: %DIAG-SP-6-DIAG_OK: Module 4: Passed Online Diagnostics
*Jun 12 05:03:21.018: %OIR-SP-6-INSCARD: Card inserted in slot 4, interfaces are now online
Jun 12 05:03:25.168: %PFREDUN-SP-6-ACTIVE: Standby initializing for SSO mode
Jun 12 05:03:25.336: %SYS-SP-3-LOGGER_FLUSHED: System was paused for 00:00:00 to ensure console debugging output.
Jun 12 05:03:36.342: %PFINIT-SP-5-CONFIG_SYNC: Sync'ing the startup configuration to the standby Router.
Jun 12 05:04:34.515: %DIAG-SP-6-RUN_MINIMUM: Module 3: Running Minimal Diagnostics...
Jun 12 05:04:37.331: %DIAG-SP-6-DIAG_OK: Module 3: Passed Online Diagnostics
Jun 12 05:04:38.335: %SVCLC-5-FWVTPMODE: VTP mode is set to non-transparent
Jun 12 05:04:38.779: %DIAG-SP-6-RUN_MINIMUM: Module 2: Running Minimal Diagnostics...
--More--                           Jun 12 05:04:39.539: %DIAG-SP-6-DIAG_OK: Module 2: Passed Online Diagnostics
Jun 12 05:04:40.395: %OIR-SP-6-INSCARD: Card inserted in slot 2, interfaces are now online
Jun 12 05:04:47.637: %OIR-SP-6-INSCARD: Card inserted in slot 3, interfaces are now online
Jun 12 05:05:51.171: %DIAG-SP-6-RUN_MINIMUM: Module 6: Running Minimal Diagnostics...
Jun 12 05:05:53.267: %DIAG-SP-6-DIAG_OK: Module 6: Passed Online Diagnostics
Jun 12 05:05:54.171: %OIR-SP-6-INSCARD: Card inserted in slot 6, interfaces are now online
*Jun 12 05:02:30.239: %SYS-SPSTBY-3-LOGGER_FLUSHED: System was paused for 00:00:00 to ensure console debugging output.
*Jun 12 05:03:02.379: %SPANTREE-SPSTBY-5-EXTENDED_SYSID: Extended SysId enabled for type vlan. The Bridge IDs of all active STP instances have been updated, which might change the spanning tree topology
Jun 12 05:05:25.371: %DIAG-SPSTBY-6-RUN_MINIMUM: Module 6: Running Minimal Diagnostics...
Jun 12 05:05:49.715: %DIAG-SPSTBY-6-DIAG_OK: Module 6: Passed Online Diagnostics
Jun 12 05:06:19.351: %SYS-SPSTBY-5-RESTART: System restarted --
Cisco IOS Software, s3223_sp Software (s3223_sp-IPBASEK9-M), Version 12.2(33)SXI1, RELEASE SOFTWARE (fc3)
Technical Support: http://www.cisco.com/techsupport
Copyright (c) 1986-2009 by Cisco Systems, Inc.
--More--                           Compiled Fri 27-Mar-09 12:03 by prod_rel_team
Jun 12 05:06:19.355: SPSTBY: Currently running ROMMON from S (Gold) region
Jun 12 05:06:32.463: %HA_CONFIG_SYNC-6-BULK_CFGSYNC_SUCCEED: Bulk Sync succeeded
Jun 12 05:06:33.019: %PFREDUN-SPSTBY-6-STANDBY: Ready for SSO mode
Jun 12 05:06:33.295: %RF-SP-5-RF_TERMINAL_STATE: Terminal state reached for (SSO)
Jun 12 05:06:33.183: %SYS-SPSTBY-3-LOGGER_FLUSHED: System was paused for 00:00:00 to ensure console debugging output.

1 Reply 1

Giuseppe Larosa
Hall of Fame
Hall of Fame

Hello Weerakoo69A,

>> Base on following logs(full log  mentioned here) can anybody tell me what  was the problem which  caused to hang the switch?

Unfortunately, this is not possible because after powering off the switch the previous log messages are lost you need to configure the device to send syslog messages to a syslog server.

if you already have configured for export of log messages you should check on the syslog server rathen then on the device

Actually, the messages that you have attached are the messages after the restart, and they do no provide any info about the previous crash.

I guess from messages that the C6500 has two supervisors installed

Jun 12 05:06:19.355: SPSTBY: Currently running ROMMON from S (Gold) region
Jun 12 05:06:32.463: %HA_CONFIG_SYNC-6-BULK_CFGSYNC_SUCCEED: Bulk Sync succeeded
Jun 12 05:06:33.019: %PFREDUN-SPSTBY-6-STANDBY: Ready for SSO mode
Jun 12 05:06:33.295: %RF-SP-5-RF_TERMINAL_STATE: Terminal state reached for (SSO)

and the boot was successful with an active SP and an hot standby SP you can check this with sh redudancy.

Edit:

I can only add that moving to a later image like 12.2(33)SXI2a is a wise step as several bugs have been fixed.

Hope to help

Giuseppe

Review Cisco Networking for a $25 gift card