cancel
Showing results for 
Search instead for 
Did you mean: 
cancel
9168
Views
10
Helpful
26
Replies

Supervisor Engine 720 errors

ahmad82pkn
Level 3
Level 3

hi, i have a standalone SUP720, no traffic is passing through it. only one port connected for remote telnet, its kind of backup device.

today i checked its log, and there are some wiered message.

is it a software bug or my SUP has gone bad?

any suggestion/help would be appreciated.

*Apr 19 18:19:02: %CONST_DIAG-SP-3-HM_TEST_FAIL: Module 6 TestSPRPInbandPing consecutive failure count:5

*Apr 19 18:19:03: %CONST_DIAG-SP-6-HM_TEST_INFO: CPU util(5sec): SP=7% RP=2% Traffic=0%

netint_thr_active[0], Tx_Rate[658], Rx_Rate[80], dev=4[IPv6, fail=5]

*Apr 19 18:20:29: %CONST_DIAG-SP-3-HM_TEST_FAIL: Module 6 TestSPRPInbandPing consecutive failure count:10

*Apr 19 18:20:30: %CONST_DIAG-SP-6-HM_TEST_INFO: CPU util(5sec): SP=3% RP=0% Traffic=0%

netint_thr_active[0], Tx_Rate[676], Rx_Rate[100], dev=4[IPv6, fail=10]

*Apr 19 18:21:56: %CONST_DIAG-SP-3-HM_TEST_FAIL: Module 6 TestSPRPInbandPing consecutive failure count:15

*Apr 19 18:21:57: %CONST_DIAG-SP-6-HM_TEST_INFO: CPU util(5sec): SP=1% RP=0% Traffic=0%

netint_thr_active[0], Tx_Rate[636], Rx_Rate[80], dev=4[IPv6, fail=15]

*Apr 19 18:23:23: %CONST_DIAG-SP-3-HM_TEST_FAIL: Module 6 TestSPRPInbandPing consecutive failure count:20

*Apr 19 18:23:24: %CONST_DIAG-SP-6-HM_TEST_INFO: CPU util(5sec): SP=7% RP=0% Traffic=0%

netint_thr_active[0], Tx_Rate[636], Rx_Rate[80], dev=4[IPv6, fail=20]

*Apr 19 18:24:50: %CONST_DIAG-SP-3-HM_TEST_FAIL: Module 6 TestSPRPInbandPing consecutive failure count:25

*Apr 19 18:24:51: %CONST_DIAG-SP-6-HM_TEST_INFO: CPU util(5sec): SP=5% RP=0% Traffic=0%

netint_thr_active[0], Tx_Rate[538], Rx_Rate[20], dev=4[IPv6, fail=25]

*Apr 19 18:26:16: %CONST_DIAG-SP-3-HM_TEST_FAIL: Module 6 TestSPRPInbandPing consecutive failure count:30

*Apr 19 18:26:17: %CONST_DIAG-SP-6-HM_TEST_INFO: CPU util(5sec): SP=3% RP=1% Traffic=0%

netint_thr_active[0], Tx_Rate[578], Rx_Rate[40], dev=4[IPv6, fail=30]

*Apr 19 18:27:43: %CONST_DIAG-SP-3-HM_TEST_FAIL: Module 6 TestSPRPInbandPing consecutive failure count:35

*Apr 19 18:27:44: %CONST_DIAG-SP-6-HM_TEST_INFO: CPU util(5sec): SP=6% RP=0% Traffic=0%

netint_thr_active[0], Tx_Rate[608], Rx_Rate[80], dev=4[IPv6, fail=35]

*Apr 19 18:29:10: %CONST_DIAG-SP-3-HM_TEST_FAIL: Module 6 TestSPRPInbandPing consecutive failure count:40

*Apr 19 18:29:11: %CONST_DIAG-SP-6-HM_TEST_INFO: CPU util(5sec): SP=5% RP=2% Traffic=0%

netint_thr_active[0], Tx_Rate[670], Rx_Rate[100], dev=4[IPv6, fail=40]

*Apr 19 18:30:36: %CONST_DIAG-SP-3-HM_TEST_FAIL: Module 6 TestSPRPInbandPing consecutive failure count:45

*Apr 19 18:30:37: %CONST_DIAG-SP-6-HM_TEST_INFO: CPU util(5sec): SP=3% RP=0% Traffic=0%

netint_thr_active[0], Tx_Rate[608], Rx_Rate[80], dev=4[IPv6, fail=45]

*Apr 19 18:32:03: %CONST_DIAG-SP-3-HM_TEST_FAIL: Module 6 TestSPRPInbandPing consecutive failure count:50

*Apr 19 18:32:04: %CONST_DIAG-SP-6-HM_TEST_INFO: CPU util(5sec): SP=2% RP=0% Traffic=0%

netint_thr_active[0], Tx_Rate[698], Rx_Rate[100], dev=4[IPv6, fail=50]

*Apr 19 18:33:30: %CONST_DIAG-SP-3-HM_TEST_FAIL: Module 6 TestSPRPInbandPing consecutive failure count:55

*Apr 19 18:33:31: %CONST_DIAG-SP-6-HM_TEST_INFO: CPU util(5sec): SP=3% RP=0% Traffic=0%

netint_thr_active[0], Tx_Rate[658], Rx_Rate[80], dev=4[IPv6, fail=55]

*Apr 19 18:34:56: %CONST_DIAG-SP-3-HM_TEST_FAIL: Module 6 TestSPRPInbandPing consecutive failure count:60

*Apr 19 18:34:57: %CONST_DIAG-SP-6-HM_TEST_INFO: CPU util(5sec): SP=3% RP=0% Traffic=0%

netint_thr_active[0], Tx_Rate[653], Rx_Rate[100], dev=4[IPv6, fail=60]

*Apr 19 18:36:23: %CONST_DIAG-SP-3-HM_TEST_FAIL: Module 6 TestSPRPInbandPing consecutive failure count:65

*Apr 19 18:36:24: %CONST_DIAG-SP-6-HM_TEST_INFO: CPU util(5sec): SP=4% RP=2% Traffic=0%

netint_thr_active[0], Tx_Rate[578], Rx_Rate[60], dev=4[IPv6, fail=65]

*Apr 19 18:36:58: %CONST_DIAG-SP-6-HM_TEST_RECOVERED: Module 6 TestSPRPInbandPing recovered after 66 consecutive failure(s)

*Apr 19 18:38:24: %CONST_DIAG-SP-3-HM_TEST_FAIL: Module 6 TestSPRPInbandPing consecutive failure count:5

*Apr 19 18:38:25: %CONST_DIAG-SP-6-HM_TEST_INFO: CPU util(5sec): SP=3% RP=0% Traffic=0%

netint_thr_active[0], Tx_Rate[653], Rx_Rate[100], dev=4[IPv6, fail=5]

*Apr 19 18:39:35: %CONST_DIAG-SP-6-HM_TEST_RECOVERED: Module 6 TestSPRPInbandPing recovered after 8 consecutive failure(s)

*Apr 19 18:42:08: %CONST_DIAG-SP-3-HM_TEST_FAIL: Module 6 TestSPRPInbandPing consecutive failure count:5

*Apr 19 18:42:09: %CONST_DIAG-SP-6-HM_TEST_INFO: CPU util(5sec): SP=4% RP=0% Traffic=0%

netint_thr_active[0], Tx_Rate[670], Rx_Rate[100], dev=4[IPv6, fail=5]

*Apr 19 18:43:34: %CONST_DIAG-SP-6-HM_TEST_RECOVERED: Module 6 TestSPRPInbandPing recovered after 9 consecutive failure(s)

26 Replies 26

Reza Sharifi
Hall of Fame
Hall of Fame

Hi,

These messages are from Module 6.  Is your sup in modules 6 or is this a line card?

What is the output of "sh module"?

HTH

yes its SUP in module 6

6506BK>sh module

Mod Ports Card Type                              Model              Serial No.

--- ----- -------------------------------------- ------------------ -----------

  1   48  SFM-capable 48 port 10/100/1000mb RJ45 WS-X6548-GE-TX     

  2   48  48 port 10/100 mb RJ-45 ethernet       WS-X6248-RJ-45     

  3   48  SFM-capable 48-port 10/100 Mbps RJ45   WS-X6548-RJ-45     

  4   48  SFM-capable 48 port 10/100/1000mb RJ45 WS-X6548-GE-TX     

  5   48  48 port 10/100 mb RJ-45 ethernet       WS-X6248-RJ-45     

  6    2  Supervisor Engine 720 (Active)         WS-SUP720-BASE     

Make sure the supervisor card is properly inserted/seated on the slot.

If it is, then contact Cisco TAC and get the card RMA-ed.

Will do this excercise and update.

In addition to what Leo suggested, it maybe also helpful if you load a different version of IOS.  It could be software issue.

HTH

I have same IOS running on my other 720 SUP/6506 with same cards and hardware. so i believe it might not be software issue?

Hi Ahmed,

This error message means that there was no communication between the

supervisor and the MSFC. This is most likely a hardware issue that could

however have been transient. The usual procedure is to monitor the router to

be sure it does not reoccur. If it happens again, then supervisor needs to

be RMA.

Whats the status of this module under show module?

Also plan for RMA of the module.

HTH

Regards

Inayath

*Plz rate all usefull posts.

Error keeps coming, they enver stops.

show module shows minor errors.

6506BK#sh module

Mod Ports Card Type                              Model              Serial No.

--- ----- -------------------------------------- ------------------ -----------

  1   48  SFM-capable 48 port 10/100/1000mb RJ45 WS-X6548-GE-TX     

  2   48  48 port 10/100 mb RJ-45 ethernet       WS-X6248-RJ-45     

  3   48  SFM-capable 48-port 10/100 Mbps RJ45   WS-X6548-RJ-45     

  4   48  SFM-capable 48 port 10/100/1000mb RJ45 WS-X6548-GE-TX     

  5   48  48 port 10/100 mb RJ-45 ethernet       WS-X6248-RJ-45    

  6    2  Supervisor Engine 720 (Active)         WS-SUP720-BASE     

Mod MAC addresses                       Hw    Fw           Sw           Status

--- ---------------------------------- ------ ------------ ------------ -------

  1  0008.7dce.fae8 to 0008.7dce.fb17  10.1   7.2(1)       8.5(0.46)RFW Ok

  2  0050.f0af.a174 to 0050.f0af.a1a3   1.1   4.2(0.24)VAI 8.5(0.46)RFW Ok

  3  0008.7dcd.f9f0 to 0008.7dcd.fa1f   6.1   6.3(1)       8.5(0.46)RFW Ok

  4  0013.1a0e.0d20 to 0013.1a0e.0d4f  10.1   7.2(1)       8.5(0.46)RFW Ok

  5  00d0.bce8.aba0 to 00d0.bce8.abcf   1.1   4.2(0.24)VAI 8.5(0.46)RFW Ok

  6  0011.21b5.6c34 to 0011.21b5.6c37   3.2   8.1(3)       12.2(18)SXF1 Ok

Mod  Sub-Module                  Model              Serial       Hw     Status

---- --------------------------- ------------------ ----------- ------- -------

  6  Policy Feature Card 3       WS-F6K-PFC3A          2.4    Ok

  6  MSFC3 Daughterboard         WS-SUP720            2.4    Ok

Mod  Online Diag Status

---- -------------------

  1  Minor Error

  2  Pass

  3  Minor Error

  4  Minor Error

  5  Pass

  6  Minor Error

Canto do RMA, since ots not under RMA contract with Cisco :-s

Hi Ahmed,

If no contract for RMA. Then I would suggest to do the Hard Reset of the module on this switch and update me the result.

1- Remove all the modules which are showing Minor Error and insert it one by one.

2- Get the following outputs:

1- Show module

2- show logging

3- show diagnostic result module all.

HTH

Regards

Inayath

Hi Inayat,

Sorry i am late here , actually had to involve onshore field support to help me out.

i have taken out all modules, and now only SUP720 and one FastEthernet module present in the Switch.

i asked Field support to remove all Cards, and then put back Sup720 and one FastEthernet module.

here is output of your requested commands, i hope you can guid me in some direction to resolve this issue

sh module

Mod Ports Card Type                              Model              Serial No.

--- ----- -------------------------------------- ------------------ -----------

  1   48  SFM-capable 48 port 10/100/1000mb RJ45 WS-X6548-GE-TX     

  6    2  Supervisor Engine 720 (Active)         WS-SUP720-BASE     

Mod MAC addresses                       Hw    Fw           Sw           Status

--- ---------------------------------- ------ ------------ ------------ -------

  1  0008.7dce.fae8 to 0008.7dce.fb17  10.1   7.2(1)       8.5(0.46)RFW Ok

  6  0011.21b5.6c34 to 0011.21b5.6c37   3.2   8.1(3)       12.2(18)SXF1 Ok

Mod  Sub-Module                  Model              Serial       Hw     Status

---- --------------------------- ------------------ ----------- ------- -------

  6  Policy Feature Card 3       WS-F6K-PFC3A          2.4    Ok

  6  MSFC3 Daughterboard         WS-SUP720             2.4    Ok

Mod  Online Diag Status

---- -------------------

  1  Minor Error

  6  Minor Error

--------------------------------------------------------------------------------------------------------------------------------

--------------------------------------------------------------------------------------------------------------------------------

6506BK#sh logging

Syslog logging: enabled (0 messages dropped, 0 messages rate-limited, 0 flushes, 0 overruns)

    Console logging: level debugging, 54 messages logged

    Monitor logging: level debugging, 0 messages logged

    Buffer logging: level debugging, 54 messages logged

    Exception Logging: size (4096 bytes)

    Count and timestamp logging messages: disabled

    Trap logging: level informational, 45 message lines logged

Log Buffer (8192 bytes):

00:01:01: curr is 0x0

00:01:01: RP: Currently running ROMMON from S (Gold) region

*May 15 14:31:36: %SYS-6-CLOCKUPDATE: System clock has been updated from 19:31:36 UTC Wed May 15 2013 to 14:31:36 EST Wed May 15 2013, configured from console by console.

*May 15 15:31:36: %SYS-6-CLOCKUPDATE: System clock has been updated from 14:31:36 EST Wed May 15 2013 to 15:31:36 EDT Wed May 15 2013, configured from console by console.

*May 15 15:32:10: %SYS-5-CONFIG_I: Configured from memory by console

*May 15 15:32:11: %SYS-5-RESTART: System restarted --

Cisco Internetwork Operating System Software

IOS (tm) s72033_rp Software (s72033_rp-IPSERVICESK9_WAN-M), Version 12.2(18)SXF17, RELEASE SOFTWARE (fc1)

Technical Support: http://www.cisco.com/techsupport

Copyright (c) 1986-2009 by cisco Systems, Inc.

Compiled Fri 25-Sep-09 10:53 by ccai

*May 15 15:32:11: %MFIB_CONST_RP-6-REPLICATION_MODE_CHANGE: Replication Mode Change Detected. Current system replication mode is Ingress

*May 15 15:32:12: %SYS-3-LOGGER_FLUSHED: System was paused for 00:00:00 to ensure console debugging output.

Firmware compiled 15-Jul-09 16:40 by integ Build [100]

*May 15 15:32:12: %OIR-SP-6-CONSOLE: Changing console ownership to route processor

*May 15 15:32:13: %SYS-SP-3-LOGGER_FLUSHED: System was paused for 00:00:00 to ensure console debugging output.

*May 15 15:32:13: %FILESYS-SP-5-MISMATCH: bootflash device programming algorithm mismatch detected, reformat is recommended

00:01:09: SP: SP: Currently running ROMMON from S (Gold) region

*May 15 15:32:14: %SYS-SP-5-RESTART: System restarted --

Cisco Internetwork Operating System Software

IOS (tm) s72033_sp Software (s72033_sp-IPSERVICESK9_WAN-M), Version 12.2(18)SXF17, RELEASE SOFTWARE (fc1)

Technical Support: http://www.cisco.com/techsupport

Copyright (c) 1986-2009 by cisco Systems, Inc.

Compiled Fri 25-Sep-09 11:01 by ccai

*May 15 15:32:14: %OIR-SP-6-INSPS: Power supply inserted in slot 1

*May 15 15:32:14: %C6KPWR-SP-4-PSOK: power supply 1 turned on.

*May 15 15:32:14: %OIR-SP-6-INSPS: Power supply inserted in slot 2

*May 15 15:32:16: %C6KENV-SP-4-FANHIOUTPUT: Version 2 high-output fan-tray is in effect

*May 15 15:32:20: %FABRIC-SP-5-FABRIC_MODULE_ACTIVE: The Switch Fabric Module in slot 6 became active.

*May 15 15:32:20: %DIAG-SP-6-RUN_MINIMUM: Module 6: Running Minimal Diagnostics...

*May 15 15:32:46: %DIAG-SP-3-MINOR: Module 6: Online Diagnostics detected a Minor Error. Please use 'show diagnostic result ' to see test results.

*May 15 15:32:46: %CONST_DIAG-SP-3-BOOTUP_TEST_FAIL: Module 6: TestFibDevices failed

*May 15 15:32:46: %CONST_DIAG-SP-3-BOOTUP_TEST_FAIL: Module 6: TestIPv6FibShortcut failed

*May 15 15:32:46: %CONST_DIAG-SP-3-BOOTUP_TEST_FAIL: Module 6: TestL3VlanMet failed

*May 15 15:32:46: %CONST_DIAG-SP-3-BOOTUP_TEST_FAIL: Module 6: TestTransceiverIntegrity failed on port(s) 1

*May 15 15:32:46: %PM_SCP-SP-3-TRANSCEIVER_UNSUPPORTED: Unsupported transceiver in LAN port 6/1

*May 15 15:32:46: %CONST_DIAG-SP-3-BOOTUP_TEST_FAIL: Module 6: TestT

*May 15 15:32:46: %PM_SCP-SP-3-TRANSCEIVER_UNSUPPORTED: Unsupported transceiver in LAN port 6/2

*May 15 15:32:47: %OIR-SP-6-INSCARD: Card inserted in slot 6, interfaces are now online

*May 15 15:32:49: %DIAG-SP-6-RUN_MINIMUM: Module 1: Running Minimal Diagnostics...

*May 15 15:33:00: %DIAG-SP-3-MINOR: Module 1: Online Diagnostics detected a Minor Error. Please use 'show diagnostic result ' to see test results.

*May 15 15:33:00: %CONST_DIAG-SP-3-BOOTUP_TEST_FAIL: Module 1: TestL3VlanMet failed

*May 15 15:33:01: %OIR-SP-6-INSCARD: Card inserted in slot 1, interfaces are now online

*May 15 15:34:14: %CONST_DIAG-SP-3-HM_TEST_FAIL: Module 6 TestSPRPInbandPing consecutive failure count:5

*May 15 15:34:15: %CONST_DIAG-SP-6-HM_TEST_INFO: CPU util(5sec): SP=9% RP=0% Traffic=0%

netint_thr_active[0], Tx_Rate[698], Rx_Rate[100], dev=4[IPv6, fail=5]

*May 15 15:35:41: %CONST_DIAG-SP-3-HM_TEST_FAIL: Module 6 TestSPRPInbandPing consecutive failure count:10

*May 15 15:35:42: %CONST_DIAG-SP-6-HM_TEST_INFO: CPU util(5sec): SP=3% RP=0% Traffic=0%

netint_thr_active[0], Tx_Rate[658], Rx_Rate[80], dev=4[IPv6, fail=10]

*May 15 15:37:08: %CONST_DIAG-SP-3-HM_TEST_FAIL: Module 6 TestSPRPInbandPing consecutive failure count:15

*May 15 15:37:09: %CONST_DIAG-SP-6-HM_TEST_INFO: CPU util(5sec): SP=3% RP=0% Traffic=0%

netint_thr_active[0], Tx_Rate[698], Rx_Rate[100], dev=4[IPv6, fail=15]

*May 15 15:38:34: %CONST_DIAG-SP-3-HM_TEST_FAIL: Module 6 TestSPRPInbandPing consecutive failure count:20

*May 15 15:38:35: %CONST_DIAG-SP-6-HM_TEST_INFO: CPU util(5sec): SP=4% RP=0% Traffic=0%

netint_thr_active[0], Tx_Rate[556], Rx_Rate[40], dev=4[IPv6, fail=20]

*May 15 15:40:01: %CONST_DIAG-SP-3-HM_TEST_FAIL: Module 6 TestSPRPInbandPing consecutive failure count:25

*May 15 15:40:02: %CONST_DIAG-SP-6-HM_TEST_INFO: CPU util(5sec): SP=7% RP=0% Traffic=0%

netint_thr_active[0], Tx_Rate[538], Rx_Rate[20], dev=4[IPv6, fail=25]

*May 15 15:40:53: %CONST_DIAG-SP-6-HM_TEST_RECOVERED: Module 6 TestSPRPInbandPing recovered after 27 consecutive failure(s)

-------------------------------------------------------------------------------------------------------------------------------------------

-------------------------------------------------------------------------------------------------------------------------------------------

6506BK#      sh diagnostic result module all

Current bootup diagnostic level: minimal

Module 1: SFM-capable 48 port 10/100/1000mb RJ45  

  Overall Diagnostic Result for Module 1 : MINOR ERROR

  Diagnostic level at card bootup: minimal

  Test results: (. = Pass, F = Fail, U = Untested)

    1) TestLoopback:

   Port  1  2  3  4  5  6  7  8  9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24

   ----------------------------------------------------------------------------

         .  .  .  .  .  .  .  .  .  .  .  .  .  .  .  .  .  .  .  .  .  .  .  .

   Port 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48

   ----------------------------------------------------------------------------

         .  .  .  .  .  .  .  .  .  .  .  .  .  .  .  .  .  .  .  .  .  .  .  .

    2) TestSynchedFabChannel -----------> .

    3) TestL3VlanMet -------------------> F

    4) TestIngressSpan -----------------> .

    5) TestEgressSpan ------------------> .

    6) TestAsicMemory ------------------> U

    7) TestFirmwareDiagStatus ----------> .

    8) TestEobcStressPing --------------> U

Module 6: Supervisor Engine 720 (Active)  

  Overall Diagnostic Result for Module 6 : MINOR ERROR

  Diagnostic level at card bootup: minimal

  Test results: (. = Pass, F = Fail, U = Untested)

    1) TestScratchRegister -------------> .

    2) TestSPRPInbandPing --------------> F

    3) TestTransceiverIntegrity:

      Port  1  2

      ----------

            F  F

    4) TestActiveToStandbyLoopback:

      Port  1  2

      ----------

            U  U

    5) TestLoopback:

      Port  1  2

      ----------

            .  .

    6) TestNewIndexLearn ---------------> .

    7) TestDontConditionalLearn --------> .

    8) TestBadBpduTrap -----------------> .

    9) TestMatchCapture ----------------> .

   10) TestProtocolMatchChannel --------> .

   11) TestFibDevices ------------------> F

   12) TestIPv4FibShortcut -------------> .

   13) TestL3Capture2 ------------------> .

   14) TestIPv6FibShortcut -------------> F

   15) TestMPLSFibShortcut -------------> .

   16) TestNATFibShortcut --------------> .

   17) TestAclPermit -------------------> .

   18) TestAclDeny ---------------------> .

   19) TestQoSTcam ---------------------> .

   20) TestL3VlanMet -------------------> F

   21) TestIngressSpan -----------------> .

   22) TestEgressSpan ------------------> .

   23) TestNetflowInlineRewrite:

      Port  1  2

      ----------

            U  U

   24) TestFabricSnakeForward ----------> .

   25) TestFabricSnakeBackward ---------> .

   26) TestTrafficStress ---------------> U

   27) TestFibTcamSSRAM ----------------> U

   28) TestAsicMemory ------------------> U

   29) TestNetflowTcam -----------------> U

   30) ScheduleSwitchover --------------> U

   31) TestFirmwareDiagStatus ----------> .

   32) TestFabricFlowControlStatus -----> U

Thank you for Every one help and interest so far