Suspecting Sup module problem (WS-C6506-E/WS-SUP720-3B)

[AnandBalakrishnan]

Hi Guys,

i have a problem here, i have Two WS-C6506-E model switches. In both switches i am getting below outputs. I some module i am seeing minor/major error.

Switch-Pri#sh module

Mod Ports Card Type                              Model              Serial No.

--- ----- -------------------------------------- ------------------ -----------

  1   24  CEF720 24 port 1000mb SFP              WS-X6724-SFP       SAL1425L2T2

  2   48  CEF720 48 port 10/100/1000mb Ethernet  WS-X6748-GE-TX     SAL1541SEA7

  3   48  CEF720 48 port 10/100/1000mb Ethernet  WS-X6748-GE-TX     SAL1425KVFM

  6    2  Supervisor Engine 720 (Active)         WS-SUP720-3B       SAL1424KDFS

Mod MAC addresses                       Hw    Fw           Sw           Status

--- ---------------------------------- ------ ------------ ------------ -------

  1  c84c.7578.316c to c84c.7578.3183   4.3   12.2(18r)S1  12.2(33)SXH5 Ok

  2  ccef.48bf.c0c0 to ccef.48bf.c0ef   4.2   12.2(18r)S1  12.2(33)SXH5 Ok

  3  5475.d0fd.3210 to 5475.d0fd.323f   3.4   12.2(18r)S1  12.2(33)SXH5 Ok

  6  0025.4539.ad00 to 0025.4539.ad03   5.10  8.5(3)       12.2(33)SXH5 Ok

Mod  Sub-Module                  Model              Serial       Hw     Status

---- --------------------------- ------------------ ----------- ------- -------

  1  Centralized Forwarding Card WS-F6700-CFC       SAL1424KJTK  4.1    Ok

  2  Centralized Forwarding Card WS-F6700-CFC       SAL114140WX  4.1    Ok

  3  Centralized Forwarding Card WS-F6700-CFC       SAL1425KQGX  4.1    Ok

  6  Policy Feature Card 3       WS-F6K-PFC3B       SAL1424KF0R  2.6    Ok

  6  MSFC3 Daughterboard         WS-SUP720          SAL1424KCMQ  4.1    Ok

Mod  Online Diag Status

---- -------------------

  1  Minor Error

  2  Minor Error

  3  Minor Error

  6  Minor Error

=================================================

Cisco IOS Software, s72033_rp Software (s72033_rp-IPSERVICESK9_WAN-M), Version 12.2(33)SXH5, RELEASE SOFTWARE (fc1)

Technical Support: http://www.cisco.com/techsupport

Copyright (c) 1986-2009 by Cisco Systems, Inc.

Compiled Thu 16-Apr-09 19:28 by prod_rel_team

ROM: System Bootstrap, Version 12.2(17r)SX6, RELEASE SOFTWARE (fc1)

Switch-Pri uptime is 1 hour, 55 minutes

Uptime for this control processor is 1 hour, 55 minutes

Time since switch-pri switched to active is 1 hour, 53 minutes

System returned to ROM by s/w reset at 17:18:42 IST Wed Jan 15 2014 (SP by bus error at PC 0x417CC1E8, address 0x0)

System restarted at 17:37:34 IST Wed Jan 15 2014

System image file is "sup-bootdisk:s72033-ipservicesk9_wan-mz.122-33.SXH5.bin"

This product contains cryptographic features and is subject to United

States and local country laws governing import, export, transfer and

use. Delivery of Cisco cryptographic products does not imply

third-party authority to import, export, distribute or use encryption.

Importers, exporters, distributors and users are responsible for

compliance with U.S. and local country laws. By using this product you

agree to comply with applicable laws and regulations. If you are unable

to comply with U.S. and local laws, return this product immediately.

A summary of U.S. laws governing Cisco cryptographic products may be found at:

http://www.cisco.com/wwl/export/crypto/tool/stqrg.html

If you require further assistance please contact us by sending email to

export@cisco.com.

cisco WS-C6506-E (R7000) processor (revision 1.2) with 458720K/65536K bytes of memory.

Processor board ID SAL1426LG8Y

SR71000 CPU at 600Mhz, Implementation 0x504, Rev 1.2, 512KB L2 Cache

Last reset from s/w reset

3 Virtual Ethernet interfaces

122 Gigabit Ethernet interfaces

1917K bytes of non-volatile configuration memory.

8192K bytes of packet buffer memory.

65536K bytes of Flash internal SIMM (Sector size 512K).

Configuration register is 0x2102

==================================================================

Switch-Pri#sh diagnostic result module all

Current bootup diagnostic level: complete

Module 1: CEF720 24 port 1000mb SFP  SerialNo : SAL1425L2T2

  Overall Diagnostic Result for Module 1 : MINOR ERROR

  Diagnostic level at card bootup: complete

  Test results: (. = Pass, F = Fail, U = Untested)

    1) TestTransceiverIntegrity:

   Port  1  2  3  4  5  6  7  8  9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24

   ----------------------------------------------------------------------------

         .  U  U  U  U  U  U  U  U  U  U  U  U  U  U  U  U  U  U  U  U  U  U  .

    2) TestLoopback:

   Port  1  2  3  4  5  6  7  8  9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24

   ----------------------------------------------------------------------------

         .  .  .  .  .  .  .  .  .  .  .  .  .  .  .  .  .  .  .  .  .  .  .  .

    3) TestTxPathMonitoring ------------> .

    4) TestSynchedFabChannel -----------> .

    5) TestL3VlanMet -------------------> F

    6) TestIngressSpan -----------------> .

    7) TestEgressSpan ------------------> .

    😎 TestAsicMemory ------------------> U

    9) TestEobcStressPing --------------> U

   10) TestFirmwareDiagStatus ----------> .

   11) TestAsicSync --------------------> .

   12) TestUnusedPortLoopback:

   Port  1  2  3  4  5  6  7  8  9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24

   ----------------------------------------------------------------------------

         U  U  U  U  U  U  U  U  U  U  U  .  U  U  U  U  U  U  U  U  U  U  U  .

   13) TestOBFL ------------------------> .

   14) TestErrorCounterMonitor ---------> .

   15) TestPortTxMonitoring:

   Port  1  2  3  4  5  6  7  8  9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24

   ----------------------------------------------------------------------------

         U  U  U  U  U  U  U  U  U  U  U  U  U  U  U  U  U  U  U  U  U  U  U  U

Module 2: CEF720 48 port 10/100/1000mb Ethernet  SerialNo : SAL1541SEA7

  Overall Diagnostic Result for Module 2 : MINOR ERROR

  Diagnostic level at card bootup: complete

  Test results: (. = Pass, F = Fail, U = Untested)

    1) TestLoopback:

   Port  1  2  3  4  5  6  7  8  9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24

   ----------------------------------------------------------------------------

         .  .  .  .  .  .  .  .  .  .  .  .  .  .  .  .  .  .  .  .  .  .  .  .

   Port 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48

   ----------------------------------------------------------------------------

         .  .  .  .  .  .  .  .  .  .  .  .  .  .  .  .  .  .  .  .  .  .  .  .

    2) TestTxPathMonitoring ------------> .

    3) TestSynchedFabChannel -----------> .

    4) TestL3VlanMet -------------------> F

    5) TestIngressSpan -----------------> .

    6) TestEgressSpan ------------------> .

    7) TestAsicMemory ------------------> U

    😎 TestEobcStressPing --------------> U

    9) TestFirmwareDiagStatus ----------> .

   10) TestAsicSync --------------------> .

   11) TestUnusedPortLoopback:

   Port  1  2  3  4  5  6  7  8  9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24

   ----------------------------------------------------------------------------

         U  U  U  U  U  U  U  U  U  U  U  U  U  U  U  U  U  U  U  U  U  U  U  U

   Port 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48

   ----------------------------------------------------------------------------

         U  U  U  U  U  U  U  U  U  U  U  U  U  U  U  U  U  U  U  U  U  U  U  U

   12) TestOBFL ------------------------> .

   13) TestErrorCounterMonitor ---------> .

   14) TestPortTxMonitoring:

   Port  1  2  3  4  5  6  7  8  9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24

   ----------------------------------------------------------------------------

         U  U  .  .  .  U  .  .  U  U  .  .  U  U  U  .  .  .  U  U  U  U  U  U

   Port 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48

   ----------------------------------------------------------------------------

         U  U  U  U  U  U  U  U  U  U  U  U  U  U  .  U  U  U  U  .  U  U  .  .

Module 3: CEF720 48 port 10/100/1000mb Ethernet  SerialNo : SAL1425KVFM

  Overall Diagnostic Result for Module 3 : MINOR ERROR

  Diagnostic level at card bootup: complete

  Test results: (. = Pass, F = Fail, U = Untested)

    1) TestLoopback:

   Port  1  2  3  4  5  6  7  8  9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24

   ----------------------------------------------------------------------------

         .  .  .  .  .  .  .  .  .  .  .  .  .  .  .  .  .  .  .  .  .  .  .  .

   Port 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48

   ----------------------------------------------------------------------------

         .  .  .  .  .  .  .  .  .  .  .  .  .  .  .  .  .  .  .  .  .  .  .  .

    2) TestTxPathMonitoring ------------> .

    3) TestSynchedFabChannel -----------> .

    4) TestL3VlanMet -------------------> F

    5) TestIngressSpan -----------------> .

    6) TestEgressSpan ------------------> .

    7) TestAsicMemory ------------------> U

    😎 TestEobcStressPing --------------> U

    9) TestFirmwareDiagStatus ----------> .

   10) TestAsicSync --------------------> .

   11) TestUnusedPortLoopback:

   Port  1  2  3  4  5  6  7  8  9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24

   ----------------------------------------------------------------------------

         U  U  U  U  U  U  U  U  U  U  U  U  U  U  U  U  U  U  U  U  U  U  U  U

   Port 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48

   ----------------------------------------------------------------------------

         U  U  U  U  U  U  U  U  U  U  U  U  U  U  U  U  U  U  U  U  U  U  U  U

   12) TestOBFL ------------------------> .

   13) TestErrorCounterMonitor ---------> .

   14) TestPortTxMonitoring:

   Port  1  2  3  4  5  6  7  8  9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24

   ----------------------------------------------------------------------------

         U  U  U  U  U  .  .  U  U  U  U  U  U  .  U  U  .  U  .  U  U  U  .  U

   Port 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48

   ----------------------------------------------------------------------------

         U  U  U  U  U  U  U  U  U  .  U  .  .  U  U  .  .  .  .  .  .  .  .  .

Module 6: Supervisor Engine 720 (Active)  SerialNo : SAL1424KDFS

  Overall Diagnostic Result for Module 6 : MINOR ERROR

  Diagnostic level at card bootup: complete

  Test results: (. = Pass, F = Fail, U = Untested)

    1) TestScratchRegister -------------> .

    2) TestSPRPInbandPing --------------> U

    3) TestTransceiverIntegrity:

      Port  1  2

      ----------

            .  .

    4) TestActiveToStandbyLoopback:

      Port  1  2

      ----------

            U  U

    5) TestLoopback:

      Port  1  2

      ----------

            .  .

    6) TestNewIndexLearn ---------------> .

    7) TestDontConditionalLearn --------> .

    😎 TestBadBpduTrap -----------------> .

    9) TestMatchCapture ----------------> .

   10) TestProtocolMatchChannel --------> .

   11) TestFibDevices ------------------> F

   12) TestIPv4FibShortcut -------------> F

   13) TestL3Capture2 ------------------> F

   14) TestIPv6FibShortcut -------------> F

   15) TestMPLSFibShortcut -------------> U

   16) TestNATFibShortcut --------------> U

   17) TestAclPermit -------------------> F

   18) TestAclDeny ---------------------> .

   19) TestQoSTcam ---------------------> F

   20) TestL3VlanMet -------------------> F

   21) TestIngressSpan -----------------> .

   22) TestEgressSpan ------------------> .

   23) TestNetflowInlineRewrite:

      Port  1  2

      ----------

            F  F

   24) TestFabricSnakeForward ----------> .

   25) TestFabricSnakeBackward ---------> .

   26) TestTrafficStress ---------------> U

   27) TestFibTcamSSRAM ----------------> U

   28) TestAsicMemory ------------------> U

   29) TestAclQosTcam ------------------> U

   30) TestNetflowTcam -----------------> U

   31) ScheduleSwitchover --------------> U

   32) TestFirmwareDiagStatus ----------> .

   33) TestAsicSync --------------------> .

   34) TestUnusedPortLoopback:

      Port  1  2

      ----------

            U  U

   35) TestErrorCounterMonitor ---------> .

   36) TestFabricFlowControlStatus -----> U

   37) TestPortTxMonitoring:

      Port  1  2

      ----------

            .  .

   38) TestSpuriousIsrDetection --------> U

   39) TestL3HealthMonitoring ----------> U

   40) TestOBFL ------------------------> .

   41) TestCFRW:

      Device  1  2

      ------------

              .  U

====================================================================

SWITCH 2

Switch-Sec#sh module all

Mod Ports Card Type                              Model              Serial No.

--- ----- -------------------------------------- ------------------ -----------

  1   24  CEF720 24 port 1000mb SFP              WS-X6724-SFP       SAL1425L2VE

  2   48  CEF720 48 port 10/100/1000mb Ethernet  WS-X6748-GE-TX     SAL1515BBK1

  3   48  CEF720 48 port 10/100/1000mb Ethernet  WS-X6748-GE-TX     SAL1126SMT9

  5    2  Supervisor Engine 720 (Active)         WS-SUP720-3B       SAL1420HTU6

Mod MAC addresses                       Hw    Fw           Sw           Status

--- ---------------------------------- ------ ------------ ------------ -------

  1  5475.d0cf.7de8 to 5475.d0cf.7dff   4.3   Unknown      Unknown      PwrDown

  2  6400.f1a4.e600 to 6400.f1a4.e62f   4.2   12.2(18r)S1  12.2(33)SXH8 Ok

  3  001b.d4cc.e550 to 001b.d4cc.e57f   2.6   12.2(14r)S5  12.2(33)SXH8 Ok

  5  0022.559e.20cc to 0022.559e.20cf   5.10  8.5(3)       12.2(33)SXH8 MajFail

Mod  Sub-Module                  Model              Serial       Hw     Status

---- --------------------------- ------------------ ----------- ------- -------

  1  Centralized Forwarding Card WS-F6700-CFC       SAL1425KQJN  4.1    PwrDown

  2  Centralized Forwarding Card WS-F6700-CFC       SAL1516BXSL  4.1    Ok

  3  Centralized Forwarding Card WS-F6700-CFC       SAL1308KB8B  4.1    Ok

  5  Policy Feature Card 3       WS-F6K-PFC3B       SAL1424KF2G  2.6    MajFail

  5  MSFC3 Daughterboard         WS-SUP720          SAL1424KCQ6  4.1    MajFail

Mod  Online Diag Status

---- -------------------

  1  Not Applicable

  2  Minor Error

  3  Minor Error

  5  Major Error

--------------------------------------------------------------------------------

Switch-Sec#sh ver

Cisco IOS Software, s72033_rp Software (s72033_rp-IPSERVICESK9_WAN-M), Version 12.2(33)SXH8b, RELEASE SOFTWARE (fc1)

Technical Support: http://www.cisco.com/techsupport

Copyright (c) 1986-2011 by Cisco Systems, Inc.

Compiled Wed 14-Sep-11 12:38 by prod_rel_team

ROM: System Bootstrap, Version 12.2(17r)SX6, RELEASE SOFTWARE (fc1)

Switch-Sec uptime is 3 weeks, 3 days, 21 hours, 58 minutes

Uptime for this control processor is 3 weeks, 3 days, 22 hours, 9 minutes

Time since Switch-Sec switched to active is 3 weeks, 3 days, 17 hours, 40 minutes

System returned to ROM by s/w reset at 21:05:45 IST Sat Dec 21 2013 (SP by bus error at PC 0x41193D10, address 0x0)

System restarted at 01:58:15 IST Sun Dec 22 2013

System image file is "disk1:s72033-ipservicesk9_wan-mz.122-33.SXH8b.bin"

This product contains cryptographic features and is subject to United

States and local country laws governing import, export, transfer and

use. Delivery of Cisco cryptographic products does not imply

third-party authority to import, export, distribute or use encryption.

Importers, exporters, distributors and users are responsible for

compliance with U.S. and local country laws. By using this product you

agree to comply with applicable laws and regulations. If you are unable

to comply with U.S. and local laws, return this product immediately.

A summary of U.S. laws governing Cisco cryptographic products may be found at:

http://www.cisco.com/wwl/export/crypto/tool/stqrg.html

If you require further assistance please contact us by sending email to

export@cisco.com.

cisco WS-C6506-E (R7000) processor (revision 1.2) with 458720K/65536K bytes of memory.

Processor board ID SAL1426LG8S

SR71000 CPU at 600Mhz, Implementation 0x504, Rev 1.2, 512KB L2 Cache

Last reset from s/w reset

3 Virtual Ethernet interfaces

170 Gigabit Ethernet interfaces

1917K bytes of non-volatile configuration memory.

8192K bytes of packet buffer memory.

65536K bytes of Flash internal SIMM (Sector size 512K).

Configuration register is 0x2102

-------------------------------------------------------------------------------------------

spidernet-GCC-sec-L3-C6506#sh diagnostic result module all

Current bootup diagnostic level: complete

Diagnostic[Module 1]: Diagnostic handle is not found for the card.

Module 2: CEF720 48 port 10/100/1000mb Ethernet  SerialNo : SAL1515BBK1

  Overall Diagnostic Result for Module 2 : MINOR ERROR

  Diagnostic level at card bootup: minimal

  Test results: (. = Pass, F = Fail, U = Untested)

    1) TestLoopback:

   Port  1  2  3  4  5  6  7  8  9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24

   ----------------------------------------------------------------------------

         .  .  .  .  .  .  .  .  .  .  .  .  .  .  .  .  .  .  .  .  .  .  .  .

   Port 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48

   ----------------------------------------------------------------------------

         .  .  .  .  .  .  .  .  .  .  .  .  .  .  .  .  .  .  .  .  .  .  .  .

    2) TestTxPathMonitoring ------------> .

    3) TestSynchedFabChannel -----------> .

    4) TestL3VlanMet -------------------> F

    5) TestIngressSpan -----------------> .

    6) TestEgressSpan ------------------> .

    7) TestAsicMemory ------------------> U

    😎 TestEobcStressPing --------------> U

    9) TestFirmwareDiagStatus ----------> .

   10) TestAsicSync --------------------> .

   11) TestUnusedPortLoopback:

   Port  1  2  3  4  5  6  7  8  9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24

   ----------------------------------------------------------------------------

         U  U  U  U  U  U  U  U  U  U  U  U  U  U  U  U  U  U  U  U  U  U  U  U

   Port 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48

   ----------------------------------------------------------------------------

         U  U  U  U  U  U  U  U  U  U  U  U  U  U  U  U  U  U  U  U  U  U  U  U

   12) TestOBFL ------------------------> .

   13) TestErrorCounterMonitor ---------> .

   14) TestPortTxMonitoring:

   Port  1  2  3  4  5  6  7  8  9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24

   ----------------------------------------------------------------------------

         .  .  .  U  .  .  .  U  U  .  .  .  U  .  .  U  .  .  .  U  U  U  U  U

   Port 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48

   ----------------------------------------------------------------------------

         U  U  .  U  U  U  .  U  U  U  U  U  U  U  U  U  U  U  .  .  .  U  .  U

Module 3: CEF720 48 port 10/100/1000mb Ethernet  SerialNo : SAL1126SMT9

  Overall Diagnostic Result for Module 3 : MINOR ERROR

  Diagnostic level at card bootup: minimal

  Test results: (. = Pass, F = Fail, U = Untested)

    1) TestLoopback:

   Port  1  2  3  4  5  6  7  8  9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24

   ----------------------------------------------------------------------------

         .  .  .  .  .  .  .  .  .  .  .  .  .  .  .  .  .  .  .  .  .  .  .  .

   Port 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48

   ----------------------------------------------------------------------------

         .  .  .  .  .  .  .  .  .  .  .  .  .  .  .  .  .  .  .  .  .  .  .  .

    2) TestTxPathMonitoring ------------> .

    3) TestSynchedFabChannel -----------> .

    4) TestL3VlanMet -------------------> F

    5) TestIngressSpan -----------------> .

    6) TestEgressSpan ------------------> .

    7) TestAsicMemory ------------------> U

    😎 TestEobcStressPing --------------> U

    9) TestFirmwareDiagStatus ----------> .

   10) TestAsicSync --------------------> .

   11) TestUnusedPortLoopback:

   Port  1  2  3  4  5  6  7  8  9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24

   ----------------------------------------------------------------------------

         U  U  U  U  U  U  U  U  U  U  U  U  U  U  U  U  U  U  U  U  U  U  U  U

   Port 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48

   ----------------------------------------------------------------------------

         U  U  U  U  U  U  U  U  U  U  U  U  U  U  U  U  U  U  U  U  U  U  U  U

   12) TestOBFL ------------------------> .

   13) TestErrorCounterMonitor ---------> .

   14) TestPortTxMonitoring:

   Port  1  2  3  4  5  6  7  8  9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24

   ----------------------------------------------------------------------------

         U  U  U  U  .  .  .  .  U  .  U  U  .  U  .  .  U  .  .  .  U  U  .  .

   Port 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48

   ----------------------------------------------------------------------------

         U  U  U  U  .  U  U  .  .  .  U  U  .  .  .  U  U  U  U  .  .  U  U  U

Module 5: Supervisor Engine 720 (Active)  SerialNo : SAL1420HTU6

  Overall Diagnostic Result for Module 5 : MAJOR ERROR

  Diagnostic level at card bootup: minimal

  Test results: (. = Pass, F = Fail, U = Untested)

    1) TestScratchRegister -------------> .

    2) TestSPRPInbandPing --------------> F

    3) TestTransceiverIntegrity:

      Port  1  2

      ----------

            .  .

    4) TestActiveToStandbyLoopback:

      Port  1  2

      ----------

            U  U

    5) TestLoopback:

      Port  1  2

      ----------

            .  .

    6) TestNewIndexLearn ---------------> .

    7) TestDontConditionalLearn --------> .

    😎 TestBadBpduTrap -----------------> .

    9) TestMatchCapture ----------------> .

   10) TestProtocolMatchChannel --------> .

   11) TestFibDevices ------------------> U

   12) TestIPv4FibShortcut -------------> U

   13) TestL3Capture2 ------------------> F

   14) TestIPv6FibShortcut -------------> F

   15) TestMPLSFibShortcut -------------> F

   16) TestNATFibShortcut --------------> F

   17) TestAclPermit -------------------> F

   18) TestAclDeny ---------------------> .

   19) TestQoSTcam ---------------------> F

   20) TestL3VlanMet -------------------> F

   21) TestIngressSpan -----------------> .

   22) TestEgressSpan ------------------> .

   23) TestNetflowInlineRewrite:

      Port  1  2

      ----------

            U  U

   24) TestFabricSnakeForward ----------> .

   25) TestFabricSnakeBackward ---------> .

   26) TestTrafficStress ---------------> U

   27) TestFibTcamSSRAM ----------------> U

   28) TestAsicMemory ------------------> U

   29) TestAclQosTcam ------------------> U

   30) TestNetflowTcam -----------------> U

   31) ScheduleSwitchover --------------> U

   32) TestFirmwareDiagStatus ----------> .

   33) TestAsicSync --------------------> .

   34) TestUnusedPortLoopback:

      Port  1  2

      ----------

            U  U

   35) TestErrorCounterMonitor ---------> .

   36) TestFabricFlowControlStatus -----> U

   37) TestPortTxMonitoring:

      Port  1  2

      ----------

            .  .

   38) TestSpuriousIsrDetection --------> U

   39) TestL3HealthMonitoring ----------> U

   40) TestOBFL ------------------------> .

   41) TestCFRW:

      Device  1  2

      ------------

              U  .

=========================================================================================

I have tried to to push the latest image and bootup.

But once i given reload command it was not even coming up with either old or new IOS. (means it is booting up and once i got the console again it falling back to ROMMODE and generates a crash dump also)

Then i changed the slot and tried. Happened the same for new IOS, but finally it booted up with old IOS itself.

Please find the attached crash-info file and help me to conclude weather it is a Sup engine hardware failure / Chassis hardware failure/ or the problem is only IOS.

Thanks in advance,

Anand

[Leo Laohoo]

I find it hard to believe that two of your Supervisor cards are showing signs of hardware failure.  Try this:

1.  Remove the rest of the line card from the chassis, if possible.

2.  Re-seat the supervisor card.  Watch the bootup sequence carefully.

3.  If problem persist, move the supervisor card to a different slot.  Again, watch the bootup sequence carefully for any signs of failure.

4.  If problem persist, RMA the supervisor card.

[AnandBalakrishnan]

Hi Leo,

Thanks for your suggestions.

I have already completed first 3 steps which you suggested (in Switch 1). But still problem was excisting. I raised RMA for one Sup engine. I will update the result post the replacement activity.

But still i am having doubt related to below points.

1. My both switchs are working fine for last 2 months, recently customer logged a complaint regarding this. But still the production is UP around 90% on both switch. All the LED in Sup module are showing green.

2. In each line card i can see diag error "TestL3VlanMet" apart from multiple error seeing in Sup engine.

3. In show version i can see "System returned to ROM by s/w reset at 17:18:42 IST Wed Jan 15 2014 (SP by bus error at PC 0x417CC1E8, address 0x0)"

4. After changing the Sup module from slot 5 to slot 6, the Major error changed to Minor error in show module output.

Is these all pointing towards hardware failure on Sup engine?

Thanks and Regards,

Anand

[Leo Laohoo]

If you move the supervisor card from one slot to another, this is to TEST that you've got a line card problem.  If you move the line card from one slot to another and the problem re-occurs, then it is confirmed to be a line card issue. 

If you move the line card from one slot to another and the problem goes away, the problem is more sinister:  The backplane of the chassis.

[AnandBalakrishnan]

Hi,

I have replaced the sup engine.. and all the errors vanished.. now i am happy..

Thanks all..

Regards,

Anand