Nexus 7010 Slot 3 problem

[josephsmar1]

Hello everyone,

 Here is my issue. We have a Nexus 7010 and for the third time now slot 3 will not allow a module to run in it. This is what we have done so far. RMA'd the module and the chassis. When the module is put into another slot it comes up without any issue. The only difference between this chassis and the other one we have is that we only have 2 of the 3 power supplies in it. The chassis itself has 2 sup mods and 3 48 port 10 gig mods. On a side note after we RMA'd the chassis slot 3 worked fine until we configured the new VDC and allocated the card to it. Any help will be appreciated.

[InayathUlla Sharieff]

Do you still have the issue with the slot3?
By reading the query after the replacment of the chasis the module 3 is getting detected.

so let us know if you still having th eissue if yes could you please capture the output of :
show module
show diagnostic result module all

[josephsmar1]

Yes I am still have this issue. Here is the output you asked for.

show diagnostic result module all

Current bootup diagnostic level: complete
Module 1: 1/10 Gbps Ethernet Module  

        Test results: (. = Pass, F = Fail, I = Incomplete,
        U = Untested, A = Abort, E = Error disabled)

         1) ASICRegisterCheck-------------> .
         2) PrimaryBootROM----------------> .
         3) SecondaryBootROM--------------> .
         4) EOBCPortLoopback--------------> .
         5) OBFL--------------------------> .
         6) PortLoopback:

          Port   1  2  3  4  5  6  7  8  9 10 11 12 13 14 15 16
          -----------------------------------------------------
                 U  U  U  U  U  U  U  U  U  U  U  U  U  U  U  U  

          Port  17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32
          -----------------------------------------------------
                 U  U  U  U  U  U  U  U  U  U  U  U  U  U  .  .  

          Port  33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48
          -----------------------------------------------------
                 .  .  .  .  .  .  .  .  .  .  .  U  U  U  .  U  

         7) RewriteEngineLoopback:

          Port   1  2  3  4  5  6  7  8  9 10 11 12 13 14 15 16
          -----------------------------------------------------
                 .  U  U  U  U  U  U  U  U  U  U  U  U  U  U  U  

          Port  17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32
          -----------------------------------------------------
                 U  U  U  U  U  U  U  U  U  U  U  U  U  U  U  U  

          Port  33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48
          -----------------------------------------------------
                 U  U  U  U  U  U  U  U  U  U  U  U  U  U  U  U  

         8) SnakeLoopback:

          Port   1  2  3  4  5  6  7  8  9 10 11 12 13 14 15 16
          -----------------------------------------------------
                 .  .  .  .  .  .  .  .  .  .  .  .  .  .  .  .  

          Port  17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32
          -----------------------------------------------------
                 .  .  .  .  .  .  .  .  .  .  .  .  .  .  .  .  

          Port  33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48
          -----------------------------------------------------
                 .  .  .  .  .  .  .  .  .  .  .  .  .  .  .  .  

         9) FIPS:

          Port   1  2  3  4  5  6  7  8  9 10 11 12 13 14 15 16
          -----------------------------------------------------
                 U  U  U  U  U  U  U  U  U  U  U  U  U  U  U  U  

          Port  17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32
          -----------------------------------------------------
                 U  U  U  U  U  U  U  U  U  U  U  U  U  U  U  U  

          Port  33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48
          -----------------------------------------------------
                 U  U  U  U  U  U  U  U  U  U  U  U  U  U  U  U  

        10) BootupPortLoopback:

          Port   1  2  3  4  5  6  7  8  9 10 11 12 13 14 15 16
          -----------------------------------------------------
                 .  .  .  .  .  .  .  .  .  .  .  .  .  .  .  .  

          Port  17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32
          -----------------------------------------------------
                 .  .  .  .  .  .  .  .  .  .  .  .  .  .  .  .  

          Port  33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48
          -----------------------------------------------------
                 .  .  .  .  .  .  .  .  .  .  .  .  .  .  .  .  

Current bootup diagnostic level: complete
Module 5: Supervisor Module-2  (Active)

        Test results: (. = Pass, F = Fail, I = Incomplete,
        U = Untested, A = Abort, E = Error disabled)

         1) ASICRegisterCheck-------------> U
         2) USB---------------------------> .
         3) NVRAM-------------------------> .
         4) RealTimeClock-----------------> .
         5) PrimaryBootROM----------------> .
         6) SecondaryBootROM--------------> .
         7) CompactFlash------------------> .
         8) ExternalCompactFlash----------> U
         9) PwrMgmtBus--------------------> .
        10) SpineControlBus---------------> .
        11) SystemMgmtBus-----------------> .
        12) StatusBus---------------------> .
        13) StandbyFabricLoopback---------> U
        14) ManagementPortLoopback--------> .
        15) EOBCPortLoopback--------------> .
        16) OBFL--------------------------> .

Current bootup diagnostic level: complete
Module 6: Supervisor Module-2  (Standby)

        Test results: (. = Pass, F = Fail, I = Incomplete,
        U = Untested, A = Abort, E = Error disabled)

         1) ASICRegisterCheck-------------> .
         2) USB---------------------------> .
         3) NVRAM-------------------------> .
         4) RealTimeClock-----------------> .
         5) PrimaryBootROM----------------> .
         6) SecondaryBootROM--------------> .
         7) CompactFlash------------------> .
         8) ExternalCompactFlash----------> U
         9) PwrMgmtBus--------------------> U
        10) SpineControlBus---------------> .
        11) SystemMgmtBus-----------------> U
        12) StatusBus---------------------> U
        13) StandbyFabricLoopback---------> .
        14) ManagementPortLoopback--------> .
        15) EOBCPortLoopback--------------> .
        16) OBFL--------------------------> .

Current bootup diagnostic level: complete
Module 8: 1/10 Gbps Ethernet Module  

        Test results: (. = Pass, F = Fail, I = Incomplete,
        U = Untested, A = Abort, E = Error disabled)

         1) ASICRegisterCheck-------------> U
         2) PrimaryBootROM----------------> .
         3) SecondaryBootROM--------------> .
         4) EOBCPortLoopback--------------> .
         5) OBFL--------------------------> .
         6) PortLoopback:

          Port   1  2  3  4  5  6  7  8  9 10 11 12 13 14 15 16
          -----------------------------------------------------
                 U  U  U  U  .  .  U  U  U  U  U  U  .  .  U  .  

          Port  17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32
          -----------------------------------------------------
                 U  U  .  .  .  .  .  .  .  .  .  .  .  .  .  .  

          Port  33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48
          -----------------------------------------------------
                 .  .  .  .  .  .  .  .  U  U  U  U  U  U  U  U  

         7) RewriteEngineLoopback:

          Port   1  2  3  4  5  6  7  8  9 10 11 12 13 14 15 16
          -----------------------------------------------------
                 .  U  U  U  U  U  U  U  U  U  U  U  U  U  U  U  

          Port  17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32
          -----------------------------------------------------
                 U  U  U  U  U  U  U  U  U  U  U  U  U  U  U  U  

          Port  33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48
          -----------------------------------------------------
                 U  U  U  U  U  U  U  U  U  U  U  U  U  U  U  U  

         8) SnakeLoopback:

          Port   1  2  3  4  5  6  7  8  9 10 11 12 13 14 15 16
          -----------------------------------------------------
                 .  .  .  .  .  .  .  .  .  .  .  .  .  .  .  .  

          Port  17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32
          -----------------------------------------------------
                 .  .  .  .  .  .  .  .  .  .  .  .  .  .  .  .  

          Port  33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48
          -----------------------------------------------------
                 .  .  .  .  .  .  .  .  U  U  U  U  U  U  U  U  

         9) FIPS:

          Port   1  2  3  4  5  6  7  8  9 10 11 12 13 14 15 16
          -----------------------------------------------------
                 U  U  U  U  U  U  U  U  U  U  U  U  U  U  U  U  

          Port  17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32
          -----------------------------------------------------
                 U  U  U  U  U  U  U  U  U  U  U  U  U  U  U  U  

          Port  33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48
          -----------------------------------------------------
                 U  U  U  U  U  U  U  U  U  U  U  U  U  U  U  U  

        10) BootupPortLoopback:

          Port   1  2  3  4  5  6  7  8  9 10 11 12 13 14 15 16
          -----------------------------------------------------
                 .  .  .  .  .  .  .  .  .  .  .  .  .  .  .  .  

          Port  17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32
          -----------------------------------------------------
                 .  .  .  .  .  .  .  .  .  .  .  .  .  .  .  .  

          Port  33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48
          -----------------------------------------------------
                 .  .  .  .  .  .  .  .  U  U  U  U  U  U  U  U 

show module
Mod  Ports  Module-Type                         Model              Status
---  -----  ----------------------------------- ------------------ ----------
1    48     1/10 Gbps Ethernet Module           N7K-F248XP-25E     ok
3    48     1/10 Gbps Ethernet Module           N7K-F248XP-25E     testing
5    0      Supervisor Module-2                 N7K-SUP2           active *
6    0      Supervisor Module-2                 N7K-SUP2           ha-standby
8    48     1/10 Gbps Ethernet Module           N7K-F248XP-25E     ok

Mod  Sw              Hw
---  --------------  ------
1    6.2(2a)         1.0     
3    6.2(2a)         1.0     
5    6.2(2a)         1.1     
6    6.2(2a)         1.0     
8    6.2(2a)         1.0     

ISSCSW7010RE-4# show module
Mod  Ports  Module-Type                         Model              Status
---  -----  ----------------------------------- ------------------ ----------
1    48     1/10 Gbps Ethernet Module           N7K-F248XP-25E     ok
3    48     1/10 Gbps Ethernet Module           N7K-F248XP-25E     testing
5    0      Supervisor Module-2                 N7K-SUP2           active *
6    0      Supervisor Module-2                 N7K-SUP2           ha-standby
8    48     1/10 Gbps Ethernet Module           N7K-F248XP-25E     ok

Mod  Sw              Hw
---  --------------  ------
1    6.2(2a)         1.0     
3    6.2(2a)         1.0     
5    6.2(2a)         1.1     
6    6.2(2a)         1.0     
8    6.2(2a)         1.0     

Mod  Online Diag Status
---  ------------------
1    Pass
3    Untested
5    Pass
6    Pass
8    Pass

Xbar Ports  Module-Type                         Model              Status
---  -----  ----------------------------------- ------------------ ----------
1    0      Fabric Module 2                     N7K-C7010-FAB-2    ok
2    0      Fabric Module 2                     N7K-C7010-FAB-2    ok
3    0      Fabric Module 2                     N7K-C7010-FAB-2    ok
4    0      Fabric Module 2                     N7K-C7010-FAB-2    ok
5    0      Fabric Module 2                     N7K-C7010-FAB-2    ok

Xbar Sw              Hw
---  --------------  ------
1    NA              1.5     
2    NA              1.5     
3    NA              1.5     
4    NA              1.5     
5    NA              1.5     

3    48     1/10 Gbps Ethernet Module           N7K-F248XP-25E     testing

3    48     1/10 Gbps Ethernet Module           N7K-F248XP-25E     initializing

Mod  Ports  Module-Type                         Model                                Status
---  -----  ----------------------------------- ------------------                       ----------
1    48     1/10 Gbps Ethernet Module    N7K-F248XP-25E          ok
3    48     1/10 Gbps Ethernet Module                                              pwr-cycld
5    0      Supervisor Module-2                 N7K-SUP2                      active *
6    0      Supervisor Module-2                 N7K-SUP2                      ha-standby
8    48     1/10 Gbps Ethernet Module    N7K-F248XP-25E         ok

Mod  Power-Status  Reason
---  ------------  ---------------------------
3    pwr-cycld      Unknown. Issue show system reset mod ...

Mod  Sw              Hw
---  --------------  ------
1    6.2(2a)         1.0     
5    6.2(2a)         1.1     
6    6.2(2a)         1.0     
8    6.2(2a)         1.0     

Mod  MAC-Address(es)                         Serial-Num
---  --------------------------------------  ----------
1    e4-c7-22-17-c0-8c to e4-c7-22-17-c0-bf  JAF1802ANEA
5    84-78-ac-1c-fa-0f to 84-78-ac-1c-fa-21  JAF1721AQPC
6    84-78-ac-14-cb-16 to 84-78-ac-14-cb-28  JAF1713BAKS
8    e8-ed-f3-38-7b-08 to e8-ed-f3-38-7b-3b  JAF1733ARPG

Mod  Online Diag Status
---  ------------------
1    Pass
5    Pass
6    Pass
8    Pass

Xbar Ports  Module-Type                         Model              Status
---  -----  ----------------------------------- ------------------ ----------
1    0      Fabric Module 2                     N7K-C7010-FAB-2    ok
2    0      Fabric Module 2                     N7K-C7010-FAB-2    ok
3    0      Fabric Module 2                     N7K-C7010-FAB-2    ok
4    0      Fabric Module 2                     N7K-C7010-FAB-2    ok
5    0      Fabric Module 2                     N7K-C7010-FAB-2    ok

Xbar Sw              Hw
---  --------------  ------
1    NA              1.5     
2    NA              1.5     
3    NA              1.5     
4    NA              1.5     
5    NA              1.5     

2014 Aug  5 11:49:19 -MAF %$ VDC-2 %$ %PLATFORM-2-MOD_DETECT: Module 3 detected () Module-Type 1/10 Gbps Ethernet Module Model N7K-F248XP-25E
2014 Aug  5 11:49:19 MAF %$ VDC-2 %$ %PLATFORM-2-MOD_PWRUP: Module 3 powered up (
2014 Aug  5 11:49:19 %$ VDC-1 %$ %PLATFORM-2-MOD_DETECT: Module 3 detected ) Module-Type 1/10 Gbps Ethernet Module Model N7K-F248XP-25E
2014 Aug  5 11:49:19 %$ VDC-1 %$ %PLATFORM-2-MOD_PWRUP: Module 3 powered up ()

[josephsmar1]

Also just grabbed this output

Show system reset-reason module 3

Time stamp   : At 748306 usecs after Mon Aug  4 07:32:16 2014

Service name : Xbar manager
Reset reason : LC insertion sequence failure => [Failures < MAX] : powercycle
Serial number: JAF1733BDQJ
Error code   : NA

Time stamp   : At 842155 usecs after Mon Aug  4 07:10:56 2014

Service name : Xbar manager
Reset reason : LC insertion sequence failure => [Failures > MAX] : powerdown
Serial number: JAF1733BDQJ
Error code   : NA

Time stamp   : At 29849 usecs after Mon Aug  4 07:08:45 2014

Service name : Xbar manager
Reset reason : LC insertion sequence failure => [Failures < MAX] : powercycle
Serial number: JAF1733BDQJ
Error code   : NA

2) At 233864 usecs after Tue Aug  5 11:49:18 2014
    Reason: Reset Requested due to Fatal Module Error
    Service: Xbar manager
    Version:
3) At 988041 usecs after Tue Aug  5 11:49:16 2014
    Reason: Unknown
    Service:
    Version:
4) At 145113 usecs after Tue Aug  5 11:47:05 2014
    Reason: Reset Requested due to Fatal Module Error
    Service: Xbar manager
    Version:

[Bashar Alali]

Hello,

I just wanted to check if there's a solution for this issue while avoiding an RMA. We have a F3 card), and it's exhibiting the same problem described above. The card works perfectly in slot 4.

The problem is, we have the small chassis (7004) and the Fabric modules are fixed with the chassis. As per Cisco TAC, the chassis has to be replaced, but I was hoping we can avoid a replacement.

Thanks.

show system reset-reason module 3
*************** module reset reason (3) *************
Time stamp   : At 214465 usecs after Thu May 21 00:22:18 2015

Service name : Xbar manager
Reset reason : LC insertion sequence failure => [Failures > MAX] : powerdown
Serial number: JAE1851025A
Error code   : NA

Time stamp   : At 29912 usecs after Thu May 21 00:19:35 2015

Service name : Xbar manager
Reset reason : LC insertion sequence failure => [Failures < MAX] : powercycle
Serial number: JAE1851025A
Error code   : NA

Time stamp   : At 210855 usecs after Thu May 21 00:16:53 2015

Service name : Xbar manager
Reset reason : LC insertion sequence failure => [Failures < MAX] : powercycle
Serial number: JAE1851025A
Error code   : NA

Time stamp   : At 344869 usecs after Wed May 20 22:06:28 2015

Service name : Xbar manager
Reset reason : LC insertion sequence failure => [Failures > MAX] : powerdown
Serial number: JAE191408TD
Error code   : NA

Time stamp   : At 976416 usecs after Wed May 20 22:03:40 2015

Service name : Xbar manager
Reset reason : LC insertion sequence failure => [Failures < MAX] : powercycle
Serial number: JAE191408TD
Error code   : NA

Time stamp   : At 240249 usecs after Wed May 20 22:00:53 2015

Service name : Xbar manager
Reset reason : LC insertion sequence failure => [Failures < MAX] : powercycle
Serial number: JAE191408TD
Error code   : NA

Time stamp   : At 354688 usecs after Wed May 20 21:03:44 2015

Service name : Xbar manager
Reset reason : LC insertion sequence failure => [Failures > MAX] : powerdown
Serial number: JAE191408TD
Error code   : NA

Time stamp   : At 845139 usecs after Wed May 20 21:00:55 2015

Service name : Xbar manager
Reset reason : LC insertion sequence failure => [Failures < MAX] : powercycle
Serial number: JAE191408TD
Error code   : NA

Time stamp   : At 54826 usecs after Wed May 20 20:58:08 2015

Service name : Xbar manager

[josephsmar]

To fix it I would go with TAC is telling you, unfortunately. If you are looking to mitigate the issue you may be able to power off that fabric module until the replacement can be put in. Sorry for the bad news.

[josephsmar1]

So in case anyone is interested we found out that a xbar module is causing the problem.