cisco 6500 power to module in slot 1 set off (FRU-power failed)

Cisco_CoE . · ‎07-19-2016

Hi guys,

Can u please help on these.

00:01:32: %C6KPWR-SP-4-PSREDUNDANTBOTHSUPPLY: in power-redundancy mode, system is operating on both power supplies.
00:01:35: %OIR-SP-6-PWRFAILURE: Module 1 is being disabled due to power convertor failure. PMB registers are dumped as follows, R0= 0x2, R1= 0x2, R2= 0x0, R3= 0x0, R4= 0x0, R5= 0x0, R6= 0x0, R7= 0x0, R8= 0x0, R9= 0x0, R10= 0x0, R11= 0x0, R12= 0x0, R13= 0x0, R14= 0x0, R15= 0x0
00:01:35: %C6KPWR-SP-4-DISABLED: power to module in slot 1 set off (FRU-power failed)

Router#sh mod
Mod Ports Card Type Model Serial No.
--- ----- -------------------------------------- ------------------ -----------
1 8 Intrusion Detection System WS-SVC-IDSM-2 SAD140203BL
3 8 Intrusion Detection System WS-SVC-IDSM-2 SAD140203AT
5 2 Supervisor Engine 720 (Active) WS-SUP720-3B SAL10382055

Mod MAC addresses Hw Fw Sw Status
--- ---------------------------------- ------ ------------ ------------ -------
1 0023.ebe6.c788 to 0023.ebe6.c78f 6.5 7.2(1) 7.0(6)E4 Ok
3 0023.ebe6.c7d8 to 0023.ebe6.c7df 6.5 7.2(1) 20.1(1)SXI8 Other
5 0016.9df6.a148 to 0016.9df6.a14b 5.3 8.5(4) 12.2(33)SXI8 Ok

Mod Sub-Module Model Serial Hw Status
---- --------------------------- ------------------ ----------- ------- -------
1 IDS 2 accelerator board WS-SVC-IDSUPG 71093240028 2.5 Ok
5 Policy Feature Card 3 WS-F6K-PFC3B SAL10360MVD 2.5 Ok
5 MSFC3 Daughterboard WS-SUP720 SAD082504EP 2.11 Ok

Mod Online Diag Status
---- -------------------
1 Pass
3 Unknown
5 Pass

Leo Laohoo · ‎07-19-2016

00:01:35: %OIR-SP-6-PWRFAILURE: Module 1 is being disabled due to power convertor failure. PMB registers are dumped as follows, R0= 0x2, R1= 0x2, R2= 0x0, R3= 0x0, R4= 0x0, R5= 0x0, R6= 0x0, R7= 0x0, R8= 0x0, R9= 0x0, R10= 0x0, R11= 0x0, R12= 0x0, R13= 0x0, R14= 0x0, R15= 0x0
00:01:35: %C6KPWR-SP-4-DISABLED: power to module in slot 1 set off (FRU-power failed)

Doesn't make sense. The log above tells me the line card in slot 1 is dead but the output in "sh module" shows an IDSM in slot 1 to be working, HOWEVER, the IDSM in slot 3 is either booting up or not powering up properly.

Cisco_CoE . · ‎07-19-2016

Hi Leo,

Problem is IDS is getting rebooting continuously. Observed same when inserted to other slots too. Same type card is working in other slot.

Here is more details.

Router#[Ash mod
Mod Ports Card Type Model Serial No.
--- ----- -------------------------------------- ------------------ -----------
1 8 Intrusion Detection System WS-SVC-IDSM-2 SAD140203BL
2 8 Intrusion Detection System WS-SVC-IDSM-2 SAD140203AT
3 4 CEF720 4 port 10-Gigabit Ethernet WS-X6704-10GE SAL1537PPNL
5 2 Supervisor Engine 720 (Active) WS-SUP720-3B SAL10382055

Mod MAC addresses Hw Fw Sw Status
--- ---------------------------------- ------ ------------ ------------ -------
1 0023.ebe6.c788 to 0023.ebe6.c78f 6.5 7.2(1) 7.0(6)E4 Ok
2 0023.ebe6.c7d8 to 0023.ebe6.c7df 6.5 7.2(1) 20.1(1)SXI8 Other
3 7081.058e.d7ec to 7081.058e.d7ef 3.2 12.2(14r)S5 12.2(33)SXI8 Ok
5 0016.9df6.a148 to 0016.9df6.a14b 5.3 8.5(4) 12.2(33)SXI8 Ok

00:16:34: %OIR-SP-6-PWRFAILURE: Module 2 is being disabled due to power convertor failure. PMB registers are dumped as follows, R0= 0x2, R1= 0x2, R2= 0x0, R3= 0x0, R4= 0x0, R5= 0x0, R6= 0x0, R7= 0x0, R8= 0x0, R9= 0x0, R10= 0x0, R11= 0x0, R12= 0x0, R13= 0x0, R14= 0x0, R15= 0x0
00:16:34: %C6KPWR-SP-4-DISABLED: power to module in slot 2 set off (FRU-power failed) 1 Pass

00:17:02: %OIR-SP-6-PWRFAILURE: Module 2 is being disabled due to power convertor failure. PMB registers are dumped as follows, R0= 0x2, R1= 0x2, R2= 0x0, R3= 0x0, R4= 0x0, R5= 0x0, R6= 0x0, R7= 0x0, R8= 0x0, R9= 0x0, R10= 0x0, R11= 0x0, R12= 0x0, R13= 0x0, R14= 0x0, R15= 0x0
00:17:02: %C6KPWR-SP-4-DISABLED: power to module in slot 2 set off (FRU-power failed)

00:17:30: %OIR-SP-6-PWRFAILURE: Module 2 is being disabled due to power convertor failure. PMB registers are dumped as follows, R0= 0x2, R1= 0x2, R2= 0x0, R3= 0x0, R4= 0x0, R5= 0x0, R6= 0x0, R7= 0x0, R8= 0x0, R9= 0x0, R10= 0x0, R11= 0x0, R12= 0x0, R13= 0x0, R14= 0x0, R15= 0x0
00:17:30: %C6KPWR-SP-4-DISABLED: power to module in slot 2 set off (FRU-power failed)sh mod

Router#sh mod
Mod Ports Card Type Model Serial No.
--- ----- -------------------------------------- ------------------ -----------
1 8 Intrusion Detection System WS-SVC-IDSM-2 SAD140203BL
2 8 Intrusion Detection System WS-SVC-IDSM-2 SAD140203AT
3 4 CEF720 4 port 10-Gigabit Ethernet WS-X6704-10GE SAL1537PPNL
5 2 Supervisor Engine 720 (Active) WS-SUP720-3B SAL10382055

Mod MAC addresses Hw Fw Sw Status
--- ---------------------------------- ------ ------------ ------------ -------
1 0023.ebe6.c788 to 0023.ebe6.c78f 6.5 7.2(1) 7.0(6)E4 Ok
2 0023.ebe6.c7d8 to 0023.ebe6.c7df 6.5 7.2(1) 20.1(1)SXI8 PwrDown
3 7081.058e.d7ec to 7081.058e.d7ef 3.2 12.2(14r)S5 12.2(33)SXI8 Ok
5 0016.9df6.a148 to 0016.9df6.a14b 5.3 8.5(4) 12.2(33)SXI8 Ok

Mod Online Diag Status
---- -------------------
1 Pass
2 Unknown
5 Pass

find the log attached

Leo Laohoo · ‎07-19-2016

Ok, the alarm logs are starting to make sense (to me) and ... you're not going to like what I'm about to say.

Sadly, if there's a problem with one slot, then I'd blame the line card. However, I'm now seeing a problem with TWO (2) slots. This is unheard of if not very rare. I believe you'll need to contact TAC to get the chassis replaced.

Cisco_CoE . · ‎07-19-2016

Hi Leo,

Thanks for the quick response.

Kindly confirm two things

1. Have you checked the log i have attached ?

2.Is there is any clear evidence we can point out before going out for a chasis replacement.

Leo Laohoo · ‎07-19-2016

Module 2 is being disabled due to power convertor failure.
Module 3 is being disabled due to power convertor failure.

Slot 2 is an IDSM. Slot 3 is a 6704-10GE line card. No way those two cards crashed by itself. It's a failure with the backplane card.

I'm fairly certain it's an RMA of the chassis.

However, there is one way of testing this. MOVE the IDSM & 6704 into different slots and see if the problem follows them to their new location or not.

Cisco_CoE . · ‎07-19-2016

Hi Leo,

Thanks for the update.

Please note that module 3 got disabled only after i insert the IDS .

00:17:48: %OIR-SP-6-PWRFAILURE: Module 2 is being disabled due to power convertor failure. PMB registers are dumped as follows, R0= 0x2, R1= 0x2, R2= 0x0, R3= 0x0, R4= 0x0, R5= 0x1, R6= 0x0, R7= 0x0, R8= 0x0, R9= 0x0, R10= 0x0, R11= 0x0, R12= 0x0, R13= 0x0, R14= 0x0, R15= 0x0
00:17:48: %C6KPWR-SP-4-DISABLED: power to module in slot 2 set off (FRU-power failed)

00:18:08: %OIR-SP-6-REMCARD: Card removed from slot 3, interfaces disabled

00:18:35: %OIR-SP-6-REMCARD: Card removed from slot 2, interfaces disabled

00:19:07: %OIR-SP-6-PWRFAILURE: Module 3 is being disabled due to power convertor failure. PMB registers are dumped as follows, R0= 0x2, R1= 0x2, R2= 0x0, R3= 0x0, R4= 0x0, R5= 0x0, R6= 0x0, R7= 0x0, R8= 0x0, R9= 0x0, R10= 0x0, R11= 0x0, R12= 0x0, R13= 0x0, R14= 0x0, R15= 0x0
00:19:07: %C6KPWR-SP-4-DISABLED: power to module in slot 3 set off (FRU-power failed)

Router#sh mod
Mod Ports Card Type Model Serial No.
--- ----- -------------------------------------- ------------------ -----------
1 8 Intrusion Detection System WS-SVC-IDSM-2 SAD140203BL
3 8 Intrusion Detection System WS-SVC-IDSM-2 SAD140203AT
5 2 Supervisor Engine 720 (Active) WS-SUP720-3B SAL10382055

Mod MAC addresses Hw Fw Sw Status
--- ---------------------------------- ------ ------------ ------------ -------
1 0023.ebe6.c788 to 0023.ebe6.c78f 6.5 7.2(1) 7.0(6)E4 Ok
3 0023.ebe6.c7d8 to 0023.ebe6.c7df 6.5 7.2(1) 20.1(1)SXI8 Other
5 0016.9df6.a148 to 0016.9df6.a14b 5.3 8.5(4) 12.2(33)SXI8 O

Leo Laohoo · ‎07-20-2016

What I meant was move the IDSM & the 6704 away from slots 2 & 3. Use other FREE slots.