6509E Fan tray 1 mode change failed

smith606306 · ‎03-06-2014

I have just installed a CEF720 8 port 10GE with DFC WS-X6708-10GE card in an existing 6509E Chassis and it shuts down due to inadequate cooling.

I have another 6509E Chassis with exactly the same setup and it works ok with that. Both are brand new.

The one transfers to High power mode and the other fails when trying to swap to restricted mode.

*Mar 5 13:55:38.527: %C6KENV-SP-4-FANHPMODE: Fan-tray 1 is operating in high power mode

*Mar 5 13:55:38.535: %C6KENV-SP-4-INSUFFCOOL: Module 9 cannot be adequately cooled

*Mar 5 13:56:38.787: %C6KENV-SP-3-FANMODEFAIL: Fan-tray 1 mode change failed

#sh env cooling

fan-tray 1:

fan-tray 1 type: WS-C6509-E-FAN

fan-tray 1 mode: Restricted-power

fan-tray 1 fan-fail: OK

chassis per slot cooling capacity: 70 cfm

ambient temperature: < 55C

module 1 cooling requirement: 30 cfm

module 2 cooling requirement: 30 cfm

module 3 cooling requirement: 30 cfm

module 5 cooling requirement: 35 cfm

module 8 cooling requirement: 70 cfm

module 9 cooling requirement: 84 cfm

environmental alarms:
no alarms

backplane:
operating clock count: 2
operating VTT count: 3
operating fan count: 1

fan-tray 1:
fan-tray 1 type: WS-C6509-E-FAN
fan-tray 1 mode: Restricted-power
fan-tray 1 fan-fail: OK
VTT 1:
VTT 1 OK: OK
VTT 1 outlet temperature: 0C
VTT 2:
VTT 2 OK: OK
VTT 2 outlet temperature: 0C
VTT 3:
VTT 3 OK: OK
VTT 3 outlet temperature: 0C
clock 1:
clock 1 OK: OK, clock 1 clock-inuse: in-use
clock 2:
clock 2 OK: OK, clock 2 clock-inuse: not-in-use
power-supply 1:
power-supply 1 fan-fail: OK
power-supply 1 power-input: AC high
power-supply 1 power-output-mode: high
power-supply 1 power-output-fail: OK
power-supply 2:
power-supply 2 fan-fail: OK
power-supply 2 power-input: AC high
power-supply 2 power-output-mode: high
power-supply 2 power-output-fail: OK
module 1:
module 1 power-output-fail: OK
module 1 outlet temperature: 27C
module 1 inlet temperature: 23C
module 1 device-1 temperature: 27C
module 1 device-2 temperature: 23C
module 1 VDB outlet temperature: 25C
module 1 VDB inlet temperature: 24C
module 1 VDB device-1 temperature: 25C
module 2:
module 2 power-output-fail: OK
module 2 outlet temperature: 26C
module 2 inlet temperature: 23C
module 2 device-1 temperature: 27C
module 2 device-2 temperature: 24C
module 2 VDB outlet temperature: 25C
module 2 VDB inlet temperature: 23C
module 2 VDB device-1 temperature: 25C
module 3:
module 3 power-output-fail: OK
module 3 outlet temperature: 27C
module 3 inlet temperature: 23C
module 3 device-1 temperature: 28C
module 3 device-2 temperature: 23C
module 3 VDB outlet temperature: 23C
module 3 VDB inlet temperature: 22C
module 3 VDB device-1 temperature: 25C
module 5:
module 5 power-output-fail: OK
module 5 outlet temperature: 32C
module 5 inlet temperature: 21C
module 5 device-1 temperature: 26C
module 5 device-2 temperature: 29C
module 5 asic-1 temperature: 51C
module 5 asic-3 temperature: 35C
module 5 asic-4 temperature: 46C
module 5 RP outlet temperature: 25C
module 5 RP inlet temperature: 26C
module 5 EARL outlet temperature: 28C
module 5 EARL inlet temperature: 23C
module 8:
module 8 power-output-fail: OK
module 8 outlet temperature: 23C
module 8 inlet temperature: 22C
module 8 device-1 temperature: 20C
module 8 device-2 temperature: 24C

chassis connector rating: 1302.00 Watts (31.00 Amps @ 42V)
chassis auxiliary connector rating: 2016.00 Watts (48.00 Amps @ 42V)
module 1
module 1 connector rating: 1260.00 Watts (30.00 Amps @ 42V)
module 1 vdb connector rating: 850.08 Watts (20.24 Amps @ 42V)
module 1 power consumption: 132.72 Watts ( 3.16 Amps @ 42V)
module 2
module 2 connector rating: 1260.00 Watts (30.00 Amps @ 42V)
module 2 vdb connector rating: 850.08 Watts (20.24 Amps @ 42V)
module 2 power consumption: 132.72 Watts ( 3.16 Amps @ 42V)
module 3
module 3 connector rating: 1260.00 Watts (30.00 Amps @ 42V)
module 3 vdb connector rating: 850.08 Watts (20.24 Amps @ 42V)
module 3 power consumption: 132.72 Watts ( 3.16 Amps @ 42V)
module 5
module 5 connector rating: 1260.00 Watts (30.00 Amps @ 42V)
module 5 power consumption: 338.10 Watts ( 8.05 Amps @ 42V)
module 8
module 8 connector rating: 2016.00 Watts (48.00 Amps @ 42V)
module 8 power consumption: 125.16 Watts ( 2.98 Amps @ 42V)
module 9
module 9 connector rating: 2016.00 Watts (48.00 Amps @ 42V)
module 9 power consumption: 444.36 Watts (10.58 Amps @ 42V)

chassis per slot cooling capacity: 70 cfm
ambient temperature: < 55C
module 1 cooling requirement: 30 cfm
module 2 cooling requirement: 30 cfm
module 3 cooling requirement: 30 cfm
module 5 cooling requirement: 35 cfm
module 8 cooling requirement: 70 cfm
module 9 cooling requirement: 84 cfm

*Mar 5 13:55:38.527: %C6KENV-SP-4-FANHPMODE: Fan-tray 1 is operating in high power mode
*Mar 5 13:55:38.535: %C6KENV-SP-4-INSUFFCOOL: Module 9 cannot be adequately cooled
*Mar 5 13:56:38.787: %C6KENV-SP-3-FANMODEFAIL: Fan-tray 1 mode change failed

#sh mod
Mod Ports Card Type                              Model              Serial No.
--- ----- -------------------------------------- ------------------ -----------
1   48 SFM-capable 48 port 10/100/1000mb RJ45 WS-X6548-GE-45AF   SAL1335XN4J
2   48 SFM-capable 48 port 10/100/1000mb RJ45 WS-X6548-GE-45AF   SAL1336Y10M
3   48 SFM-capable 48 port 10/100/1000mb RJ45 WS-X6548-GE-45AF   SAL1539QUPY
5    5 Supervisor Engine 720 10GE (Active)    VS-S720-10G        SAL1532LPPY
8   24 CEF720 24 port 1000mb SFP              WS-X6724-SFP       SAL1227WDGA
9    8 CEF720 8 port 10GE with DFC            WS-X6708-10GE      SAL1746G73Z

Mod MAC addresses                       Hw    Fw           Sw           Status
--- ---------------------------------- ------ ------------ ------------ -------
1 0027.0d00.2910 to 0027.0d00.293f 12.5   7.2(1)       12.2(33)SXJ Ok
2 0027.0d00.2880 to 0027.0d00.28af 12.5   7.2(1)       12.2(33)SXJ Ok
3 44d3.ca8b.bed0 to 44d3.ca8b.beff 12.7   7.2(1)       12.2(33)SXJ Ok
5 68ef.bd83.524c to 68ef.bd83.5253   4.0   8.5(4)       12.2(33)SXJ Ok
8 0021.d88c.1e10 to 0021.d88c.1e27   3.2   12.2(18r)S1 12.2(33)SXJ Ok
9 c067.afc4.09b8 to c067.afc4.09bf   3.6   Unknown      Unknown      PwrDown

Mod Sub-Module                  Model              Serial       Hw     Status
---- --------------------------- ------------------ ----------- ------- -------
1 IEEE Voice Daughter Card    WS-F6K-48-AF       SAL1336Y0SH 2.4    Ok
2 IEEE Voice Daughter Card    WS-F6K-48-AF       SAL1336Y01P 2.4    Ok
3 IEEE Voice Daughter Card    WS-F6K-48-AF       SAL1538QHUV 2.5    Ok
5 Policy Feature Card 3       VS-F6K-PFC3C       SAL1531L4CR 1.1    Ok
5 MSFC3 Daughterboard         VS-F6K-MSFC3       SAL1531LH76 5.1    Ok
8 Centralized Forwarding Card WS-F6700-CFC       SAL1227W50M 4.1    Ok
9 Distributed Forwarding Card WS-F6700-DFC3C     SAL1744FFSW 1.6    PwrDown

Mod Online Diag Status
---- -------------------
1 Pass
2 Pass
3 Pass
5 Pass
8 Pass

Jose Solano · ‎03-06-2014

Hi see the below information:

About the Fan Tray Mode: This high power mode may change to restricted power mode depending upon the environment temperature, the fan may be working at High or Restricted mode depending on the cooling needed at the moment.

Restricted Power mode: This is a default mode for a fan-tray operation. This mode ensures that there is an adequate amount of cooling provided to the chassis to keep its temperature in control.

High Power mode: This is another mode which is used for future line cards if there an increase in temperature due to addition of modules. In our case, this mode only gets in use when there is WS-X6708-10GE module inserted. Logs are getting displayed as new module is causing rise in chassis temperature and fan-tray has to accommodate more cooling thereby changing its operating mode.

The WS-X6708-10GE requires 84 cfm. Installing a second fan module in the chassis should bring the chassis cooling capacity up to 84 cfm per line card. WS-X6708-10GE is supported in non-E Series chassis at operating temperature up to 40 C only.

Hope this helps.

smith606306 · ‎03-07-2014

Thanks Jose.

The 6509E chassis doesnt support a 2nd fan tray and there is no upgrade.

Is there a way of forcing the fan into high power mode?

Mar 5 13:55:38.527: %C6KENV-SP-4-FANHPMODE: Fan-tray 1 is operating in high power mode

*Mar 5 13:55:38.535: %C6KENV-SP-4-INSUFFCOOL: Module 9 cannot be adequately cooled

*Mar 5 13:56:38.787: %C6KENV-SP-3-FANMODEFAIL: Fan-tray 1 mode change failed

This all works ok on our other 6500s

Jose Solano · ‎03-10-2014

Hi,

As far as I know there is no way to force the the fan to work at the high power mode, now can you may be try to insert the module on another slot in the chassis and see if you get the same issue?

smith606306 · ‎03-11-2014

I have tried it in every slot and it is still the same

Rafatur Rahman · ‎05-24-2015

Hello Smith,

Greetings! As per the logs, the available cooling capacity per slot is less then the required by module in slot 9. I would recommend following steps.

-- The insertion of the new module should trigger high power mode, however, it seems to be stuck in restricted mode.

Recommendations:

-- Reset Fan-tray if not tried already.

-- Or reload the switch.

.

I hope this helps.

Regards,