ASR 9k power outage behavior

[Eduard Gheorghiu]

Hello folks,

Not a clearly documented behavior - what happens with an ASR 9k chassis that runs in non protected power mode when one of the power supply fails? What linecard is powered down if such event would happen?

Does disabling the power budget enforcement (via power budget enforcement disable) influence the behavior when losing some of the power?

Thanks,

Eduard

[xthuijs]

the budget enforcement command merely verifies, at boot time, whether the LC's rated wattage can be accommodated against the available nominal power provided to the system.

Say you have an LC configuration that on paper draws say 5kW and you have 2 AC modules (so providing 6kW max). The actual draw for that may be way less then 5kW (check admin show env power) and it may only pull say 2.8kW.

Then when one module dies the following scenario can exist:

1) actual draw is less then what one module can provide -> system continues to operate as normal

2) actual draw is more then what one module can provide ->

    a) breaker trips provided to the module because of overdraw and system shuts off

    b) breaker does not trip and module browns out or shuts down itself

In scenario 1 and 2b when the chassis reloads, the power monitoring detects that the nomimal power required is 5kW and only has 3kW available and boots RSP and FAN's first. then starting from LC0 upward it will start to boot the linecards until it reaches the nominal max of 3kW

and leaves the remaining LC's in an unpowered state.

(check for more detail on power and all that the cisco live 2904 from sanfran 2014/orlando 2013 (different decks)

xander

[Eduard Gheorghiu]

Hello Xander,

Thank you the answer. One more question - at boot time the starting order is clear. In case of oversubscription there is no mechanisms for shutting down selectively the linecards,right?

In case 2 a) the entire system will shut down or more precisely it will reload?

Thanks,

Eduard

[xthuijs · Accepted solution]

hi eduard,

correct there is no mechanism for that. Funnily that was our original behavior when we first designed it, but our first insertion users of the A9K hated that behavior (not that they ever ran into it though) but just didnt like that "proactive" behavior, so we ripped it out based on those substantial requests...

cheers!

xander

[atif_hafeez]

Hi Xander,

Thank you very much for your usual support.

In scenario 2a, breaker tripping will happen on all the power modules as all modules are providing equal share to the bus under high load conditions. So, breaker tripping case should actually shuts down the whole router.

In scenario 2b, if there is a detection mechanism for shutting down the power module, why the module will be burnt? Can you please elaborate the case. We have a scheduled Lab testing for ASR9922 which includes power outage scenarios, so it is very important for us to understand this specific behavior to avoid damaging the hardware.

Best regards!

[xthuijs]

Atif:

re 2a) yeah in an ideal world it is perfect loadshare, but that is not guaranteed; due to component tolerances and the loadbalancing logic, input voltage skews etc, one might see some slight variation there. Those little differences may make not every breaker to trip right at the same time and it could very well be, timing dependent one breaker tripping, RP noticing voltage drop and inducing reload before the next breaker can trip.

2b) I guess this is not the right place to make a joke :) Never seen it burn. These modules are rather well designed.

General note:

You can surely test all this, but I want to make sure you understand that the actual results "may vary" in terms of full shutdown due to all breakers tripped, chassis reload things like that. Also very much depending on the actual power needed at that time on occurrence.

(Driven by optics, optic types, traffic switching, forwarding interfaces, temperatures-< fan speed etc).

cheers

xander

[atif_hafeez]

Xander,

Thanks for the clarifications.

So in case 2, we can run into the following two situations:

1. Breaker tripping causing the power module to shut off.
2. RP inducing reload due to the voltage drop.

In both cases, power modules are protected. And RP never induce complete shutting off the node in the case under discussion. Please confirm that i have understood the behavior correctly.

Best regards.

[xthuijs]

that is correct atif!

if the system reloads with lesser modules, then some cards will not be booted, budget depending. cards boot in order from LC0, LC1...LCx

xander
 

[atif_hafeez]

Xander,

Is this behavior same across all ASR9XXX platforms (ASR9006, ASR9010 & ASR9922)?

The platforms we will be testing are ASR9922 & ASR9006.

Best regards!

[xthuijs]

also correct there Atif, the behavior is the same between 9922, 9010, 9006, 9912 etc.

the power manager sw doesn't differentiate between chassis.

xander