Re: [asr99 boots with RP1]

Carlos A. Silva · ‎06-13-2020

Hi, All:

I'm installing an ASR 9912 from scratch for the first time. Because of an IOS downgrade and SMUs, I've seen it boot 3 times and every time it starts with RP1 as the primary processor. I've software tested redundancy switchover and everything seems to be fine, as far as I can tell.

I thought I'd ask: is is normal for RP1 to be the primary every time it boots up? I have never seen it before with an ASR9000. Maybe a command controls this now. Any ideas?

Thanks!

/c.

smilstea · ‎06-13-2020

That is odd, RP0, specifically RP0 on rack0, is given preference in ASR9K (as well as CRS). In both platforms there is a timer that other RPs have to wait for RP0 to start, if RP0 doesn't start and isn't detected by the other RPs within that time then they will assume mastership. It's possible RP0 took longer to reload or to boot back up and validate its MBI.

Note I can't tell you the time the RPs wait for RP0 because that has changed over the years and between ASR9K and CRS.

Sam

Carlos A. Silva · ‎06-15-2020

I reloaded the box, just for the fun of it. Should I open a case?

BEFORE RELOAD

RP/0/RP0/CPU0:XXXX#sh platform
Mon Jun 15 14:48:20.745 UTC
Node Type State Config state
--------------------------------------------------------------------------------
0/RP0/CPU0 A99-RP3-TR(Active) IOS XR RUN NSHUT
0/RP1/CPU0 A99-RP3-TR(Standby) IOS XR RUN NSHUT
0/FT0 ASR-9912-FAN OPERATIONAL NSHUT
0/FT1 ASR-9912-FAN OPERATIONAL NSHUT
0/FC0 A99-SFC3 OPERATIONAL NSHUT
0/FC1 A99-SFC3 OPERATIONAL NSHUT
0/FC2 A99-SFC3 OPERATIONAL NSHUT
0/FC3 A99-SFC3 OPERATIONAL NSHUT
0/FC4 A99-SFC3 OPERATIONAL NSHUT
0/FC5 A99-SFC3 OPERATIONAL NSHUT
0/FC6 A99-SFC3 OPERATIONAL NSHUT
0/PT0 A9K-DC-PEM-V3 OPERATIONAL NSHUT
0/PT1 A9K-DC-PEM-V3 OPERATIONAL NSHUT
0/PT2 A9K-DC-PEM-V3 OPERATIONAL NSHUT
RP/0/RP0/CPU0:XXXX#admin reload rack 0
Mon Jun 15 14:48:24.986 UTC
Reload node ? [no,yes] yes
result Rack graceful reload request on 0 acknowledged.
RP/0/RP0/CPU0:XXXX#
RP/0/RP0/CPU0:XXXX#
The semaphore timeout period has expired.

AFTER RELOAD

RP/0/RP1/CPU0:XXXX#show platform
Mon Jun 15 14:57:34.258 UTC
Node Type State Config state
--------------------------------------------------------------------------------
0/RP0/CPU0 A99-RP3-TR(Standby) IOS XR RUN NSHUT
0/RP1/CPU0 A99-RP3-TR(Active) IOS XR RUN NSHUT
0/FT0 ASR-9912-FAN OPERATIONAL NSHUT
0/FT1 ASR-9912-FAN OPERATIONAL NSHUT
0/FC0 A99-SFC3 OPERATIONAL NSHUT
0/FC1 A99-SFC3 OPERATIONAL NSHUT
0/FC2 A99-SFC3 OPERATIONAL NSHUT
0/FC3 A99-SFC3 OPERATIONAL NSHUT
0/FC4 A99-SFC3 OPERATIONAL NSHUT
0/FC5 A99-SFC3 OPERATIONAL NSHUT
0/FC6 A99-SFC3 OPERATIONAL NSHUT
0/PT0 A9K-DC-PEM-V3 OPERATIONAL NSHUT
0/PT1 A9K-DC-PEM-V3 OPERATIONAL NSHUT
0/PT2 A9K-DC-PEM-V3 OPERATIONAL NSHUT
RP/0/RP1/CPU0:XXXX#show redundancy summ
Mon Jun 15 14:57:43.357 UTC
Active Node Standby Node
----------- ------------
0/RP1/CPU0 0/RP0/CPU0 (Node Not Ready, NSR:Not Ready)
RP/0/RP1/CPU0:XXXX#
RP/0/RP1/CPU0:XXXX#show ver
Mon Jun 15 15:00:12.350 UTC
Cisco IOS XR Software, Version 6.6.3
Copyright (c) 2013-2019 by Cisco Systems, Inc.

Build Information:
Built By : hlo
Built On : Fri Dec 13 17:04:06 PST 2019
Built Host : iox-ucs-032
Workspace : /auto/srcarchive15/prod/6.6.3/asr9k-x64/ws
Version : 6.6.3
Location : /opt/cisco/XR/packages/

cisco ASR9K () processor
System uptime is 5 minutes

RP/0/RP1/CPU0:XXXX#

smilstea · ‎06-18-2020

Sorry my response was for 32-bit, and you are running 64-bit where I just found that the behavior is different.

active RP elect when boot up depends on which RP is 1st boot, it doesn’t have mandatory sequence that must RP0 boot up 1st then boot up RP1, a possibility is RP1 booted up before RP0 start booting, and RP0 detected RP1 when it booting up and try to acquire master

So its all about which card is quicker to boot, no timer to wait for RP0 in 64-bit as multichassis systems don't exist in 64-bit.

Sam

smilstea · ‎06-18-2020

Also what happens if RP1 is active and you do the admin reload rack 0? Do you get the same result?

It makes sense when RP0 is active and you reload as we would want to reload all other cards first before the active RP.

Sam

Carlos A. Silva · ‎06-18-2020

Hi, Sam:

Thanks again for taking the time. Just did an admin reload with RP1 as the active RP and RP1 is still coming up as the active processor.
See below.

Before:

RP/0/RP1/CPU0:XXXX#show platform
Thu Jun 18 14:51:51.028 CDT
Node Type State Config state
--------------------------------------------------------------------------------
0/RP0/CPU0 A99-RP3-TR(Standby) IOS XR RUN NSHUT
0/RP1/CPU0 A99-RP3-TR(Active) IOS XR RUN NSHUT
0/FT0 ASR-9912-FAN OPERATIONAL NSHUT
0/FT1 ASR-9912-FAN OPERATIONAL NSHUT
0/FC0 A99-SFC3 OPERATIONAL NSHUT
0/FC1 A99-SFC3 OPERATIONAL NSHUT
0/FC2 A99-SFC3 OPERATIONAL NSHUT
0/FC3 A99-SFC3 OPERATIONAL NSHUT
0/FC4 A99-SFC3 OPERATIONAL NSHUT
0/FC5 A99-SFC3 OPERATIONAL NSHUT
0/FC6 A99-SFC3 OPERATIONAL NSHUT
0/PT0 A9K-DC-PEM-V3 OPERATIONAL NSHUT
0/PT1 A9K-DC-PEM-V3 OPERATIONAL NSHUT
0/PT2 A9K-DC-PEM-V3 OPERATIONAL NSHUT
RP/0/RP1/CPU0:XXXX #admin
Thu Jun 18 14:52:48.261 CDT

cisco connected from 127.0.0.1 using console on sysadmin-vm:0_RP0
sysadmin-vm:0_RP0# reload rack 0
Thu Jun 18 19:52:54.662 UTC+00:00
Reload node ? [no,yes] yes
result Rack graceful reload request on 0 acknowledged.
sysadmin-vm:0_RP0#
sysadmin-vm:0_RP0#

AFTER:

RP/0/RP1/CPU0:XXXXX#sh platfo
Thu Jun 18 15:02:15.652 CDT
Node Type State Config state
--------------------------------------------------------------------------------
0/RP0/CPU0 A99-RP3-TR(Standby) IOS XR RUN NSHUT
0/RP1/CPU0 A99-RP3-TR(Active) IOS XR RUN NSHUT
0/FT0 ASR-9912-FAN OPERATIONAL NSHUT
0/FT1 ASR-9912-FAN OPERATIONAL NSHUT
0/FC0 A99-SFC3 OPERATIONAL NSHUT
0/FC1 A99-SFC3 OPERATIONAL NSHUT
0/FC2 A99-SFC3 OPERATIONAL NSHUT
0/FC3 A99-SFC3 OPERATIONAL NSHUT
0/FC4 A99-SFC3 OPERATIONAL NSHUT
0/FC5 A99-SFC3 OPERATIONAL NSHUT
0/FC6 A99-SFC3 OPERATIONAL NSHUT
0/PT0 A9K-DC-PEM-V3 OPERATIONAL NSHUT
0/PT1 A9K-DC-PEM-V3 OPERATIONAL NSHUT
0/PT2 A9K-DC-PEM-V3 OPERATIONAL NSHUT
RP/0/RP1/CPU0:XXXXXX#

RP/0/RP1/CPU0:XXXXX#show redundancy summ
Thu Jun 18 15:05:09.815 CDT
Active Node Standby Node
----------- ------------
0/RP1/CPU0 0/RP0/CPU0 (Node Ready, NSR:Not Ready)
RP/0/RP1/CPU0:XXXXX#

smilstea · ‎06-19-2020

I'll let Yuriry the TAC engineer respond further via email, but from what I can tell eXR behaves differently and it appears to be behaving appropriately here. Why it is always RP1 is something we will raise with BU though to double check, I'll work with him on that request.

One test to perform is to physically change RP0 and RP1 and see if the issue follows the hardware or still behaves the same with RP1 always taking over.

Sam

Carlos A. Silva · ‎06-19-2020

Thank you very much, Sam. Yuriy already replied, but it’d be nice for everybody involved to have the BU’s comments on this.

I will ask if it’s OK for us to switch the RPs and report back.

Carlos A. Silva · ‎06-19-2020

I don’t know what to say. I guess, one of the RPs is just faster than the other.
We switched them and the same RP became active.

RP/0/RP0/CPU0:XXXXXX#show platform
Fri Jun 19 13:32:13.259 CDT
Node Type State Config state
--------------------------------------------------------------------------------
0/RP0/CPU0 A99-RP3-TR(Active) IOS XR RUN NSHUT <--
0/RP1/CPU0 A99-RP3-TR(Standby) IOS XR RUN NSHUT
0/FT0 ASR-9912-FAN OPERATIONAL NSHUT
0/FT1 ASR-9912-FAN OPERATIONAL NSHUT
0/FC0 A99-SFC3 OPERATIONAL NSHUT
0/FC1 A99-SFC3 OPERATIONAL NSHUT
0/FC2 A99-SFC3 OPERATIONAL NSHUT
0/FC3 A99-SFC3 OPERATIONAL NSHUT
0/FC4 A99-SFC3 OPERATIONAL NSHUT
0/FC5 A99-SFC3 OPERATIONAL NSHUT
0/FC6 A99-SFC3 OPERATIONAL NSHUT
0/PT0 A9K-DC-PEM-V3 OPERATIONAL NSHUT
0/PT1 A9K-DC-PEM-V3 OPERATIONAL NSHUT
0/PT2 A9K-DC-PEM-V3 OPERATIONAL NSHUT
RP/0/RP0/CPU0:XXXXX#