cancel
Showing results for 
Search instead for 
Did you mean: 
cancel
Announcements

Migrating from RP1 to RP2 on ASR9000

2029
Views
1
Helpful
11
Comments

Migrating from RP1 to RP2 on ASR9000

 

1) Prerequisites:

 

Console access to the router and Router must be running release 5.3.2 or higher

System must not have 800G (tomahawk) line cards, (see caveat (a) in list below)

Before the upgrade process starts, and prior to removing any RP from the chassis, make sure that “show redundancy” shows NSR and standby card is ready.

Make sure any none 5.3.x software is removed from the system (ie admin install remove *4.3.4*)

Enable console logging on the router and on the terminal application (optional)

 

2) Caveats:

 

  1. a)The system must NOT have Tomahawk(400G or 800G) Line cards in the chassis prior to attempting this as the EOBC link between Tomahawk and newly inserted RP2 will be down which will interfere with the failover from RP1 to RP2. If there are Tomahawk cards in the chassis, they need to be removed or powered down for the upgrade to be successful.  ‘admin-config #hw-module power disable location 0/1/cpu0’

 

3) Upgrade process:

 

 

1)      Issue show redundancy and make sure NSR and Standby card is ready

 

RP/0/RP1/CPU0:vkg#show redundancy summary

Wed Jun 17 11:18:51.391 PST

  Active/Primary   Standby/Backup

  --------------   --------------

  0/RP0/CPU0(A)   0/RP1/CPU0(S) (Node Ready, NSR: Ready)

  0/RP0/CPU0(P)   0/RP1/CPU0(B) (Proc Group Ready, NSR: Ready)

RP/0/RP1/CPU0:vkg#       

 

RP/0/RP1/CPU0:vkg#show redundancy

Wed Jun 17 11:20:19.086 PST

Redundancy information for node 0/RP1/CPU0:

==========================================

Node 0/RP1/CPU0 is in ACTIVE role

Node Redundancy Partner (0/RP0/CPU0) is in STANDBY role

Standby node in 0/RP0/CPU0 is ready

Standby node in 0/RP0/CPU0 is NSR-ready

Node 0/RP1/CPU0 is in process group PRIMARY role

Process Redundancy Partner (0/RP0/CPU0) is in BACKUP role

Backup node in 0/RP0/CPU0 is ready

Backup node in 0/RP0/CPU0 is NSR-ready

 

Group            Primary         Backup          Status

---------        ---------       ---------       ---------

v6-routing       0/RP1/CPU0     0/RP0/CPU0     Ready

mcast-routing    0/RP1/CPU0     0/RP0/CPU0     Ready

netmgmt          0/RP1/CPU0     0/RP0/CPU0     Ready

v4-routing       0/RP1/CPU0     0/RP0/CPU0     Ready

central-services 0/RP1/CPU0     0/RP0/CPU0     Ready

dlrsc            0/RP1/CPU0     0/RP0/CPU0     Ready

dsc              0/RP1/CPU0     0/RP0/CPU0     Ready

 

Active node reload "Cause: Initiating switch-over."

Standby node reload "Cause: Initiating switch-over."

 

RP/0/RP1/CPU0:vkg#                                                       

 

 

2)     Remove the standby RP1

 

3)      Insert RP2 (A) 

 

4)      Issue Ctrl-c to break into ROMMON of inserted RP2(A)

                (If this is missed, the RP2(A) will go into a boot cycle loop, and there will be another chance to break into rommon on the next bootup, no side effects apart from syslog msgs on the active RP).

 

5)      From ROMMON prompt on RP2 (A), set ROMMON variable to force 1GE Mode for Peer RSP communication (RSP_LINK_1G=1), and check config register is set correctly in ROMMON

               rommon B1> RSP_LINK_1G=1

               rommon B1> sync

 

6)      Reset RP2 (A) card, it will boot up to standby state which will synchronize configuration from Active RP1

               rommon B1> reset -h

 

7)      Verify the RPs have reached full synchronization with “show redundancy” command and the groups are in “ready” state.

               show redundancy

 

8)      Verify the RPs have synchronized the SNMP engine ID and SNMP ifindex-table

               #more disk0:snmp/ifindex-table loc 0/rp0/cpu0

               #more disk0:snmp/ifindex-table loc 0/rp1/cpu0

               #more disk0:snmp/snmp_persist loc 0/rp0/cpu0

               #more disk0:snmp/snmp_persist loc 0/rp1/cpu0

               manually copy any eem scripts from RP1 to RP2 disks if applicable

 

 

9)      Failover from active RP1 to standby RP2

                “redundancy switchover”

 NOTE: A CLI switchover is needed, a physical OIR of the RP will not do the trick.

10)    Verify the RP2 becomes active RP and configuration is present

               At this point the RP1 must be removed/eject.

 

11)   Insert RP2 (B) and allow it to boot to standby.  Do *not* set the ROMMON variable

 

12)  Issue show redundancy to confirm the RPs have sync’d up and in correct state

'show platform’

'show redundancy'

manually copy any eem scripts from Active RP to Standby disks if applicable

 

  13)   On Active RP2 Clear ROMMON variable from XR

                 run nvram_rommonvar RSP_LINK_1G 0 (root-system access is needed)

                  more nvram:/classic-rommon-var

In case newly inserted second RP2 fails to boot up, park standby RSP4/RP2 in Rommon.

Type “set” to list Rommon variables. Check for the presence of Rommon variable “RSP_LINK_1G=1”.

If set, unset RSP_LINK_1G and reset card by following below steps.

 

rommon B1> priv

rommon B1> unset RSP_LINK_1G

rommon B1> sync

rommon B1> sync

rommon B1> reset -h

 

14)    Perform FPD upgrade on the RP2 (optional)

admin upgrade hw-module fpd all location 0/rp0/cpu0

                       admin upgrade hw-module fpd all location 0/rp1/cpu0

 

15)    Reload the router.

 

admin reload location all

 

16)    Check the EOBC ports are running at 10G with the following four commands, the output should be “10G/KR/1-lane”

 

show controllers epm-switch port-status 58 loc 0/rp0/cpu0 | in Mode

show controllers epm-switch port-status 58 loc 0/rp0/cpu0 | in Mode

show controllers epm-switch port-status 59 loc 0/rp1/cpu0 | in Mode

show controllers epm-switch port-status 59 loc 0/rp1/cpu0 | in Mode

 

17)    Final checks to ensure system is optional and functional

 

          Note; this operation has been tested to be graceful with NSR configured

 

Comments
Cisco Employee

Perfect doc Eddie!! Thank you!!!

Beginner

Hello,

Does it has the difference if we started migration when 0/RP0/CPU0 in ACTIVE role?

Cisco Employee

To Customers and Partners

Please pay attentions for the following rommon bug if you are considering the migration from RP1 to RP2 by method that there are RP1 and RP2 in same Chassis.

The fix is in rommon 14.23 and later.

 CSCva30444: ASR9922 RP1 to RP2 In Service Upgrade fail with rommon version > 14.11

Symptom:
While performing ISSU following procedure listed in below doc, the TFTP download on RP2 is really slow and gets timeout. So it cannot boot up and download software from Active RP (RP1)

https://supportforums.cisco.com/document/12751781/migrating-rp1-rp2-asr9000

Conditions:
Rommon version greater than 14.11

Workaround:
Downgrade to 14.10

Enthusiast

Hi and thank you for the info. It came in the right time. 

We are waiting for five ASR9912 chassis with RP2 and TH LC. We will run 6.2.x as soon it's out.

Not sure about rommon upgrade on A9K. Does it automatically upgrade when we do an IOS-XR upgrade?

In which version do we have rommon 14.23?

Cisco Employee

Automatic FPD upgrades can be enabled with the following command in admin config:

(admin-config)#fpd auto-upgrade

Enthusiast

Ah ok, FPD is covering this. So it's not like rommon on C7600.

Yes, we are using "fpd auto-upgrade" for every IOS-XR upgrade.

What about 14.23, in which version can we expect to have it?

Cisco Employee

The FPD which has the fix is in 6.1.2, so if your on 6.1.2 and you have upgraded your FPD to whats bundled in 6.1.2+ your good. Your good if you go to 6.2.x as you mentioned earlier. 

Beginner

How to downgrade to 14.10

it related to this bug.

https://bst.cloudapps.cisco.com/bugsearch/bug/CSCva30444/?referring_site=bugquickviewclick

Cisco Employee

We don't recommend downgrading the rommon. Are you running into this issue while trying to migrate from RP1 to RP2?

Beginner

Yes, 

what is the best way to this.

I am disabling watchdog but that takes to much time to sync.

i am running 5.3.3.

could you give me some advice.

Cisco Employee

In that case you can downgrade the rommon on the RP2 in the lab and then take it to the production router to perform the seamless migration. It would be the best if you would open a TAC Service Request to have this task performed under TAC engineer guidance.

regards,

/Aleksandar

CreatePlease to create content
Content for Community-Ad
August's Community Spotlight Awards