ROMMON upgrade/migration fails on ESP40 with Config ASR1006, RP2, SIP40 (redundant)

[User404]

Hello,

I have a problem at upgrading the ROMMON of two ESP40.
I posted this comment in an existing thread, but without feedback.

I have some ASR1006 chassis with 2x RP2, 2x ESP40 and 2x SIP40 (2x SPA-10G cards in the SIPs).

During the migration I completely followed the official Cisco instructions: https://www.cisco.com/c/en/us/td/docs/routers/asr1000/release/notes/xe-16-9/asr1000-rel-notes-xe-16-9.html#concept_zdm_2nx_5bb

Especially regarding the recommended ROMMON versions:

Cisco IOS XE Release	ASR 1000 RP2	ASR 1000-SIP40	ASR 1000-ESP40
16.9.x	16.9(5r)	16.3(2r)	16.3(2r)

All RPs run on IOS XE 16.9.3 and ROMMON Verion 16.9(5r).
All SIP40s run on ROMMON version 16.3(2r).

Only a few (not all) ESP40 can not be upgraded to 16.3(2r).

The following error message appears:

Target copying rom-monitor image file

cp: omitting directory '/'

md5sum: /: Is a directory

md5sum: /bootflash/EFI/UpdateCapsule/UpdateCapsuleFile.bin: No such file or directory

Does anyone have any idea where the problem is or how to solve it?
Below you can see the whole CLI output:

Router#    sh ver
Cisco IOS XE Software, Version 16.09.03
Cisco IOS Software [Fuji], ASR1000 Software (X86_64_LINUX_IOSD-UNIVERSALK9-M), Version 16.9.3, RELEASE SOFTWARE (fc2)
Technical Support: http://www.cisco.com/techsupport
Copyright (c) 1986-2019 by Cisco Systems, Inc.
Compiled Wed 20-Mar-19 08:02 by mcpre

Router# sh platform 
Chassis type: ASR1006             

Slot      Type                State                 Insert time (ago) 
--------- ------------------- --------------------- ----------------- 
0         ASR1000-SIP40       ok                    00:04:27      
 0/0      SPA-1X10GE-L-V2     ok                    00:03:05      
 0/2      SPA-1X10GE-L-V2     ok                    00:02:57      
1         ASR1000-SIP40       ok                    00:04:27      
 1/0      SPA-1X10GE-L-V2     ok                    00:03:05      
 1/2      SPA-1X10GE-L-V2     ok                    00:02:53      
R0        ASR1000-RP2         ok, active            00:04:27      
R1        ASR1000-RP2         ok, standby           00:04:27      
F0        ASR1000-ESP40       ok, active            00:04:27      
F1        ASR1000-ESP40       ok, standby           00:04:27      
P0        ASR1006-PWR-AC      ok                    00:03:56      
P1        ASR1006-PWR-AC      ok                    00:03:55      

Slot      CPLD Version        Firmware Version                        
--------- ------------------- --------------------------------------- 
0         00200800            16.3(2r)                            
1         00200800            16.3(2r)                            
R0        10021901            16.9(5r)                            
R1        10021901            16.9(5r)                            
F0        1003190E            16.2(1r)                            
F1        1003190E            15.3(3r)S                           

Thanks!

[Leo Laohoo]

---

@User404 wrote:

All RPs run on IOS XE 16.9.3

---

Be careful of CSCvo36188.

According to Table 5, ASR 1K with ESP40 should be running a minimum of 16.3(2r).  Only the standby ESP40 is not loaded with this version.  You may need to failover to the secondary ESP40 and run the ROMMON upgrade again.

[User404]

Hi,

thanks for the hint!!

Regarding the ROMMON version, 16.3(2r) is the recommended release, the minimum version is 15.0(1r)S according to table 2.

I will try your suggestion and of course let you know if it worked :-)

[User404 · Accepted solution]

FYI
In our case, the solution was to downgrade the RP2 to get the stubborn ESP40 to the desired ROMMON.

In Step 1: downgrade the RP2 to ROMMON 16.2.1 and the corresponding IOS XE 16.2.x

In Step 2: upgrade to ROMMON 16.3.2 card per card.