IOS XR Upgrade ERROR: Failed to install hostos rpm asr9k-sysadmin-host

[MANFRED LIEBCHEN]

I tried to upgrade our ASR 9904 RSP5 system from 6.6.3 to 7.5.2(64-bit)

According to Cisco´s Upgrade Procedure I installed the following  
    Upgrade Mandatory IOS XR  6.6.3 SMUs       
           
    asr9k-x64-6.6.3.CSCvr47102.tar    done from XR VM     Pre-requisites:
    Install Method:          ISSU                  None
    SMU Install Impact:      ISSU/Reload       
    asr9k-x64-6.6.3.CSCvu75507.tar    done from XR VM     Pre-requisites:
    Install Method:          ISSU                  asr9k-x64-6.6.3.CSCvr47102
    SMU Install Impact:      ISSU/Reload       
    asr9k-x64-6.6.3.CSCvw57276.tar   (Bridge SMU)    done from XR VM     Pre-requisites:
    Install Method:          ISSU                  asr9k-x64-6.6.3.CSCvr47102
    SMU Install Impact:      ISSU/Reload                  asr9k-x64-6.6.3.CSCvu75507
tion of the Bridge SMU

Nevertheless Installing 7.5.2 results in an Error Message

RP/0/RSP0/CPU0internet-gw#show install log 131
Jul 05 06:21:53 Install operation 131 started by liebchen:
  install prepare id 130
Jul 05 06:21:53 Package list:
Jul 05 06:21:53     asr9k-ospf-x64-1.0.0.0-r752.x86_64
Jul 05 06:21:53     asr9k-mgbl-x64-2.0.0.0-r752.x86_64
Jul 05 06:21:53     asr9k-mcast-x64-2.1.0.0-r752.x86_64
Jul 05 06:21:53     asr9k-mini-x64-7.5.2
Jul 05 06:21:53     asr9k-k9sec-x64-2.2.0.0-r752.x86_64
Jul 05 06:22:00 Action 1: install prepare action started
Jul 05 06:22:00 Install operation will continue in the background
Jul 05 06:22:41 The prepared software is set to be activated with reload upgrade
Jul 05 06:22:42 Start preparing new VM for reload upgrade
Jul 05 06:24:25 Cannot proceed with system upgrade.
 ERROR: Failed to install hostos rpm asr9k-sysadmin-hostos-7.5.2-r752.host.x86_64.rpm.
Jul 05 06:24:25 Action 1: install prepare action aborted
Jul 05 06:24:27 Install operation 131 aborted
Jul 05 06:24:27 Ending operation 131

According to Cisco´s Bug Reports, the error message results from BUGID CSCvq46421

CSCvq46421  

Description

Symptom: System upgrade or ISSU will fail due to new image structure in this release. Install prepare operation will fail Conditions: If Software upgrade or ISSU is tried from older release to this release then it will be failed Workaround: No Workaround Bridge SMU for this work is tracked with DDTS CSCvw57276

Further Problem Description: Install operation will fail during system upgrade with following error message In optimised prepare error message would be :

 2020-09-24 11:17:57:: Triggering prepare operation. This may take a while...

2020-09-24 11:24:23:: Following error(s) occurred in sysadmin vm(s) during install operation:

2020-09-24 11:24:23:: Preparation failed on 0/1 2020-09-24 11:24:23:: Preparation failed on 0/RSP1 2020-09-24 11:24:23:: Preparation failed on 0/2 2020-09-24 11:24:23:: Preparation failed on 0/RSP0

2020-09-24 11:24:23:: Error stack for 0/1 : #1 Error in preparing partition of host vm #2 Error in preparing partition of calvados vm #3 Error in preparing partition of xr vm Error stack for 0/RSP1 : #1 Error in preparing partition of host vm #2 Error in preparing partition of calvados vm #3 Error in preparing partition of xr vm Error stack for 0/2 : #1 Error in preparing partition of host vm #2 Error in preparing partition of calvados vm #3 Error in preparing partition of xr vm Error stack for 0/RSP0 : #1 Error in preparing partition of host vm #2 Error in preparing partition of calvados vm #3 Error in preparing partition of xr vm Please collect 'show tech-support install one-showtech' from XR and 'show tech-support ctrace' from Admin and pass this information to your TAC representative for support. In non optimised prepare error message would be:

=======================================

Sep 24 11:51:57 Install operation will continue in the background

Sep 24 11:52:55 The prepared software is set to be activated with reload upgrade

Sep 24 11:52:56 Start preparing new VM for reload upgrade Sep 24 11:54:32 Cannot proceed with system upgrade. ERROR: Failed to install hostos rpm asr9k-sysadmin-hostos-7.3.1.25I-r73125I.host.x86_64.rpm. Sep 24 11:54:32 Action 1: install prepare action aborted

QUESTION What is wrong with my installation of Bridge SMU

asr9k-x64-6.6.3.CSCvw57276.tar   (Bridge SMU)

Must I install Bridge SMU from XR VM or from Sysadmin or BOTH ?

Thanks in advance greetings

Manfred

[smilstea · Accepted solution]

Can you provide a show install active and show install commit?

Also did you install both asr9k-x64-6.6.3.CSCvw57276.tar and asr9k-sysadmin-6.6.3.CSCvw57276.tar?

Thanks,

Sam

[MANFRED LIEBCHEN]

Hallo Sam

I did not install
*asr9k-sysadmin-6.6.3.CSCvw57276.tar*

I have only installed
asr9k-x64-6.6.3.CSCvw57276.tar

I have to install *asr9k-sysadmin-6.6.3.CSCvw57276.tar in Sysadmin
context; is this correct ?
*Installation steps are the same as for asr9k-x64-6.6.3.CSCvw57276.tar
in XR VM context ?

*Step 0 - Change to sysadmin*

*Step 1 - install add source ...
*
**Step 2 - install prepare issu id id#
**
***Step 3 - install activate issu*

[smilstea]

We allow all installations from the XR VM, you should be able to install the sysadmin rpm from XR. If you have to go to the admin VM you have to worry about copying files between VMs etc.
Do make sure to run an install commit after the install.

Let me know if you continue to see any further issues.

Sam

[rahuver3]

Hi Manfred,

We installed the SMU for CSCvs54713 (including prerequisite SMUs) but are still getting this error on some routers running older SMU packs. Were you able to get this fixed with something else? 

Thanks

Rahul

[smilstea]

Rahul,

Can you share show install active from XR VM and sysadmin VM?
Also the show install log detail with the error you encountered?

Thanks,
Sam

[rahuver3]

Hi Sam, 

Here you go. 

show install active summary

asr9k-xr-6.6.3 version=6.6.3 [Boot image]

asr9k-mcast-x64-2.0.0.0-r663

asr9k-k9sec-x64-2.1.0.0-r663

asr9k-mpls-te-rsvp-x64-3.1.0.0-r663

asr9k-mgbl-x64-2.0.0.0-r663

asr9k-ospf-x64-1.0.0.0-r663

asr9k-9000v-nV-x64-1.0.0.0-r663

asr9k-bng-x64-1.0.0.0-r663

asr9k-isis-x64-1.2.0.0-r663

asr9k-eigrp-x64-1.0.0.0-r663

asr9k-m2m-x64-2.0.0.0-r663

asr9k-optic-x64-1.0.0.0-r663

asr9k-mpls-x64-2.0.0.0-r663

asr9k-li-x64-1.1.0.0-r663

asr9k-mpls-x64-2.0.0.1-r663.CSCvs24151

asr9k-bng-x64-1.0.0.2-r663.CSCvs62931

asr9k-common-pd-fib-64-1.0.0.1-r663.CSCvs09980

asr9k-gcp-fwding-64-1.0.0.1-r663.CSCvs09980

asr9k-mgbl-x64-2.0.0.2-r663.CSCvw32825

asr9k-isis-x64-1.2.0.3-r663.CSCvu96269

asr9k-iosxr-fwding-64-3.0.0.9-r663.CSCvr47102

asr9k-fwding-64-5.0.0.9-r663.CSCvr47102

asr9k-base-64-5.0.0.6-r663.CSCvx57128

asr9k-iosxr-os-64-3.0.0.5-r663.CSCvs07298

asr9k-iosxr-infra-64-3.0.0.19-r663.CSCvs54713

asr9k-os-supp-64-5.0.0.9-r663.CSCvs54713

admin show install active sum

Tue Jul 26 12:32:57.533 IST

     Active Packages: 1

        asr9k-sysadmin-6.6.3 version=6.6.3 [Boot image]

Install log

Jul 26 13:21:12 Install operation 16 started by rooto:

  install activate pkg asr9k-goldenk9-x64-7.5.2-v1 replace

Jul 26 13:21:12 Package list:

Jul 26 13:21:12     asr9k-goldenk9-x64-7.5.2-v1

Jul 26 13:21:18 Action 1: install prepare action started

Jul 26 13:21:38 Following rpms will be part of operation

Jul 26 13:21:40 XR RPMs

Jul 26 13:21:40          1:asr9k-li-x64-1.1.0.0-r752.x86_64

Jul 26 13:21:40          2:asr9k-mcast-x64-2.1.0.0-r752.x86_64

Jul 26 13:21:40          3:asr9k-isis-x64-1.1.0.0-r752.x86_64

Jul 26 13:21:40          4:asr9k-mpls-te-rsvp-x64-2.1.0.0-r752.x86_64

Jul 26 13:21:40          5:asr9k-gcp-fwding-64-1.0.0.1-r752.CSCwc01059.x86_64

Jul 26 13:21:40          6:asr9k-k9sec-x64-2.2.0.0-r752.x86_64

Jul 26 13:21:40          7:asr9k-base-64-5.0.0.1-r752.CSCwb99279.x86_64

Jul 26 13:21:40          8:asr9k-ospf-x64-1.0.0.0-r752.x86_64

Jul 26 13:21:40          9:asr9k-bng-supp-x64-1.0.0.0-r752.x86_64

Jul 26 13:21:40          10:asr9k-eigrp-x64-1.0.0.0-r752.x86_64

Jul 26 13:21:40          11:asr9k-isis-x64-1.1.0.1-r752.CSCvz64405.x86_64

Jul 26 13:21:40          12:asr9k-services-x64-1.0.0.0-r752.x86_64

Jul 26 13:21:40          13:asr9k-fwding-64-4.0.0.2-r752.CSCwc01059.x86_64

Jul 26 13:21:40          14:asr9k-m2m-x64-2.0.0.0-r752.x86_64

Jul 26 13:21:40          15:asr9k-mgbl-x64-2.0.0.0-r752.x86_64

Jul 26 13:21:40          16:asr9k-ospf-x64-1.0.0.1-r752.CSCwb98104.x86_64

Jul 26 13:21:40          17:asr9k-9000v-nV-x64-1.0.0.0-r752.x86_64

Jul 26 13:21:40          18:asr9k-bng-x64-1.0.0.0-r752.x86_64

Jul 26 13:21:40          19:asr9k-iosxr-routing-64-1.0.0.1-r752.CSCwb84175.x86_64

Jul 26 13:21:40          20:asr9k-mpls-x64-2.0.0.0-r752.x86_64

Jul 26 13:21:40          21:asr9k-optic-x64-1.0.0.0-r752.x86_64

Jul 26 13:22:52 The prepared software is set to be activated with reload upgrade

Jul 26 13:22:53 This install operation will reload the system, continue?

 [yes/no]:[yes] yes

Jul 26 13:22:53 Install operation will continue in the background

Jul 26 13:22:53 Start preparing new VM for reload upgrade

Jul 26 13:24:31 Cannot proceed with system upgrade.

 ERROR: Failed to install hostos rpm asr9k-sysadmin-hostos-7.5.2-r752.host.x86_64.rpm.

Jul 26 13:24:31 Action 1: install prepare action aborted

Jul 26 13:24:33 Install operation 16 aborted

Jul 26 13:24:33 Ending operation 16

Thanks, 

Rahul

[smilstea]

I don't see the sysadmin SMU for the issue in the show install active summary, can you please confirm you installed both the XR rpm and the sysadmin RPM for the issue. They are two different downloads on the download site.

Sam

[rahuver3]

We've installed the sysadmin SMU and now getting this error during the upgrade. 

Jul 27 16:21:05 The prepared software is set to be activated with reload upgrade

Jul 27 16:21:06 This install operation will reload the system, continue?

 [yes/no]:[yes] yes

Jul 27 16:21:06 Install operation will continue in the background

Jul 27 16:21:06 Start preparing new VM for reload upgrade

Jul 27 16:21:14 Cannot proceed with system upgrade.

 Error! No enough diskspace in install repository to proceed.

 Install remove packages from install repository to free up disk space

and retry install operation.

Jul 27 16:21:14 Action 1: install prepare action aborted

Jul 27 16:21:16 Install operation 64 aborted

Jul 27 16:21:16 Ending operation 64