C9500 Install Mode issue

[philasam79]

Hi folks. I'm trying to upgrade my C9500 that is in VSS to IOS image Cat9k_iosxe.17.03.04.SPA.bin in install mode. when i issue the command "install add file flash:Cat9k_iosxe.17.03.04.SPA.bin activate ISSU commit. It fails to perform the upgrade. I discovered that the "packages.conf" file isn't in the flash. 

How can i get it back in the flash to be able to perform the ISSU upgrade successfully?

[marce1000]

                > It fails to perform the upgrade.

 Can you also show subsequent errors after the install command ?

 M.

[philasam79]

Hi Marce1000,

Thank you for responding very quickly. the summary is below.

I had tried to perform the ISSU before and it failed. So, i decided to set the boot variable in other for the device to boot with an image because everytime i tried to boot, it boots into ROMMON. After i have fully booted the device to the image i wanted, i then tried again to convert that image to install as it's currently in Bundle mode.

Each attempt failed. See the error i get "FAILED: install_add_activate_commit : Install ISSU in progress. Abort the ISSU upgrade to proceed".

Then, i tried to abort the ISSU by issuing the following command "install abort issu". system completes the abort successfully, but when i go back to try ISSU again, the error is the same. "FAILED: install_add_activate_commit : Install ISSU in progress. Abort the ISSU upgrade to proceed".

These are all the install info below:

Directory of flash:/

442381 -rw- 616 Nov 25 2021 16:33:49 +00:00 stby-vlan.dat
81921 drwx 4096 Nov 25 2021 16:32:04 +00:00 .installer
49155 -rw- 2097152 Nov 25 2021 16:31:49 +00:00 nvram_config_bkup
49154 -rw- 2097152 Nov 25 2021 16:31:49 +00:00 nvram_config
49157 -rw- 0 Nov 25 2021 16:31:47 +00:00 dope_hist
442383 -rw- 16984 Nov 25 2021 16:31:47 +00:00 rdope_out.txt
442384 -rw- 89 Nov 25 2021 16:31:46 +00:00 rdope.log
442387 -rw- 5242880 Nov 25 2021 16:30:21 +00:00 ssd
442376 -rwx 2049 Nov 25 2021 16:30:01 +00:00 svl_ipc.tcl
442377 -rw- 134458 Nov 25 2021 16:30:01 +00:00 memleak.tcl
49156 -rw- 312 Nov 25 2021 16:29:51 +00:00 boothelper.log
131073 drwx 4096 Nov 25 2021 16:29:50 +00:00 dc_profile_dir
442375 -rw- 1480 Nov 25 2021 16:29:41 +00:00 bootloader_evt_handle.log
98305 drwx 4096 Nov 25 2021 14:17:07 +00:00 .prst_sync
229377 drwx 4096 Nov 25 2021 13:47:57 +00:00 .rommon_sync
49160 -rw- 616 Nov 25 2021 13:32:36 +00:00 vlan.dat
147457 drwx 4096 Nov 24 2021 23:02:23 +00:00 .dbpersist
49161 -rw- 12242 Nov 24 2021 23:01:55 +00:00 running_config
49158 -rw- 907450095 Nov 24 2021 18:33:51 +00:00 cat9k_iosxe.17.03.04.SPA.bin
221185 drwx 4096 Nov 18 2021 03:06:34 +00:00 Tbot
81967 drwx 4096 Nov 18 2021 03:06:28 +00:00 sys_report
81922 drwx 4096 Nov 18 2021 03:06:28 +00:00 tech_support
122881 drwx 4096 Nov 18 2021 03:06:28 +00:00 gs_script
81960 drwx 4096 Nov 18 2021 03:06:26 +00:00 ss_disc
442380 -rw- 116459 Nov 17 2021 22:39:07 +00:00 pnp-tech-discovery-summary
442379 -rw- 35 Nov 17 2021 22:39:04 +00:00 pnp-tech-time
155649 drwx 4096 Apr 30 2021 00:20:10 +00:00 onep
90113 drwx 4096 Jan 28 2019 10:32:38 +00:00 core
114689 drwx 4096 Jan 28 2019 10:31:12 +00:00 .rollback_timer

11353194496 bytes total (9827840000 bytes free)
!
!
!
c9500_core#sho boot
---------------------------
Switch 2
---------------------------
Current Boot Variables:
BOOT variable = flash:cat9k_iosxe.17.03.04.SPA.bin;packages.confi;

Boot Variables on next reload:
BOOT variable = flash:cat9k_iosxe.17.03.04.SPA.bin;packages.confi;
Manual Boot = no
Enable Break = no
Boot Mode = DEVICE
iPXE Timeout = 0
!
!
!
c9500_core#sh install summary
[ Switch 1 2 ] Installed Package(s) Information:
State (St): I - Inactive, U - Activated & Uncommitted,
C - Activated & Committed, D - Deactivated & Uncommitted
--------------------------------------------------------------------------------
Type St Filename/Version
--------------------------------------------------------------------------------
IMG C 17.03.04.0.5557

--------------------------------------------------------------------------------
Auto abort timer: inactive
--------------------------------------------------------------------------------
!
!
!
c9500_core#$ file flash:cat9k_iosxe.17.03.04.SPA.bin activate issu commit
install_add_activate_commit: START Thu Nov 25 16:37:04 UTC 2021
FAILED: install_add_activate_commit : Install ISSU in progress. Abort the ISSU upgrade to proceed
c9500_core#
*Nov 25 16:37:04.944: %INSTALL-5-INSTALL_START_INFO: Switch 2 R0/0: install_engine: Started install one-shot ISSU flash:cat9k_iosxe.17.03.04.SPA.bin
*Nov 25 16:37:05.242: %INSTALL-3-OPERATION_ERROR_MESSAGE: Switch 2 R0/0: install_engine: Failed to install_add_activate_commit package flash:cat9k_iosxe.17.03.04.SPA.bin, Error: install_add_activate_commit : Install ISSU in progress. Abort the ISSU upgrade to proceed
!
!
!
c9500_core#install abort issu
install_abort: START Thu Nov 25 16:37:42 UTC 2021

*Nov 25 16:37:43.390: %INSTALL-5-INSTALL_START_INFO: Switch 2 R0/0: install_engine: Started install abort ISSU
NOTE: Going to start Abort ISSU install process

STAGE 0: System Level Sanity Check
===================================================
/usr/binos/conf/install_util.sh: line 688: /bootflash/.installer/install_add_pkg_list.txt: No such file or directory
--- Verifying install_issu supported ---
--- Verifying booted from the valid media ---
--- Verifying AutoBoot mode is enabled ---
--- Verifying Platform specific ISSU admission criteria ---
Finished Initial System Level Sanity Check

SUCCESS: install_abort Thu Nov 25 16:37:51 UTC 2021

c9500_core#
*Nov 25 16:37:51.863: %INSTALL-5-INSTALL_COMPLETED_INFO: Switch 2 R0/0: install_engine: Completed install abort ISSU

!

Thank you for your switf response.

[philasam79]

Directory of flash:/

442381 -rw- 616 Nov 25 2021 16:33:49 +00:00 stby-vlan.dat
81921 drwx 4096 Nov 25 2021 16:32:04 +00:00 .installer
49155 -rw- 2097152 Nov 25 2021 16:31:49 +00:00 nvram_config_bkup
49154 -rw- 2097152 Nov 25 2021 16:31:49 +00:00 nvram_config
49157 -rw- 0 Nov 25 2021 16:31:47 +00:00 dope_hist
442383 -rw- 16984 Nov 25 2021 16:31:47 +00:00 rdope_out.txt
442384 -rw- 89 Nov 25 2021 16:31:46 +00:00 rdope.log
442387 -rw- 5242880 Nov 25 2021 16:30:21 +00:00 ssd
442376 -rwx 2049 Nov 25 2021 16:30:01 +00:00 svl_ipc.tcl
442377 -rw- 134458 Nov 25 2021 16:30:01 +00:00 memleak.tcl
49156 -rw- 312 Nov 25 2021 16:29:51 +00:00 boothelper.log
131073 drwx 4096 Nov 25 2021 16:29:50 +00:00 dc_profile_dir
442375 -rw- 1480 Nov 25 2021 16:29:41 +00:00 bootloader_evt_handle.log
98305 drwx 4096 Nov 25 2021 14:17:07 +00:00 .prst_sync
229377 drwx 4096 Nov 25 2021 13:47:57 +00:00 .rommon_sync
49160 -rw- 616 Nov 25 2021 13:32:36 +00:00 vlan.dat
147457 drwx 4096 Nov 24 2021 23:02:23 +00:00 .dbpersist
49161 -rw- 12242 Nov 24 2021 23:01:55 +00:00 running_config
49158 -rw- 907450095 Nov 24 2021 18:33:51 +00:00 cat9k_iosxe.17.03.04.SPA.bin
221185 drwx 4096 Nov 18 2021 03:06:34 +00:00 Tbot
81967 drwx 4096 Nov 18 2021 03:06:28 +00:00 sys_report
81922 drwx 4096 Nov 18 2021 03:06:28 +00:00 tech_support
122881 drwx 4096 Nov 18 2021 03:06:28 +00:00 gs_script
81960 drwx 4096 Nov 18 2021 03:06:26 +00:00 ss_disc
442380 -rw- 116459 Nov 17 2021 22:39:07 +00:00 pnp-tech-discovery-summary
442379 -rw- 35 Nov 17 2021 22:39:04 +00:00 pnp-tech-time
155649 drwx 4096 Apr 30 2021 00:20:10 +00:00 onep
90113 drwx 4096 Jan 28 2019 10:32:38 +00:00 core
114689 drwx 4096 Jan 28 2019 10:31:12 +00:00 .rollback_timer

11353194496 bytes total (9827840000 bytes free)
!
!
!
c9500_core#sho boot
---------------------------
Switch 2
---------------------------
Current Boot Variables:
BOOT variable = flash:cat9k_iosxe.17.03.04.SPA.bin;packages.confi;

Boot Variables on next reload:
BOOT variable = flash:cat9k_iosxe.17.03.04.SPA.bin;packages.confi;
Manual Boot = no
Enable Break = no
Boot Mode = DEVICE
iPXE Timeout = 0
!
!
!
c9500_core#sh install summary
[ Switch 1 2 ] Installed Package(s) Information:
State (St): I - Inactive, U - Activated & Uncommitted,
C - Activated & Committed, D - Deactivated & Uncommitted
--------------------------------------------------------------------------------
Type St Filename/Version
--------------------------------------------------------------------------------
IMG C 17.03.04.0.5557

--------------------------------------------------------------------------------
Auto abort timer: inactive
--------------------------------------------------------------------------------
!
!
!
c9500_core#$ file flash:cat9k_iosxe.17.03.04.SPA.bin activate issu commit
install_add_activate_commit: START Thu Nov 25 16:37:04 UTC 2021
FAILED: install_add_activate_commit : Install ISSU in progress. Abort the ISSU upgrade to proceed
c9500_core#
*Nov 25 16:37:04.944: %INSTALL-5-INSTALL_START_INFO: Switch 2 R0/0: install_engine: Started install one-shot ISSU flash:cat9k_iosxe.17.03.04.SPA.bin
*Nov 25 16:37:05.242: %INSTALL-3-OPERATION_ERROR_MESSAGE: Switch 2 R0/0: install_engine: Failed to install_add_activate_commit package flash:cat9k_iosxe.17.03.04.SPA.bin, Error: install_add_activate_commit : Install ISSU in progress. Abort the ISSU upgrade to proceed
!
!
!
c9500_core#install abort issu
install_abort: START Thu Nov 25 16:37:42 UTC 2021

*Nov 25 16:37:43.390: %INSTALL-5-INSTALL_START_INFO: Switch 2 R0/0: install_engine: Started install abort ISSU
NOTE: Going to start Abort ISSU install process

STAGE 0: System Level Sanity Check
===================================================
/usr/binos/conf/install_util.sh: line 688: /bootflash/.installer/install_add_pkg_list.txt: No such file or directory
--- Verifying install_issu supported ---
--- Verifying booted from the valid media ---
--- Verifying AutoBoot mode is enabled ---
--- Verifying Platform specific ISSU admission criteria ---
Finished Initial System Level Sanity Check

SUCCESS: install_abort Thu Nov 25 16:37:51 UTC 2021

c9500_core#
*Nov 25 16:37:51.863: %INSTALL-5-INSTALL_COMPLETED_INFO: Switch 2 R0/0: install_engine: Completed install abort ISSU

[balaji.bandi]

what is the current version you running ? ( post complete upgrade steps to understand the issue)

read the release notes : (In Service Software Upgrade (ISSU) with Cisco StackWise Virtual)

https://www.cisco.com/c/en/us/td/docs/switches/lan/catalyst9500/software/release/17-3/release_notes/ol-17-3-9500.html#task_b1z_hwr_43b

[philasam79]

I'm currently running "cat9k_iosxe.17.03.04.SPA.bin". This is currently in BUNDLE mode as you can see below. However, i'm trying to convert it to "INSTALL MODE".

Switch Ports Model SW Version SW Image Mode
------ ----- ----- ---------- ---------- ----
1 50 C9500-40X 17.03.04 CAT9K_IOSXE BUNDLE
* 2 50 C9500-40X 17.03.04 CAT9K_IOSXE BUNDLE

!

c9500_core#sho boot
---------------------------
Switch 2
---------------------------
Current Boot Variables:
BOOT variable = flash:cat9k_iosxe.17.03.04.SPA.bin;packages.confi;

Boot Variables on next reload:
BOOT variable = flash:cat9k_iosxe.17.03.04.SPA.bin;packages.confi;
Manual Boot = no
Enable Break = no
Boot Mode = DEVICE
iPXE Timeout = 0

I GET THE FOLLOWING ERROR WHEN I TRY TO RUN A NEW INSTALL UPGRADE EVEN AFTER I HAVE TRIED TO ABORT THE PREVIOUS ATTEMPT!.

!

c9500_core#$install add file flash:cat9k_iosxe.17.03.04.SPA.bin activate issu commit
install_add_activate_commit: START Thu Nov 25 16:37:04 UTC 2021
FAILED: install_add_activate_commit : Install ISSU in progress. Abort the ISSU upgrade to proceed
c9500_core#
*Nov 25 16:37:04.944: %INSTALL-5-INSTALL_START_INFO: Switch 2 R0/0: install_engine: Started install one-shot ISSU flash:cat9k_iosxe.17.03.04.SPA.bin
*Nov 25 16:37:05.242: %INSTALL-3-OPERATION_ERROR_MESSAGE: Switch 2 R0/0: install_engine: Failed to install_add_activate_commit package flash:cat9k_iosxe.17.03.04.SPA.bin, Error: install_add_activate_commit : Install ISSU in progress. Abort the ISSU upgrade to proceed
!
!
!
c9500_core#install abort issu
install_abort: START Thu Nov 25 16:37:42 UTC 2021

*Nov 25 16:37:43.390: %INSTALL-5-INSTALL_START_INFO: Switch 2 R0/0: install_engine: Started install abort ISSU
NOTE: Going to start Abort ISSU install process

STAGE 0: System Level Sanity Check
===================================================
--- Verifying install_issu supported ---
--- Verifying booted from the valid media ---
--- Verifying AutoBoot mode is enabled ---
--- Verifying Platform specific ISSU admission criteria ---
Finished Initial System Level Sanity Check

SUCCESS: install_abort Thu Nov 25 16:37:51 UTC 2021

c9500_core#
*Nov 25 16:37:51.863: %INSTALL-5-INSTALL_COMPLETED_INFO: Switch 2 R0/0: install_engine: Completed install abort ISSU

[Leo Laohoo]

See attached file.

IMPORTANT: 

• Before running the Install Mode script, make sure the boot variable string is pointing to "packages.conf" file and not the BIN file. 

Use this command to unpack the packages files from the BIN file: 

install add file bootflash:cat9k_iosxe.17.03.04.SPA.bin activate commit

WARNING:  This command will reboot the 9500 to complete the process.  

[philasam79]

Hi Team,

The problem here is that, i couldn't attempt a new INSTALL because these two errors.

1.There is not packages.conf" in the flash: or bootflash:

2. "*Nov 25 16:37:05.242: %INSTALL-3-OPERATION_ERROR_MESSAGE: Switch 2 R0/0: install_engine: Failed to   install_add_activate_commit package flash:cat9k_iosxe.17.03.04.SPA.bin, Error: install_add_activate_commit : Install ISSU in progress. Abort the ISSU upgrade to proceed"

I have tried to abort the install by issuing this command "install abort issu". The switch will perform the abort and successfully complete the abort. However, when i try to do another install, the error message comes up again that another install is already in progress and subsequently abort it.

This is my dilemma.

[Leo Laohoo]

---

@philasam79 wrote:

The problem here is that, i couldn't attempt a new INSTALL because these two errors.

---

Wrong. 
The problem is no one bothered to read the PDF document I have attached.  
Read it. 

[philasam79]

Hi Leo,

I did read the PDF. Your PDF stated to install SMU. SMU is a patch which is needed to be installed after a compatible base image has been installed. In this case, i haven't even been successful in installing the image. packages.conf doesn't exist in my "flash". Therefore, if i set my boot variable as packages.conf, when the router reboots, it looks for packages.conf in my bootflash or flash. In this case, since it's not in the flash, the boot is aborted and the switch goes into ROMMON. I have posted my flash dir above. As i have mentioned earlier. I think, for me to be able to point the switch to boot with packages.conf, it must have exist in the bootflash/flash. How can i get it downloaded to the flash? Is it a file i can download from cisco or do i have to generate it within the switch before the install? That is the part that is not talked about in your PDF.

[Leo Laohoo]

What command are you using to do the Install Mode.

[philasam79]

install add file flash:cat9k_iosxe.17.03.04.SPA.bin activate issu commit.

[Leo Laohoo]

---

@philasam79 wrote:

install add file flash:cat9k_iosxe.17.03.04.SPA.bin activate issu commit.

---

Please read the PDF document I have uploaded.  This command is not in that document.  

[philasam79]

Hi Leo,

I read the PDF. I see this command "install add file bootflash:filename.bin activate commit". However, i'm trying to upgrade a VSS switch. I have tried both of the command. Each time i tried it, i get this error message right after i hit enter key "*Nov 25 16:37:05.242: %INSTALL-3-OPERATION_ERROR_MESSAGE: Switch 2 R0/0: install_engine: Failed to install_add_activate_commit package flash:cat9k_iosxe.17.03.04.SPA.bin, Error: install_add_activate_commit : Install ISSU in progress. Abort the ISSU upgrade to proceed".

I also tried to abort the issu by entering this command "install abort issu" many times. It will complete the abort process successfully. then when i tried again to initiate a new install, i get the same error again. Almost as if the issu process is still running and cannot be stopped.

[Leo Laohoo]

Read between the lines:  There is a reason why the documents never mention (anything about upgrading using) ISSU, FSU/eFSU.  

(I am not troubleshooting any IOS-XE upgrades using ISSU, FSU/eFSU methods.)