Solved: Re: Catalyst 9500 IOS_XE upgrade missed reboot prompt

vv0bbLeS · ‎10-26-2021

Hello all,

This question probably shows my lack of understanding on what actually happens with the activate keyword, but I uploaded a new IOS_XE 17.3.4 image .bin file to a Catalyst 9500 switch, and I ran the install add file flash:cat9k_iosxe.17.03.04.SPA.bin activate commit command, and then got sidetracked doing something else, and when I came back I missed where the switch tells me that operation may require a reload and asks me to confirm. So I got put back to the main Switch# prompt, so I tried running the same install command as above but it told me the Super package was already added (not sure what that means to be honest).

So, I'm not sure how to proceed from here to finish this Switch upgrade, and I guess what I'm looking to learn is:

What all has actually taken place so far on this Switch?
What does it mean on my second run of the install command when it says Super package already added. Add operation not allowed. ?
What do I need to do now to finish the upgrade to 17.3.4 on this switch?

Switch#$ file flash:cat9k_iosxe.17.03.04.SPA.bin activate commit
install_add_activate_commit: START Tue Oct 26 21:05:23 UTC 2021
install_add_activate_commit: Adding PACKAGE
install_add_activate_commit: Checking whether new add is allowed ....

--- Starting Add ---
Performing Add on Active/Standby
  [1] Add package(s) on R0
  [1] Finished Add on R0
Checking status of Add on [R0]
Add: Passed on [R0]
Finished Add

Image added. Version: 17.03.04.0.5557
install_add_activate_commit: Activating PACKAGE
Following packages shall be activated:
/bootflash/cat9k-wlc.17.03.04.SPA.pkg
/bootflash/cat9k-webui.17.03.04.SPA.pkg
/bootflash/cat9k-srdriver.17.03.04.SPA.pkg
/bootflash/cat9k-sipspa.17.03.04.SPA.pkg
/bootflash/cat9k-sipbase.17.03.04.SPA.pkg
/bootflash/cat9k-rpboot.17.03.04.SPA.pkg
/bootflash/cat9k-rpbase.17.03.04.SPA.pkg
/bootflash/cat9k-lni.17.03.04.SPA.pkg
/bootflash/cat9k-guestshell.17.03.04.SPA.pkg
/bootflash/cat9k-espbase.17.03.04.SPA.pkg
/bootflash/cat9k-cc_srdriver.17.03.04.SPA.pkg

This operation may require a reload of the system. Do you want to proceed? [y/n]install_add_activate_commit: No User Response, Request Timeout

FAILED: install_add_activate_commit /bootflash/cat9k-wlc.17.03.04.SPA.pkg /bootflash/cat9k-webui.17.03.04.SPA.pkg /bootflash/cat9k-srdriver.17.03.04.SPA.pkg /bootflash/cat9k-sipspa.17.03.04.SPA.pkg /bootflash/cat9k-sipbase.17.03.04.SPA.pkg /bootflash/cat9k-rpboot.17.03.04.SPA.pkg /bootflash/cat9k-rpbase.17.03.04.SPA.pkg /bootflash/cat9k-lni.17.03.04.SPA.pkg /bootflash/cat9k-guestshell.17.03.04.SPA.pkg /bootflash/cat9k-espbase.17.03.04.SPA.pkg /bootflash/cat9k-cc_srdriver.17.03.04.SPA.pkg: Cannot proceed activate because of user input
Switch#
Switch#
Switch#
Switch#$ file flash:cat9k_iosxe.17.03.04.SPA.bin activate commit
install_add_activate_commit: START Tue Oct 26 21:28:15 UTC 2021
install_add_activate_commit: Adding PACKAGE
install_add_activate_commit: Checking whether new add is allowed ....
FAILED: install_add_activate_commit : Super package already added. Add operation not allowed. 'install remove inactive' can be used to discard added packages
Switch#

0xD2A6762E

Leo Laohoo · ‎10-27-2021

Is this a single unit or a VSS?
If this is a stand-alone/non-VSS unit, do the following:

rename flash:packages.conf flash:packages.conf.00-
rename flash:cat9k_iosxe.17.03.04.SPA.conf flash:packages.conf

After 1 & 2 are done, post the complete output (1 output page only) to the following command: more flash:packages.conf | begin for CAT

Also post the complete output to the command "sh boot".

View solution in original post

Leo Laohoo · ‎02-15-2023

In the "more flash:packages.conf", start from the second "page" and confirm that the packages are the intended version, i. e. it does not contain the OLD version. See photo below:

In the "sh boot", the boot variable string should be pointing to "packages.conf" only and nothing else. Another thing to look out for in the "sh boot" output is the next line below, MANUAL_BOOT must be "no". See the photo below:

Please read the attached file. I have made the document easy-to-understand and I have thoroughly tested and applied the method for the last 5 years without a single failure.

View solution in original post

Leo Laohoo · ‎10-26-2021

@vv0bbLeS wrote:

This operation may require a reload of the system. Do you want to proceed? [y/n]

Someone hit "n" at this prompt, didn't they?

I am not at all concerned. This is a "safety catch": The only way to be able to schedule a reload time when using the "install software" command.

Can I see the complete output to the command "dir"?

vv0bbLeS · ‎10-27-2021

@Leo Laohoo wrote:

Someone hit "n" at this prompt, didn't they?

It's hard to read in my output above, but it actually just timed out as I got sidetracked and never gave any input to the prompt:

This operation may require a reload of the system. Do you want to proceed? [y/n]install_add_activate_commit: No User Response, Request Timeout

Here is the output of dir, and also output of show install sum :

Switch#dir
Directory of bootflash:/

275809  drwx             4096  Oct 26 2021 21:28:31 +00:00  .installer
381267  -rw-             7715  Oct 26 2021 21:06:59 +00:00  cat9k_iosxe.17.03.04.SPA.conf
381278  -rw-         47407779  Oct 26 2021 21:06:59 +00:00  cat9k-rpboot.17.03.04.SPA.pkg
381277  -rw-             9220  Oct 26 2021 21:06:15 +00:00  cat9k-wlc.17.03.04.SPA.pkg
381276  -rw-         14431236  Oct 26 2021 21:06:15 +00:00  cat9k-webui.17.03.04.SPA.pkg
381275  -rw-         29717512  Oct 26 2021 21:06:15 +00:00  cat9k-srdriver.17.03.04.SPA.pkg
381274  -rw-         57082884  Oct 26 2021 21:06:15 +00:00  cat9k-sipspa.17.03.04.SPA.pkg
381273  -rw-         34874376  Oct 26 2021 21:06:15 +00:00  cat9k-sipbase.17.03.04.SPA.pkg
381272  -rw-        599221252  Oct 26 2021 21:06:15 +00:00  cat9k-rpbase.17.03.04.SPA.pkg
381271  -rw-             5124  Oct 26 2021 21:06:14 +00:00  cat9k-lni.17.03.04.SPA.pkg
381270  -rw-          2262024  Oct 26 2021 21:06:14 +00:00  cat9k-guestshell.17.03.04.SPA.pkg
381269  -rw-        104526856  Oct 26 2021 21:06:14 +00:00  cat9k-espbase.17.03.04.SPA.pkg
381268  -rw-         18289676  Oct 26 2021 21:06:14 +00:00  cat9k-cc_srdriver.17.03.04.SPA.pkg
210930  -rw-        907450095  Oct 26 2021 21:04:21 +00:00  cat9k_iosxe.17.03.04.SPA.bin
210929  -rw-                0  Oct 26 2021 20:54:59 +00:00  dope_hist
210927  -rw-            17197  Oct 26 2021 20:54:59 +00:00  rdope_out.txt
210928  -rw-               91  Oct 26 2021 20:54:58 +00:00  rdope.log
210926  -rwx             2049  Oct 26 2021 20:54:32 +00:00  svl_ipc.tcl
210925  -rw-           134458  Oct 26 2021 20:54:32 +00:00  memleak.tcl
210924  -rw-              396  Oct 26 2021 20:54:12 +00:00  boothelper.log
275854  drwx             4096  Oct 26 2021 20:54:10 +00:00  dc_profile_dir
210921  -rw-              111  Oct 26 2021 20:53:52 +00:00  bootloader_evt_handle.log
210914  drwx             4096  Jul 28 2021 10:09:06 +00:00  .prst_sync
210931  -rw-              396  Jul 28 2021 10:08:30 +00:00  boothelper.log.old
275811  drwx             4096  Jul 28 2021 10:05:39 +00:00  .rollback_timer
275889  -rw-             7715  Jul 28 2021 10:03:36 +00:00  packages.conf
275900  -rw-         47364227  Jul 28 2021 10:03:36 +00:00  cat9k-rpboot.17.03.03.SPA.pkg
275899  -rw-             9220  Jul 28 2021 10:02:52 +00:00  cat9k-wlc.17.03.03.SPA.pkg
275898  -rw-         14427140  Jul 28 2021 10:02:52 +00:00  cat9k-webui.17.03.03.SPA.pkg
275897  -rw-         28738568  Jul 28 2021 10:02:52 +00:00  cat9k-srdriver.17.03.03.SPA.pkg
275896  -rw-         57529348  Jul 28 2021 10:02:52 +00:00  cat9k-sipspa.17.03.03.SPA.pkg
275895  -rw-         34792456  Jul 28 2021 10:02:52 +00:00  cat9k-sipbase.17.03.03.SPA.pkg
275894  -rw-        595178500  Jul 28 2021 10:02:52 +00:00  cat9k-rpbase.17.03.03.SPA.pkg
275893  -rw-             5124  Jul 28 2021 10:02:52 +00:00  cat9k-lni.17.03.03.SPA.pkg
275892  -rw-          2262024  Jul 28 2021 10:02:52 +00:00  cat9k-guestshell.17.03.03.SPA.pkg
275891  -rw-        104428552  Jul 28 2021 10:02:52 +00:00  cat9k-espbase.17.03.03.SPA.pkg
275890  -rw-         17392652  Jul 28 2021 10:02:52 +00:00  cat9k-cc_srdriver.17.03.03.SPA.pkg
210915  drwx             4096  Jul 28 2021 10:01:17 +00:00  .dbpersist
275872  drwx             4096  Jul 28 2021 09:58:41 +00:00  onep
275861  drwx             4096  Jul 28 2021 09:58:07 +00:00  Tbot
275856  drwx             4096  Jul 28 2021 09:58:02 +00:00  sys_report
275825  drwx             4096  Jul 28 2021 09:58:01 +00:00  tech_support
275823  drwx             4096  Jul 28 2021 09:58:01 +00:00  ss_disc
210923  -rw-          5242880  Jul 28 2021 09:58:01 +00:00  ssd
275822  drwx             4096  Jul 28 2021 09:58:01 +00:00  gs_script
275820  drwx             4096  Jul 28 2021 09:58:00 +00:00  core
275810  drwx             4096  Jul 28 2021 09:57:43 +00:00  .ssh
210922  -rw-             1442  Jul 28 2021 09:51:59 +00:00  2021_07_28T09_51_13.157067.xml
210920  -rw-        901741775  Jul 28 2021 09:51:15 +00:00  cat9k_iosxe.17.03.03.SPA.bin

11250098176 bytes total (6952480768 bytes free)
Switch#
Switch#show install sum
[ R0 ] Installed Package(s) Information:
State (St): I - Inactive, U - Activated & Uncommitted,
            C - Activated & Committed, D - Deactivated & Uncommitted
--------------------------------------------------------------------------------
Type  St   Filename/Version
--------------------------------------------------------------------------------
IMG   C    17.03.03.0.4762
IMG   I    17.03.04.0.5557

--------------------------------------------------------------------------------
Auto abort timer: inactive
--------------------------------------------------------------------------------


Switch#

So when I run install add file , does the switch just unpack the .bin file into the various .conf and .pkg files, and thus the install for that version is in an "inactive" state? Looking again at the switch output from my first post above, it looks like the switch "added" the file and then it was going to "activate" the file but it requested my input before doing so. Am I correct in my thinking there?

So would all I need to do now is "activate" the added file? If so, what file would I activate, the .bin file or one of the .pkg files?

Also, how would I "commit" this install? ( I have to admit it is cool learning about how all this works :-)) )

0xD2A6762E

Leo Laohoo · ‎10-27-2021

Is this a single unit or a VSS?
If this is a stand-alone/non-VSS unit, do the following:

rename flash:packages.conf flash:packages.conf.00-
rename flash:cat9k_iosxe.17.03.04.SPA.conf flash:packages.conf

After 1 & 2 are done, post the complete output (1 output page only) to the following command: more flash:packages.conf | begin for CAT

Also post the complete output to the command "sh boot".

vv0bbLeS · ‎10-28-2021

@Leo Laohoo thank you, this is a standalone unit.

I may have jumped the gun, but yesterday before your reply I ran the install activate command, and after the reload I ran the install commit command, and now when I run a show install I see my 17.3.4 software is Activated and Committed. So I'm guessing my issue may be solved, but perhaps I was just lucky that the above commands worked, and that I didn't cause a bigger problem? Your method of renaming the .conf files makes me think I may have just been lucky LOL

0xD2A6762E

Leo Laohoo · ‎10-28-2021

For consistency purposes, different Cisco platforms have different behaviours.

The biggest issue with the "install" command is the inability to schedule the platform to reboot. As soon as the commit is entered, the platform will reboot.
The method I am using is a workaround.

vv0bbLeS · ‎02-15-2023

@Leo Laohoo I realize this thread is over a year old but I've ran into this issue again about "add failed," and I wanted to ask you about your neat workaround method above as I never asked you for the final steps of it. So if I were to use your workaround steps of:

rename flash:packages.conf flash:packages.conf.00-
rename flash:cat9k_iosxe.17.03.04.SPA.conf flash:packages.conf

Then per your instructions I would then (1) run "more flash:packages.conf | begin for CAT", and (2) look at the output of "show boot".

My question is, what would I be looking for in the "more" and "show boot" commands, and what would my next steps be to complete this workaround? And would I change anything if it is a VSS pair vs. a standalone unit? Basically looking for all the steps in this workaround so I can add this to my toolbox! : )

Thanks!

0xD2A6762E

Leo Laohoo · ‎02-15-2023

In the "more flash:packages.conf", start from the second "page" and confirm that the packages are the intended version, i. e. it does not contain the OLD version. See photo below:

In the "sh boot", the boot variable string should be pointing to "packages.conf" only and nothing else. Another thing to look out for in the "sh boot" output is the next line below, MANUAL_BOOT must be "no". See the photo below:

Please read the attached file. I have made the document easy-to-understand and I have thoroughly tested and applied the method for the last 5 years without a single failure.

vv0bbLeS · ‎02-16-2023

@Leo Laohoo wow thank you so much! That is excellent content, exactly what I needed! Thanks again!

0xD2A6762E

the-lebowski · ‎11-10-2023

@Leo Laohoo

Can you avoid rebooting when you add the new package by running this command?

install add file bootflash:cat9k_iosxe.16.12.01.SPA.bin

Then run install activate whenever you are ready to do reboot/install it? I ask because your post/document makes it seem like you can't avoid rebooting a stack when run the install add file command.

Reference doc: https://www.cisco.com/c/en/us/support/docs/switches/catalyst-9300-series-switches/216231-upgrade-guide-for-cisco-catalyst-9000-sw.html#toc-hId-236511669

Leo Laohoo · ‎11-10-2023

@the-lebowski wrote:

Can you avoid rebooting when you add the new package by running this command?

I have documented this.

About 5 minutes into the command, there process will pause to ask a "y/n" question. Hit "n" and the process stops.