Upgrade IOS-XE image pending Reboot.

Charlie Grey · ‎03-23-2020

for ios-xe, is it possible to be like IOS that we can do all the preparation works (pre-upload the image + setup the boot variable) and only at deseriable time then we proceed with the reboot?

reason for asking is i will like to reduce the downtime needed for the upgrade to just the reboot duration.

i notice when doing the ios-xe upgrade, i need to upload the image, run the command to install and the entire procedure must be perform all the same time.

Leo Laohoo · ‎03-23-2020

Depends on the hardware but yes.
I have done it many times, unpack the firmware and schedule the appliance to reboot at, say, 4:00 am the next day.

Charlie Grey · ‎03-25-2020

Hi,

can share wat platform and how to do it?

i did the upgrade on a C9200.

seem that after i launched the 'install add file' command, there was only one prompt to that ask if u want to proceed. IF u choose 'y', the switch will reboot and upgrade will be successful.

if u choose 'n', upgrade will be aborted and relaunching the same command resulted in an error.

i will need to run ‘install remove inactive’ to remove all previous extracted .pkg files and initiate the entire process all over again.

please share how u unpack and do the install and wat command to run to initiate the reboot to carry on from where u left off. thanks.

Switch# install add file flash:cat9k_iosxe.16.06.02.SPA.bin activate commit

....

This operation requires a reload of the system. Do you want to proceed? [y/n] y

NathanBailey44536 · ‎08-21-2020

Found this while working on similar and wanted to add the steps I used. There may be a better way, if someone with more knowledge can respond.

The process is as follows:

1. copy file to local flash
2. verify checksum of file
3. install add file flash:ios.version.whatever.bin
4. install activate | This is your reload step and you will be prompted as such.
5. install commit
6. install remove inactive.

After step 3 you can come back at anytime and perform step 4. I have yet to find something similar to "reload at time date" or "install activate /force at time date". IMHO it's a missing and necessary feature for parity and usability.

HTH.

Leo Laohoo · ‎08-21-2020

For some insane reason there is no way to "schedule" an firmware upgrade on 9200/9200L and 9500 high-performance switches. The "only" way is to use a very expensive system called DNAc.
Unlike the 9300 and "regular" 9500, the firmware can be unpacked with the available option of "on-reboot" which actually means "no-reboot" (`tis a "bug": CSCve94966).
The "install" command does not have this same option.
Like I said, the only way is to use DNAc.

jedavis · ‎03-02-2021

I am upgrading a 9300 from 12.16.3 to 12.16.5 and I too want to separate the install from the reboot. Unfortunately I am am doing this on a production switch so I can't play around with it as much as I want to. But I think this is what is happening.

Switch#install add file flash:/cat9k_iosxe.16.12.05.SPA.bin
This just does some validation, unbundles the .pkg files and creates a new cat9k_iosxe.16.12.05.SPA.conf file.

Switch#install activate file flash:/cat9k_iosxe.16.12.05.SPA.bin
This is going to copy the cat9k_iosxe.16.12.05.SPA.conf file to packages.conf and do a reload. So why can't I just do this manually?

Rename the packages.conf file file to cat9k_iosxe.16.12.03.SPA.conf
rename cat9k_iosxe.16.12.05.SPA.conf to packages.conf
reload

Would that be a valid upgrade procedure? What else does the install activate process do? There is a rollback timer that is set somewhere so that if install commit is not entered in 120 minutes (default) it reverts to the previous image. But what else? It would be nice to have some more comprehensive documentation. The release notes actually instruct you to issue a reload command after the install activate is done. This document seems like it was written by someone while actually performing the process and there is no mention of a separate reload step:

https://www.cisco.com/c/en/us/support/docs/switches/catalyst-9300-series-switches/216231-upgrade-guide-for-cisco-catalyst-9000-sw.html

What have I gotten wrong?

Leo Laohoo · ‎03-02-2021

9300 can be done. The install and the reboot can be "separate" but you'll need to use a different command:

request platform software package install switch all file flash:cat3k_caa-universalk9.16.12.05.SPA.bin ON-REBOOT new auto-copy verbose

Like I've mentioned above, the 9300 and "regular" 9500 firmware can be unpacked without any reboot using the option of "on-reboot" which actually means "no-reboot" (`tis a "bug": CSCve94966).

Hope this helps.

NOTE: Yes, I take great pleasure highlighting the "on-reboot" bug and reluctance to fix it.

jedavis · ‎03-02-2021

Thanks for the reply Leo.

I realize that the install and the reboot can be done in separate steps. I guess my question is, after the install, is manipulating the packages.conf file a valid way to proceed? By replacing the packages.conf file with the one from the new version it would allow you to do a "reload at XX:XX" command for an unattended reboot.

And to be fair the request platform software commands are deprecated starting from IOS XE Gibraltar 16.10.1 so there is not much point in fixing it.

https://www.cisco.com/c/en/us/td/docs/switches/lan/catalyst9300/software/release/16-12/release_notes/ol-16-12-9300.html#id_67619

Leo Laohoo · ‎03-02-2021

@jedavis wrote:

I guess my question is, after the install, is manipulating the packages.conf file a valid way to proceed? By replacing the packages.conf file with the one from the new version it would allow you to do a "reload at XX:XX" command for an unattended reboot.

I am glad this question is asked.

Whatever method is used to "unpack" the BIN file, the automated method is meant to change the "packages.conf" file to point to the new package files, however, I have seen times where the script will say either one of the following:

1. The packages have been successfully unpacked and packages.conf file has been changed to reflect this, HOWEVER, double-checking shows the packages.conf file is still pointing to the old package files.

2. The packages have been successfully unpacked, however, the packages.conf "could not be changed" so the system has created another file. And if the operator is paying attention to this error message he/she must rename the file to "packages.conf".

Where I work, I have made it a point to double-check the contents of the packages.conf file to make sure it is pointing to the correct package files. The command to do the check is simple:

more flash-X:packages.conf

The command above must be checked in every switch member of the stack. No one can afford for one stack member to be booting into a different firmware version.

Make sure the contents all point to the new package files.

Also check the boot-variable string is pointing to the "packages.conf" file.

Leo Laohoo · ‎03-02-2021

@jedavis wrote:

And to be fair the request platform software commands are deprecated starting from IOS XE Gibraltar 16.10.1 so there is not much point in fixing it.

And the other command does not have a "no-reboot" option either. And the developers are not entertaining the notion of adding this feature.

benjamin.lowe.3 · ‎09-09-2022

The keyword "on-reboot" is correct syntax because it's telling the switch to install the new software "on the next reboot" rather than now. Nothing is actually installed yet, it's just pre-staged. The installation actually occurs "on-reboot".

Leo Laohoo · ‎09-09-2022

@benjamin.lowe.3 wrote:
The keyword "on-reboot" is correct syntax because it's telling the switch to install the new software "on the next reboot" rather than now.

No, that is not what "on-reboot" means. Look at it. Type the command and enter the "?". It specifically states "do not reboot". The developers made a type-o and are refusing to fix it.

benjamin.lowe.3 · ‎09-12-2022

If I log into any of my devices running IOS-XE ranging from 16.3.6 to 17.6.4, typing the "?" after the file prompt shows "on-reboot" with the description "Delay effects until RP reboot", which I find to be consistent with my description of "on-reboot" above, meaning "on the next reboot".

The bug report you referenced is for 16.3.3, which I unfortunately do not have on hand, but the report shows the description of "Suppress reload prompt and do not reload after installation". I agree this is inconsistent with "on the next reboot".

If you perform a verbose installation with the reboot suppressed, you see that all that happens is the software is unpacked and a new packages.conf file is created, pointing to the new .pkg files that have been unpacked. Any kind of microcode update or other installation does not happen until the device is rebooted.

So yes, when first implemented, the command and description were not consistent with each other, but it appears to be the description that was wrong, not the command. And now it looks like they've fixed it at least as early as 16.3.6.

jocelyngallant · ‎10-18-2023

Removed response: Wrong thread.

FK85 · ‎10-24-2023

@jocelyngallant

"it is possible to set the prompt-level to none at the end of a install command and it does not ask you yes/no for any questions (normally at the end of the commit command of the upgrade, it asks you if you want to reload). If you set the prompt-level to none, the switch reloads as soon as it is ready to do so."

https://www.cisco.com/c/en/us/support/docs/switches/catalyst-9300-series-switches/216231-upgrade-guide-for-cisco-catalyst-9000-sw.html

That option does exactly the opposite. We are searching for a way to install and commit the new version WITHOUT reloading the switch immediately. That way you could do all the preparatory work during production and when the maintenance window started you only need to reload the switch.