Cisco Catelyst 9800-CL image upgrade

[Sjalle]

Hello everyone  

I hope there is someone there maybe can answer on a issue i have with the image upgrade. 

We are trying to upgrade our WLC with the "ISSU Upgrade(HA Upgrade)" ON, so we dont have so much downtime while we are upgrade it. But when the image has been predownloaded we hit a error.

We dont understand why, because when we research on the internet its say the version we have now can be upgraded to 17.06 version. 

From the log: 

--- Verifying install_issu supported ---
--- Verifying standby is in Standby Hot state ---
--- Verifying booted from the valid media ---
--- Verifying AutoBoot mode is enabled ---
--- Verifying Platform specific ISSU admission criteria ---
--- Verifying Image ISSU Compatibility ---
ERROR: Added image 17.06.03.0.3629 is not compatible with running image, exiting ISSU
FAILED: Image ISSU Compatibility
FAILED: install_activate exit(1) Thu Aug 25 08:05:18 CET 2022

[Leo Laohoo]

Never, ever, believe in the hype about ISSU, FSU/eFSU.  

ISSU, FSU/eFSU do not (always) work in production -- It will work in a "lab" environment but, in the wild, it normally fails most of the time. 

Are the APs located in sites which are 24x7?

[Sjalle]

Hello  

Yes, the APs run 24/7. its a production environment 

[Arshad Safrulla]

From which code to which code are you doing ISSU? Base image must support ISSU and your WLC pair must be in SSO HA. ISSU feature is supported only within and between major releases, for example, 17.3.x (within a release) and 17.3.x to 17.6.x (among major releases)

As of today you can do ISSU only from 17.3.X to 17.6.x or 17.9.x or withing these release only.

Is your switch IOS running in install mode or bundle mode? Both must be in isntall mode for it to work. You can use show version and show boot to verify this.

As Leo mentioned I have tested this in my Lab with positive results, but never on production as I never had successful ISSU upgrades with other Cisco devices (not 9800).

[Sjalle]

We want to upgrade from version 17.4.1 to version 17.06.03.0.3629. And as we can read from Cisco there should not be a problem to upgrade from that version to the other one? - But maybe its a problem because we want it with "Issu".  

[Arshad Safrulla]

ISSU not supported in 17.4.1. You must do an distruptive upgrade. What is your HA setup and how many total AP’s you have.

[Rich R]

Even in lab I've had odd errors trying to use ISSU even when it should be 100% supported - all requirements met but still throws errors.  I haven't got time to waste on a TAC case - we just have a brief outage when we do the upgrade.  We have dual SSO pairs of WLC so we rely on that when doing upgrades - APs fail over to the other WLC pair.  And when ISSU fails it tends to leave you with a mess of install state which can sometimes be really difficult to undo! 
You didn't say what version you were trying to upgrade from so we can't say whether the error you got was correct or not?  @Arshad Safrulla has detailed the minimum criteria to meet for ISSU and you'll find those in the config guide and release notes too.
If you've confirmed that you definitely meet all the criteria and still get the error then you'll have to open a TAC case and work through it with TAC.
As @Leo Laohoo says long experience with Cisco's various attempts at ISSU on different platforms over the years is that it rarely works as well as the demo's they show you, so every network I've worked on has been designed to provide availability without relying on ISSU features because they've never been reliable enough for a real 24x7x365 environment.
9800 looked very impressive in the early demo's by the BU and TAC but reality has proved to be quite disappointing so far.

One small thing to look out for - if you've ever had a SMU installed in the past - you must uninstall that and remove it because even though it doesn't show anywhere in the install info it's still hidden and lurking there in the install registry (even after the file is gone you'll see console logs about it on booting) and it can interfere with future installs forever after.

[Sjalle]

Hello  

We have version 17.4.1 right now and want to upgrade to 17.06.03.0.3629. 

[Leo Laohoo · Accepted solution]

Ok, I have a way to do a one-shot upgrade and will include an AP Pre-Download.  Here are the steps:
 
1.  Clean up the WLC flash using the command "install remove inactive".
2.  Download the firmware into the WLC. 

NOTE:  No need to verify the MD5.  If the file is corrupt, the next stage will fail.

3.  Unpack the files:  install add file bootflash:filename.bin activate commit

IMPORTANT: 
Keep your hands off the keyboard.  About two minutes into this process, there will be a "y/n" question.  Press "N".  (DO NOT PRESS "Y" or the WLC will reload prematurely.)

4.  After pressing "N", the process will stop with an error message.  Do not worry about this error message because it is still part of a "process".  
5.  Look into the WLC flash for files that have an extension of "conf":  dir bootflash:*.conf
6.  Rename the existing packages.conf file:  rename bootflash:packages.conf bootflash:packages.conf.00-
7.  Remember Step 5?  One of those "conf" files was created with the new firmware was unpacked.  Rename that file to be the new packages.conf file:  rename bootflash:filename.conf packages.conf
8.  Make sure the boot variable string is pointing to the packages.conf file:  sh boot | include BOOT variable
9.  Initiate AP Predownload:  ap image predownload
10.  Wait for the APs to finish pre-downloading by checking the progress:  sh ap image
11.  After the APs have finished pre-downloading, save the config of the WLC and reboot. 

NOTE:  Alternatively, in Step 11 it is also possible to reboot the WLC at a later time-and-date.