C9800 IOS upgrade stuck in Upgrading standby

[Ram1100]

Hi can someone help me on this?

I'm upgrading the IOS the WLC - C9800-CL-K9 in ISSU mode and it got stuck in  Upgrading standby. Please suggest.

Aborted once and tried again, same issue now.

Please refer to the attached image.

[Leo Laohoo]

conf t
 service internal
end
clear install state

[Ram1100]

i have already tried that, this is the 3rd attempt to upgrade, again and again it getting stuck at the same place.

[Leo Laohoo]

Stop the ISSU upgrade and stop using ISSU to upgrade. 

3 failed attempts at ISSU?  I would've stopped at the 2nd failed attempt.  

[Ram1100]

will it cause the down time if we upgrade without ISSU?

Generally how long it will take to upgrade the WLC?

[Scott Fella]

Upgrading a controller, you should always plan for a downtime, because you never know what will happen.  If things go well, I would say 5-15 minutes as long as you don't have to push the image to the ap's.  Every environment is different especially if the controller is not local to a site and the transfer might take longer.

Since you are having problems already, open a proactive TAC case so they can be available when you plan your upgrade and help in case something goes wrong.

[Leo Laohoo]

---

@Ram1100 wrote:

will it cause the down time if we upgrade without ISSU?

Generally how long it will take to upgrade the WLC?

---

Look at what is happening right now.  3 failed ISSU attempts and going on to 4th failed attempts.  

We did N+1 Hitless Upgrade and the hospital never knew when we started at 10:00 in the morning during business hours until we finished around 16:00 that afternoon.  The hospital staff and patients did not experience any outage.  

It is highly recommended to raise a Proactive TAC case before embarking on an ISSU upgrade.  

[Ram1100]

Thanks for the suggestion,

This is the current state, The new image that I'm going to upgrade is in inactive state

[ Chassis 1/R0 ] Installed Package(s) Information:
State (St): I - Inactive, U - Activated & Uncommitted,
C - Activated & Committed, D - Deactivated & Uncommitted
--------------------------------------------------------------------------------
Type St Filename/Version
--------------------------------------------------------------------------------
IMG C 17.06.05.0.5797
IMG I 17.09.06.0.7681

--------------------------------------------------------------------------------
Auto abort timer: inactive
--------------------------------------------------------------------------------

[Leo Laohoo]

We've been told (by TAC) not to perform ISSU from 17.3.X, 17.6.X and 17.9.X because there is guaranteed failure rate is in the high 80%.

[balaji.bandi]

Sure and Agreed, i am not 100% confident ISSU works all the time.

[Rich R]

> This is the current state, The new image that I'm going to upgrade is in inactive state
install remove ... the 17.09.06 image and then use the command Leo already provided to clear the corrupted install database.

Then upgrade without ISSU as others have already suggested.  You can still pre-download the APs to minimise upgrade time.  Some APs will still re-download even after pre-download (one of the many bugs).  Note that after upgrading to 17.9.6 you will still need to install the latest APSP - refer to the TAC recommended link below.  That will require all the APs to reload.  Note that there is a bug in the GUI which prevents pre-download of APSP to APs - that can only be done from CLI.

> will it cause the down time if we upgrade without ISSU?
Less than what you have already caused with 3 attempts at ISSU followed by 3 install database clears (each resulting in a full reload).

> Generally how long it will take to upgrade the WLC?
If you pre-download the APs 15 minutes at most, usually 5-10 minutes.

If you decide to try ISSU again (not recommended since I'd say you have a 100% chance of it failing again) then make sure you have TAC on WebEx with you before you start. If/when it fails they will just tell you to do what Leo has already told you.  Like he said they already know that it is expected to fail.  If you have already covered all the warnings and precautions covered in the release notes https://www.cisco.com/c/en/us/td/docs/wireless/controller/9800/17-9/release-notes/rn-17-9-9800.html#id_136398 + https://bst.cloudapps.cisco.com/bugsearch/bug/CSCwk97948 and it still failed 3 times then unlikely that it will ever succeed.