C9800-L-F: Upgrade fails 'Route processor type (qwlc) not valid'

[PErampas]

I'm trying to upgrade a C9800-L-F controller from 17.3.3 to 17.3.4c using ISSU.
The controller hardware is:
cisco C9800-L-F-K9 (KATAR) processor (revision KATAR) with 1702951K/6147K bytes of memory.
The software I'm using is:
C9800-L-universalk9_wlc.17.03.04c.SPA.bin

I've tried ISSU from CLI and WebUI, the result is:

 --- Starting candidate package set construction ---
  Verifying existing software set
  Processing candidate provisioning file
  Production Image
  ERROR: Route processor type (qwlc) not valid for this route processor (KATAR)
  Failure processing package C9800-mono-universalk9_wlc.17.03.04c.SPA.pkg
  background verification failure. rc: 1. see /tmp/cs_debug32183.txt

Why is this failing because of an invalid route processor?
I double checked the downloaded software, I should have the right image.

There is a newer Rommon  16.12(3r) available, the controller is running 16.12(1r).
Could this be the problem?

[marce1000]

 >...There is a newer Rommon  16.12(3r) available, the controller is running 16.12(1r). >...Could this be the problem?

   - I would at least give it a try ; when you reload the controller there should be a line similar too :

               QWLC-1GL platform with 33554432 Kbytes of main memory 

 What is shown in your case ? You may also connect to the controller with (SSH) : https://cway.cisco.com/cli , at the upper left run System Diagnostics , see if anything comes  up. I further think you may need to escalate this (TAC) Also note : https://bst.cloudapps.cisco.com/bugsearch/bug/CSCvs01135?rfs=iqvred (this is probably not related to your case).

 M.

[Leo Laohoo]

Do not use ISSU in the upgrade.

[Rich R]

All the TAC calls I've been on they've said upgrading firmware on 9800L is an absolute prerequisite to avoid any number of problems including some serious ones so you should be checking ROMMON and ethernet PHY are updated before anything else.

Did you read all the following before you started?

https://www.cisco.com/c/en/us/support/docs/wireless/catalyst-9800-series-wireless-controllers/214749-tac-recommended-ios-xe-builds-for-wirele.html#anc25

https://www.cisco.com/c/en/us/td/docs/wireless/controller/9800/17-3/config-guide/b_wl_17_3_cg/m_issu.html

https://www.cisco.com/c/en/us/td/docs/wireless/controller/9800/config-guide/b_upgrade_fpga_c9800.html

https://www.cisco.com/c/en/us/td/docs/wireless/controller/9800/17-3/release-notes/rn-17-3-9800.html

Note ISSU is only supported in install mode - is your 9800 already running in install mode?  If not you'd need to convert to install mode before attempting ISSU so you might as well just do a regular installation in install mode then next time ISSU would be possible.

[Scott Fella]

You have a lot of information here... but some of these, you could have tested and validated.  The upgrade without ISSU is one, upgrading the ROMMON is the other.  Performing these test on your own will make you a better engineer and really will help you understand troubleshooting issues specific to upgrading rather than try one thing.  I'm a fan of testing and documenting what is working and whats not working and really reaching out when I have exhausted my resources.  It's fine to ask the question, but trying things always helps me learn better.

[PErampas]

I have upgraded Rommon to '16.12(3r)', but the issue is still there.
I checked beforehand, if any Rommon upgrade or other prerequisites are required, but there is nothing mentioned in the release note.
The system is ISSU ready and all checks during ISSU Stage 0 pass. The issue happens during Stage 1.

@marce1000
Cisco CLI Analyzer doesn't show any issues.
During reload, I can only see:

C9800-L-X-K9 platform with 16777216 Kbytes of main memory

What is "QWLC" and "KATAR"? I tried googling it in a Cisco WLC context, but I can't find anything pointing to specific hardware platforms or product codes.

[marce1000]

Try upgrading without ISSU.
M.

[Rich R]

KATAR seems to be the Cisco internal codename for the 9800L platform.  They usually have codenames like that for platforms and modules from early development stages and those names often stick even after the product is released and has an official PID.

If all firmware is up to date then your next port of call is Cisco TAC.

For interest what does it say in that /tmp/cs_debug32183.txt file? (it might give more detail)

[marce1000 · Accepted solution]

 - Ref : https://www.cisco.com/c/en/us/td/docs/wireless/controller/9800/17-1/config-guide/b_wl_17_11_cg/b_wl_17_11_cg_chapter_010000110.html#id_128296

 >...

ISSU is supported only on the following Cisco Catalyst 9800 Series Wireless Controllers, and supports upgrade and downgrade.

• Cisco Catalyst 9800-80 Wireless Controller

• Cisco Catalyst 9800-40 Wireless Controller

• Cisco Catalyst 9800-L Wireless Controller

• Cisco Catalyst 9800-CL Wireless Controller (Private Cloud)

                  From that I conclude that your model is not supported for ISSU

 M.

[Scott Fella]

ISSU is not supported on engineering special releases and .s (or similar) images.  You can go from 17.3.3 to 17.3.4 but not to 17.3.4c.  This is why we said to not use ISSU to upgrade to that release.

ISSU is supported between minor versions in EM and between EM.  ISSU is not supported from EM to SM or between SM version and to special release versions.

[Scott Fella]

Keep in mind.  When you look at the downloads for a device, look at the end of support.  This will tell you which major release is EM (Extended Maintenance) and SM (Standard Maintenance).

Take a look at the 17.x releases that call our the EM and SM:

Extended Fast Software Upgrade Quick Reference Guide (cisco.com)

[Rich R]

@Scott Fella I think Extended Fast Software Upgrade only applies to switches?

And I seem to recall noticing that it requires DNA advantage license.

[Scott Fella]

@Rich R I just wanted to show the EM and SM versions.  This was the best link I found to show that.  Yes.... it also depends on your DNA license you buy and what you get as perpetual after DNA Advantage expires.  May of these features like ISSU and rolling upgrades do require Advantage.

[Rich R]

17.3.5 will be out soon so if you're hellbent on using ISSU you might want to wait for that.

[PErampas]

Ok, so it seems ISSU with the Catalyst 9800-L-F simply isn't supported (at least now), because of the KATAR processor.

Would have been nice if this came out during "Stage 0: System Level Sanity Checks" of the ISSU and not past that in "Stage 1:Installing software on Standby".
Luckily, there is no outage during rollback.

ISSU from 17.3.3 to 17.3.4c on a 9800-40 worked perfectly without issues.

Upgrading the 9800-L-F without ISSU worked also, but I really would prefer ISSU.