Cisco 9800-80 HA upgrade from 17.9.3 to 17.12.4

[patrickroberts]

Hi All

Currently have a 9800-80 HA setup with approx 800 + APs (info below)

Wanting to upgrade from 17.9.3 to the latest version Dublin-17.12.4 MD due to bugs/issues .

I keep seeing issue with AP image memory and checking on the cisco site for any known pit falls, should i raise this with TAC first and ask for help on the upgrade ,with so many APs its a major outage if they don't come back on.

Has anyone done this recently who can share there thoughts with the AP pre download and controller upgrade

9800-80 (HA)

Cisco IOS XE Software, Version 17.09.03
Cisco IOS Software [Cupertino], C9800 Software (C9800_IOSXE-K9), Version 17.9.3, RELEASE SOFTWARE (fc6)

APs

C9130AXI-E

Version
Primary Software Version17.9.3.50
Predownloaded StatusN/A
Predownloaded VersionN/A
Next Retry TimeN/A
Boot Version1.1.2.4
IOS Version17.9.3.50
Mini IOS Version0.0.0.0

C9136I-ROW

Primary Software Version17.9.3.50
Predownloaded StatusN/A
Predownloaded VersionN/A
Next Retry TimeN/A
Boot Version1.1.2.4
IOS Version17.9.3.50
Mini IOS Version0.0.0.0

[marce1000]

   >....I keep seeing issue with AP image memory
               - What issue are you talking about ?

 M.

[patrickroberts]

APs running Cisco IOS-XE 17.9.3 or later can encounter issues when attempting to upgrade their software due to insufficient space in the /tmp directory. When the /tmp space on the AP becomes full, it prevents the download of the new AP image. In such instances, it is recommended that you reboot the AP.

[patrickroberts]

APs running Cisco IOS-XE 17.9.3 or later can encounter issues when attempting to upgrade their software due to insufficient space in the /tmp directory. When the /tmp space on the AP becomes full, it prevents the download of the new AP image. In such instances, it is recommended that you reboot the AP.

[marce1000]

   = Not  being offensive but I have never heard of /tmp on access points, is this from official Cisco information ?

  M.

[patrickroberts]

I’m reading this Cisco document?

https://www.cisco.com/c/en/us/support/docs/wireless/catalyst-9800-series-wireless-controllers/216838-upgrade-and-downgrade-of-catalyst-9800-c.html

[marce1000]

 - Ok , personally it's one of those issues where my attitude would be : let's wait and see
   More interesting according to  me is a checkup of the controller before and after with the
   CLI command show tech wireless (not 'show tech')  and feed the output from that into Wireless Config Analyzer

    Check for instance if any of the advisories can impact upgrading and after the upgrade ; check if there
    are any new advisories.   

  M.

[Rich R]

You're right - there have been a number of bugs relating to AP /tmp running out of space for various reasons:
https://bst.cloudapps.cisco.com/bugsearch/bug/CSCwm07499
https://bst.cloudapps.cisco.com/bugsearch/bug/CSCwe15172
https://bst.cloudapps.cisco.com/bugsearch/bug/CSCwe97901
https://bst.cloudapps.cisco.com/bugsearch/bug/CSCwm72142
In most cases the workaround is to reload the AP so if you want to avoid the problem simply reload all the APs before starting your upgrades.

[patrickroberts]

Hi Rich thanks for the reply , for the AP reload did you pre download the image to the APs then on the upgrade reload them all and then start the WLC upgrade ?

[Rich R]

No you'll need to reload the AP before you do the pre-download otherwise the pre-download could fail because of the full /tmp partition.

The APs always reload when the WLC is upgraded because they need to switch to the new code version which can only be done with a reload.

[patrickroberts]

 i see ok we just had a outage last week and all the APs reloaded so would there be a way to check a AP full/tmp status ? or is it a case of the day/night we do the upgrade is to reload them before hand and once back on start the normal HA upgrade 

[Rich R]

If you want to check them you can login to the AP and use this or just reload before you start:
show filesystems

[Leo Laohoo]

---

@patrickroberts wrote:
I keep seeing issue with AP image memory and checking on the cisco site for any known pit falls

---

Use Hitless Upgrade.  If the Cheetah OS APs are moved manually, there is a strong (>60%) chance they will hit the "/tmp" bugs @Rich R has enumerated.  Rebooting the APs is the only solution.  

I know this because we have been using Hitless Upgrade but I can sometimes "not" follow instructions and when I do it "my way", I hit the bugs.  

[Leo Laohoo]

FN74222 - Full or Partial Cisco 9800 Series Wireless Controller Configuration Loss after High-Availability Stateful Switchover Failover