07-24-2018 02:44 AM - edited 07-05-2021 08:53 AM
I recently attempted to upgrade our WLC environment (5508 and 15+ 2602 APs) and hit a few issues and was wondering if anyone else has encountered anything similar?
My starting point was 7.3.112.0 on our WLC with a planned upgrade path of:
7.3.112.0 > 7.4.150 > 8.0.152.0 > 8.3.143.0 > 8.5.131.0
In case of issues and to allow a swift rollback, I powered off all APs except 2 during each upgrade hop
I updated FUS to 1.9.0.0 first and then WLC to 7.4.145.0 which was fine.
I then updated the WLC to 8.0.152.0.
When the 2 test APs attempted to update, they bricked with a red flashing LED.
Console access to the devices showed the following output:
--------------------------------------------
IOS Bootloader - Starting system.
flash is writable
FLASH CHIP: Numonyx Mirrorbit (0089)
Xmodem file system is available.
flashfs[0]: 13 files, 5 directories
flashfs[0]: 0 orphaned files, 0 orphaned directories
flashfs[0]: Total bytes: 31997952
flashfs[0]: Bytes used: 12638208
flashfs[0]: Bytes available: 19359744
flashfs[0]: flashfs fsck took 16 seconds.
Reading cookie from SEEPROM
Base Ethernet MAC address: 4c:4e:35:03:4b:fb
Ethernet speed is 1000 Mb - FULL Duplex
Loading "flash:/ap3g2-k9w8-mx.152-2.JB6/ap3g2-k9w8-mx.152-2.JB6"...flash:/ap3g2- k9w8-mx.152-2.JB6/ap3g2-k9w8-mx.152-2.JB6: no such file or directory
Error loading "flash:/ap3g2-k9w8-mx.152-2.JB6/ap3g2-k9w8-mx.152-2.JB6"
Interrupt within 5 seconds to abort boot process.
Loading "flash:/update/ap3g2-k9w8-mx.153-3.JA12/ap3g2-k9w8-mx.153-3.JA12"...#### #####################
File "flash:/update/ap3g2-k9w8-mx.153-3.JA12/ap3g2-k9w8-mx.153-3.JA12" uncompressed and installed, entry point: 0x2003000
executing...
Secondary Bootloader - Starting system.
Tide MB - 32MB of flash
Xmodem file system is available.
flashfs[0]: 13 files, 5 directories
flashfs[0]: 0 orphaned files, 0 orphaned directories
flashfs[0]: Total bytes: 31997952
flashfs[0]: Bytes used: 12638208
flashfs[0]: Bytes available: 19359744
flashfs[0]: flashfs fsck took 9 seconds.
flashfs[1]: 0 files, 1 directories
flashfs[1]: 0 orphaned files, 0 orphaned directories
flashfs[1]: Total bytes: 12257280
flashfs[1]: Bytes used: 1024
flashfs[1]: Bytes available: 12256256
flashfs[1]: flashfs fsck took 0 seconds.
Base Ethernet MAC address: 4c:4e:35:03:4b:fb
2600/3600 AP, PID: 'AIR-CAP2602I-E-K9 '. Checking for BL upgrade...
BL: C2600 Boot Loader (AP3G2-BOOT-M) LoaderVersion 12.4(25e)JA1, RELEASE SOFTWARE (fc1)
Technical Support: http://www.cisco.com/techsupport
Compiled Wed 30-May-12 03:34 by binugopa
BL Build year: 12
De-compress 'flash:/ap3g2-k9w8-mx.152-2.JB6/ap3g2-bl-2600' to 'ram:/ap3g2-bl'...
File not found: flash:/ap3g2-k9w8-mx.152-2.JB6/ap3g2-bl-2600
Boot CMD: 'boot flash:/ap3g2-k9w8-mx.152-2.JB6/ap3g2-k9w8-xx.152-2.JB6;flash:/update/ap3g2-k9w8-mx.153-3.JA12/ap3g2-k9w8-xx.153-3.JA12'
Loading "flash:/ap3g2-k9w8-mx.152-2.JB6/ap3g2-k9w8-xx.152-2.JB6"...flash:/ap3g2-k9w8-mx.152-2.JB6/ap3g2-k9w8-xx.152-2.JB6: no such file or directory
Error loading "flash:/ap3g2-k9w8-mx.152-2.JB6/ap3g2-k9w8-xx.152-2.JB6"
Interrupt within 5 seconds to abort boot process.
Unable to locate IOS image with name **xx**.
Boot CMD: 'flash:/update/ap3g2-k9w8-mx.153-3.JA12/ap3g2-k9w8-xx.153-3.JA12'
Loading "flash:/update/ap3g2-k9w8-mx.153-3.JA12/ap3g2-k9w8-xx.153-3.JA12"...###################
Error loading "flash:/update/ap3g2-k9w8-mx.153-3.JA12/ap3g2-k9w8-xx.153-3.JA12"
Interrupt within 5 seconds to abort boot process.
Boot process failed...
The system is unable to boot automatically. The BOOT
environment variable needs to be set to a bootable
image.
C2600 Boot Loader (AP3G2-BOOT-M) LoaderVersion 12.4(25e)JA1, RELEASE SOFTWARE (fc1)
Technical Support: http://www.cisco.com/techsupport
Compiled Wed 30-May-12 03:34 by binugopa
-------------------------------
This appears to be IOS corruption. I was eventually able to recover the AP’s by dismounting them, and factory resetting them via the mode button but, as my maintenance window had then expired, I rolled back the WLC and AP’s to 7.4.150.
Has anyone else encountered issues with 2602’s when going from 7.4.150.0 > 8.0.152.0?
I have looked into maybe pre downloading the AP image to the APs instead to see if this helps, but this isn’t supported for this upgrade hop (7.4.150 > 8.0.152.0) as per this document:
If you upgrade from a release that is prior to Release 7.5 directly to Release 7.6.X or a later release, the predownload process on Cisco AP2600 and AP3600 fails. After the Cisco WLC is upgraded to Release 7.6.X or a later release, the new image is loaded on Cisco AP2600 and AP3600. After the upgrade to a Release 7.6.X image, the predownload functionality works as expected. The predownload failure is only a one-time failure.
Solved! Go to Solution.
07-24-2018 03:10 AM
@PJR_CDF wrote:
Loading "flash:/update/ap3g2-k9w8-mx.153-3.JA12/ap3g2-k9w8-mx.153-3.JA12"...####
The controller wasn't finish with the pre-download when the AP was rebooted.
Console into the AP and post the complete output to the command "dir flash:".
07-24-2018 03:10 AM
@PJR_CDF wrote:
Loading "flash:/update/ap3g2-k9w8-mx.153-3.JA12/ap3g2-k9w8-mx.153-3.JA12"...####
The controller wasn't finish with the pre-download when the AP was rebooted.
Console into the AP and post the complete output to the command "dir flash:".
07-24-2018 03:14 AM
Flash Dir below - this is post recovery of the AP
Directory of flash:/
2 -rwx 77777 Jul 22 2018 18:13:27 +00:00 event.log
3 -rwx 64 Mar 1 1993 00:01:05 +00:00 sensord_CSPRNG1
4 -rwx 7192 Jul 24 2018 10:12:31 +00:00 private-multiple-fs
5 -rwx 64 Mar 1 1993 00:01:03 +00:00 sensord_CSPRNG0
45 drwx 0 Mar 1 1993 00:00:25 +00:00 configs
46 -rwx 0 Jan 30 2018 09:58:40 +00:00 ce
7 drwx 960 Jul 22 2018 18:13:17 +00:00 ap3g2-k9w8-mx.152-2.JB6
51 -rwx 380 Jul 23 2018 07:30:18 +00:00 env_vars
13 drwx 192 Feb 4 2013 13:18:02 +00:00 ap3g2-rcvk9w8-mx
31739904 bytes total (13079552 bytes free)
07-25-2018 01:36 AM
07-25-2018 01:43 AM
Thanks for your response - I resolved the issue by using the Mode button to reset the AP's (which presumably boots the AP into the RCV image?)
Now I know the cause of the issue is related to a reboot of the AP mid update rather than an upgrade path bug/incompatibility, I can schedule another window and ensure the APs are not rebooted during their upgrade.
Out of interest if an AP updates to 8.0.152 and for some unforeseen reason I need to rollback the WLC to 7.4.145.0, will the AP downgrade its software gracefully automatically in the same way they upgrade?
07-25-2018 02:05 AM
07-25-2018 02:20 AM
@PJR_CDF wrote:
Out of interest if an AP updates to 8.0.152 and for some unforeseen reason I need to rollback the WLC to 7.4.145.0, will the AP downgrade its software gracefully automatically in the same way they upgrade?
Correct.
Discover and save your favorite ideas. Come back to expert answers, step-by-step guides, recent topics, and more.
New here? Get started with these tips. How to use Community New member guide