2800 APs and 3800 Controller - Upgrade problem

[AlfianZaini]

Hi Guys,

I have an issue during performing WLC & AP upgrade via 3800 Mobility Express GUI. 

During the upgrade, it seems like the upgrade failed (based on the screenshot attached). Im not too sure is it related with tftp blocksize or whatelse? because based on my observation, from the TFTP the process kept restarting after reaching approximately 30%. This cycle continued repeatedly for the entire 12-hour process.  I’ve attached the updated logs from TFTP for your further reference. Any ideas how to solve this?

[Flavio Miranda]

@AlfianZaini 

 Logs are not that helpful. The only thing  I can think of is some issue with the TFTP server. Maybe you can try another one.  You can also try to transfer using CLI to make sure the problem is not related to browser. 

[eglinsky2012]

Has performing the upgrade in this way ever worked for you, or is this your first time attempting it? If it's worked previously, is there anything different about the setup or the network between the APs and the TFTP server?

Are there any site to site VPNs or lossy/congested links between the AP and the TFTP server? How is the wiring between the APs and the switch ports and the TFTP server and the switch port, any CRC errors from either side?

Are you able to use FTP rather than TFTP? FTP uses TCP rather than UDP, so will be more resilient if you're transferring over a link that is susceptible to loss or packet fragmentation.

On second glance, I see the interface you're using on your TFTP server program is a FortiNet virtual adapter, so I assume that's a VPN and the computer it's running on is at a different site than the APs. Try using a local TFTP server that the APs can reach directly without a VPN, and/or using FTP.

[AlfianZaini]

thank you sir. I have bypassed the VPN and I am able to proceed. however I have a weird situation after that. The status shows the update is completed but the version hasn't changed. It still remains at ver 8.5.131.0. The upgrade should be from 8.5.131.0 to 8.10.196.0. Any reason why is it so?

[MHM Cisco World]

for WLC 9800 

show plat resource 

show process memory sort 

MHM

[AlfianZaini]

I have a weird situation after done the firmware update. The status shows the update is completed but the version hasn't changed. It still remains at ver 8.5.131.0. The upgrade should be from 8.5.131.0 to 8.10.196.0. Any reason why is it so?

[Flavio Miranda]

@AlfianZaini 

 Did you reload the master AP?

[AlfianZaini]

Yes i did 

[Flavio Miranda]

One thing about Cisco WLC is that it validates the firmware before install. What could explain this is the firmware is not fine. You were facing transfer failure.

 Try to download and tranfer the firmware again. Mobility express upgrade process is straitforward. It is hard to imagine something was done incorrectly. And this is not like switches that you need to change the boot or something.

[AlfianZaini]

As you can see from my screenshot earlier, the upgrade was successful. Even though I used a different version, it had the same case.

[Flavio Miranda]

What I can see from the screenshot is that the download was completed.  The installation is another thing.

 Put a console on the Master AP and look what is going on during the upgrade  process.

[Rich R]

You pre-downloaded the 8.10.196.0 software to the subordinate APs.
Did you install the 8.10.196.0 software to the ME WLC AP before rebooting it?
Your screenshot still shows it running 8.5.131.0 software so it doesn't look like you upgraded it, or the upgrade failed.

There's a well known bug that ME runs out of space on flash when trying to upgrade.  The workaround is to backup your config, convert to CAPWAP mode, upgrade to 8.5.182.0 CAPWAP image https://software.cisco.com/download/home/286304536/type/286288051/release/15.3.3-JF15 then convert the AP back to ME mode running 8.10.196.0 using the tar file from https://software.cisco.com/download/home/286304536/type/286289839/release/8.10.196.0 and then restore your backed up config.

Once the ME WLC is running 8.10.196.0 then APs should all reload onto the pre-downloaded 8.10.196.0 software automatically.  If they don't reload automatically you might need to swap image and reload them manually. (there was a bug in old 8.5 code which prevented them doing this automatically)