WLC 5508 OS 8.5.182.7 - issue with AP 3700 after upgrade

[karol.krzyzyk]

HI All

Yesterday I made upgrade WLC controller from 8.5.2.180.0 to .7 ( issue with SSO and ) and problem with jumping AP between primary and secondary WLC. But after upgrade again AP 3702 stuck in downloading mode. I thought this was fixed in version 8.5.182.0 but no.

Workaround with changing date to December 2022 not helped. 

Still downloading.  Any idea. From last week WLC drive ma crazy with many anomalies. AP joining - unjoining, going to backup WLC - now this case. 

[Leo Laohoo]

What is the status of the 3700, are they all in "Downloading" status?

[karol.krzyzyk]

Yes. All in downloading status, . Other 1600,2600,2800 are already upgraded and registered

[Leo Laohoo]

How many APs are in the Downloading state? 

If the APs are rebooted, did this make any difference?

[karol.krzyzyk]

this model only 7 , After reboot still the same , are installed 12m high under the roof, will be difficult to get access to console today, When I made upgrade WLC from 171 to 182 workaround with time helped. this time no

[Leo Laohoo]

Ok, 7 is not a bad number.  I have a solution and it is deemed "NSFW".  

To get this to work, SSH or Telnet access to the AP is mandatory.  

Here are the steps (and why it is "NSFW"):  

1.  Download the file RCV (Recovery) file called "ap3g2-rcvk9w8-tar.153-3.JPO.tar" and put this file into a TFTP server. 
2.  Telnet or SSH into the AP and, in enable mode, do the enter the following commands:

debug capwap console cli
delete /f /r flash:ap3g2*
archive tar /x tftp://<TFTP_IP_ADDRESS>/ap3g2-rcvk9w8-tar.153-3.JPO.tar flash:

3.  After the AP finishes downloading the files, reboot the AP.  

Hope this helps.  

EXPLANATION:  

I call the above steps "NSFW" for the following reasons: 

1.  Never use the command "archive tar /x" option unless one knows what this option cannot do.  
2.  The workaround (above) are not supported by Cisco TAC nor WLBU.  I formulated the steps myself.  

The "RCV" file is a recovery file.  After Step #2, the AP will reboot and load the only firmware it can find -- The RCV firmware for IOS-XE version 17.10.1.  Even though this version is for IOS-XE, this version has the necessary fix for FN-72524.  The AP will then contact the controller and download the correct CAPWAP firmware.

The option "tar /x" is a very crude and archaic option that forces the contents of the TAR file to be unpacked at a specified location.  What is bad about the "tar /x" option is that, at the end of the process, it ends without any verifying.  This option is just a notch "higher" than merely copying the files across without checking if the files being exported are corrupt or not.  So use the option sparingly or don't.  

[karol.krzyzyk]

This solution is not possible for me - I don't have service contract and can't download mentioned file 

[Leo Laohoo]

---

@karol.krzyzyk wrote:
This solution is not possible for me - I don't have service contract and can't download mentioned file 

---

I have a solution for this:  

1.  Read Cisco Wireless LAN Controller HTTP Parsing Engine Denial of Service Vulnerability.
2.  Scroll down to the "Customers Without Service Contracts" section, where it specifically states: 

Customers who purchase directly from Cisco but do not hold a Cisco service contract and customers who make purchases through third-party vendors but are unsuccessful in obtaining fixed software through their point of sale should obtain upgrades by contacting the Cisco TAC.

Customers should have the product serial number available and be prepared to provide the URL of this advisory as evidence of entitlement to a free upgrade.

---

3.  Email TAC.  Provide the filename of the AP, "ap3g2-rcvk9w8-tar.153-3.JPO.tar", the HTML location and the Security Bulletin.  

NOTE:  Do not call Cisco TAC.  Send them an email. 

[karol.krzyzyk]

Hello Again. 

Many thanks for your advise ,

Also found one spare part box with 3700 and in found the reason for this problem

May  3 15:53:18.875: %PKI-3-CERTIFICATE_INVALID_NOT_YET_VALID: Certificate chain validation has failed.  The certificate (SN: 02A79669ACDDF395D2103895880438649829) is not yet valid   Validity period starts on 16:53:06 UTC Dec 7 2022

In this case workaround with the date before 4 Dec  2022 was wrong. Certificate started from 7. I already set 10 Dec, reboot AP and now it works, AP got image from Controller

[Rich R]

That's correct - the workaround was only for software without the fix for the cert expiry.

Check that you also have "config ap cert-expiry-ignore mic enable" configured on the WLC?

Did all the remaining 3700 download after you set the correct date?