AIR-CAP3702I is not joining 5508 WLC after upgrade to 8.2.121.0

[rbestuzhev]

Hi All,

I followed the process https://supportforums.cisco.com/discussion/12169071/upgrade-wlcs-firmware-high-availability-ap-sso-enabled#comment-9628051, before reboot all APs looked like this:

Predownload Predownload Flexconnect
AP Name Primary Image Backup Image Status Version Next Retry Time Retry Count Predownload
------------------ -------------- -------------- --------------- -------------- ---------------- ------------ ---------
AP01 8.2.121.0 7.6.130.0 Complete 8.2.121.0 NA NA

After reboot most APs successfully joined WLC except 4. Those 4 have network connectivity to WLC but can't join for some reason:

00:62:ec:c1:f5:b0   N A                  AP01            X.X.X.2         Not Joined

I enabled capwap debugging and got the following:

*spamApTask0: Aug 20 09:31:17.130: [SA] 00:62:ec:c1:f5:b0 ApModel: AIR-CAP3702I-Z-K9

*spamApTask0: Aug 20 09:31:17.130: [SA] Could not find image version of bundled AP(apType: 36)!!!

*spamApTask0: Aug 20 09:31:17.130: [SA] Unable to get AP Bundled Version. Using Controller Version!!!

*spamApTask0: Aug 20 09:31:17.131: [SA] 00:62:ec:c1:f5:b0 ApModel: AIR-CAP3702I-Z-K9

*spamApTask0: Aug 20 09:31:17.131: [SA] Could not find image version of bundled AP(apType: 36)!!!

*spamApTask0: Aug 20 09:31:17.131: [SA] Unable to get AP Bundled Version. Using Controller Version!!!

I don't have access to AP via SSH and no console access as it is a remote site but it could be possible to reset AP and we have DHCP server that can issue IP address to AP. Currently AP has a static IP.

Based on the above details what could be an issue and what would be options?

Thanks,

Roman

[Leo Laohoo]

Post the complete output to the WLC command "sh sysinfo" and "sh time".

[rbestuzhev]

Thanks for feedback, please see an output:

(WLC01) >show sysinfo

Manufacturer's Name.............................. Cisco Systems Inc.
Product Name..................................... Cisco Controller
Product Version.................................. 8.2.121.0
Bootloader Version............................... 1.0.1
Field Recovery Image Version..................... 6.0.182.0
Firmware Version................................. FPGA 1.3, Env 1.6, USB console 1.27
Build Type....................................... DATA + WPS

System Name...................................... WLC01
System Location.................................. <Location>
System Contact................................... <Contact>
System ObjectID.................................. 1.3.6.1.4.1.9.1.1069
Redundancy Mode.................................. SSO
IP Address....................................... X.X.10.10
IPv6 Address..................................... ::
Last Reset....................................... Software reset
System Up Time................................... 2 days 14 hrs 25 mins 10 secs
System Timezone Location......................... (GMT+10:00) Sydney, Melbourne, Canberra
System Stats Realtime Interval................... 5
System Stats Normal Interval..................... 180

Configured Country............................... AU - Australia

--More-- or (q)uit
Operating Environment............................ Commercial (0 to 40 C)
Internal Temp Alarm Limits....................... 0 to 65 C
Internal Temperature............................. +36 C
External Temperature............................. +19 C
Fan Status....................................... OK

State of 802.11b Network......................... Enabled
State of 802.11a Network......................... Enabled
Number of WLANs.................................. 7
Number of Active Clients......................... 509

Burned-in MAC Address............................ 64:00:F1:F5:C0:00
Power Supply 1................................... Present, OK
Power Supply 2................................... Absent
Maximum number of APs supported.................. 500
System Nas-Id....................................
WLC MIC Certificate Types........................ SHA1

(WLC01) >

(WLC01) >show time

Time............................................. Mon Aug 22 10:58:49 2016

Timezone delta................................... 0:0
Timezone location................................ (GMT+10:00) Sydney, Melbourne, Canberra

NTP Servers
NTP Polling Interval......................... 600

Index NTP Key Index NTP Server Status NTP Msg Auth Status
------- ----------------------------------------------------------------------------------------------
1 0 X.82.10.250 In Sync AUTH DISABLED
2 0 X.34.10.1 Not Tried AUTH DISABLED
3 0 X.18.15.39 Not Tried AUTH DISABLED

[Leo Laohoo]

Unable to get AP Bundled Version. Using Controller Version!!!

Re-load the image to the controller and reboot the controller.  

Bootloader Version............................... 1.0.1

Think about upgrading the bootstrap while you're at it.

[rbestuzhev]

Hi Leo,

Thank you for feedback. Do you mean that I need to repeat the upgrade process? 98% of APs successfully joined WLC and work fine, may be I can just re-set those 4 APs to default and let them connect to WLC again? For Bootstrap upgrade, is there any documentation?

Thanks,

Roman

[Leo Laohoo]

98% of APs successfully joined WLC and work fine, may be I can just re-set those 4 APs to default and let them connect to WLC again

I misunderstood that it's only affecting four APs and the rest are fine.  If this is the case, then my question is this:  Can you remote into the four APs?  If you can then do this: 

The flash of the APs sometimes contain up to three IOS:  Two full IOS and one RCV IOS.  

Remote into the affected AP and delete the directory of the two full IOS and leave the IOS with the "rcv" in the subdirectory.  To delete the subdirectory, you'll need to do the following commands: 

debug capwap console cli
delete /f /r flash:subdirectory_1
delete /f /r flash:subdirectory_2
reload

For Bootstrap upgrade, is there any documentation?

Release Notes for Cisco Wireless LAN Controller Field Upgrade Software for Release 1.9.0.0

[rbestuzhev]

OK, thanks, we don't have remote access now, we'll try to get console access. If we can do it, we'll try your suggestion. In case the person can't access console, the only option would be to re-set AP. The router on that site supports DHCP so we can pass IP/WLC settings via DHCP like in below example:

ip dhcp pool POOL
network X.X.X.0 255.255.255.0

option 60 ascii "Cisco AP c3700"
option 43 hex f104<WLC IP in hex>

So AP is going to receive IP in X.X.X.0/24 range and WLC IP specified in opt 43.

Can you please confirm it as another option for worse case scenario?

[Leo Laohoo]

Can you please confirm it as another option for worse case scenario?

My suggestion is THE worst-case-scenario.

[rbestuzhev]

Ok, just wanted to say thanks for all suggestions. Fortunately we did not have to go the "reset AP" way.

It appeared that those APs for some reason did not swap their images and joined other controller with old version. That was something totally unexpected as all APs point to only one WLC which has been upgraded. Anyways, after finding this out and swapping images on APs and reloading APs remotely we were able to re-join them on the correct WLC.

[sahpat Patil]

Hi, even I am facing same issue

 Ap is getting IP throu DHCP on switch.IP is pingable from controller

but I am trying for AP to join controller.Its not passing the join phase.

 AP is brand new AP (3802I) and controller is 5520 software version is 8.2

I am seeing discovery response sent after that its throwing below error

*spamApTask5: Nov 18 13:56:59.142: apType: Ox34 bundleApImageVer:
*spamApTask5: Nov 18 13:56:59.142: Could not find image version of bundled AP(apType: 52)!!!
*spamApTask5: Nov 18 13:56:59.142: Unable to get AP Bundled Version. Using Controller Version!!!