AP failing to rejoin controller after 3850 switch upgrades

[pwwiddicombe]

We've had a number of occasions recently where AP's fail to reassociate with a WLC after upgrading 3850 switch stacks to correct for the PoE bugs.   These have been running normally for some time, and upon updating the 3850 switch, they fail to rejoin.  Not all AP's fail; but one stack upgrade and we lost 18 AP's.

Predominantly 3702 AP's, but have also "lost" 1142's.  In most cases a factory reset of the AP followed by basic IP and controller gets them going again; some have gone dead (apparently flash failure), and needed full RMA.

I heard rumors that there may have been a range of 3702's that may be problematic, but I have been unable to locate that information searching.  The 1142 loss may be due to expired MIC.

[Scott Fella]

Might be a bug with the controller code and not really the switch upgrade. We have seen this with 3702’s when the switch gets reloaded or power outage. Really anytime the AP reboots. Upgrading to 8.5.135.0 helped reduce this issue for us.

[Haydn Andrews]

Ill second that it is probably a bug.

I had one where the AP flash was corrupt but they keep working until a power cycle

[Leo Laohoo]

---

@pwwiddicombe wrote:

I heard rumors that there may have been a range of 3702's that may be problematic, but I have been unable to locate that information searching.

---

What firmware is the controller running on? 

With the 3702, can you check if they boot up correctly on the switch:  Do the command "sh power inline" and see if the port(s) are showing up the AP model number and not "Ieee PD".  

Next, if the APs are showing up correctly in the switch, get their IP address (using "sh cdp neight <PORT> detail") and try to remote into them.  

Once you're in there look at the log of the AP and see if the AP is searching for the controller using DHCP Option 43 or not.  

Finally, Cisco has released a Python script to identify which AP is prone to the "IOS corruption" issue.