Clients disconnect 8.0.140.0

eglinsky2012 · ‎02-08-2017

Hi everyone,

I came across an issue with the 8.0.140.0 firmware on our WLC and thought I'd ask to see if anyone else has experienced it or if it's a known issue.

As background info, we had release 8.0.121.0 running on a 5508 WLC with a mix of 3500, 3600, and 3700 APs. Two SSIDs are 802.1x, one is PSK. We peak at around 1900 clients across 40 APs. One DHCP/NPS server.

This fall, we started using the Digium Switchvox VoIP app on iPhones, and when walking down hallways, calls would get disconnected. So I enabled 802.11r over the DS, didn't help.

Then I realized that there was a known bug with Apple devices roaming that was fixed in 8.0.140.0, so I upgraded. Then we experienced issues with devices (not sure if just the iPhones/iPads or all devices) randomly getting disconnected, and not reconnect until waiting for a minute or two. This is a school and one teacher said he had a class of 22 with no issues, then his next class that had 12 kids had 3 kids who were kicked off in the middle of class. He observed that it seemed as though a couple would get kicked off and a couple could join. We do use the default 200 client hard limit per radio, though we don't usually exceed 60 or so clients per radio (with 3700s), which still produces adequate throughput for each client.

We also have two iPads, both near low-traffic APs, and both are stationary (check-in kiosks), that kept losing connection and making the app show no connection.

I tried disabling 802.11r over the DS, then 802.11r entirely, didn't help. So, last week, I reverted to 8.0.121.0 and the problems have gone away, no other config changes. The kiosk iPads have stayed connected all week. Somehow the VoIP app doesn't drop calls anymore, either (perhaps an update to that app or the PBX fixed that issue).

So I think I'm going to stick with 8.0.121.0 for a while, but I thought I'd reach out, see if anyone else had the same experience or any idea what the issue could have been.

Rasika Nayanajith · ‎02-08-2017

So I think I'm going to stick with 8.0.121.0 for a while, but I thought I'd reach out, see if anyone else had the same experience or any idea what the issue could have been.

If something works as you expected, I would stick to that software version as long as I can. That is the general recommendation for WLC software :)

As you already identified, there is no perfect solution, every code you upgrade you will hit with number of issues. Always stay with lowest problematic code

HTH

Rasika

Michael Combs · ‎02-09-2017

Sounds like you should open a TAC case on this issue, as there's no guarantee that anything is addressed until troubleshooting is conducted and a proper root cause is determined. Whether it ends up being a bug or similar on the Cisco AP/WLC, Apple iOS device(s), or even a combination of all the above and more.

That said, you can find the latest AireOS WLC code recommendations from Cisco TAC and HTTS at the URL below, which we update frequently:

https://www.cisco.com/c/en/us/support/docs/wireless/wireless-lan-controller-software/200046-TAC-Recommended-AireOS.html?cachemode=refresh

Also, for highly mobile devices like iOS clients and the like, my recommendation is to use FT over-the-air method rather than FT over-the-DS. You can find the enterprise best practices guide for Apple iOS devices on Cisco WLANs below as well:

http://www.cisco.com/c/dam/en/us/td/docs/wireless/controller/technotes/8-3/Enterprise_Best_Practices_for_Apple_Devices_on_Cisco_Wireless_LAN.pdf

I hope that helps!

/Michael

eglinsky2012 · ‎02-09-2017

Thanks for that insight, Rasika. Whenever I do upgrades (a couple times a year), I use the release that TAC recommends as denoted with the gold star on the download page. That's why I went with 8.0.140.0 when I found the bug with Apple devices, and that's still the recommended 8.0 release shown in the link Michael sent.

However, under the 8.0 release, they recommend customers with 3700 APs use release 8.2 due to CSCus83638, though I'm not sure if that's applicable to me because I haven't changed the default RX-SOP values are set to Auto.

https://bst.cloudapps.cisco.com/bugsearch/bug/CSCus83638

It did seem as though the clients were experiencing an association issue, but it's strange that they would get kicked off after they were connected and stationary (unless they regularly go through the authentication process? Noob question, I know). I also only seemed to get complaints where there were 3700s, not 3500s.

They cite the bug as being fixed in several releases not ending in .0; does that mean those are special releases that must be obtained from TAC? The only one ending in .0 is 8.2.130.0, so it seems that's the only one I can download on my own. Does that mean any release after 8.2.130.0 will be free of the bug?

They say it was fixed in 8.0.140.2 so I guess that means 8.0.140.0 was affected.

They also cite the bug as being affected in 8.0.110.0; does that mean every subsequent release of the firmware would be affected until the exact specified release that it was fixed in?

Sorry for the rapid fire questions but I've been wondering these things for some time.

Unfortunately I won't be able to do testing and provide feedback to Cisco since we just have a single 5508 in production and I've downgraded the firmware, so I have nothing to test with.

Rasika Nayanajith · ‎02-09-2017

If you have look that bug detail, it is a very severe issue. There are 94 TAC cases for the same.

They cite the bug as being fixed in several releases not ending in .0; does that mean those are special releases that must be obtained from TAC? The only one ending in .0 is 8.2.130.0, so it seems that's the only one I can download on my own. Does that mean any release after 8.2.130.0 will be free of the bug?

Yes, this mean a code that has the fix only available via TAC. Those are called engineering releases and target for a solution for particular critical issues.

Any public releases always end with .0 at the end (8.0.100.0, 80.0.135.0 or 8.0.140.0, etc)

They say it was fixed in 8.0.140.2 so I guess that means 8.0.140.0 was affected.

Yes, that mean 8.0.140.0 is affected and next maintenance release (may be 8.0.150.0) will be have the fix.

HTH

Rasika

Jaycel West · ‎05-03-2017

eglinsky2012,

Did you find any more information on what was needed to fix this issue? We are also experiencing the same issue randomly and cannot find a root cause for it. We are on WISM2 8.0.140 using 3500/3600 APs.

Thanks!

eglinsky2012 · ‎05-03-2017

Jaycel, it's been a while but I don't think I found any explanation. We downgraded to 8.0.121.0 which eliminated the random disconnect issues. A few weeks ago, we upgraded to 8.2.151.0, which is now the Cisco-recommended version, and it's been fine so far. Occasionally, iOS devices take a while to associate after unlocking and/or roaming, though I'm beginning to suspect that that's because of something else (long, irrelevant explanation).

eglinsky2012 · ‎05-03-2017

Also, Jaycel, check out what Rasika shared about the bug affecting the 5 GHz radios (in 3600s like you have, and 3700s like we have). Sounds you might want to upgrade to 8.2.151.0 to avoid it.