WLC 5508 - Client Intermittent Drop Out

[E07837_2]

We have an intermittent issue where clients seeem to drop their connectivity intermittently and I wondered if anyone else have seen this.

When the client drops out it is associated to the AP at good signal strength -45 -> -49 dbm. All our clients are running Win7. Once the issue occurs a yellow exclamation mark appear across the system tray wireless icon and a message appears in the "Network and Sharing Centre" that either says “no internet connection” or “DNS server not responding". The association with the AP appears to be intact. This seems to be confirmed as the client can ping its default gateway and can be pinged from another device but no connectivity above layer 3 appear to be working. Really the best way to describe it is that all communication above layer 3 stops. This behaviour is extremely intermittent.

The wireless clients are a small cross section of HP business laptops. HP8460p, HP6930p and a few other models. The laptops are all running the latest wireless drivers available (that was the first thing we tried). We have two 5508 wireless controllers operating 60 APs between them. Most of the APs in the area where we are seeing the issue are AIR-LAP1242 we do run the AIR-CAP3502 also but not in this area. We only have 2.4GHz enabled but with the extended data rates (to 130mbps). We have multiple WLANs but the one that seems to be causing the issue is the only one running WPA2.

The only way to restore full connectivity to the client is to hard reset the wireless adapter.

We have run debugs on the client and ran a sniffer trace between the controller and the authentication server but so far this has not turned up anything. The wireless controller is showing the client still associated but a link test from the controller to the client failed.

Any ideas would be welcome. I can provide further information if required.

[grabonlee]

Hi

What EAP type do you have deployed, as I guess you are using 802.1X?

Does this problem occur during client roaming?

Is the problem specific to only HP devices or are other laptop vendors in the mix?

I would suggest that you perform a packet capture on one of the wireless clients when the problem happens. You could use wireshark. Perform the following tests during the capture:

1. Ping a remote device on a different subnet.

2. Perform a layer 4 and above activity such as opening a web page or FTP.

In the interim, you could try the following:

1. Disable Aironet IE on the WLAN SSID Advanced tab

2. Increase the packet max-retries timeout value on the dot11radio interface.

[E07837_2]

Hi

Thanks for your answer.

Yes, we are using 802.1x.

The problem seems to occur when the client is stationary.

We only have HP laptops on our network and no ability to connect anything else right now.

I agree on the packet capture but in the last couple of days I have failed to catch in real time as it is extremely intermittent.

We already have Aironet IE disabled but have not yet tired to Increase the packet max-retries timeout value on the dot11radio interface. Thanks for your suggestion.

[Scott Fella]

Make sure you are only using WPA2/AES or WPA/TKIP with 802.1x. I would also disable session timeout and Client Exclusion for testing. See if that helps. Also upgrade your client drivers if you haven't already.

Sent from Cisco Technical Support iPhone App

[E07837_2]

FURTHER INFORMATION:

After days of trying to capture the issue with a packet sniffer we finally got it. What we see is that TCP sessions are torn down and once these are gone we are only really getting DNS and NBNS requests and responses. The client can ping within and outside its own subnet. The controller shows the client in RUN state and with a signal strength at -57dbm. A link test from the controller to the client is successfull. We tried HTTP and FTP and none of them work. We tried nslookup to resolve a hostname to IP address and gets a server error.

So the client has a good connection with the AP but it is almost like the controller is stopping some connectivity. Could this be a bug? We are running 7.0.98.0 on the controller.

[Scott Fella]

Yes it can be. That code has been deferred by Cisco. So I would recommend you upgrade to the latest 7.0.235.3 code

Sent from Cisco Technical Support iPhone App

[grabonlee]

It would've helped if you could post the packet capture. You can blank out the first three octets of your network IPs if you don't want them revealed. I would rather you find the root cause of the problem before you upgrade the software. We all used 7.0.98 and never experienced such problem.

If I may ask, when the problem occurs, do you have diffulty in accessing local resources or only external web resources. Are all your wireless clients on DHCP?

Did you specify option 15 for DNS name on the DHCP server?

Do you have a local host entry for the Controller on the DNS server?

If you are having problem resolving hostname to IP, you check your DNS services.

[johnbcsc34]

We are also experiencing the same problem on our 5508 WLC running 7.0.253.3, using 1142 and a few 1242 LAPs.  The WLC trap log quickly fills up with mostly "Client Authenticated" and some "Client Deauthenticated" entries.  The random anomalies occur with both roaming and stationary clients, that are mostly using Dell Latitude laptops (Intel and Windows wireless clients), Windows7.  So far our TAC case has not offered a remedy.  Has anyone found a definitive culprit for this yet?  Please advise.

[Scott Fella]

Can you start a new thread and explain exatly what is happeining along with some loggs and what clients, how is your ssid configured, etc.

Thanks,

Scott

Help out other by using the rating system and marking answered questions as "Answered"

[johnbcsc34]

Also (in addition to my initil responding post) we have only 2 WLANS (Corp. and guest) that were migrated over from an older 4402 controller onto our current 5508.  The corp. SSID is using WPA, TKIP, all radios, and 802.1X authenticating to a Cisco ACS1121 (back to an AD).  The Guest SSID is unfortunately still using WEP with PSK and we are predictably getting decrypt errors there as well.  Overall the wireless network works pretty well except for the random disconnects (deauthenticated / reauthenticated).

Still just wondering if there was a common definitive culprit that has been found, or if this seems to be more of an isolated problem.

[E07837_2]

Sorry for the long silence.

In answer to osita N's questions....

No - I will not post the packet capture online for the world to see.

In answer to the questions on DHCP the answer to all three is yes. From the packet capture we can see the DNS server replying to requests so I am not sure this is a DNS error, I think "DNS server not responding" is a Microsoft generic error?

Since we are not getting anywhere with a solution we have conceded and will upgrade to 7.4.100.0. We are just running some testing now  and then hopefully will deploy the upgrade next week. Will let you know how we get on.

[Xiao Yi Steven FAN]

Hi Tina,

did you get the problems fixed by upgrading the code? i am having the same issue but running code 7.2.111.3

[Scott Fella]

Xiao,

I would open a new thread also, but add more detail like what model AP/WLC, what clients are affected, is all the clients affected or some, what troubleshooting you have done, also post the show WLAN of the WLAN that is affected.

Sent from Cisco Technical Support iPhone App

[Xiao Yi Steven FAN]

Ok, will update the thing tomorrow, cause will debug when all the clients come back

[E07837_2]

Sorry Guys, I really should have updated you all a lot sooner. Yes, this issue was fixed by upgrading the wireless controller to 7.4.100.0. But remember that WCS will not support this version (if you use WCS) you would have to upgrade to Cisco Prime for the management platform.