Issues with WLC 7.0.116 and iPhone/iPads iOS devices

[simonwynn]

We are using a WLC 2106 running 7.0.116 and notice that when iPhones are streaming data (e.g. YouTube) during WPA re-auths we get the following log entry:

Max EAPOL-key M3 retransmissions exceeded for client xxxxx

And the iPhone prompts for the username/password  for EAP-PEAP.

This occurs on two different lab setups, one local auth, one MS IAS. Changing EAPOL timeout settings does not fix the problem.

Note that we rolled back to WLC 6.x software and the problem appears fixed. Is this a known issue in 7.0.116 that others have seen?

thanks,

Simon

[George Stefanick]

We have 8,000+ wireless devices. We also have a large base of apple devices including iPhones and iPads. We've done extensive testing prior to the roll out and I can tell you we do not have this specific issue. We have controllers on 7.0.166 and 7.0.98

Our set up is EAP-PEAP MsChapv2. As I mentioned in one of your last post we did chnage the idle and session timeout because we did see the Apples i devices reat

In addition you are using WPA without EAP correct ? This means no radius server even comes into the mix. Did you do a layer 2 capture of this ? I would love to take a peek at it ..

[simonwynn]

We are using EAP-PEAP hence the local auth and IAS references.

I don't have a capture unfotuinately and have reverted to 6.x which appears to work just fine. Increating your session timout of course just reduces the likelyhood of this happening. It's only happeing when the iOS device is fairly CPU bound so you may have never encountered this.

Simon

[George Stefanick]

Its speculation at this point stating its tied to CPU processing. If you obtain a capture of both low CPU and high CPU you could then state its CPU bound.

Specific to the session timeout, Increasing or disabling it makes sense. The only purpose of the session timeout is to regen the MSK keys which then seed the PMK,GMK. Cisco recommends to disable session timeout on voice because of its known problems with voice reliability.

In addition, client idle timeout impacts Apple "i" deivces more than any other divice. Do a lab and capture the traffic coming out of a iPad for exmaple. These devices dont chatter on the network like other devices.

Its not uncommon for iPads to drop off the network with default settngs on the WLC (but this doesnt explain your 7 vs 6 code). In fact we lab this where iPads client records get deleted after 300 seconds (default timer) and have to reauth becuase of this very issue. An adjustment of the timers aided this situation.

Call TAC see what they tell you ... I suspect they will review these same items and the EAP timeout, that you mentioned already.

I just checked and we have 768 ipads on our network at this very moment  including mine.

I just moved my auth time to 5 minutes and I am pulling youtube and watched 3 auths and there is no problems.

You need to do a L2 capture and see what the iPad is doing. If the ipad sends its EAP creds and the WLC doesnt respond then you know its the WLC.

[simonwynn]

Weird, this is so reproducible. I've tried every setting I can imagine in terms of timouts, plus CCKM, non-CCKM. All combinations of WPA, 1X and PSK etc... No issues with 6.x.

How do I enable a l2 packet trace on the controller?

btw thanks for your help here..

Simon

[George Stefanick]

Simon,

What are you using to load down (stress) the i device? I can test some more in my lab, but what little testing Ive done so far and the amount of i devices i have today Im not having this issue.

l2 debug on the WLC may not yield much. Controller isn't very good (in my experience) to do l2 captures for 802.11 frames.  Do you have a 802.11 snifter?

Also, did you open a TAC case?

Leo / BG / Steve,

Do you have any input here?

[simonwynn]

We don't have Cisco support for this device, so no TAC.

We are going to do a L2 trace next week - my company developed a CCX supplicant so we have a lot of knowledge here. We have seen some issues with WLCs in the past with customer devices.

Simon

[Leo Laohoo]

No comment here George.  Will try this out on Tuesday (Monday is a holiday).

[Leo Laohoo]

Hey George,

Our session timeout is set for 1800.

[simonwynn]

Ok, we ran wireshark and basically when other traffic is passing to an iPad we get 4 M3 message attempts and then a deassociation upon 1x re-auth timeouts. Using wireshark we couldn't;t decrypt the packets so we are not sure what was going on.

Anyhow, we've just turned offsession timeout for 1X, which results in a default timeout of 24 hrs, which appears to be fine for our clients.

Simon