arp delay during wifi client roaming

[BelSkorpio]

We have on several type of WLC controllers (2500, 5508, 5520) in different remote sites all the same problem.

When a wifi client roams from 1 LWAP to another LWAP we notice:

- a very consistent loss of 2 pings, if we ping from a wired device to the wifi client

- no loss of pings when we ping from the wifi client to a wired device

So from the point of view of the wifi client all is fine.

Our wifi clients all use DHCP and I know that the WLC acts as an ARP proxy for the wifi clients.

Is it possible that there is some kind of ARP delay in the WLC between the capwap tunnels of the different registered LWAPs when a wifi client roams ?

[Flavio Miranda]

Hi

 I dont think so. The problem about roaming is that this depends on a lot on the client algorithm, which is in most case failure. 

 This is an eternal discussion but basically, if client does not perform the roam process like it should and had to re-authenticate then the process takes longer than it should. 

 It is necessary to troubleshoot in order to affirm something but most case rely on the fact that client is not actually roaming but associating to another AP. 

 You can start with debug client <client_MAC-address>" If you see too much "Association" log, probably roaming is not happen as expected. 

 On the WLC side you can perform some ajustments on the roaming process in "Client Roaming" under WIRELESS tab. You can also play with "Fast Transition" but all this depends on the client capability.

-If I helped you somehow, please, rate it as useful.-

[BelSkorpio]

We have 802.11r,k,v BSS fast transitioning with optimized roaming in place. It works like a charm.

Like already said, form the point of view of the wifi client we are NOT loosing a ping.

As long as the wifi client starts the communication, everything goes very smooth.

There is NO re-authentication.

The issue is when we ping from a wired device to the wifi client we very consistently loose 2 pings during  each roaming. It is not sometimes 1 or none. No, each time we loose 2 pings (time frame of +/- 6 secs). 100% reproducible EVERY time.

[Ric Beeching]

Sounds like arp-caching potentially... are they running in Local or FlexConnect mode?

[BelSkorpio]

Local mode.

[Ric Beeching]

Well that's my bright idea out the window. Are all the WLCs using the same firmware/software.. no way to upgrade one and test to see if any different?

For whatever reason the local APs aren't updating the WLC upon roam so the initial few arps fail. Have you also tried on a test WLAN without 802.11r running?

Ric

[BelSkorpio]

They all have different versions. :)

8.5.105.0

8.4.100.0

8.2.141.0

8.0.120.0

We've run for a long time without 802.11r,k,v.

We even have a few wifi client that don't support these roaming protocols.

The behavior is each time exactly the same.

Actually, it is not a real problem for us, because our wifi clients are always setting up a connection to the wired server.

I bring it up to this forum (and not to the official CISCO TAC) only for 2 reasons:

- purely academic

- when we follow/debug our wifi clients, we often launch a ping from a wired device

[patoberli]

Uhoh, that's not good. Don't forget that 8.2 and 8.0 releases don't support adaptive 802.11r and I think 8.0 doesn't support 802.11r at all (but I might be wrong here).
That means you do have different SSID configurations possibly, always bad for roaming!
My first recommendation is to get all controllers to at least 8.3.140.0, which supports adaptive 802.11r and also update the 8.4 and 8.5 releases to 8.5.120.0, because the contain a ton of bugs. 8.4 is actually already killed off and will not anymore get any bugfix releases.
Do the controllers make L2 or L3 roaming of the clients? This has an effect on the ARP cache of the router (in case of L3 roaming) and possibly firewall. Do you have DHCP-Required enabled on the SSID?

[BelSkorpio]

Yes of course, I know that 8.0 does not support 802.11r,k,v.

The thing that I want to make clear is that it does not matter in what way we are roaming, i.e. assisted or not by the AP, decision made purely by the client, with or without help of ccx, pure old-fashion classic roaming .... It just does not matter.

When we ping from a wired device we consistently loose 2 pings.

When we ping from the wifi client we loose NO pings. 

[Flavio Miranda]

Try to run debug client while roaming.

 debug ft events enable

-If I helped you somehow, please, rate it as useful.-

[BelSkorpio]

Like said in the previous reply, I don't think it has anything to do with the used roaming protocol. But because you insist.

debug ft events enable:

*apfMsConnTask_0: Feb 27 13:46:13.731: 28:16:ad:03:8c:1b Doing preauth for thisclient over the Air
*apfMsConnTask_0: Feb 27 13:46:13.731: 28:16:ad:03:8c:1b Doing local roaming for destination address 34:a8:4e:42:b8:dd
*apfMsConnTask_0: Feb 27 13:46:13.731: 28:16:ad:03:8c:1b Got 1 AKMs in RSNIE
*apfMsConnTask_0: Feb 27 13:46:13.731: 28:16:ad:03:8c:1b RSNIE AKM matches with PMK cache entry :0x3
*apfMsConnTask_0: Feb 27 13:46:13.731: 28:16:ad:03:8c:1b pmkRoName derived sucessfully
*apfMsConnTask_0: Feb 27 13:46:13.732: 28:16:ad:03:8c:1b Validate FTIE for R0KH-ID, Store SNonce passed
*apfMsConnTask_0: Feb 27 13:46:13.732: 28:16:ad:03:8c:1b FT Auth over the ds. Generate the Anonce ..BB:DE...49:29
*apfMsConnTask_0: Feb 27 13:46:13.732: 28:16:ad:03:8c:1b Created a new preauth entry for AP:34:a8:4e:42:b8:dd
*apfMsConnTask_0: Feb 27 13:46:13.732: 4e:42:b8:dd:34:a8 Sending FT Response to station on BSSID 34:a8:4e:42:b8:d0 (status 0)
*apfMsConnTask_0: Feb 27 13:46:13.734: 28:16:ad:03:8c:1b Processing assoc-req station:28:16:ad:03:8c:1b AP:34:a8:4e:42:b8:d0-01 ssid : COBPKI thread:1b77ce80
*apfMsConnTask_0: Feb 27 13:46:13.734: 28:16:ad:03:8c:1b Marking this mobile as TGr capable.
*apfMsConnTask_0: Feb 27 13:46:13.734: R1KHID+S1KHID: (12)
*apfMsConnTask_0: Feb 27 13:46:13.734:      [0000] 80 e8 6f 03 d1 c0 28 16 ad 03 8c 1b
*apfMsConnTask_0: Feb 27 13:46:13.734: 28:16:ad:03:8c:1b Roaming succeed for this client.
*apfMsConnTask_0: Feb 27 13:46:13.735: 28:16:ad:03:8c:1b Including FT Mobility Domain IE (length 5) in reassociation assoc Resp to mobile

[patoberli]

One more question, this also happens if the two APs are on the same WLC?
Or only if you do controller-roaming?

[BelSkorpio]

Yes, the 2 APs are always on the same controller during my tests.

[BelSkorpio]

We don't do controller roaming.

[patoberli · Accepted solution]

Hmm ok, I've actually never tested this, maybe it's even "normal".
Are your test clients by any chance Windows based? Just asking as they might switch the firewall profile after a second or two after connecting, while one profile allows and the other blocks ICMP. You shouldn't see this with a mobile phone though.
Do you have any outage because of this?