Cisco WLC - 4 way handshake timeout/failure - Troubleshooting

[JRD1213]

Dear Cisco Technology & Support Community,

We are experiencing an issue with our Cisco 5508 Wireless LAN controller (Software Version 8.3.143.61) and Cisco Wireless access points. After the WLC is operational for a period of 30-60 days or so, clients begin experiencing the inability to authenticate new devices with the wireless network. For example, a windows client may attempt to connect to the wireless network and provide the correct password, the authentication process with hang, and then fail without an error (on the windows machine). Sometimes the windows machine may say "invalid password" even though it is not the case of an incorrect password.

We are using the following security authentication: [WPA2][Auth(PSK)]

When we visit the WLC to review error(s) we see errors of this nature: "[DATE] Client Deauthenticated: MACAddress:[X] 1f Base Radio MAC:[Y] Slot: 0 User Name: unknown Ip Address: unknown Reason:4-Way Handshake timeout ReasonCode: 15"

For troubleshooting:

1.) Power recycling the services in the past has solved our issue. 

2.) We had thought in the past that this was an issue with DHCP and something to do with the inability to lease an IP address from our Cisco switch, however, we have moved the DHCP to both the firewall for one wireless network, and kept the DHCP on the switch for another wireless network, which ruled out DHCP, because DHCP leases from both the firewall and switch are functional.

3.) I have reviewed some articles on the support community about adjusting the eap eapol key timeout and retries via the following commands:

config advanced eap eapol-key-timeout
config advanced eap eapol-key-retries

However, we are not certain about what key 'timeout' or key 'retry' adjustments would be necessary against default settings and more importantly why, as this appears to be stop-gap-fix measure against a larger issue concerning wireless network uptime(s).

Would be happy to provide additional logs or additional troubleshooting information as required.

Any assistance concerning this matter would be much appreciated.

Thanks,

--

JD

[marce1000]

 - You may do some client (mac) debugging and have it then analyzed with :

            https://cway.cisco.com/tools/WirelessDebugAnalyzer/

  Also use the latest and or last-forever supported software version on the 5508 (as it is EOL)

 M.

[JRD1213]

Hi there, thank you for your reply.

I parsed some logs through the Wireless Debug Analyzer.

We are seeing a great number of these connection failures from all over the network.

(Multiple access points. Multiple BSSID(s).)

We have the maximum EAP connection timeout thresholds set.

It would appear that the client fails to respond to the M2 handshake.

If I power cycle the hardware, things generally work fine and this issue "goes away".

Any thoughts?

Jun 01 04:53:21.329 *apfMsConnTask_2 Client made new Association to AP/BSSID BSSID aa:bb:cc:dd:ee:ff AP LWAPxx
Jun 01 04:53:21.331 *apfMsConnTask_2 The WLC/AP has found from client association request Information Element that claims PMKID Caching support
Jun 01 04:53:21.331 *apfMsConnTask_2 Client is entering the 802.1x or PSK Authentication state
Jun 01 04:53:21.331 *apfMsConnTask_2 Client has successfully cleared AP association phase
Jun 01 04:53:21.331 *apfMsConnTask_2 Client is entering PSK Dot1x or WEP authentication phase
Jun 01 04:53:21.332 *apfMsConnTask_2 WLC/AP is sending an Association Response to the client with status code 0 = Successful association
Jun 01 04:53:21.336 *Dot1x_NW_MsgTask_5 4-Way PTK Handshake, Sending M1
Jun 01 04:53:26.345 *osapiBsnTimer 4-Way PTK Handshake, Client did not respond with M2
Jun 01 04:53:26.345 *Dot1x_NW_MsgTask_5 4-Way PTK Handshake, Retransmitting M1 retry #1
Jun 01 04:53:31.345 *osapiBsnTimer 4-Way PTK Handshake, Client did not respond with M2
Jun 01 04:53:31.345 *Dot1x_NW_MsgTask_5 4-Way PTK Handshake, Retransmitting M1 retry #2
Jun 01 04:53:36.345 *osapiBsnTimer 4-Way PTK Handshake, Client did not respond with M2
Jun 01 04:53:36.345 *Dot1x_NW_MsgTask_5 4-Way PTK Handshake, Retransmitting M1 retry #3
Jun 01 04:53:41.345 *osapiBsnTimer 4-Way PTK Handshake, Client did not respond with M2
Jun 01 04:53:41.345 *Dot1x_NW_MsgTask_5 4-Way PTK Handshake, Retransmitting M1 retry #4
Jun 01 04:53:46.345 *osapiBsnTimer 4-Way PTK Handshake, Client did not respond with M2
Jun 01 04:53:46.346 *Dot1x_NW_MsgTask_5 Client has been deauthenticated
Jun 01 04:53:46.346 *Dot1x_NW_MsgTask_5 Client expiration timer code set for 10 seconds. The reason: Roaming failed due to WLAN security policy mismatch between controllers (configuration error). It can also be used to report EAPoL retry errors, and GTK rotation failure (in 8.5)
Jun 01 04:53:56.345 *apfReceiveTask Client session has timed out
Jun 01 04:53:56.345 *apfReceiveTask Client expiration timer code set for 10 seconds. The reason: Client was marked for deletion, and it was on associated, power save or blacklist state. Other message would provide reason for delete
Jun 01 04:54:06.345 *apfReceiveTask Client session has timed out

[marce1000]

 - Which authenticating-policy-platform are you using ? Are the intended policy rules being hit ? Check the logs for the clients on the wireless network (expecting to get authenticated).

 M.

[JRD1213]

Everything is handled locally by the wireless LAN controller.

Layer 2 Security = WPA+WPA2

WPA+WPA2 Parameters = WPA2 Policy, WPA2 Encryption (AES)

Authentication Key Management = PSK

Thanks,

[Matt Kahle]

Hi All,

Since I am assisting JRD1213 with this issue. I can confirm the previous comment that WPA2 is being used. The WLC is locally preforming the authentication for all WiFi clients. Is there something we should be checking on the WLC for Auth? 

Regards,

[rayyaanfayker0006]

Hi were you able to resolve this issue, I am getting multiple scanners dropping off the same time failing m2 handshake. Then only after the scanners are rebooted does it come back up.

[JRD1213]

No, we have not found a solution for this issue.
Our only workaround is to restart the access points.

[rayyaanfayker0006]

Hi JRD

did you find a fix for this issue.