Client devices reporting 'Incorrect Wi-Fi password' on PEAP network

[t3rebello]

Hi folks,

Running a Catalyst 9800-40 WLC. 1463 WAP's connected, predominantly C9136I WAP's. 8,817 active clients with 4,796 clients on our SSID using PEAP authentication via ISE on the back end.  

We are observing clients will frequently disconnect from the network despite having strong RSSI and SNR levels. The behavior manifests as 'Incorrect Wi-Fi Password', even though the user has been previously connected with no issues and their wireless profile is saved on their phone. I have seen this exclusively on iPhone devices at this time. The DNAC log timestamps seem to lineup with the log 'Client has requested it be deleted' 

To remediate, users can simply hit cancel and wait, and they will reconnect to the network. I am working a TAC case parallel to this community post. Just wanted to throw this out there in case other people were seeing this in their environment. 

[Dustin Anderson]

I'm on a 5520 still and not seen it, but wanted to ask if you have been able to narrow it down.

did you change any WLC code, or has it been since the 17 or 17.1 iOS version?

Is it certain models of phone/device?

[t3rebello]

We began the semester on 17.11.1 and have tried upgrading to 17.12.1 with no change in behavior. With 17.12.1, we are seeing fewer AP radio crashes, but that seems to just be one small issue resolved of the many we're feeling. 

I am seeing the issue mostly on iPhones. 

[Leo Laohoo]

---

@t3rebello wrote:
I am seeing the issue mostly on iPhones. 

---

This is interesting but could be related:  iOS 17.2 fixes Wi-Fi slowdowns and connectivity issues, Apple says

[marce1000]

                         ....Incorrect Wi-Fi Password
   -  Check  (PEAP authentication)  ISE   authentication logs (when this is seen)

  - Have a review of the  9800-40 WLC configuration with the CLI command show tech wireless ; feed the output into 
                                                                Wireless Config Analyzer

   - Perform client debugging according to https://logadvisor.cisco.com/logadvisor/wireless/9800/9800ClientConnectivity
     You can have client debugs (so called RadioActive Traces) analyzed with : https://cway.cisco.com/tools/WirelessDebugAnalyzer/

  -  Follow-up  on overall client behavior with : https://www.cisco.com/c/en/us/support/docs/wireless/catalyst-9800-series-wireless-controllers/217738-monitor-catalyst-9800-kpis-key-performa.html#anc5

 M.
                                                                                           

[t3rebello]

PEAP authentication from ISE does not indicate an authentication failure at the time that the client device is reporting 'Invalid WiFi password' 

Regarding the wireless config analyzer, my output of show tech wireless is 168.9MB large and the analyzer will not seem to parse my output.

Regarding the client debugging, I am working on reproducing and collecting debugs now. The issue is intermittent in nature and not easily reproduceable to capture.

Regarding the KPI's, everything seems healthy as far as I can tell. 

[marce1000]

   >...Regarding the wireless config analyzer, my output of show tech wireless is 168.9MB large and the analyzer will not seem to parse my output.
   Reminder WirelessAnalyzer (Wireless Config Analyzer)  needs the output of show tech wireless ; not simple-single show tech 
   Good to engage further on it ,  if you can , as this customer once reported too : This is so good

  Also have a look into recommended wlan timeout settings as advised here :
              https://community.cisco.com/t5/wireless/ideal-wlan-timeout-settings-9800cl/m-p/4412468#M230030

  M.

[t3rebello]

Thank you Marce.

We currently have our WLAN timeout set to 54000 and our idle timeout set to 3600. 

I have managed to get the Wireless Analyzer tool to work and I am putting in some changes based on the recommendations from the tool.

However, I did not see anything in the tool that was relevant to this issue. I am going to continue looking into the RA trace with the debug analyzer for the moment. 

[marce1000]

 - Great work, as Leo pointed out , if this is only seen with a particular brand/os of the devices then it could be an intrinsic bug related to the device , 

 M.

[t3rebello]

Something doesn't quite seem right with this flow. The screenshot of the wireless debug analyzer below is at the exact time where the issue occurs on the client device.

After the iPhone gets into a 'run' state, why is it requesting deletion from the controller after it's already connected?

CO_CLIENT_DELETE_REASON_USER_REQUEST

[marce1000]

               - Client buggy behavior ?

 M.

[Scott Fella]

It will be interesting what you find.  We do have a mix of 5500's and 9800's and have not seen issues with iPhones and PEAP.

[marce1000]

     >....After the iPhone gets into a 'run' state, why is it requesting deletion from the controller after it's already connected?
                   Have a look into : https://bst.cloudapps.cisco.com/bugsearch/bug/CSCwe81775

 M.

[Leo Laohoo]

Wireless Debug Analyzer should not be trusted at all.  Read THIS.  

[Scott Fella]

@Leo Laohoo too funny.  I'm 50/50 with the tools, but for not so experienced engineers, it's a good start.