Cisco 9800-cl 802.1x radius authentication

[Djeten]

Hello,

I want to configure 802.1x authentication on an SSID on a 9800-CL hosted in Azure.

I have configured the radius server, server group and method list and set this in the AAA section of the WLAN.

In Wireshark I can see Access-Request coming from the WLC to the NPS, and Access-Challenge from the NPS to the WLC, but nothing more.

I suppose I should see some Accept packet coming from the controller.

Did anyone have this issue before?

[marce1000]

 - For starters  have a checkup of the 9800-CL Azure based controller  configuration with the
   CLI command show tech wireless and feed the output into : Wireless Config Analyzer

   M.

[Djeten]

Thx for your reply, but this did not reveal any issues regarding Radius

[marce1000]

 - In any case make sure that the overall wlc check-results (Tab) contain no errors red flagged, because for sure these must always be corrected.
             For the rest : what's in the native NPS server's logs for authenticating request from the 9800-cl ?

 M.

[Rich R]

If you do the packet capture on the WLC do you see the WLC receiving the challenge from the NPS?
If not check routing/ACL/firewall etc between NPS/WLC.

If WLC receives the challenge then debug on WLC to see what is happening when it receives the challenge.

[xes]

Hi, got same issue,

@Djeten have you found the solution ?

WLC in Azure 17.9.5, WLC stops replying to second 'access-challenge' packet from NPS (MS Windows). 

When disabling 'central authentication' it authenticates correctly.

Capture when fail:

[eglinsky2012]

@xes @Djeten Is this a sporadic or a constant issue, and when did it start? After upgrading to 17.9.5? Are any APs/clients able to authenticate successfully?

In addition to checking routing/ACL/firewalls and doing a capture from the WLC as Rich suggested, you could do an OTA capture or a capture from a client trying to authenticate. That would give more information on where the issue lies if the WLC capture shows that the expected packets are not arriving from the client. Is the AP not sending the challenge to the client, or is the AP not sending the client's second request back out to the WLC? Etc.

Also, what model WAP are you using? If it's certain 9100 series, you may be hitting this bug:

https://bst.cloudapps.cisco.com/bugsearch/bug/CSCwj45141

We haven't yet confirmed whether we're experiencing the bug, but we've found that 5 GHz radios on our 9130, 9166, and 9136 APs stop authenticating after prolonged high client count situations. x700 and x800 series don't seem to be affected. You could confirm by rebooting the WAP, or temporarily disabling the 5 GHz radio, which will force clients on 2.4 GHz instead. Note that this is a time-based issue in my experience; it's fine when the AP boots up, but within 2-3 days of high client count (say over 30-50 clients on an AP), the issue occurs.

[Rich R]

@eglinsky2012 did you notice that they've changed CSCwj45141 status to Unreproducible which basically means TAC were unable to reproduce it themselves so have thrown their arms in the air and given up.  Meaning they won't do any more work on it until a big enough customer escalates and they'll then put more effort into repro and/or fixing it.  With similar bugs in the past they usually address it by introducing extra debugs in the code to help diagnose the issue and then eventually a fix but seems they're getting lazy and have just decided to give up on this one! It's only got 6 TAC cases attached <sarcasm> to the bug so it would be helpful if everybody observing the issue opens a TAC case and gets it attached to the bug so they can see it's not a rare and isolated issue.

[Djeten]

Hello @eglinsky2012 

This is a new WLC we have deployed in Azure to replace an older on prem WLC. So it hasn't worked before on this WLC.

There are different types of AP's registered on the controller: C9115AXI-E, AP1832I and AP1852I.

I will do a capture on the WLC and on a client to get more info and will get back to you.

[xes]

@Djeten 

i think i solved my issue setting radius Framed MTU to 1300 

Configurable Framed-MTU on Cisco Catalyst 9800 - Cisco

Setting Framed MTU on NPS in Network Policy settings also seems to help.

[Djeten]

Hello,

I have been testing some more but I did not fix the issue yet. 

Changing the MTU to 1300 did not fix it.

In the Wireshark I can see:
1. the association request from the client to the AP
2. EAP request identity from AP to client
3. EAP response identity from client to AP
4. Radius Access Request form WLC to Radius server
5. Radius Access Challenge from Radius server to WLC
6. EAP Challenge request from AP to client
7. EAP Challenge response from client to AP
8. Radius Access Request from WLC to radius

Then I should recieve an Radius Access Accept from the radius, but I see a new Access challenge coming from the radius server.

And then the process starts again...

[Djeten]

I have been doing further testing and have opened a case with TAC.

It seems it is not a Radius issue. We have different types of AP's. The SSID is working fine on C9115AXI-E AP's, but does not work on AP1832I AP's.

Traces on the client, controller, AP and switch show that Radius works, but the 4-way handshake fails. 
We see that step 4 of the 4-way handshake is not sent from the AP to the switch, resulting in a failure of the connection.

Logs from the AP (AP recieves EAPOL frame 4 from client):

[*05/29/2024 11:55:05.1760] Client(00:d4:9e:54:6e:8d): apr1v2: 802.11: start SAE authentication (RX commit, status=126 (SAE_HASH_TO_ELEMENT))
[*05/29/2024 11:55:05.1760] Client(00:d4:9e:54:6e:8d): apr1v2: 802.11: SAE authentication (RX confirm, status=0 (SUCCESS))
[*05/29/2024 11:55:05.1860] Client(00:d4:9e:54:6e:8d): apr1v2: MLME: MLME-AUTHENTICATE.indication(00:d4:9e:54:6e:8d, unknown)
[*05/29/2024 11:55:05.1860] Client(00:d4:9e:54:6e:8d): apr1v2: MLME: MLME-DELETEKEYS.request(00:d4:9e:54:6e:8d)
[*05/29/2024 11:55:05.1860] Client(00:d4:9e:54:6e:8d): apr1v2: WPA: event 0 notification
[*05/29/2024 11:55:05.1860] Client(00:d4:9e:54:6e:8d): apr1v2: WPA: event 1 notification
[*05/29/2024 11:55:05.1860] Client(00:d4:9e:54:6e:8d): apr1v2: WPA: start authentication
[*05/29/2024 11:55:05.1860] Client(00:d4:9e:54:6e:8d): apr1v2: 802.1X: unauthorizing port
[*05/29/2024 11:55:05.1860] Client(00:d4:9e:54:6e:8d): apr1v2: WPA: sending 1/4 msg of 4-Way Handshake
[*05/29/2024 11:55:05.1860] Client(00:d4:9e:54:6e:8d): apr1v2: WPA: received EAPOL-Key frame (2/4 Pairwise)
[*05/29/2024 11:55:05.1960] Client(00:d4:9e:54:6e:8d): apr1v2: WPA: sending 3/4 msg of 4-Way Handshake
[*05/29/2024 11:55:05.1960] Client(00:d4:9e:54:6e:8d): apr1v2: WPA: received EAPOL-Key frame (4/4 Pairwise)
[*05/29/2024 11:55:05.1960] Client(00:d4:9e:54:6e:8d): apr1v2: 802.1X: authorizing port
[*05/29/2024 11:55:05.1960] Client(00:d4:9e:54:6e:8d): apr1v2: WPA: pairwise key handshake completed (RSN)
[*05/29/2024 11:55:05.2660] Client(00:d4:9e:54:6e:8d): apr1v2: RADIUS: starting accounting session 0FD827614105CB66

PCAP on the switchport (Switch does not recieve message 4 from the AP):

So there is a problem on the AP, investigation by TAC still ongoing

[Rich R]

Apologies if I missed it - but what software version are you using @Djeten ?

[Djeten]

Well this is weird...

When I deployed the WLC, it was on version 17.7.1.
Because of these issues, I upgraded to version 17.9.5.
When I check the version now, it says it is back at 17.7.1... I am 100% sure the upgrade to 17.9.5 was completed succesfully, because I checked before I opened the TAC case...

[marce1000]

 - Check content of bootflash:  and examine the boot setting(s)  (boot variable) in the running configuration , 

 M.