dot1x Clients can't get authenticated du to Cred Fail on Cisco 9800

[FreddyJay]

Hi!

clients cannot join our dot1x SSIDs. We get below messages in our WLC 9800:

WLC1#
Jan 17 10:46:44.155: %DOT1X-5-FAIL: Chassis 1 R0/0: wncd: Authentication failed for client (ee23.093e.5580) with reason (Cred Fail) on Interface capwap_90000002 AuditSessionID 1964900A000006A7BF56A2A1 Username: 
Jan 17 10:46:44.155: %SESSION_MGR-5-FAIL: Chassis 1 R0/0: wncd: Authorization failed or unapplied for client (ee23.093e.5580) on Interface capwap_90000002 AuditSessionID 1964900A000006A7BF56A2A1. Failure reason: Authc fail. Authc failure reason: Cred Fail.

our platform is: Cisco IOS Software [Bengaluru], C9800 Software (C9800_IOSXE-K9), Version 17.6.4, RELEASE SOFTWARE (fc1)

Would you please guide me what is the remedy?

Best regards

Farkhan

[marce1000]

                   >....Authc fail. Authc failure reason: Cred Fail.
 - The reason(s) seems obvious , client credentials are incorrect, if you have backend radius authenticating server(s) , then check the radius logs too.
   You may also find these commands useful on the 9800 controller  :
                   show wireless stats client delete reasons
                   show wireless stats client detail
                   show wireless client summary 

    Also review the 9800   configuration with the CLI command : show  tech   wireless , have the output analyzed by  https://cway.cisco.com/tools/WirelessAnalyzer/  , please note do not use classical show tech-support (short version) , use the command denoted in green for Wireless Analyzer.               Checkout all advisories!

 M.

[Gaurav Kansal]

Hello Farkhan

Have you checked the Logs in your AAA server, what logs its showing.Because its credentials failed.Please checked the policy also you have applied for dot1x. And Please confirm all user are facing issue or some particular clients.

[zachhoiberg]

I am seeing the same issue on 17.9.2 using 2802I and 2802E APs. Central authentication for a DOT1X WLAN does not work and no Radius requests are sent from any interface on the C9800-CL device to the radius servers. Tested this with a packet capture at both ends. It is not a communication issue between the C9800-CL and the radius servers, as the "test aaa" command sends radius packets just fine.

Any legitimate connection attempt to the SSID will instantly result in the (Cred Fail) log, even if the credentials are correct.

Disabling central authentication allows the APs themselves to send the DOT1X requests which pass as expected.

I have an open ticket with TAC regarding this issue.

You may want to test with non-central authentication, provided that you setup the wireless client on your RADIUS servers to accommodate the APs doing the authentication themselves.

[Schulda]

Hi

Have you had any success with TAC so far? We seem to have a similar issue now in our environment..

Thanks

[Scott Fella]

@Schulda you should start a new thread.  Majority of folks have a different setup and that makes a big difference.  Start a new thread and add as much information as possible.  Add what troubleshooting you have done, if this is a new setup or existing, has it worked before, is it with a specific device type or authorization type, etc.

[zachhoiberg]

I have not had any luck thus far. It seems like the EAP request is sent between the AP and the Controller during Central Authentication, and I see the username passing, but once that packet hits the Controller (which is nearly instantly), the controller doesn't forward it to the defined radius servers and marks it as a failed authentication attempt. 

I hope to get more information from TAC next week when I have more time to troubleshoot this further. Additionally, I'd like to state that I am using 2800 series APs for all of this testing, incase this happens to be a model specific issue. I will finally get some 9120s in next week for testing as well.

[Scott Fella]

What error do you see on the radius server?  Are you using ISE?

[zachhoiberg]

I saw nothing on the radius server logs and no traffic hitting the radius server with a packet capture running on the server. We are using Windows NPS, not ISE.

In some more testing this week, Central Authentication is now working at a separate site with a stronger WAN connection. The testing setup used originally was using a cell router for its WAN. Additionally, on that weaker WAN connection, I would occasionally see the CAPWAP tunnels drop and re-establish, presumably due to that weaker connection. I have not seen the same behavior at the new site.

It could be that there is some hidden threshold that the test setup was not meeting that the new site now is, and I am waiting to hear more from TAC after reporting my latest findings.

[marce1000]

            @zachhoiberg      >...seeing the same issue...
                           Did you also check the same advisories :
                     >....Authc fail. Authc failure reason: Cred Fail.
 - The reason(s) seems obvious , client credentials are incorrect, if you have backend radius authenticating server(s) , then check the radius logs too.
   You may also find these commands useful on the 9800 controller  :
                   show wireless stats client delete reasons
                   show wireless stats client detail
                   show wireless client summary 

    Also review the 9800   configuration with the CLI command : show  tech   wireless , have the output analyzed by  https://cway.cisco.com/tools/WirelessAnalyzer/  , please note do not use classical show tech-support (short version) , use the command denoted in green for Wireless Analyzer.               Checkout all advisories!

- If still not resolved perform client debugging as described in : https://logadvisor.cisco.com/logadvisor/wireless/9800/9800ClientConnectivity , you can have client debugs analyzed with : https://cway.cisco.com/wireless-debug-analyzer

 M.

[gaurav mahajan1]

Any update from TAC

even i am observing the same issue

Authentication failed for client (70a8.d39b.f5c8) with reason (Cred Fail) on Interface capwap_90000010 AuditSessionID 0408FA0A000869CBB2E052E7 Username: host/d3187f91-6ca9-4969-ae54-9fe6e0015c2a.beig.birdseyeiglo.com

[CSCO11177789]

same environment cisco 9800 and nps server as a radius side.

and taking similer logs like;

May 7 20:43:24: %SESSION_MGR-5-FAIL: Chassis 2 R0/0: wncd: Authorization failed or unapplied for client (0028.f88f.63f3) on Interface capwap_90000005 AuditSessionID F003320A0000011EF74F7F31. Failure reason: Authc fail. Authc failure reason: Cred Fail.

any update from tac ?

[zachhoiberg]

Nothing concrete, but I did update to 17.9.3 and tested the same configuration from a different site (with a stronger WAN signal, our test setup was using a cell modem for its primary connection), and I've been unable to replicate the same behavior since.

I'm unsure if it was simply the update to 17.9.3, the stronger WAN signal (as I was observing the CAPWAP tunnels occasionally timeout/drop from the test setup, which I have not seen since), or a combination of the two.

Do you guys also see the CAPWAP tunnels timing out within your logs periodically?

[GEgishira]

Hello All,

I was facing the same issue in my production environment, and just found a solution for my issue. I am working with a 9800-L Wireless Controller and NPS as our radius server. 

After checking the logs on my failed authentication requests, I found that under the Authentication Details the request was calling the wrong Network Policy Name. We use a policy that grants us privilege 15 to cisco command lines,  and that is what was being called. I moved my new policy ( the one I'm trying to get to work) above the one being called in processing order, and I was able to authenticate finally. 

This was a unique situation, but I hope this info helps some. 

[TriAngel]

我在现网也碰到了相同的问题，我检查了客户端证书，发现证书过期了，更新了证书后客户端就可以正常使用网络。