12511 Unexpectedly received TLS alert message; treating as a rejection by the client

[Larry Bernard]

ISE Version: 1.2.0.899 (Running in VMware)

WLC: 5508 ver 7.6.100.0

I have a WLAN created that uses dot1x authentication. The WLAN points to ISE for RADIUS AAA. I cannot get any windows computer to connect to it (7,8 or 8.1 tested), but android, ios and osx are all able to connect. I have a 3rd party cert (GoDaddy) installed in my local store in ISE, which is valid and not expired. I do not understand why windows machines are failing.

I am migrating to this new ISE server and my old ISE server has the same configuration (as far as I can tell) for this WLAN and it works for all devices, including windows. The difference is that it is on a different domain (the reason for the migration is we changed domains).

Here is the ISE error:

Event: 5400 Authentication failed

Failure Reason: 12511 Unexpectedly received TLS alert message; treating as a rejection by the client

Resolution: Ensure that the ISE server certificate is trusted by the client, by configuring the supplicant with the CA certificate that signed the ISE server certificate. It is strongly recommended to not disable the server certificate validation on the client!

Root cause: While trying to negotiate a TLS handshake with the client, ISE received an unexpected TLS alert message. This might be due to the supplicant not trusting the ISE server certificate for some reason. ISE treated the unexpected message as a sign that the client rejected the tunnel establishment.

Here is the WLC error:

AAA Authentication Failure for UserName:Domain\User User Type: WLAN USER

Here is the windows event viewer error:

Source:        Microsoft-Windows-Security-Auditing
Event ID:      5632

Description:
A request was made to authenticate to a wireless network.

Subject:
    Security ID:        NULL
    Account Name:        User
    Account Domain:        Domain

Network Information:
    Name (SSID):        IT-Test

Additional Information:
    Reason Code:        Explicit Eap failure received (0x50005)
    Error Code:        0x80420014
    EAP Reason Code:    0x80420100
    EAP Root Cause String:    Network authentication failed\nThe user certificate required for the network can't be found on this computer.

    EAP Error Code:        0x80420014

On the ISE server that is working you are presented with a window that asks you to connect or terminate based on the certificate not being validated. I don't know why that isn't happening with this new ISE server, it just fails without prompting the user to connect or terminate. Both certs are from GoDaddy.

A difference between the certs is the old has a cert that was generated through ISE and the new server has an imported wildcard cert.

Anyway, I hope that is enough information to understand the issue. I appreciate the time anyone takes in assisting me with this issue. I did setup a copy of the WLAN so that I can test as needed and not have to wait for a maintenance window.

[Saurav Lodh · Accepted solution]

Some endpoint devices (Windows OS)  have issues with wildcard cert when CN contains * (start) as wildcard
>
> the PEAP authentication fails due to "12511 Unexpectedly received TLS alert message; treating as a rejection by the client"
>

> <B>Conditions:</B>
> when the  wildcard cert  contains  * (start) as wildcard in CN
>
> <B>Workaround:</B>
>
> create wildcard with * (start)
> e.g. CN= aaa.cisco.com

[Larry Bernard]

Thanks for your prompt reply. If I understand you correctly, the workaround is to essentially NOT use a wildcard certificate?

Here is another thing. In the certificate operations section I moved EAP to the self-signed certificate and the behavior is the same, but the error is different. The self-signed cert isn't a wildcard and it still fails on windows only.

ISE Error:

Event: 5400 Authentication failed
Failure Reason: 12321 PEAP failed SSL/TLS handshake because the client rejected the ISE local-certificate
Resolution: Check whether the proper server certificate is installed and configured for EAP in the Local Certificates page ( Administration > System > Certificates > Local Certificates ). Also ensure that the certificate authority that signed this server certificate is correctly installed in client's supplicant. Check the previous steps in the log for this EAP-TLS conversation for a message indicating why the handshake failed. Check the OpenSSLErrorMessage and OpenSSLErrorStack for more information.
Root cause: PEAP failed SSL/TLS handshake because the client rejected the ISE local-certificate

Obviously the self-signing CA isn't in the local machines store.

[Larry Bernard]

Nevermind, I get it now. I found the answer spelled out right here:

http://www.cisco.com/c/en/us/td/docs/security/ise/1-2/user_guide/ise_user_guide/ise_man_cert.html#pgfId-1171626

[Qingxin Yang]

Hi,Larry:

   I face this issue same to you. Could you tell me how did you fix this problem? very tks.

Can you send me E-mail for a solution? Email:liuyifeibuai@126.com

[ndemers]

This is because the client cannot verify the certificate chain (roots ca) of ISE.  Basically ask yourself this.  (This is all just an example) If you someone shows you an ID you, in theory, should be able to validate that ID via a 3rd party.  You cant trust the ID until you verify with a seperate 3rd party that says you can trust it.

The root ca is itself so its as if some shady guy said, " Dude, you can trust me."

The only way this would work is if you took the ISE root ca cert (itself) and imported it onto the client under trusted root ca.

[ndemers]

I noticed you said it was godaddy.  They by default replace the CN with the wildcard SAN

This will not work because the CN needs to be a host for Windows machines.   

CSR Example:

CN: ise01.ise.example.com

SAN: ise01.ise.example.com , *.ise.example.com

CERTIFICATE GIVEN:

CN: *.ise.example.com

SAN: *.ise.example.com, ise.example.com