SSL handshake failure

pnavratil · ‎07-03-2017

Hi all,

am triing to implement Lets Encrypt certificate enrollment for Cisco ASA - I am using guide

so I configured all fof the required things but finaly I stuck on SSL handsake failure on ASA.

Small descryption: in process of cerificate enrolment the selfsigned certificate with proper attributes is generated by the script and uploaded to ASA - then LetsEncrypt verify that on the FQDN the right certificate is published - it is proove that I am administrator of the site.

So certificate is uploaded to ASA

I can see this config on ASA:

---

fw# sh runn | in acme
crypto ca trustpoint acme_challenge_707c7c706236003ae45722d237055cc9
keypair acme_challenge_707c7c706236003ae45722d237055cc9
crypto ca certificate chain acme_challenge_707c7c706236003ae45722d237055cc9
ssl trust-point acme_challenge_707c7c706236003ae45722d237055cc9 domain 5bc92a506e34158f9ed025d47e2f37c7.c3770b4529f6e561168f9e3244374368.acme.invalid

----

but when the Lets Encrypt tries to check it - the ASA sends back handshake failure.

I sniffed all the comunication and I can see SSL helo packet from let encrypt verification server - the only specific thing is the server name attribute is filled - so it use SNI for identification.

I checked the SNI requested by SSL client and it is identical to the string set in "ssl trust-point .." command.

But ASA answer is "Handshake failure"

I tried to debug it on ASA (ssl debug) but the only message a can get about it is:

---

error:1408A0C1:SSL routines:ssl3_get_client_hello:no shared cipher@s3_srvr.c:2053

---

I upload packet sniff from ASA where the whole communiaction is visible.

We found one bug

https://bst.cloudapps.cisco.com/bugsearch/bug/CSCve20346

which could cause this problem (I am using ASA-OS 9.8.1 and I need at least 9.7.1 - as VTI is used in ASA config)

Can somebody verify it?

Regards

Pavel

Marvin Rhoads · ‎07-03-2017

The bug is webvpn-specific and should not affect you.

Is host 66.33.139.36 the Let's Encrypt server? I see it proposing a good-sized list of strong ciphers. Then 82.113.54.14 (your ASA?) responds back with the handshake failure.

Can you share the ciphers your ASA supports? "show run ssl" and "show ssl ciphers" should give us the necessary information.

chrismarget · ‎07-03-2017

I don't believe the problem is webvpn specific, regardless of what the bug notes say.

The issue is with the TLS implementation. Heck, how could we know if the TLS client has webvpn intentions at the point it fails?

If the TLS client specifies a name (SNI extension) which matches an "ssl trust-point X domain X" directive, then the handshake fails and the "no shared cipher" error is logged.

Change the name in the client's SNI header, or switch from domain-based to interface-based server certificate selection and the the TLS handshake goes through with no problem.

pnavratil · ‎07-04-2017

First to answer Marvin questions:

Yes - you are right -

66.33.139.36 is Lets Encrypt Server
82.113.54.14 is outside interface of ASA

All ciphers setting is left in default - it is medium - this is output from command

show ssl cipher:

---

default (medium):
ECDHE-ECDSA-AES256-GCM-SHA384
ECDHE-RSA-AES256-GCM-SHA384
DHE-RSA-AES256-GCM-SHA384
AES256-GCM-SHA384
ECDHE-ECDSA-AES256-SHA384
ECDHE-RSA-AES256-SHA384
DHE-RSA-AES256-SHA256
AES256-SHA256
ECDHE-ECDSA-AES128-GCM-SHA256
ECDHE-RSA-AES128-GCM-SHA256
DHE-RSA-AES128-GCM-SHA256
AES128-GCM-SHA256
ECDHE-ECDSA-AES128-SHA256
ECDHE-RSA-AES128-SHA256
DHE-RSA-AES128-SHA256
AES128-SHA256
DHE-RSA-AES256-SHA
AES256-SHA
DHE-RSA-AES128-SHA
AES128-SHA
tlsv1 (medium):
DHE-RSA-AES256-SHA
AES256-SHA
DHE-RSA-AES128-SHA
AES128-SHA
tlsv1.1 (medium):
DHE-RSA-AES256-SHA
AES256-SHA
DHE-RSA-AES128-SHA
AES128-SHA
tlsv1.2 (medium):
ECDHE-ECDSA-AES256-GCM-SHA384
ECDHE-RSA-AES256-GCM-SHA384
DHE-RSA-AES256-GCM-SHA384
AES256-GCM-SHA384
ECDHE-ECDSA-AES256-SHA384
ECDHE-RSA-AES256-SHA384
DHE-RSA-AES256-SHA256
AES256-SHA256
ECDHE-ECDSA-AES128-GCM-SHA256
ECDHE-RSA-AES128-GCM-SHA256
DHE-RSA-AES128-GCM-SHA256
AES128-GCM-SHA256
ECDHE-ECDSA-AES128-SHA256
ECDHE-RSA-AES128-SHA256
DHE-RSA-AES128-SHA256
AES128-SHA256
DHE-RSA-AES256-SHA
AES256-SHA
DHE-RSA-AES128-SHA
AES128-SHA
dtlsv1 (medium):
DHE-RSA-AES256-SHA
AES256-SHA
DHE-RSA-AES128-SHA
AES128-SHA

---

I just made several tests and it seems to mi Chris is right - it is probably not only webvpn (webvpn portal) related bug - or it is some other bug.

I tried to shutdown webvpn portal, then also run portal on different port then 443 - I allowed connection to admin portal (ASDM )from the Lets Encrypt IP - but I obtain always the same error.

From my communication with Chris (the author of the Lets Encrypt ASA enrolment script - thank you for helping to solve the issue)

he tried to use ASA-OS version 9.3.2 and no issue appeared - unfortunately I cannot verify this as I need to use at least 9.7.1.

I will try to investigate the issue more - if I will find anything interesting - I will post the info.

Marvin Rhoads · ‎07-04-2017

I agree - it certainly sounds like the bug is not just webvpn-specific.

Thanks to Chris for the enlightening comments.

Pavel, your ciphers are all in order and, if the ASA was behaving properly, it should select the strongest common cipher between it and the Let's Encrypt server.

Please do keep us updated with the TAC's findings. This bug can potentially affect lots of customers.

Regards.

- Marvin

pnavratil · ‎07-04-2017

I can confirm it is not only webvpn related. I played a little bit with SNI (domain) setting on ASA and tried to connect from linux cli with openssl.
I setup testing SNI record pointing to internal certificate - used with no problem in other cases.
If I try connection without SNI (-servername option) or with non-existent servername host, the https connection is succesfull and returns certificate from interface specification.
Once I specify existing SNI }defined on ASA] in -servername option - connection fails:----
root@kozel:~# openssl s_client -connect 192.168.1.1:443 -showcerts -servername pokus.cz
CONNECTED(00000003)
3073996476:error:14077410:SSL routines:SSL23_GET_SERVER_HELLO:sslv3 alert handshake failure:s23_clnt.c:757:

----

producing the same debug error on ASA:

---

error:1408A0C1:SSL routines:ssl3_get_client_hello:no shared cipher@s3_srvr.c:2053

----

Then I tried to disable webvpn portal on accessing ASA interface - unless I allowed https admin access to ASA the packets were silently dropped on ASA (proove it was really admin interface) - when I allowed admin access from testing IP, the situation is the same - ssl connection is successful only in case no servername option or non-existenn SNI specified - once I use SNI defined on ASA, ssl connetion fails with the same error.
In case you will need some more detail diagnostic or anything other (techsupp ..) - let me know.

pnavratil · ‎07-14-2017

Only small note - I tried to upgrade to latest ASA-OS - 9.8.1-5 with the same result / issue.

I just checked the bug and it was updated with info it will be solved in 9.8.(1.6).

Marvin Rhoads · ‎07-14-2017

Thanks for updating the thread for all to be apprised of the latest status. That's quite helpful.

pnavratil · ‎08-04-2017

New ASA-OS had been released on October 1st - 9.8(1)7 - yesterday I upgraded our ASA and I can confirm the issue is gone - I was succesfull in obtaining Lets Encrypt certificate for our ASA.

Marvin Rhoads · ‎08-04-2017

Thanks for the update.

It appears they documented it as a bug:

https://bst.cloudapps.cisco.com/bugsearch/bug/CSCve20346

The release notes confirm the fix as well:

http://www.cisco.com/web/software/280775065/138735/ASA-981-Interim-Release-Notes.html

chrismarget · ‎08-04-2017

If you have some feedback about the plugin or its documentation, I'd be delighted to hear it.

The walkthrough (that I wrote) makes perfect sense to me :)

Other perspectives would be helpful.

I'm going to be doing some maintenance to add support for TLS-SNI-02 soon, could make other tweaks if they're needed.

jacob.mmchan · ‎07-19-2017

i am seeing the interim release in 9.6 and 9.4 series have covered the bug fix for this issue. unfortunately, i am still looking into 9.7 / 9.8 series.

i hope it can be released soon.