Anyconnect: CRL unreachable with Always-on

[mhost-vejle]

Hello,

We are using the AnyConnect Secure Mobility Client with an ASA 5520 and self-signed certificates.

Initial connections work fine.

But if the clients do not connect for some time, the Windows CRL cache apparently expires, CRL check fails and the client does not connect.

It seems that "Always-on" prevents the client from accessing the CRL URL, since it is on an untrusted network - and when revocation check is unavailable, the Anyconnect client fails with certificate error.

We can recreate the problem by clearing the CRL cache (certutil -urlcache crl delete)

- We can also work around the problem by temporarily disabling Always-On - allowing the client to perform revocation check.

How do we allow the client to access the CRL URL prior to etablishing the VPN connection?

Split tunneling? Exceptions?

Best regards

Martin

[micromedicreturns]

Hi, just thought I would add to this discussion.  We have the same problem as you, down to the letter.  We also suspected the CRL checking, but when looking at some Wireshark traces, the client does not appear to be even attempting to contact the CRL, at least when it is failling to connect in Always On mode.

There must be more to it, as it would be strange if the AC client blocked itself from checking, given that it must be allowed to check the CRL for Always On.

I will check the split tunneling option and let you know how it goes.

[micromedicreturns]

A guy in another thread said he talked to Cisco TAC and they said there was a bug in the CRL checking where the CRL URL is load balanced.  I realised that split tunneling was not the answer as that would require a connection first.

[mhost-vejle]

Thanks for the replies.

The Wireshark result is quite interesting. Wonder what the issue is then?

I saw the thread about load balanced CRL URL as well, but discarded it at first, since we host our CRL on a single webserver.

I take it hat "Load balanced CRL URL" means that the CRL is hosted on load balanced web servers?

We are considering opening a case with Cisco. If so, I will post the result.

/Martin

[micromedicreturns]

No problems - the certutil command you posted pinned it down for us also.  It is very quirky as before we enable "always on", the trace reveals the client looking up the CRL in DNS successfuly, then checking one or two CRLs.  Once "always on" is enable and the cache cleared, it still looks up DNS, but does no more.  The CRL we check is actually an alias, and may even be re-directing.  I would guess that the load balancing refers to DNS records, and that the always on feature doesn't like the redirecting, but that is just a guess.

There is a similar sounding bug on the Cisco website, and is addressed in a forthcoming release of the client, but nothing definite or conclusive is on there.

[mhost-vejle]

Well...

The "CRL Distribution Points" in our certificate contains multiple values:

- An LDAP URL

- An internally accessible CRL URL

- An externally accessible CRL URL (using public IP-address)

So maybe this will cause the "load balance" issue as well.

We have opened a case with our danish Cisco vendor and have sent them DART-logs.
- will post result.

If the case takes too long to solve, I might consider testing a free certificate from startssl.com.

/Martin

[micromedicreturns]

In case you haven't received a response yet from your vendor - the latest version of the client was made available to download today.  So far it appears to function properly, even after deleting the CRL cache.  We did have a slight issue with ASDM though, so I upgraded it to the latest version also.  Then I opened the Client profile stored on the ASA to make sure it was still valid with the latest schema (included when the image of the client was uploaded to the ASA).  Deleting the XML file on the test PC and letting it download a new one seemed to be a necessary step, since I did have some problems connecting.

If it still functions on Monday, I will assume the problem is solved!

[mhost-vejle]

Hi,

Thank you for the update.

We have just had the opportunity to test v3.0.4235 here.

Unfortunately it does not seem to solve our problem - although the certificate error is a bit different now.

So we will await a response in the case we opened.

/Martin

[mhost-vejle]

Just another update.

We tried with a new certificate on the ASA - Just a free one from StartSSL, which is trusted by most browsers.

But we still get cert errors. Strange...

[micromedicreturns]

Definitely weird - we did have multiple issues with our system, of which the CRL checking was the last to be discovered.  Perhaps you have DNS lookup errors, for example.  Did you have a look at a Wireshark trace?  The client should be visibly looking up at least the IP address of the CRL, if not the CRL itself.  Will be interesting to see what is failing still.

[mhost-vejle]

There is certainly a problem med CRL checking of our self-signed cert.

Anyconnect client seems to ignore the multiple CRL URLs - and only tries to resolve the internal URLs.

(These are, of course, unreachable)

But I guess that we are back to the load balanced CRL URL here.

I did however edit a client's hosts file, redirecting the CRL URLs to a public IP-address, where the CRLs are available.

This stopped the failed DNS lookups. But still resulted in the error below:

Description: FILEDOWNLOADER_ERROR_UNTRUSTED_CERTIFICATE_WITH_ALWAYS_ON

Failed to download from https://ciscoasa.vejle.dk/CACHE/stc/4/VPNManifest.xml to C:\Users\mhost\AppData\Local\Temp\19384.tmp\VPNManifest.xml

It was the same result with the StartSSL certificate.

Maybe there is an issue with the XML-file on the ASA itself.

So we may try to upload a new client and XML-file to the ASA.

Strangely though, the problem still only exists when Always-On is used.

/Martin

[shn]

Hi

Did anyone find a solution for this problem

[mhost-vejle]

Hi Søren,

We currently have an open case with Netdesign here in Denmark.

I have just received a mail from them stating that they might have a solution for the problem, which they would like to test.

I am hoping to test it tomorrow.

Will get back to you after that.

Best regards

Martin

[mhost-vejle]

We accidentally found a workaround for our problem yesterday.

It has been tested on a handful of clients today – and seems to work without problems.

Firstly we have upgraded to AnyConnect client version 3.0.4235.

This solved the CRL-issue, since this version does not perform CRL checks by default.

Secondly we disabled client Auto Update (for other reasons).

This happened to solve our problems entirely, since the client failed when trying to check for updates.

We will still have to push the new client and client profile (with “AutoUpdate=False”) to our users.

But we have decided to accept the workaround for now.

Best regards

Martin

[bo.morgan]

I know this may seem overly simplistic but we are having issues with auto udpate and I can recreate the FILEDOWNLOADER_ERROR_UNTRUSTED_CERTIFICATE_WITH_ALWAYS_ON error by changing the logged on user's rights from modify to read and execute on the user's TEMP location.  If you have group policies or scripts that change NTFP permissions this may be causing an issue.  Verify that the user has modify rights to both c:\programdata\cisco\cisco anyconnect secure mobility client\profile and the user's TEMP location.

Good Luck!

~Bo