AnyConnect Random Disconnects for some users.

[TM876]

Hi all,

I have some users that are facing some disconnects from AnyConnect.

This is the error they get:

The server certificate received from the secure gateway during the reconnect attempt differs from the one received during the initial connection. A new connection is necessary, which requires re-authentication.

They are using AnyConnect version 4.4.02039. If anyone has any pointers on what to look for then that would be greatly appreciated.

Thanks in advance.

[Mike.Cifelli]

The server certificate received from the secure gateway during the reconnect attempt differs from the one received during the initial connection. A new connection is necessary, which requires re-authentication.

-A few thoughts that may assist:

--Test installing a newer/later AC client and see if issue goes away.  4.4 is ancient and vulnerable.

--Have you verified that the presented cert is actually different? 

--Not sure of your setup.  Can you determine that on reconnect you are hitting same VPN box? 

--Run DART on a troubled client and check event viewer log to see if anything pops out there that may assist

[TM876]

Hi Mike,

Thanks for replying. I'm not very experienced so please bear with me if I make any mistakes. I noticed that the disconnects happen around the same time so for one user logs in at 8AM then gets disconnected at around 9:40AM~, then another disconnect at 11:20AM~. This then continues and happens throughout the day. The pattern seems to repeat daily where the user get disconnected after an hour and 30 mins to 2 hours. Also some further information, it seems to be only affecting 2 users out of around 10 or so and there is also a 30 minute idle timeout for the VPN. Could it just be the idle timeout all this time?

• We have tried to update to 4.10 and the issue still happens.
• Not sure how I would go about checking if the certs are different on client side.
• I will try double check this but it should be hitting the same box.
• Got the DART bundle. I'll paste below some stuff that may help. It's from AnyConnect event viewer logs. In time ascending order.

Line: 2153 Invoked Function: ::WSAGetOverlappedResult Return Code: 10054 (0x00002746) Description: An existing connection was forcibly closed by the remote host. Associated overlapped operation is ::WSARecv

Line: 801 Invoked Function: CSocketTransport::readSocket Return Code: -31588312 (0xFE1E0028) Description: SOCKETTRANSPORT_ERROR_GET_RESULT_FAILURE:The system get result call for the socket failed.

Tunnel level reconnect reason code 6: Disruption of the VPN connection to the secure gateway. Caching the default reconnect reason for SSL

The Primary SSL connection to the secure gateway is being re-established.

Line: 432 Invoked Function: ::WSASend Return Code: 10054 (0x00002746) Description: An existing connection was forcibly closed by the remote host.

Line: 1378 Invoked Function: CSocketTransport::internalWriteSocket Return Code: -31588341 (0xFE1E000B) Description: SOCKETTRANSPORT_ERROR_WRITE Local Addr: [192.168.0.31]:50275, Remote Addr: [{VPN Server IP}]:443

Line: 1077 Invoked Function: CSocketTransport::writeSocket Return Code: -31588341 (0xFE1E000B) Description: SOCKETTRANSPORT_ERROR_WRITE

The VPN client has sent the following close message to the gateway: Reconnecting the VPN tunnel.

A SSL Alert was sent by the client during a write operation. Severity: warning Description: close notify

VPN state: Reconnecting Network state: Network Accessible Network control state: Network Access: Restricted Network type: Undefined

Failed to verify Server Certificate. Certificate differs from previously verified.

Description: CERTSTORE_ERROR_HASH_MISMATCH

A SSL Alert was sent by the client during a write operation. Severity: fatal Description: certificate unknown

Invoked Function: SSL_do_handshake Return Code: 337047686 (0x1416F086) Description: error:1416F086:SSL routines:tls_process_server_certificate:certificate verify failed

Invoked Function: initialHandshake Return Code: -31457268 (0xFE20000C) Description: CERTSTORE_ERROR_HASH_MISMATCH

Description: CERTSTORE_ERROR_HASH_MISMATCH SSL tunnel state 0

Line: 1600 The HTTPS probe to {IP address of server} resulted in a redirect.

Line: 1814 SG ({IP address of server}) contacted

Termination reason code 101: The server certificate received from the secure gateway during the reconnect attempt differs from the one received during the initial connection.

[Mike.Cifelli]

Some additional thoughts:

-Is the 2 troubled users configuration different than the working users? I would take a known good and known bad and start with comparisons: do they use the same tunnel group? Same ASA group policy? Same AC versions? Same AC VPN profile (can be found here on win clients: C:\ProgramData\Cisco\Cisco AnyConnect Secure Mobility Client\Profile), Same third party software on client image, etc.

-To check/add certs to local client trust stores you can do some on Windows clients via MMC (microsoft management console); the respective chain would need to be in the trust store in order to be trusted by remote clients; see here for example: How to Use the Certificates Console - TechNet Articles - United States (English) - TechNet Wiki (microsoft.com)

-Check the ASA group policy to determine if there is a max connect time configured

-The HTTPS probe to {IP address of server} resulted in a redirect. --IMO this answers the concern that you are hitting another box; Are the ASAs behind a load balancer or in a cluster?

-The server certificate received from the secure gateway during the reconnect attempt differs from the one received during the initial connection. --Verify the identity certs being presented;  Actually would be surprised if the chain was different between ASAs, but something to check.

-Engage TAC