Function BSafeESPDecrypt() failed with an error code of 0xffffffff

[virtualcomputer]

I am using VPN client version 5.0.05.0290.  After connection established, log file will have same error message continuously added (two repeats shown below).  The connection will stay up, VPN IP address and routes all look correct, but no communication between my PC and server can be achieved via VPN.

325    15:23:17.190  01/16/11  Sev=Warning/2 IPSEC/0xE3700010

BSafe ESP Decrypt HMAC mismatch.

326    15:23:17.190  01/16/11  Sev=Warning/2 IPSEC/0xE3700003

Function BSafeESPDecrypt() failed with an error code of 0xffffffff (IpSecDrvTransform.c:1776)

327    15:23:17.190  01/16/11  Sev=Info/4 IPSEC/0x63700019

Activate outbound key with SPI=0xadc41c07 for inbound key with SPI=0x829c6898

328    15:23:17.190  01/16/11  Sev=Warning/2 IPSEC/0xE3700010

BSafe ESP Decrypt HMAC mismatch.

329    15:23:17.190  01/16/11  Sev=Warning/2 IPSEC/0xE3700003

Function BSafeESPDecrypt() failed with an error code of 0xffffffff (IpSecDrvTransform.c:1776)

330    15:23:17.190  01/16/11  Sev=Info/4 IPSEC/0x63700019

Activate outbound key with SPI=0xadc41c07 for inbound key with SPI=0x829c6898

Thank you for your help!

Best regards,

Yuhua

[Jennifer Halim]

It doesn't seem to VPN Client issue at this stage.

Base on the error logs, the VPN Client driver is receiving the ESP packet with mismatch of the hash, that's why it's not working.

The ESP packet seems to be modified along the way which causes the hash to not match, and hence error log: "BSafe ESP Decrypt HMAC mismatch".

I saw issue with some NIC card on PC where there is some security feature that is turned on that causes that issue. You might want to check the NIC profile/feature, and try to turn off the security feature and see if that makes any difference. Otherwise, if you happen to have any firewall in front of your pc, it's worth a check as well to see if it makes any modification to ESP packet.

[virtualcomputer]

Thank you for your help!  I will check the driver.  On the other hand, since I have same PC that worked once but stop working (without changing anything), would the driver problem cause such behavior, or is it possible that the problem is caused by something else?

[Jennifer Halim]

It would be difficult to say unless we further troubleshoot the issue. Have you installed anything else in the PC since then? Have you tried with other PC in the same network?

[virtualcomputer]

No, I have not made any changes in PC and I tried on different PCs, results are the same.  I have also checked device dirver setting and no security features are enabled.  The deiver is xen network driver (so this is happening on VM).

On the server side, I have log messages listed below.  Can you tell from the error message on server and ones on the client side what can be potential cause of such problem?  some kind of key got switched at client before negotiation with server?

Much apprecaited!

Yuhua

Jan 17 2011 10:48:36: %ASA-5-713130: Group = ABC, Username = ylu, IP = 65.140.58.60, Received unsupported transaction mode attribute: 5
Jan 17 2011 10:48:41: %ASA-5-713119: Group = ABC, Username = ylu, IP = 65.140.58.60, PHASE 1 COMPLETED
Jan 17 2011 10:48:41: %ASA-3-713133: Group = ABC, Username = ylu, IP = 65.140.58.60, Mismatch: Overriding phase 2 DH Group(DH group 0) with phase 1 group(DH group 2)
Jan 17 2011 10:48:41: %ASA-3-713133: Group = ABC, Username = ylu, IP = 65.140.58.60, Mismatch: Overriding phase 2 DH Group(DH group 0) with phase 1 group(DH group 2)
Jan 17 2011 10:48:41: %ASA-3-713133: Group = ABC, Username = ylu, IP = 65.140.58.60, Mismatch: Overriding phase 2 DH Group(DH group 0) with phase 1 group(DH group 2)
Jan 17 2011 10:48:41: %ASA-3-713133: Group = ABC, Username = ylu, IP = 65.140.58.60, Mismatch: Overriding phase 2 DH Group(DH group 0) with phase 1 group(DH group 2)
Jan 17 2011 10:48:41: %ASA-3-713133: Group = ABC, Username = ylu, IP = 65.140.58.60, Mismatch: Overriding phase 2 DH Group(DH group 0) with phase 1 group(DH group 2)
Jan 17 2011 10:48:41: %ASA-3-713133: Group = ABC, Username = ylu, IP = 65.140.58.60, Mismatch: Overriding phase 2 DH Group(DH group 0) with phase 1 group(DH group 2)
Jan 17 2011 10:48:41: %ASA-3-713133: Group = ABC, Username = ylu, IP = 65.140.58.60, Mismatch: Overriding phase 2 DH Group(DH group 0) with phase 1 group(DH group 2)
Jan 17 2011 10:48:41: %ASA-3-713133: Group = ABC, Username = ylu, IP = 65.140.58.60, Mismatch: Overriding phase 2 DH Group(DH group 0) with phase 1 group(DH group 2)
Jan 17 2011 10:48:41: %ASA-3-713133: Group = ABC, Username = ylu, IP = 65.140.58.60, Mismatch: Overriding phase 2 DH Group(DH group 0) with phase 1 group(DH group 2)
Jan 17 2011 10:48:41: %ASA-3-713133: Group = ABC, Username = ylu, IP = 65.140.58.60, Mismatch: Overriding phase 2 DH Group(DH group 0) with phase 1 group(DH group 2)
Jan 17 2011 10:48:41: %ASA-5-713075: Group = ABC, Username = ylu, IP = 65.140.58.60, Overriding Initiator's IPSec rekeying duration from 2147483 to 28800 seconds
Jan 17 2011 10:48:41: %ASA-5-713049: Group = ABC, Username = ylu, IP = 65.140.58.60, Security negotiation complete for User (ylu)  Responder, Inbound SPI = 0x03543c63, Outbound SPI = 0x104939b6
Jan 17 2011 10:48:41: %ASA-5-713120: Group = ABC, Username = ylu, IP = 65.140.58.60, PHASE 2 COMPLETED (msgid=334aa610)

[Jennifer Halim]

No issue with the server side. As far as the server is concern the vpn tunnel is connected, that is why we are getting the error message on the vpn client. It is failing to decrypt the ESP packet.

Are any other users having the issue? or you are the only one who connects to the vpn?

I would also test from a different internet connection and see if you are having the same issue. This is to isolate where the issue is.a

[virtualcomputer]

Thank you Jennifer.  Yes, this problem happens to multiple external users from home office and on the road, I am able to reproduce the issue.

[Jennifer Halim]

Sorry, do you mean to say it happens to every users that connect to the VPN? or multiple users but not every user who connects to the vpn?

What is common between those users who are having the issue? Are they running the same version, same OS, etc.?

[virtualcomputer]

My apology for not explaining it clearer.  Please feel free to ask.  Yes, it worked on a couple of users for once, and stopped working after that.  I don't know if they keep trying, would it work again, but for the rest of users/PCs (dozens), it never worked.  Yes, every user has identical PC (same windows 7 virtual machine and same hardware).

[Jennifer Halim]

Apology for not spotting it earlier, but VPN Client on Virtual Machine is not supported. It may have worked by chance the first time, however, it is not a supported platform to run VPN Client on.

[virtualcomputer]

ok, I understand and I appreciate your help.  If you have any suggestion as to which part of IPSEC stack we should look into, it would be really helpful.

[Jennifer Halim]

Base on the error messages on the vpn client, the ESP packet might have been somehow modified, therefore the hash does not match.

You can try to configure just encryption but not hashing policy for the IPSec transform set, however, that would apply to all your other remote vpn clients too.

Eg: crypto ipsec transform-set myset esp-3des

If you have "esp-md5" or "esp-sha" configured, try without it and see if it makes any difference.