Ipsec vpn tunnel goes down on one side only

michellp · ‎09-21-2012

Hello,

We have a ipsec vpn tunnel between two locations. On our end we use a cisco Asa. The remote end uses Juniper netscreen. Now, the vpn tunnel works for a period of time but then is torn down, as it appears, from the asa side due to loss of service after not receiving DPD R-U-THERE-ACK on 3 consecutive DPD R-U-THERE's. There is no event that takes place that causes this. It just randomly happens. The DPD traffic might go back and forth for hours and then out of no where BOOM it stops.

The netscreen however seems to believe the tunnel is up and when its sending traffic it is discarded by our ASA, as can be seen in the last part of the log. This issue, as it seems, can only be solved by bringing down the vpn on the remote end (netscreen) and then it starts working as normal as ipsec phase 1 is established and then phase 2 and so on.

See logging:

Sep 20 2012 21:59:05 FW1 : %ASA-7-715075: Group = 190.90.24.10, IP = 190.90.24.10, Received keep-alive of type DPD R-U-THERE-ACK (seq number 0x5d0455e6)

Sep 20 2012 21:59:25 FW1 : %ASA-7-715036: Group = 190.90.24.10, IP = 190.90.24.10, Sending keep-alive of type DPD R-U-THERE (seq number 0x5d0455e7)

Sep 20 2012 21:59:25 FW1 : %ASA-7-715046: Group = 190.90.24.10, IP = 190.90.24.10, constructing blank hash payload

Sep 20 2012 21:59:25 FW1 : %ASA-7-715046: Group = 190.90.24.10, IP = 190.90.24.10, constructing qm hash payload

Sep 20 2012 21:59:25 FW1 : %ASA-7-713236: IP = 190.90.24.10, IKE_DECODE SENDING Message (msgid=f290ed30) with payloads : HDR + HASH (8) + NOTIFY (11) + NONE (0) total length : 84

Sep 20 2012 21:59:25 FW1 : %ASA-7-713236: IP = 190.90.24.10, IKE_DECODE RECEIVED Message (msgid=99b31b2e) with payloads : HDR + HASH (8) + NOTIFY (11) + NONE (0) total length : 84

Sep 20 2012 21:59:25 FW1 : %ASA-7-715047: Group = 190.90.24.10, IP = 190.90.24.10, processing hash payload

Sep 20 2012 21:59:25 FW1 : %ASA-7-715047: Group = 190.90.24.10, IP = 190.90.24.10, processing notify payload

Sep 20 2012 21:59:25 FW1 : %ASA-7-715075: Group = 190.90.24.10, IP = 190.90.24.10, Received keep-alive of type DPD R-U-THERE-ACK (seq number 0x5d0455e7)

Sep 20 2012 21:59:45 FW1 : %ASA-7-715036: Group = 190.90.24.10, IP = 190.90.24.10, Sending keep-alive of type DPD R-U-THERE (seq number 0x5d0455e8)

Sep 20 2012 21:59:45 FW1 : %ASA-7-715046: Group = 190.90.24.10, IP = 190.90.24.10, constructing blank hash payload

Sep 20 2012 21:59:45 FW1 : %ASA-7-715046: Group = 190.90.24.10, IP = 190.90.24.10, constructing qm hash payload

Sep 20 2012 21:59:45 FW1 : %ASA-7-713236: IP = 190.90.24.10, IKE_DECODE SENDING Message (msgid=123d5289) with payloads : HDR + HASH (8) + NOTIFY (11) + NONE (0) total length : 84

Sep 20 2012 21:59:47 FW1 : %ASA-7-715036: Group = 190.90.24.10, IP = 190.90.24.10, Sending keep-alive of type DPD R-U-THERE (seq number 0x5d0455e9)

Sep 20 2012 21:59:47 FW1 : %ASA-7-715046: Group = 190.90.24.10, IP = 190.90.24.10, constructing blank hash payload

Sep 20 2012 21:59:47 FW1 : %ASA-7-715046: Group = 190.90.24.10, IP = 190.90.24.10, constructing qm hash payload

Sep 20 2012 21:59:47 FW1 : %ASA-7-713236: IP = 190.90.24.10, IKE_DECODE SENDING Message (msgid=815f7542) with payloads : HDR + HASH (8) + NOTIFY (11) + NONE (0) total length : 84

Sep 20 2012 21:59:49 FW1 : %ASA-7-715036: Group = 190.90.24.10, IP = 190.90.24.10, Sending keep-alive of type DPD R-U-THERE (seq number 0x5d0455ea)

Sep 20 2012 21:59:49 FW1 : %ASA-7-715046: Group = 190.90.24.10, IP = 190.90.24.10, constructing blank hash payload

Sep 20 2012 21:59:49 FW1 : %ASA-7-715046: Group = 190.90.24.10, IP = 190.90.24.10, constructing qm hash payload

Sep 20 2012 21:59:49 FW1 : %ASA-7-713236: IP = 190.90.24.10, IKE_DECODE SENDING Message (msgid=bf6a9681) with payloads : HDR + HASH (8) + NOTIFY (11) + NONE (0) total length : 84

Sep 20 2012 21:59:51 FW1 : %ASA-3-713123: Group = 190.90.24.10, IP = 190.90.24.10, IKE lost contact with remote peer, deleting connection (keepalive type: DPD)

Sep 20 2012 21:59:51 FW1 : %ASA-7-713906: Group = 190.90.24.10, IP = 190.90.24.10, IKE SA MM:ac53953d rcv'd Terminate: state MM_ACTIVE flags 0x00000042, refcnt 1, tuncnt 1

Sep 20 2012 21:59:51 FW1 : %ASA-7-720041: (VPN-Primary) Sending Phase 1 Terminate message (type L2L, remote addr 190.90.24.10, my cookie AC53953D, his cookie D85D5EAA) to standby unit

Sep 20 2012 21:59:51 FW1 : %ASA-7-713906: Group = 190.90.24.10, IP = 190.90.24.10, sending delete/delete with reason message

Sep 20 2012 21:59:51 FW1 : %ASA-7-715046: Group = 190.90.24.10, IP = 190.90.24.10, constructing blank hash payload

Sep 20 2012 21:59:51 FW1 : %ASA-7-715046: Group = 190.90.24.10, IP = 190.90.24.10, constructing IPSec delete payload

Sep 20 2012 21:59:51 FW1 : %ASA-7-715046: Group = 190.90.24.10, IP = 190.90.24.10, constructing qm hash payload

Sep 20 2012 21:59:51 FW1 : %ASA-7-713236: IP = 190.90.24.10, IKE_DECODE SENDING Message (msgid=3c62f12c) with payloads : HDR + HASH (8) + DELETE (12) + NONE (0) total length : 68

Sep 20 2012 21:59:51 FW1 : %ASA-7-713906: Group = 190.90.24.10, IP = 190.90.24.10, Active unit receives a delete event for remote peer 190.90.24.10.

Sep 20 2012 21:59:51 FW1 : %ASA-7-715009: Group = 190.90.24.10, IP = 190.90.24.10, IKE Deleting SA: Remote Proxy 190.90.25.11, Local Proxy 60.20.128.40

Sep 20 2012 21:59:51 FW1 : %ASA-7-713906: Group = 190.90.24.10, IP = 190.90.24.10, IKE SA MM:ac53953d terminating: flags 0x01000002, refcnt 0, tuncnt 0

Sep 20 2012 21:59:51 FW1 : %ASA-7-713906: Group = 190.90.24.10, IP = 190.90.24.10, sending delete/delete with reason message

Sep 20 2012 21:59:51 FW1 : %ASA-6-602304: IPSEC: An inbound LAN-to-LAN SA (SPI= 0x66B9DDCE) between 60.20.128.30 and 190.90.24.10 (user= 190.90.24.10) has been deleted.

Sep 20 2012 21:59:51 FW1 : %ASA-6-602304: IPSEC: An outbound LAN-to-LAN SA (SPI= 0xC349DFD3) between 60.20.128.30 and 190.90.24.10 (user= 190.90.24.10) has been deleted.

Sep 20 2012 21:59:51 FW1 : %ASA-7-715046: Group = 190.90.24.10, IP = 190.90.24.10, constructing blank hash payload

Sep 20 2012 21:59:51 FW1 : %ASA-7-715046: Group = 190.90.24.10, IP = 190.90.24.10, constructing IKE delete payload

Sep 20 2012 21:59:51 FW1 : %ASA-7-715046: Group = 190.90.24.10, IP = 190.90.24.10, constructing qm hash payload

Sep 20 2012 21:59:51 FW1 : %ASA-7-713236: IP = 190.90.24.10, IKE_DECODE SENDING Message (msgid=623d25cb) with payloads : HDR + HASH (8) + DELETE (12) + NONE (0) total length : 80

Sep 20 2012 21:59:51 FW1 : %ASA-4-113019: Group = 190.90.24.10, Username = 190.90.24.10, IP = 190.90.24.10, Session disconnected. Session Type: IPsec, Duration: 0h:57m:46s, Bytes xmt: 841, Bytes rcv: 1626, Reason: Lost Service

Sep 20 2012 22:00:36 FW1 : %ASA-7-710006: ESP request discarded from 190.90.24.10 to outside:60.20.128.30

Sep 20 2012 22:00:39 FW1 : %ASA-7-710006: ESP request discarded from 190.90.24.10 to outside:60.20.128.30

Sep 20 2012 22:00:45 FW1 : %ASA-7-710006: ESP request discarded from 190.90.24.10 to outside:60.20.128.30

Sep 20 2012 22:01:52 FW1 : %ASA-6-302016: Teardown UDP connection 254132500 for outside:190.90.24.10/500 to identity:60.20.128.30/500 duration 57:53:26 bytes 1937468

Sep 20 2012 22:01:52 FW1 : %ASA-7-609002: Teardown local-host outside:190.90.24.10 duration 57:53:26

Sep 20 2012 23:13:27 FW1 : %ASA-7-710006: ESP request discarded from 190.90.24.10 to outside:60.20.128.30

Sep 20 2012 23:13:30 FW1 : %ASA-7-710006: ESP request discarded from 190.90.24.10 to outside:60.20.128.30

Sep 20 2012 23:13:36 FW1 : %ASA-7-710006: ESP request discarded from 190.90.24.10 to outside:60.20.128.30

During the period that the ESP request is discarded I have debugging on both isakmp and ipsec. However the debugging logs show ZERO entries for this specific vpn, which is kind of logical since there is NO failure in bringing up the tunnel. The remote end (netscreen) sees the tunnel as up and thus does not attempt to establish a tunnel, so indeed nothing in the debug log. Seems plausible to me.

Partial config ASA:

object-group network Customer_subs

network-object host 190.90.25.11

crypto map outside_map 8 match address Customer_cryptomap

crypto map outside_map 8 set peer 190.90.24.10

crypto map outside_map 8 set transform-set ESP-3DES-SHA

crypto map outside_map 8 set security-association lifetime seconds 86400

crypto map outside_map 8 set security-association lifetime kilobytes 4608000

access-list Customer_cryptomap extended permit ip host 60.20.128.40 object-group Customer_subs log

tunnel-group 190.90.24.10 type ipsec-l2l

tunnel-group 190.90.24.10 ipsec-attributes

pre-shared-key *

So what is happening here?

Javier Portuguez · ‎09-21-2012

Hi Michell,

I can see the Phase II lifetime is 86400, May I know the Phase I lifetime value?

By default Cisco ASAs use a lifetime of 86400 seconds for Phase I and 28800 for Phase II, so it wouldn't be a good idea to have Phase I & II using the same values.

Thanks.

Portu.

michellp · ‎09-26-2012

Hi Portu,

Sorry for the late reply, it is indeed 24 hours (86400). Somewhere I agree withthis logic, but why not? What would this cause?

BTW,

I believe to have found what is wrong or at least is not right. There was no 'nonat' rule in place for the protected traffic on our side. Although I find it strange that the tunnel is ok once it's setup from the remote end. They have no problems accessing our network here. I would expect them to have issues if the 'nonat' rule is missing, but they do not. It's only after a couple of days that the tunnel goes down from our end and stays up at the remote end, which of course leads to them not being able to access our network. SO hopefully this is it.

Regards,

michellp · ‎10-19-2012

This issue has been resolved, although we never found the cause of this. We rebuild the tunnel from scratch and since then it has never failed. This discussion is closed

Javier Portuguez · ‎10-19-2012

Hi Michell,

Thanks for the heads up, I am glad to know it is working now.

Please mark this post as answered and rate any helpful posts

Portu.