Site to Site VPN not working

[EnsercaAdmin]

I have two ASA 5505 devices that I am using for a site-to-site VPN. When I went to the remote site, I was able to connect everything and had it working until about two weeks ago. This was about 3 weeks after the install.

Here are the results from "show cry ipsec sa" that I ran on both devices. The area that looks strange to me is that the "inbound esp sas:" is different on each device. I have another site-to-site VPN configured with another office and they are exactly the same. I have deleted and recreated the connection a couple of times but cannot get them to work again.

Main office config:

Result of the command: "show cry ipsec sa"

interface: outside
    Crypto map tag: outside_map, seq num: 3, local addr: 72.16.195.3

      access-list outside_3_cryptomap permit ip Lak-Int-Network 255.255.255.0 STX-Int-Network 255.255.255.0
      local ident (addr/mask/prot/port): (Lak-Int-Network/255.255.255.0/0/0)
      remote ident (addr/mask/prot/port): (STX-Int-Network/255.255.255.0/0/0)
      current_peer: STX-Firewall-.80

      #pkts encaps: 0, #pkts encrypt: 0, #pkts digest: 0
      #pkts decaps: 28, #pkts decrypt: 28, #pkts verify: 28
      #pkts compressed: 0, #pkts decompressed: 0
      #pkts not compressed: 0, #pkts comp failed: 0, #pkts decomp failed: 0
      #pre-frag successes: 0, #pre-frag failures: 0, #fragments created: 0
      #PMTUs sent: 0, #PMTUs rcvd: 0, #decapsulated frgs needing reassembly: 0
      #send errors: 0, #recv errors: 0

      local crypto endpt.: 72.16.195.3, remote crypto endpt.: STX-Firewall-.80

      path mtu 1500, ipsec overhead 58, media mtu 1500
      current outbound spi: C1B18926

    inbound esp sas:
      spi: 0x3C8A10BB (1015681211)
         transform: esp-3des esp-sha-hmac no compression
         in use settings ={L2L, Tunnel, }
         slot: 0, conn_id: 4263936, crypto-map: outside_map
         sa timing: remaining key lifetime (kB/sec): (4373996/28731)
         IV size: 8 bytes
         replay detection support: Y
         Anti replay bitmap:
          0x00000000 0x1FFFFFFF
    outbound esp sas:
      spi: 0xC1B18926 (3249637670)
         transform: esp-3des esp-sha-hmac no compression
         in use settings ={L2L, Tunnel, }
         slot: 0, conn_id: 4263936, crypto-map: outside_map
         sa timing: remaining key lifetime (kB/sec): (4374000/28731)
         IV size: 8 bytes
         replay detection support: Y
         Anti replay bitmap:
          0x00000000 0x00000001

Remote office config:

Result of the command: "show cry ipsec sa"

interface: outside
    Crypto map tag: outside_map, seq num: 1, local addr: STX-Firewall-.80

      access-list outside_1_cryptomap extended permit ip 192.168.15.0 255.255.255.0 192.168.0.0 255.255.255.0
      local ident (addr/mask/prot/port): (STX-Int-Network/255.255.255.0/0/0)
      remote ident (addr/mask/prot/port): (Lak-Int-Network/255.255.255.0/0/0)
      current_peer: 72.16.195.3

      #pkts encaps: 2, #pkts encrypt: 2, #pkts digest: 2
      #pkts decaps: 0, #pkts decrypt: 0, #pkts verify: 0
      #pkts compressed: 0, #pkts decompressed: 0
      #pkts not compressed: 2, #pkts comp failed: 0, #pkts decomp failed: 0
      #pre-frag successes: 0, #pre-frag failures: 0, #fragments created: 0
      #PMTUs sent: 0, #PMTUs rcvd: 0, #decapsulated frgs needing reassembly: 0
      #send errors: 0, #recv errors: 0

      local crypto endpt.: STX-Firewall-.80, remote crypto endpt.: 72.16.195.3

      path mtu 1500, ipsec overhead 58, media mtu 1500
      current outbound spi: 3C8A10BB
      current inbound spi : C1B18926

    inbound esp sas:
      spi: 0xC1B18926 (3249637670)
         transform: esp-3des esp-sha-hmac no compression
         in use settings ={L2L, Tunnel, }
         slot: 0, conn_id: 909312, crypto-map: outside_map
         sa timing: remaining key lifetime (kB/sec): (3915000/28784)
         IV size: 8 bytes
         replay detection support: Y
         Anti replay bitmap:
          0x00000000 0x00000001
    outbound esp sas:
      spi: 0x3C8A10BB (1015681211)
         transform: esp-3des esp-sha-hmac no compression
         in use settings ={L2L, Tunnel, }
         slot: 0, conn_id: 909312, crypto-map: outside_map
         sa timing: remaining key lifetime (kB/sec): (3914999/28784)
         IV size: 8 bytes
         replay detection support: Y
         Anti replay bitmap:
          0x00000000 0x00000001

I found a couple of posts that mentioned that it might be the NAT configs. I have removed the NAT's that I had set up on the remote office to see if this resolved the issue.

any assistance would be greatly appreciated.

[Julio Carvajal]

Hello Mike,

What version are you running on the box that has the issues with the other devices?

Are you sure the Crypto ACL are 100 % good

So you have the right NAT 0 o No_Nat rules right?

Check this out

This guy

72.16.195.3 seems to not be encrypting traffic, what version is it running?

#pkts encaps: 0, #pkts encrypt: 0, #pkts digest: 0

#pkts decaps: 0, #pkts decrypt: 0, #pkts verify: 0

[EnsercaAdmin]

thank you for the quick response,

I am not sure what you are asking "Are you sure the Crypto ACL are 100 % good?"

Here is a screenshot of the NAT config that I have on the remote machine:

Here are the versions for both devices:

The main office (72.16.195.3) is running ASDM 6.2(1) and ASA 8.2(1)

The remote office (71.97.229.80) is running ASDM 6.4(5) and ASA 8.2(5)

Let me know if you have any additonal questions.

[Julio Carvajal]

Hello,

I mean that the ACL being referenced on the crypto map is properly setup.

8.2.1 not able to encrypt traffic ( bug ) 

CSCtd36473

There you go,

Let me know if you need something else

[EnsercaAdmin]

I have another firewall (same type) that is running ASDM 6.4(5) and ASA 8.2(5) with the one at the main office running the 8.2 and I don't have any issue with that one. Why would one work and the other would not?

[Julio Carvajal]

The one with the issue is the one running 8.2.1 that is the one showing 0 encryptions right?????

[EnsercaAdmin]

The one showing 0 encryptions is the main office. It is not able to connect with the remote office but it can connect to another remote office. The remote offices as the ones that have the 8.4 version and the main office has the 8.2 version.

[EnsercaAdmin]

I have tried to access the but you noted "CSCtd36473" but the system keeps telling me that I need to log in. I am logged in but it still won't let me see anything.

[Julio Carvajal]

Hello Mike,

You should be able to see that bug

Can you share the show crypto ipsec sa of the main office

[EnsercaAdmin]

Result of the command: "show crypto ipsec sa"

interface: outside
    Crypto map tag: outside_map, seq num: 3, local addr: 72.16.x.x

      access-list outside_3_cryptomap permit ip Lak-Int-Network 255.255.255.0 STX-Int-Network 255.255.255.0
      local ident (addr/mask/prot/port): (Lak-Int-Network/255.255.255.0/0/0)
      remote ident (addr/mask/prot/port): (STX-Int-Network/255.255.255.0/0/0)
      current_peer: STX-Firewall-.80

      #pkts encaps: 0, #pkts encrypt: 0, #pkts digest: 0
      #pkts decaps: 746, #pkts decrypt: 744, #pkts verify: 744
      #pkts compressed: 0, #pkts decompressed: 0
      #pkts not compressed: 0, #pkts comp failed: 0, #pkts decomp failed: 0
      #pre-frag successes: 0, #pre-frag failures: 0, #fragments created: 0
      #PMTUs sent: 0, #PMTUs rcvd: 0, #decapsulated frgs needing reassembly: 0
      #send errors: 0, #recv errors: 0

      local crypto endpt.: 72.16.x.x, remote crypto endpt.: STX-Firewall-.80

      path mtu 1500, ipsec overhead 58, media mtu 1500
      current outbound spi: 52EDAF7D

    inbound esp sas:
      spi: 0xC5DC2E55 (3319541333)
         transform: esp-3des esp-sha-hmac no compression
         in use settings ={L2L, Tunnel, }
         slot: 0, conn_id: 4321280, crypto-map: outside_map
         sa timing: remaining key lifetime (kB/sec): (4373881/26678)
         IV size: 8 bytes
         replay detection support: Y
         Anti replay bitmap:
          0xF7FFFBFF 0xFFFFFFFF
    outbound esp sas:
      spi: 0x52EDAF7D (1391308669)
         transform: esp-3des esp-sha-hmac no compression
         in use settings ={L2L, Tunnel, }
         slot: 0, conn_id: 4321280, crypto-map: outside_map
         sa timing: remaining key lifetime (kB/sec): (4374000/26678)
         IV size: 8 bytes
         replay detection support: Y
         Anti replay bitmap:
          0x00000000 0x00000001

    Crypto map tag: outside_map, seq num: 1, local addr: 72.16.x.x

      access-list outside_1_cryptomap permit ip Lak-Int-Network 255.255.255.0 Tul-Int-Network 255.255.255.0
      local ident (addr/mask/prot/port): (Lak-Int-Network/255.255.255.0/0/0)
      remote ident (addr/mask/prot/port): (Tul-Int-Network/255.255.255.0/0/0)
      current_peer: Tul-Firewall-.3

      #pkts encaps: 531052, #pkts encrypt: 531052, #pkts digest: 531052
      #pkts decaps: 381811, #pkts decrypt: 381811, #pkts verify: 381811
      #pkts compressed: 0, #pkts decompressed: 0
      #pkts not compressed: 531052, #pkts comp failed: 0, #pkts decomp failed: 0
      #pre-frag successes: 0, #pre-frag failures: 0, #fragments created: 0
      #PMTUs sent: 0, #PMTUs rcvd: 0, #decapsulated frgs needing reassembly: 0
      #send errors: 0, #recv errors: 0

      local crypto endpt.: 72.16.195.3, remote crypto endpt.: Tul-Firewall-.3

      path mtu 1500, ipsec overhead 58, media mtu 1500
      current outbound spi: 6DDD74E9

    inbound esp sas:
      spi: 0xCCDC457E (3436987774)
         transform: esp-3des esp-sha-hmac no compression
         in use settings ={L2L, Tunnel, }
         slot: 0, conn_id: 4247552, crypto-map: outside_map
         sa timing: remaining key lifetime (kB/sec): (3865568/12259)
         IV size: 8 bytes
         replay detection support: Y
         Anti replay bitmap:
          0xFFFFFFFF 0xFFFFFFFF
    outbound esp sas:
      spi: 0x6DDD74E9 (1843229929)
         transform: esp-3des esp-sha-hmac no compression
         in use settings ={L2L, Tunnel, }
         slot: 0, conn_id: 4247552, crypto-map: outside_map
         sa timing: remaining key lifetime (kB/sec): (3361153/12259)
         IV size: 8 bytes
         replay detection support: Y
         Anti replay bitmap:
          0x00000000 0x00000001

[JohnTylerPearce]

Mike,

Have you checked to see traffic is being matched to an ACL, before it gets to the crypto ACL?

Also if you run the following debug commands, they were give you more information.

debug crypto condition peer x.x.x.x (Remote Peer IP)

After this is done, you can run the following commands

debug crypto isakmp 127

debug crypto ipsec 127

This will allow you to see debugs ONLY on that peer, not any other VPNs you have configured, that way you don't blow up your router.

Also, it could very well be a bug. I had a site-to-site VPN, just stop encrypting packets for some reason, I had to install a minor update, and it fixed the issue. So that is completely possible.

[Julio Carvajal · Accepted solution]

Hello Mike,

Exactly what I was expecting

IPsec: Outbound context may be deleted prematurely

Symptom: Outbound encryption traffic in an IPsec tunnel may fail, even if inbound decryption traffic is working. Conditions: This issue has been observed on an IPsec connection after multiple rekeys, but the trigger condition is not clear. The presence of this issue can be established by checking the output of "show asp drop" and verifying that the Expired VPN context counter is increasing for each outbound packet sent. Workaround: None.

You must go to a version that fix this (8.2.5 as an example).

Allthough some tunnels work you are going to see this behavior with this version

[EnsercaAdmin]

Thanks for your help guys.