Unable to connect LAN to LAN between IPSEC VPN - both ASA devices

[habeebuddin786]

Hello Folks,

I am trying to bring up the connectivity between LAN to LAN over IPSEC VPN tunnel and we are using both devices as ASA. Dallas ASA is using 8.2(2) version and SJC ASA is using 8.2(1) version.

I tried every possibility to test the connection but encrypt packets are not seen on the SJC VPN side and decrypt packet are not seen on the dallas side. However the packet tracer from one phase to another is allowing everything and nothing is blocking. I checked the routes as well on the FWSM behind the VPN on both sides and looks good and routes are pointing to VPNs. Both sides are seen that the packet are receiving from internal. Attached are the statistics of packet-tracer and ipsec sa of both VPNs for your reference.

Below are the recommendations of Cisco and followed to fix it. Suspecting the bug in the code of SJC-VPN-ASA as it is using the version as 8.2(1) and dallas one is using as 8.2(2). But would like to confirm if anything i missed to try.

Following are the things I tried so far:

1) reset the phase 1

2) reset the phase 2

3) clear xlate

4) packet-tracert input inside icmp src-ip(dallas/SJC) and dst-ip (SJC/Dallas) detailed (every phase shows ALLOW)

Attached are the configs of both the VPN.

Approciate your time and help in this to fix the issue.

Thank you.

-Ahmed

[Jennifer Halim]

On the Dallas end, add the following:

management-access prod-vpn-inside

Try to ping from the ASA:

From SJC: ping inside 10.120.188.3

From Dallas: ping prod-vpn-inside 10.126.188.5

If you are able to ping and seeing encrypts and decrypts, that means nothing wrong with the ASA config.

[nkarthikeyan]

Hi habbeb,

I have checked your configuration and found few things in your configuration. In dallas end you do not have the no nat configured. i mean the NAT command is missing. Also please check on the NAT traversal feature.... if you are getting logs for NAT reverse path denies the traffic then make changes on the NAT traversal...

Hope this will make your VPN up and running.

Try establising the connection in this way also from your inside segment switch to generate the intersting traffic.

telnet /source-interface vlan

here are my findings on both the firewalls.

Please do rate if the given information helps.

SJC End

=========

interface GigabitEthernet0/0

speed 1000

duplex full

nameif outside

security-level 0

ip address 205.140.203.10 255.255.255.248

!

access-list inside_nat0_outbound extended permit ip 10.126.0.0 255.255.0.0 10.120.0.0 255.255.0.0

!

access-list SJC-DAL-prod-vpn extended permit ip 10.126.0.0 255.255.0.0 10.120.0.0 255.255.0.0

!

crypto map outside_map 30 match address SJC-DAL-prod-vpn

crypto map outside_map 30 set peer 205.216.36.134

crypto map outside_map 30 set transform-set ESP-AES-256-SHA

crypto map outside_map 30 set security-association lifetime seconds 28800

crypto map outside_map 30 set security-association lifetime kilobytes 4608000

!

crypto map outside_map interface outside

crypto isakmp identity address

crypto isakmp enable outside

!

tunnel-group 205.216.36.134 type ipsec-l2l

tunnel-group 205.216.36.134 ipsec-attributes

pre-shared-key *

!

crypto isakmp policy 1

authentication pre-share

encryption 3des

hash md5

group 2

lifetime 86400

crypto isakmp policy 5

authentication pre-share

encryption 3des

hash sha

group 2

lifetime 86400

crypto isakmp policy 10

authentication pre-share

encryption des

hash sha

group 2

lifetime 86400

crypto isakmp policy 30

authentication pre-share

encryption 3des

hash md5

group 1

lifetime 86400

!

crypto isakmp nat-traversal 30

!

nat (inside) 0 access-list inside_nat0_outbound

Dallas End

=============

interface GigabitEthernet0/0

nameif prod-vpn-outside

security-level 0

ip address 205.216.36.134 255.255.255.128 standby 205.216.36.135

!

access-list nonat extended permit ip 10.120.0.0 255.255.0.0 10.126.0.0 255.255.0.0

!

access-list Dal-SJC-prod-vpn extended permit ip 10.120.0.0 255.255.0.0 10.126.0.0 255.255.0.0

!

crypto ipsec transform-set ESP-AES-256-SHA esp-aes-256 esp-sha-hmac

!

crypto map outside-map 10 match address Dal-SJC-prod-vpn

crypto map outside-map 10 set peer 205.140.203.10

crypto map outside-map 10 set transform-set ESP-AES-256-SHA

crypto map outside-map 10 set security-association lifetime seconds 28800

crypto map outside-map 10 set security-association lifetime kilobytes 4608000

!

crypto isakmp enable prod-vpn-outside

!

crypto isakmp policy 1

authentication pre-share

encryption aes-256

hash sha

group 5

lifetime 86400

crypto isakmp policy 30

authentication pre-share

encryption 3des

hash md5

group 2

lifetime 86400

crypto isakmp policy 100

authentication pre-share

encryption 3des

hash sha

group 2     

lifetime 86400

!

no crypto isakmp nat-traversal

!

tunnel-group 205.140.203.10 type ipsec-l2l

tunnel-group 205.140.203.10 ipsec-attributes

pre-shared-key *****

!

By

Karthik

[Ramraj Sivagnanam Sivajanam]

Hi Bro

Shown below are the configuration changes that you'll need to cut and paste into your respective Firewalls;

Dallas FW
========

config t
nat (prod-vpn-inside) 0 access-list nonat

access-list outside permit icmp any any echo

access-list outside permit icmp any any echo-reply

access-group outside-in in interface prod-vpn-outside

SJC FW
======

config t
crypto isakmp policy 40
authentication pre-share
encryption aes-256
hash sha
group 2
lifetime 86400

In both FW
=========

config t
sysopt connection permit-vpn
clear threat-detection shun
crypto isakmp keepalive 10
crypto isakmp nat-traversal 30

Once you've done that, issue the command clear crypto isakmp sa and clear crypto ipsec sa, and try to access both sides e.g. PING. If it doesn't work, paste back here the latest config on both the Firewalls.

[habeebuddin786]

Hello brothers,

Thank you very much for your response and helping me in this one. I tried the steps recommended by Ramraj but no luck. No changes in the behaviour.  As recommeded I am attaching the latest config of both side Firewalls with IPSEC sa output and Packet tracer.

Please suggest next plan of action.

Thanks

-Ahmed

[nkarthikeyan]

Hi Ahmed,

Please find the troubleshooting info for your issue. Please try with the steps provided in the last.

You need to check crypto acl matches @ both the ends. NAT 0 exempt traffic is correct on both the ends. Routing is find on both the ends.

Also try enabling the sysopt permit connection-vpn / sysopt connection permit-vpn

Unable to Pass Traffic Across VPN Tunnel

Problem

You are unable to pass traffic across a VPN tunnel.

Solution

This issue occurs due to the problem described in Cisco bug ID CSCtb53186 (registered customers only) . In order to resolve this issue, reload the ASA. Refer to the bug for more information.

This issue might also occur when the ESP packets are blocked. In order to resolve this issue, reconfiguring the VPN tunnel.

This issue might occur when data is not encrypted, but only decrypted over the VPN tunnel as shown in this output:

ASA# sh crypto ipsec sa peer x.x.x.x
peer address: y.y.y.y
    Crypto map tag: IPSec_map, seq num: 37, local addr: x.x.x.x
      access-list test permit ip host xx.xx.xx.xx host yy.yy.yy.yy
      local ident (addr/mask/prot/port): (xx.xx.xx.xx/255.255.255.255/0/0)
      remote ident (addr/mask/prot/port): (yy.yy.yy.yy/255.255.255.255/0/0)
      current_peer: y.y.y.y

      #pkts encaps: 0, #pkts encrypt: 0, #pkts digest: 0
      #pkts decaps: 393, #pkts decrypt: 393, #pkts verify: 393
      #pkts compressed: 0, #pkts decompressed: 0
      #pkts not compressed: 0, #pkts comp failed: 0, #pkts decomp failed: 0
      #pre-frag successes: 0, #pre-frag failures: 0, #fragments created: 0
      #PMTUs sent: 0, #PMTUs rcvd: 0, #decapsulated frgs needing reassembly: 0
      #send errors: 0, #recv errors: 0

In order to resolve this issue, check the following:

1. If the crypto access-lists match with the remote site, and that NAT 0 access-lists are correct.

2. If routing is correct and traffic does hit outside interface passing  through inside. The sample output shows that decryption is done, but  encryption does not occur.

3. If the sysopt permit connection-vpn command has been configured on the ASA. If not configured, configure  this command because it allows the ASA to exempt the encrypted/VPN  traffic from interface ACL checking.

Please do rate for the helpful posts.

By

Karthik

[Ramraj Sivagnanam Sivajanam]

Hi Bro

Please paste your latest Firewall config here (both units). I want to verify your config.

Worst case, we may need to add this line in PROD-VPN-ASA Firewall to solve the issue;

route outside 10.120.0.0 255.255.0.0 205.140.203.1

[habeebuddin786]

Sorry I though I pasted the latest config.

Here are the lates config of both the firewalls.

Regards,

-Ahmed

[habeebuddin786]

Seems this is relate to BUG as per the following link:

http://tools.cisco.com/Support/BugToolKit/search/getBugDetails.do?method=fetchBugDetails&bugId=CSCtb53186

SJC-VPN (ASA5520) is running version 8.2(1) and this bug is found first in 8.2(1) and as per the IT deployment team this tunnel is broken 6 months back and since then no network engineer is worked  on it as the earlier network engineer left the job. I have recently joined and this task is priority to me.

Also can I have steps or the link in which shows upgrade of software from 8.2(1) to 8.2(4). Much appreciate your time and effort in this to fix this one.

Below is the software version of SJC VPN:

Cisco Adaptive Security Appliance Software Version 8.2(1)

Device Manager Version 6.2(1)

Compiled on Tue 05-May-09 22:45 by builders

System image file is "disk0:/asa821-k8.bin"

Config file at boot was "startup-config"

PROD-VPN-ASA up 2 years 342 days

Hardware:   ASA5520, 512 MB RAM, CPU Pentium 4 Celeron 2000 MHz

Internal ATA Compact Flash, 256MB

Slot 1: ATA Compact Flash, 128MB

BIOS Flash M50FW080 @ 0xffe00000, 1024KB

Encryption hardware device : Cisco ASA-55x0 on-board accelerator (revision 0x0)

                             Boot microcode   : CN1000-MC-BOOT-2.00

                             SSL/IKE microcode: CNLite-MC-SSLm-PLUS-2.03

                             IPSec microcode  : CNlite-MC-IPSECm-MAIN-2.04

0: Ext: GigabitEthernet0/0  : address is 001e.f760.a2a4, irq 9

1: Ext: GigabitEthernet0/1  : address is 001e.f760.a2a5, irq 9

2: Ext: GigabitEthernet0/2  : address is 001e.f760.a2a6, irq 9

3: Ext: GigabitEthernet0/3  : address is 001e.f760.a2a7, irq 9

4: Ext: Management0/0       : address is 001e.f760.a2a8, irq 11

5: Int: Not used            : irq 11

6: Int: Not used            : irq 5

Licensed features for this platform:

Maximum Physical Interfaces  : Unlimited

Maximum VLANs                : 150      

Inside Hosts                 : Unlimited

Failover                     : Active/Active

VPN-DES                      : Enabled  

VPN-3DES-AES                 : Enabled  

Security Contexts            : 2        

GTP/GPRS                     : Disabled 

SSL VPN Peers                : 2        

Total VPN Peers              : 750      

Shared License               : Disabled

AnyConnect for Mobile        : Disabled 

AnyConnect for Linksys phone : Disabled 

AnyConnect Essentials        : Disabled 

Advanced Endpoint Assessment : Disabled 

UC Phone Proxy Sessions      : 2        

Total UC Proxy Sessions      : 2        

Botnet Traffic Filter        : Disabled 

This platform has an ASA 5520 VPN Plus license.

Serial Number: JMX1209L1QH

Running Activation Key: 0x0b13d651 0x10bee714 0x24326da4 0xbc10ccfc 0x4201a5aa

Configuration register is 0x10001

Configuration last modified by enable_15 at 23:36:09.498 GMT Tue Aug 28 2012

[Ramraj Sivagnanam Sivajanam]

Hi Bro           

Frankly, I doubt this is a bug related issue. However, you could refer to this Cisco URL http://www.cisco.com/en/US/products/ps6120/products_configuration_example09186a008067e9f9.shtml#maintask1, in particular the  Ugrade a Software Image and ASDM Image using CLI section, if you want to upgrade both sides of the Firewall to version 8.2.4.

By the way, can you kindly make the configurations changes as shown below;

Dallas FW
---------------
access-list outside-in line 1 permit icmp any any
access-list outside-in extended deny ip any any
clear threat-detection shun
no crypto isakmp ipsec-over-tcp port 10000
sysopt connection permit-vpn

SJC FW
-------------
clear threat-detection shun
access-list outside_access_in line 1 permit icmp any any
sysopt connection permit-vpn