L2L Ipsec issue: 0 Decrypt pkts

[patelparth3]

Scenario is like this:
I have site A and Site B. I am trying to get connectivity between Site A and site B by configuring IPSEC site-to-site tunnel. On site-A I am using cisco asa 5510 ver 7.0 and on site-B cisco asa 5500 ver 9.2. In short one FW is running old version while the second one is running newer version. 2 hosts(192.168.10.41 and 192.168.10.42 from site-A should be able to connect to host(10.10.10.20) of site-B.

Issue: From site-A, on my ASDM when I ping from site A to site B or vice-versa the tunnel goes up but I don't see any ping reply. Neither it could ping from site-A to site-B nor from site-B to site-A. But the tunnels between the site-A and B shows up without any errors.

Notes to be taken while you suggest any solutions:
1. I have another site-to-site IPSEC vpn configured on the same firewall and it works fine. I am able to ping the remote host. It is just that I have excluded from the below posted configuration.
2. Tried rebooting the firewall, clearing xlate, ipsec sa, isakmp sa but no luck.
3. Double checked NAT exempt and ACL on both firewalls

Site A:
Public IP - 1.2.3.4
Private - 192.168.10.0/24

Site B:
Public IP - X.X.X.X
Private - 10.10.10.0/24

Site-A Configuration:

ASA Version 7.0(8)
!
dns-guard
!
interface Ethernet0/0
nameif OUTSIDE
security-level 0
ip address 1.2.3.4 255.255.255.252
!
interface Ethernet0/1
nameif INSIDE
security-level 100
ip address 192.168.10.1 255.255.255.0
!
ftp mode passive
!
access-list OUTSIDE_IN extended permit icmp any interface ISP echo-reply
access-list INSIDE_nat0_outbound extended permit ip host 192.168.10.41 host 10.10.10.20
access-list INSIDE_nat0_outbound extended permit ip host 192.168.10.42 host 10.10.10.20
access-list OUTSIDE_cryptomap_20 extended permit ip host 192.168.10.41 host 10.10.10.20
access-list OUTSIDE_cryptomap_20 extended permit ip host 192.168.10.42 host 10.10.10.20
!
icmp deny any OUTSIDE
!
global (OUTSIDE) 100 interface
nat (INSIDE) 0 access-list INSIDE_nat0_outbound
nat (INSIDE) 100 192.168.10.0 255.255.255.0
!
access-group OUTSIDE_IN in interface OUTSIDE
!
route OUTSIDE 0.0.0.0 0.0.0.0 1.2.3.4 1
timeout xlate 3:00:00
timeout conn 1:00:00 half-closed 0:10:00 udp 0:02:00 icmp 0:00:02
timeout sunrpc 0:10:00 h323 0:05:00 h225 1:00:00 mgcp 0:05:00
timeout mgcp-pat 0:05:00 sip 0:30:00 sip_media 0:02:00
timeout uauth 0:05:00 absolute
!
crypto ipsec transform-set ESP-AES-256-SHA esp-aes-256 esp-sha-hmac
crypto ipsec security-association lifetime seconds 28800
crypto ipsec security-association lifetime kilobytes 4608000
crypto map OUTSIDE_map 20 match address OUTSIDE_cryptomap_20
crypto map OUTSIDE_map 20 set peer X.X.X.X
crypto map OUTSIDE_map 20 set transform-set ESP-AES-256-SHA
crypto map OUTSIDE_map 20 set security-association lifetime seconds 28800
crypto map OUTSIDE_map 20 set security-association lifetime kilobytes 4608000
crypto map OUTSIDE_map 20 set nat-t-disable
!
isakmp identity address
isakmp enable OUTSIDE
isakmp policy 20 authentication pre-share
isakmp policy 20 encryption aes-256
isakmp policy 20 hash sha
isakmp policy 20 group 5
isakmp policy 20 lifetime 86400
!
tunnel-group X.X.X.X type ipsec-l2l
tunnel-group X.X.X.X ipsec-attributes
pre-shared-key *

: end

On my ASDM it shows #pkts encrypt: 5 and #pkts decrypt: 0 , Byte Tx: 0 and Bytes Rx: 180

Please help, I am new to the Cisco Firewall.

[Marius Gunnerud]

Would you be able to post the Site-B configuration also as it is quite difficult to troubleshoot a VPN connection with only one side of the configuration?

When the tunnel is up, could you please provide the output of show crypto isakmp sa (for older ASA version) and show crypto ikev1 sa (for newer ASA version) for both ASAs.

Also, run a debug and teardown the tunnel and reestablish the tunnel.

debug crypto condition peer x.x.x.x  (replace x.x.x.x with the peer IP)

debug crypto ikev1 sa 127

debug crypto ipsec 127

This should give you a slightly better idea of where the issue is located.  If phase one is being established you should see a message saying something like Phase1 complete 

A message saying something like QM FSM Error indicates an issue with your IPsec / phase 2 configuration

[patelparth3]

Thank you Marius for your reply.

1. Would you be able to post the Site-B configuration also as it is quite difficult to troubleshoot a VPN connection with only one side of the configuration?

I won't be able to post Site-B configuration as this the B2B(Business to business) VPN. So, I dont have access to Site-B.

2.debug crypto condition peer x.x.x.x  (replace x.x.x.x with the peer IP)

I can not run this command as I am using the old Cisco ASA.

3. debug crypto iskamp sa

Active SA: 1
Rekey SA: 0 (A tunnel will report 1 Active and 1 Rekey SA during rekey)
Total IKE SA: 1

1 IKE Peer: X.X.X.X
Type : L2L Role : initiator
Rekey : no State : MM_ACTIVE

4. debug crypto iskamp sa 127

Jan 13 15:20:21 [IKEv1]: Group = X.X.X.X, IP = X.X.X.X, PHASE 1 COMPLETED

5. debug crypto ipsec 127

IPSEC: New embryonic SA created @ 0x03AF0230,
SCB: 0x03B1F9B0,
Direction: inbound
SPI : 0xDDC49DD6
Session ID: 0x0000001E
VPIF num : 0x00000001
Tunnel type: l2l
Protocol : esp
Lifetime : 240 seconds
IPSEC: New embryonic SA created @ 0x03AE26F0,
SCB: 0x03B03D08,
Direction: outbound
SPI : 0xB31AF406
Session ID: 0x0000001E
VPIF num : 0x00000001
Tunnel type: l2l
Protocol : esp
Lifetime : 240 seconds
IPSEC: Completed host OBSA update, SPI 0xB31AF406
IPSEC: Creating outbound VPN context, SPI 0xB31AF406
Flags: 0x00000005
SA : 0x03AE26F0
SPI : 0xB31AF406
MTU : 1500 bytes
VCID : 0x00000000
Peer : 0x00000000
SCB : 0x03B03D08
Channel: 0x01135E58
IPSEC: Completed outbound VPN context, SPI 0xB31AF406
VPN handle: 0x03A982A0
IPSEC: New outbound encrypt rule, SPI 0xB31AF406
Src addr: 192.168.10.42
Src mask: 255.255.255.255
Dst addr: 10.10.10.20
Dst mask: 255.255.255.255
Src ports
Upper: 0
Lower: 0
Op : ignore
Dst ports
Upper: 0
Lower: 0
Op : ignore
Protocol: 0
Use protocol: false
SPI: 0x00000000
Use SPI: false
IPSEC: Completed outbound encrypt rule, SPI 0xB31AF406
Rule ID: 0x03A83638
IPSEC: New outbound permit rule, SPI 0xB31AF406
Src addr: 1.2.3.4
Src mask: 255.255.255.255
Dst addr: X.X.X.X
Dst mask: 255.255.255.255
Src ports
Upper: 0
Lower: 0
Op : ignore
Dst ports
Upper: 0
Lower: 0
Op : ignore
Protocol: 50
Use protocol: true
SPI: 0xB31AF406
Use SPI: true
IPSEC: Completed outbound permit rule, SPI 0xB31AF406
Rule ID: 0x0196A9A0
IPSEC: Completed host IBSA update, SPI 0xDDC49DD6
IPSEC: Creating inbound VPN context, SPI 0xDDC49DD6
Flags: 0x00000006
SA : 0x03AF0230
SPI : 0xDDC49DD6
MTU : 0 bytes
VCID : 0x00000000
Peer : 0x03A982A0
SCB : 0x03B1F9B0
Channel: 0x01135E58
IPSEC: Completed inbound VPN context, SPI 0xDDC49DD6
VPN handle: 0x03AE0878
IPSEC: Updating outbound VPN context 0x03A982A0, SPI 0xB31AF406
Flags: 0x00000005
SA : 0x03AE26F0
SPI : 0xB31AF406
MTU : 1500 bytes
VCID : 0x00000000
Peer : 0x03AE0878
SCB : 0x03B03D08
Channel: 0x01135E58
IPSEC: Completed outbound VPN context, SPI 0xB31AF406
VPN handle: 0x03A982A0
IPSEC: Completed outbound inner rule, SPI 0xB31AF406
Rule ID: 0x03A83638
IPSEC: Completed outbound outer SPD rule, SPI 0xB31AF406
Rule ID: 0x0196A9A0
IPSEC: New inbound tunnel flow rule, SPI 0xDDC49DD6
Src addr: 10.10.10.20
Src mask: 255.255.255.255
Dst addr: 192.168.10.42
Dst mask: 255.255.255.255
Src ports
Upper: 0
Lower: 0
Op : ignore
Dst ports
Upper: 0
Lower: 0
Op : ignore
Protocol: 0
Use protocol: false
SPI: 0x00000000
Use SPI: false
IPSEC: Completed inbound tunnel flow rule, SPI 0xDDC49DD6
Rule ID: 0x03B50C78
IPSEC: New inbound decrypt rule, SPI 0xDDC49DD6
Src addr: X.X.X.X
Src mask: 255.255.255.255
Dst addr: 1.2.3.4
Dst mask: 255.255.255.255
Src ports
Upper: 0
Lower: 0
Op : ignore
Dst ports
Upper: 0
Lower: 0
Op : ignore
Protocol: 50
Use protocol: true
SPI: 0xDDC49DD6
Use SPI: true
IPSEC: Completed inbound decrypt rule, SPI 0xDDC49DD6
Rule ID: 0x02E35FF8
IPSEC: New inbound permit rule, SPI 0xDDC49DD6
Src addr: X.X.X.X
Src mask: 255.255.255.255
Dst addr: 1.2.3.4
Dst mask: 255.255.255.255
Src ports
Upper: 0
Lower: 0
Op : ignore
Dst ports
Upper: 0
Lower: 0
Op : ignore
Protocol: 50
Use protocol: true
SPI: 0xDDC49DD6
Use SPI: true

6. A message saying something like QM FSM Error indicates an issue with your IPsec / phase 2 configuration

I checked ASDM logs and could not find this QM FSM error.

[Sheraz.Salim]

On my ASDM it shows #pkts encrypt: 5 and #pkts decrypt: 0 , Byte Tx: 0 and Bytes Rx: 180

seem ASDM ASA running version 7 is encrypt the traffic but does not decry the traffic. double check the routing/staic route are in place accordingly.

If an ASA is getting encaps but not decaps, this means it is encrypting the data and sending it but has not received anything to decrypt in return. Verify the other end has a route outside for the interesting traffic. Check that both VPN ACL’s are not mismatched. Double check NAT’s to make sure the traffic is not NAT’ing correctly.
Is what you are trying to ping even responding back? Often what you’re sending traffic to is not able to accept or is not responding to this traffic. I prefer to put a packet capture on the remote end firewall to see if the traffic is coming back into that firewall.

instead of icmp have to try any other protocol.

on ASA give it a command

!

ASA(config)# fixup protocol icmp
ASA(config)# fix protocol icmp error

!

also use this command and share the output

packet-tracer input inside tcp x.x.x.x x.x.x.x 80 det

[Marius Gunnerud]

As Sheraz has mentioned, verify the remote side configuration.  Have the network team at site B check that routing to Site A is in place and, if there are any firewalls between the ASAs and the subnets in the encryption domain, make sure that traffic is allowed through these firewalls.  Also, If you are using NAT on the ASAs, make sure that you have a twice-NAT rule / NAT-exempt rule for the interesting traffic as well as check to make sure that this traffic flow is not hitting another NAT rule.