ICMP not encrypted through VPN tunnel

ciscosysadm01 · ‎08-21-2013

Hi guys

I've run into a problem I really don't know the answer to and need your help.

We've set up a site-to-site VPN tunnel between a Cisco ASA and a Juniper firewall.

The tunnel itself works as a charm. It is comming up and traffic is flowing as needed. However, ICMP isn't working across the tunnel. I am now wondering what might be wrong.

Fact about setup and debugging info

- we are running no-NAT and both our side and remote side is permitted through "permit ip source destination". As mentioned TCP/UDP traffic is working as designed.

access-list nonat line 2 extended permit ip host 2.2.2.2 host 1.1.1.1

- interesting traffic ACL is also allowed through "permit ip source destination". Again, this is working as designed.

access-list ToSiteX line 4 extended permit ip host 2.2.2.2 host 1.1.1.1(hitcnt=17360)

- inspect ICMP and ICMP error has been added to policy-map global_policy - class inspection_default

- if I do a TCPdump on the firewall behind the VPN GW, I can see the ICMP traffic, when I try to ping from remote site (1.1.1.1) to local host (2.2.2.2). I also see the echo-reply, but

it isn't forwarded through the tunnel. Routing is OK since TCP/UDP traffic is working.

eth3.56:I[60]: 1.1.1.1 -> 2.2.2.2 (ICMP) len=60 id=9090 ICMP: type=8 code=0 echo request id=1 seq=2472

eth2:O[60]: 1.1.1.1 -> 2.2.2.2 (ICMP) len=60 id=9090 ICMP: type=8 code=0 echo request id=1 seq=2472

eth2:I[60]: 2.2.2.2 -> 1.1.1.1 (ICMP) len=60 id=2819 ICMP: type=0 code=0 echo reply id=1 seq=2472

eth3.56:O[60]: 2.2.2.2 -> 1.1.1.1 (ICMP) len=60 id=2819 ICMP: type=0 code=0 echo reply id=1 seq=2472

- packet-tracer fails on VPN encryption

Phase: 9

Type: VPN

Subtype: encrypt

Result: DROP

Config:

Additional Information:

Forward Flow based lookup yields rule:

out id=0xcca41ad0, priority=70, domain=encrypt, deny=false

hits=146995, user_data=0x0, cs_id=0xcc9f1960, reverse, flags=0x0, protocol=1

src ip=0.0.0.0, mask=0.0.0.0, port=0

dst ip=0.0.0.0, mask=0.0.0.0, port=0, dscp=0x0

My question is, why? What is wrong here?

Thank you for your help :o)

ciscosysadm01 · ‎08-22-2013

No one who can help me. I would really appreciate it.

FYI the ASA is running 8.2(5)33. Is it a bug, misconfiguration etc.?

I've also attached a sanitized running-config:

ASA Version 8.2(5)33

!

names

!

interface GigabitEthernet0/0

description *** OUTSIDE ***

nameif outside

security-level 0

ip address *.*.*.*

!

interface GigabitEthernet0/1

description *** INSIDE ***

nameif inside

security-level 100

ip address *.*.*.* 255.255.255.224 standby *.*.*.*

!

interface GigabitEthernet0/3

description LAN/STATE Failover Interface

!

interface Management0/0

shutdown

!

boot system disk0:/asa825-33-k8.bin

!

access-list nonat extended permit ip object-group inside-addresses object-group outside-addresses

access-list nonat remark == Cleanup Rules

access-list nonat extended deny ip any any log

!

access-list FromOutside extended permit icmp any any echo-reply

access-list FromOutside remark == Cleanup Rules

access-list FromOutside extended deny ip any any log

!

access-list ACLname extended permit icmp any any (added for testing purposes but didn't do any difference - ICMP anyway should be included in the IP statement below)

access-list ACLname extended permit ip object-group inside-interesting object-group outside-interesting

!

pager lines 24

logging enable

logging timestamp

logging buffer-size 8000

logging monitor debugging

logging buffered informational

logging trap informational

logging queue 0

logging host inside *

mtu outside 1500

mtu inside 1500

!

no monitor-interface outside

icmp unreachable rate-limit 1 burst-size 1

icmp permit any echo-reply outside

icmp permit any inside

no asdm history enable

arp timeout 14400

global (outside) 1 interface

nat (inside) 0 access-list nonat

nat (inside) 1 0.0.0.0 0.0.0.0

access-group FromOutside in interface outside

!

route inside 2.2.2.0 255.255.255.128 y.y.y.y 1

route outside 1.1.1.192 255.255.255.248 x.x.x.x 1

route outside x.x.x.x 255.255.255.255 x.x.x.x 1 (remote peer address)

timeout xlate 0:05:00

timeout conn 1:00:00 half-closed 0:10:00 udp 0:02:00 icmp 0:00:02

timeout sunrpc 0:10:00 h323 0:05:00 h225 1:00:00 mgcp 0:05:00 mgcp-pat 0:05:00

timeout sip 0:30:00 sip_media 0:02:00 sip-invite 0:03:00 sip-disconnect 0:02:00

timeout sip-provisional-media 0:02:00 uauth 0:05:00 absolute

timeout tcp-proxy-reassembly 0:01:00

timeout floating-conn 0:00:00

dynamic-access-policy-record DfltAccessPolicy

!

sysopt connection tcpmss 1300

service resetoutside

crypto ipsec transform-set AES256-SHA esp-aes-256 esp-sha-hmac

crypto ipsec security-association lifetime seconds 28800

crypto ipsec security-association lifetime kilobytes 4608000

!

crypto map ToSite 85 match address ACLname

crypto map ToSite 85 set pfs

crypto map ToSite 85 set peer x.x.x.x

crypto map ToSite 85 set transform-set AES256-SHA

crypto map ToSite 85 set security-association lifetime seconds 3600

crypto map ToSite 85 set security-association lifetime kilobytes 2000000

crypto map ToSite interface outside

crypto isakmp identity hostname

crypto isakmp enable outside

crypto isakmp policy 1

authentication pre-share

encryption aes-256

hash sha

group 2

lifetime 86400

console timeout 0

threat-detection basic-threat

threat-detection statistics access-list

no threat-detection statistics tcp-intercept

group-policy DfltGrpPolicy attributes

vpn-idle-timeout none

!

tunnel-group x.x.x.x type ipsec-l2l

tunnel-group x.x.x.x ipsec-attributes

pre-shared-key *****

isakmp keepalive threshold 300 retry 2

!

class-map inspection_default

match default-inspection-traffic

!

policy-map type inspect dns preset_dns_map

parameters

message-length maximum client auto

message-length maximum 512

policy-map global_policy

class inspection_default

inspect dns preset_dns_map

inspect ftp

inspect h323 h225

inspect h323 ras

inspect ip-options

inspect netbios

inspect rsh

inspect rtsp

inspect skinny

inspect sqlnet

inspect sunrpc

inspect tftp

inspect sip

inspect xdmcp

inspect http

inspect icmp error

inspect icmp

!

service-policy global_policy global

prompt hostname context

no call-home reporting anonymous

harshisi_2 · ‎08-22-2013

Hi,

Could you also attach the "sh cry ipsec sa peer " along with the contents of the object groups

inside-interesting

outside-interesting

and i am sure you must have tried to use TCP in place of ICMP in the packet trace, just confirming that it worked fine keeping same source and destination.?

~Harry

ciscosysadm01 · ‎08-22-2013

Thank you for your answer.

Sure. Here is my SA (real addresses masked but the host-bit is maintained in the below output). This SA is in this case for TCP/UDP traffic between 2.2.2.26 and 1.1.1.197. Still ICMP between those two isn't working (neither way).

sh cry ipsec sa pe x.x.x.x

peer address: x.x.x.x

Crypto map tag: crypto-map, seq num: 85, local addr: y.y.y.y

access-list ACLname extended permit ip 2.2.2.16 255.255.255.240 1.1.1.192 255.255.255.248

local ident (addr/mask/prot/port): (2.2.2.16/255.255.255.240/0/0)

remote ident (addr/mask/prot/port): (1.1.1.192/255.255.255.248/0/0)

current_peer: x.x.x.x

#pkts encaps: 37270, #pkts encrypt: 37270, #pkts digest: 37270

#pkts decaps: 36857, #pkts decrypt: 36857, #pkts verify: 36857

#pkts compressed: 0, #pkts decompressed: 0

#pkts not compressed: 37270, #pkts comp failed: 0, #pkts decomp failed: 0

#pre-frag successes: 0, #pre-frag failures: 0, #fragments created: 0

#PMTUs sent: 0, #PMTUs rcvd: 0, #decapsulated frgs needing reassembly: 0

#send errors: 0, #recv errors: 0

local crypto endpt.: y.y.y.y, remote crypto endpt.: x.x.x.x

path mtu 1500, ipsec overhead 74, media mtu 1500

current outbound spi: 1D6004C1

current inbound spi : 3BCFA744

inbound esp sas:

spi: 0x3BCFA744 (1003464516)

transform: esp-aes-256 esp-sha-hmac no compression

in use settings ={L2L, Tunnel, PFS Group 2, }

slot: 0, conn_id: 109871104, crypto-map: crypto-map

sa timing: remaining key lifetime (kB/sec): (1898378/1476)

IV size: 16 bytes

replay detection support: Y

Anti replay bitmap:

0xFFFFFFFF 0xFFFFFFFF

outbound esp sas:

spi: 0x1D6004C1 (492831937)

transform: esp-aes-256 esp-sha-hmac no compression

in use settings ={L2L, Tunnel, PFS Group 2, }

slot: 0, conn_id: 109871104, crypto-map: crypto-map

sa timing: remaining key lifetime (kB/sec): (1898321/1476)

IV size: 16 bytes

replay detection support: Y

Anti replay bitmap:

0x00000000 0x00000001

object-group network inside_interesting

network-object 2.2.2.16 255.255.255.240

!

object-group network outside_interesting

network-object 1.1.1.192 255.255.255.248

Santhosha Shetty · ‎08-25-2013

Hi,

The initial PING test you did between host 1.1.1.1 and 2.2.2.2 would not be going over the tunnel as the crypto ACL does not include these hosts(unless you allow them on both ends).

I see yoiu have mentioned ICMp traffic fails though , TCP/UDP traffic between the same host 1.1.1.26 and 2.2.2.197 succeeds.

Could you paste the complete packet tracer output for non-working ICMP packet on ASA?

Thanks.

ciscosysadm01 · ‎08-25-2013

My first post was only an example in regards to source/destination IP address. I reality the ping is between x.x.x.26 and y.y.y.197. Sorry for the confusion.

Packet-tracer output below:

packet-tracer in inside icmp 2.2.2.26 8 0 1.1.1.197

Phase: 1

Type: ROUTE-LOOKUP

Subtype: input

Result: ALLOW

Config:

Additional Information:

in 1.1.1.192 255.255.255.248 outside

Phase: 2

Type: IP-OPTIONS

Subtype:

Result: ALLOW

Config:

Additional Information:

Phase: 3

Type: INSPECT

Subtype: np-inspect

Result: ALLOW

Config:

Additional Information:

Phase: 4

Type: INSPECT

Subtype: np-inspect

Result: ALLOW

Config:

class-map inspection_default

match default-inspection-traffic

policy-map global_policy

class inspection_default

inspect icmp error

service-policy global_policy global

Additional Information:

Phase: 5

Type: DEBUG-ICMP

Subtype:

Result: ALLOW

Config:

Additional Information:

Phase: 6

Type: NAT-EXEMPT

Subtype:

Result: ALLOW

Config:

match ip inside 2.2.2.16 255.255.255.240 outside 1.1.1.192 255.255.255.248

NAT exempt

translate_hits = 4361, untranslate_hits = 55171

Additional Information:

Phase: 7

Type: NAT

Subtype:

Result: ALLOW

Config:

nat (inside) 1 0.0.0.0 0.0.0.0

match ip inside any outside any

dynamic translation to pool 1 (x.x.x.x [Interface PAT])

translate_hits = 283802, untranslate_hits = 127

Additional Information:

Phase: 8

Type: NAT

Subtype: host-limits

Result: ALLOW

Config:

nat (inside) 1 0.0.0.0 0.0.0.0

match ip inside any outside any

dynamic translation to pool 1 (x.x.x.x [Interface PAT])

translate_hits = 283802, untranslate_hits = 127

Additional Information:

Phase: 9

Type: VPN

Subtype: encrypt

Result: DROP

Config:

Additional Information:

Result:

input-interface: inside

input-status: up

input-line-status: up

output-interface: outside

output-status: up

output-line-status: up

Action: drop

Drop-reason: (acl-drop) Flow is denied by configured rule

NAT config:

nat (inside) 0 access-list nonat

nat (inside) 1 0.0.0.0 0.0.0.0

!

...

access-list nonat line 11 extended permit ip 2.2.2.16 255.255.255.240 1.1.1.192 255.255.255.248

Santhosha Shetty · ‎08-27-2013

Hi,

Could you please get me the detailed version of packet-tracer ?

packet-tracer in inside icmp 2.2.2.26 8 0 1.1.1.197 detailed

And also colelct the following:

sh vpn-session-db detail l2l .

Thanks

ciscosysadm01 · ‎08-27-2013

Hi

Thank you for your assistance so far.

Here is the packet-tracer detailed output:

packet-tracer in in icmp 2.2.2.26 8 0 1.1.1.197 det

Phase: 1

Type: ROUTE-LOOKUP

Subtype: input

Result: ALLOW

Config:

Additional Information:

in 1.1.1.192 255.255.255.248 outside

Phase: 2

Type: IP-OPTIONS

Subtype:

Result: ALLOW

Config:

Additional Information:

Forward Flow based lookup yields rule:

in id=0xcc8b4490, priority=0, domain=inspect-ip-options, deny=true

hits=12062386, user_data=0x0, cs_id=0x0, reverse, flags=0x0, protocol=0

src ip=0.0.0.0, mask=0.0.0.0, port=0

dst ip=0.0.0.0, mask=0.0.0.0, port=0, dscp=0x0

Phase: 3

Type: INSPECT

Subtype: np-inspect

Result: ALLOW

Config:

class-map inspection_default

match default-inspection-traffic

policy-map global_policy

class inspection_default

inspect icmp

service-policy global_policy global

Additional Information:

Forward Flow based lookup yields rule:

in id=0xcd32f460, priority=70, domain=inspect-icmp, deny=false

hits=27158, user_data=0xcd3224a0, cs_id=0x0, use_real_addr, flags=0x0, protocol=1

src ip=0.0.0.0, mask=0.0.0.0, port=0

dst ip=0.0.0.0, mask=0.0.0.0, port=0, dscp=0x0

Phase: 4

Type: INSPECT

Subtype: np-inspect

Result: ALLOW

Config:

Additional Information:

Forward Flow based lookup yields rule:

in id=0xcd3dbf08, priority=70, domain=inspect-icmp-error, deny=false

hits=5895, user_data=0xcca600c0, cs_id=0x0, use_real_addr, flags=0x0, protocol=1

src ip=0.0.0.0, mask=0.0.0.0, port=0

dst ip=0.0.0.0, mask=0.0.0.0, port=0, dscp=0x0

Phase: 5

Type: DEBUG-ICMP

Subtype:

Result: ALLOW

Config:

Additional Information:

Forward Flow based lookup yields rule:

in id=0xcd3394b0, priority=12, domain=debug-icmp-trace, deny=false

hits=290886, user_data=0x0, cs_id=0x0, reverse, flags=0x0, protocol=1

src ip=0.0.0.0, mask=0.0.0.0, port=0

dst ip=0.0.0.0, mask=0.0.0.0, port=0, dscp=0x0

Phase: 6

Type: NAT-EXEMPT

Subtype:

Result: ALLOW

Config:

match ip inside 2.2.2.16 255.255.255.240 outside 1.1.1.192 255.255.255.248

NAT exempt

translate_hits = 4885, untranslate_hits = 62245

Additional Information:

Forward Flow based lookup yields rule:

in id=0xcd4b7a38, priority=6, domain=nat-exempt, deny=false

hits=25770, user_data=0xcd4b7978, cs_id=0x0, use_real_addr, flags=0x0, protocol=0

src ip=2.2.2.16, mask=255.255.255.240, port=0

dst ip=1.1.1.192, mask=255.255.255.248, port=0, dscp=0x0

Phase: 7

Type: NAT

Subtype:

Result: ALLOW

Config:

nat (inside) 1 0.0.0.0 0.0.0.0

match ip inside any outside any

dynamic translation to pool 1 (212.88.71.13 [Interface PAT])

translate_hits = 318901, untranslate_hits = 127

Additional Information:

Forward Flow based lookup yields rule:

in id=0xccf01f10, priority=1, domain=nat, deny=false

hits=9004247, user_data=0xccf01e50, cs_id=0x0, flags=0x0, protocol=0

src ip=0.0.0.0, mask=0.0.0.0, port=0

dst ip=0.0.0.0, mask=0.0.0.0, port=0, dscp=0x0

Phase: 8

Type: NAT

Subtype: host-limits

Result: ALLOW

Config:

nat (inside) 1 0.0.0.0 0.0.0.0

match ip inside any outside any

dynamic translation to pool 1 (212.88.71.13 [Interface PAT])

translate_hits = 318901, untranslate_hits = 127

Additional Information:

Forward Flow based lookup yields rule:

in id=0xccf02268, priority=1, domain=host, deny=false

hits=18992638, user_data=0xccf01e50, cs_id=0x0, reverse, flags=0x0, protocol=0

src ip=0.0.0.0, mask=0.0.0.0, port=0

dst ip=0.0.0.0, mask=0.0.0.0, port=0, dscp=0x0

Phase: 9

Type: VPN

Subtype: encrypt

Result: DROP

Config:

Additional Information:

Forward Flow based lookup yields rule:

out id=0xcca41ad0, priority=70, domain=encrypt, deny=false

hits=174013, user_data=0x0, cs_id=0xcc9f1960, reverse, flags=0x0, protocol=1

src ip=0.0.0.0, mask=0.0.0.0, port=0

dst ip=0.0.0.0, mask=0.0.0.0, port=0, dscp=0x0

Result:

input-interface: inside

input-status: up

input-line-status: up

output-interface: outside

output-status: up

output-line-status: up

Action: drop

Drop-reason: (acl-drop) Flow is denied by configured rule

***************************************************************************************************

Connection :

Index : 26824 IP Addr :

Protocol : IKE IPsec

Encryption : AES256 Hashing : SHA1

Bytes Tx : 34077921 Bytes Rx : 19305198

Login Time : 08:11:29 CET+1 Tue Aug 20 2013

Duration : 7d 7h:20m:48s

IKE Tunnels: 1

IPsec Tunnels: 7

IKE:

Tunnel ID : 26824.1

UDP Src Port : 500 UDP Dst Port : 500

IKE Neg Mode : Main Auth Mode : preSharedKeys

Encryption : AES256 Hashing : SHA1

Rekey Int (T): 14400 Seconds Rekey Left(T): 9483 Seconds

D/H Group : 2

Filter Name :

IPsec:

Tunnel ID : 26824.3

Local Addr : 2.2.2.16/255.255.255.240/0/0

Remote Addr : 1.1.1.192/255.255.255.248/0/0

Encryption : AES256 Hashing : SHA1

Encapsulation: Tunnel PFS Group : 2

Rekey Int (T): 3600 Seconds Rekey Left(T): 1562 Seconds

Rekey Int (D): 2000000 K-Bytes Rekey Left(D): 1999910 K-Bytes

Idle Time Out: 0 Minutes Idle TO Left : 0 Minutes

Bytes Tx : 34077921 Bytes Rx : 19305198

Pkts Tx : 110721 Pkts Rx : 109692

NAC:

Reval Int (T): 0 Seconds Reval Left(T): 0 Seconds

SQ Int (T) : 0 Seconds EoU Age(T) : 631322 Seconds

Hold Left (T): 0 Seconds Posture Token:

Redirect URL :

Santhosha Shetty · ‎08-27-2013

Hi,

from the packet-tracer output it looks like, the packet could not locate an SA.

Fom the earlier crypto ipsec sa output, the idents are listed as below:

sh cry ipsec sa pe x.x.x.x

peer address: x.x.x.x

Crypto map tag: crypto-map, seq num: 85, local addr: y.y.y.y

access-list ACLname extended permit ip 2.2.2.16 255.255.255.240 1.1.1.192 255.255.255.248

local ident (addr/mask/prot/port): (2.2.2.16/255.255.255.240/0/0)

remote ident (addr/mask/prot/port): (1.1.1.192/255.255.255.248/0/0)

current_peer: x.x.x.x

The vpn-session-db details you have snet seems to have different idents:

IPsec:

Tunnel ID : 26824.3

Local Addr : 172.20.0.16/255.255.255.240/0/0<<<<<<<<<<<<<<<<<<<<<<<<<<<<<

Remote Addr : 192.49.251.192/255.255.255.248/0/0<<<<<<<<<<<<<<<<<<<<<<<<

Encryption : AES256 Hashing : SHA1

Encapsulation: Tunnel PFS Group : 2

Rekey Int (T): 3600 Seconds Rekey Left(T): 1562 Seconds

Rekey Int (D): 2000000 K-Bytes Rekey Left(D): 1999910 K-Bytes

Idle Time Out: 0 Minutes Idle TO Left : 0 Minutes

Bytes Tx : 34077921 Bytes Rx : 19305198

Pkts Tx : 110721 Pkts Rx : 109692

Did you collect the session output for correct peer?Please veirfy the same and send again, "sh cry ipsec sa peer" and "show vpn-session-db...." output

Thanks

ciscosysadm01 · ‎08-28-2013

It is from correct peer. Just the "anonymising" missing a bit . For i.e. the 172.20.0.16/28 is in fact 2.2.2.16/28 and so on. I know it is confusing, but I wanted to edit some of the addresses since they are non RFC-1918.

It has been corrected in my previous output.

Santhosha Shetty · ‎08-28-2013

Hi,

Thanks for the clarifications, It is indeed confusing :-) !

We need to check asp table for crypto to see if there is a null entry which might be causing this issue.collect the following at the same time (for the same ipsec SA)

packet-tracer in in icmp 2.2.2.26 8 0 1.1.1.197 det

sh asp table vpn-context detail

sh asp table classify crypto

sh asp drop

Thanks

ciscosysadm01 · ‎08-28-2013

Hello,

Requested output below.

packet-tracer in in icmp 2.2.2.26 8 0 1.1.1.197 det

Phase: 1

Type: ROUTE-LOOKUP

Subtype: input

Result: ALLOW

Config:

Additional Information:

in 1.1.1.192 255.255.255.248 outside

Phase: 2

Type: IP-OPTIONS

Subtype:

Result: ALLOW

Config:

Additional Information:

Forward Flow based lookup yields rule:

in id=0xcc8b4490, priority=0, domain=inspect-ip-options, deny=true

hits=12344700, user_data=0x0, cs_id=0x0, reverse, flags=0x0, protocol=0

src ip=0.0.0.0, mask=0.0.0.0, port=0

dst ip=0.0.0.0, mask=0.0.0.0, port=0, dscp=0x0

Phase: 3

Type: INSPECT

Subtype: np-inspect

Result: ALLOW

Config:

class-map inspection_default

match default-inspection-traffic

policy-map global_policy

class inspection_default

inspect icmp

service-policy global_policy global

Additional Information:

Forward Flow based lookup yields rule:

in id=0xcd32f460, priority=70, domain=inspect-icmp, deny=false

hits=31549, user_data=0xcd3224a0, cs_id=0x0, use_real_addr, flags=0x0, protocol=1

src ip=0.0.0.0, mask=0.0.0.0, port=0

dst ip=0.0.0.0, mask=0.0.0.0, port=0, dscp=0x0

Phase: 4

Type: INSPECT

Subtype: np-inspect

Result: ALLOW

Config:

Additional Information:

Forward Flow based lookup yields rule:

in id=0xcd3dbf08, priority=70, domain=inspect-icmp-error, deny=false

hits=10286, user_data=0xcca600c0, cs_id=0x0, use_real_addr, flags=0x0, protocol=1

src ip=0.0.0.0, mask=0.0.0.0, port=0

dst ip=0.0.0.0, mask=0.0.0.0, port=0, dscp=0x0

Phase: 5

Type: DEBUG-ICMP

Subtype:

Result: ALLOW

Config:

Additional Information:

Forward Flow based lookup yields rule:

in id=0xcd3394b0, priority=12, domain=debug-icmp-trace, deny=false

hits=318341, user_data=0x0, cs_id=0x0, reverse, flags=0x0, protocol=1

src ip=0.0.0.0, mask=0.0.0.0, port=0

dst ip=0.0.0.0, mask=0.0.0.0, port=0, dscp=0x0

Phase: 6

Type: NAT-EXEMPT

Subtype:

Result: ALLOW

Config:

match ip inside 2.2.2.16 255.255.255.240 outside 1.1.1.192 255.255.255.248

NAT exempt

translate_hits = 5292, untranslate_hits = 67563

Additional Information:

Forward Flow based lookup yields rule:

in id=0xcd4b7a38, priority=6, domain=nat-exempt, deny=false

hits=27969, user_data=0xcd4b7978, cs_id=0x0, use_real_addr, flags=0x0, protocol=0

src ip=2.2.2.16, mask=255.255.255.240, port=0

dst ip=1.1.1.192, mask=255.255.255.248, port=0, dscp=0x0

Phase: 7

Type: NAT

Subtype:

Result: ALLOW

Config:

nat (inside) 1 0.0.0.0 0.0.0.0

match ip inside any outside any

dynamic translation to pool 1 ( [Interface PAT])

translate_hits = 341951, untranslate_hits = 127

Additional Information:

Forward Flow based lookup yields rule:

in id=0xccf01f10, priority=1, domain=nat, deny=false

hits=9166808, user_data=0xccf01e50, cs_id=0x0, flags=0x0, protocol=0

src ip=0.0.0.0, mask=0.0.0.0, port=0

dst ip=0.0.0.0, mask=0.0.0.0, port=0, dscp=0x0

Phase: 8

Type: NAT

Subtype: host-limits

Result: ALLOW

Config:

nat (inside) 1 0.0.0.0 0.0.0.0

match ip inside any outside any

dynamic translation to pool 1 ( [Interface PAT])

translate_hits = 341951, untranslate_hits = 127

Additional Information:

Forward Flow based lookup yields rule:

in id=0xccf02268, priority=1, domain=host, deny=false

hits=19374746, user_data=0xccf01e50, cs_id=0x0, reverse, flags=0x0, protocol=0

src ip=0.0.0.0, mask=0.0.0.0, port=0

dst ip=0.0.0.0, mask=0.0.0.0, port=0, dscp=0x0

Phase: 9

Type: VPN

Subtype: encrypt

Result: DROP

Config:

Additional Information:

Forward Flow based lookup yields rule:

out id=0xcca41ad0, priority=70, domain=encrypt, deny=false

hits=178431, user_data=0x0, cs_id=0xcc9f1960, reverse, flags=0x0, protocol=1

src ip=0.0.0.0, mask=0.0.0.0, port=0

dst ip=0.0.0.0, mask=0.0.0.0, port=0, dscp=0x0

Result:

input-interface: inside

input-status: up

input-line-status: up

output-interface: outside

output-status: up

output-line-status: up

Action: drop

Drop-reason: (acl-drop) Flow is denied by configured rule

**********************************************************************************

local ident (addr/mask/prot/port): (2.2.2.16/255.255.255.240/0/0)

remote ident (addr/mask/prot/port): (1.1.1.192/255.255.255.248/0/0)

current_peer:

#pkts encaps: 141520, #pkts encrypt: 141520, #pkts digest: 141520

#pkts decaps: 138879, #pkts decrypt: 138824, #pkts verify: 138824

#PMTUs sent: 0, #PMTUs rcvd: 0, #decapsulated frgs needing reassembly: 0

current outbound spi: 1D60C9B6

current inbound spi : DBA27288

spi: 0xDBA27288 (3684856456)

spi: 0x1D60C9B6 (492882358)

**********************************************************************************

sh asp table vpn-context detail

Peer IP = 1.1.1.192

Pointer = 0xCCB46240

State = UP

Flags = DECR+ESP

SA = 0x83A1F4E5

SPI = 0xDBA27288

Group = 4

Pkts = 138927

Bad Pkts = 0

Bad SPI = 0

Spoof = 0

Bad Crypto = 0

Rekey Pkt = 209

Rekey Call = 315

VPN Filter =

VPN CTX = 0x0C587584

Peer IP = 1.1.1.192

Pointer = 0xCD4AAA90

State = UP

Flags = ENCR+ESP

SA = 0x838469D7

SPI = 0x1D60C9B6

Group = 3

Pkts = 141569

Bad Pkts = 0

Bad SPI = 0

Spoof = 0

Bad Crypto = 0

Rekey Pkt = 209

Rekey Call = 209

VPN Filter =

VPN CTX = 0x0C584324

**********************************************************************************

sh asp table classify crypto

Interface inside:

Interface outside:

in id=0xccb4c550, priority=69, domain=ipsec-tunnel-flow, deny=false

hits=161, user_data=0xc58821c, cs_id=0x0, reverse, flags=0x0, protocol=0

src ip=1.1.1.192, mask=255.255.255.248, port=0

dst ip=2.2.2.16, mask=255.255.255.240, port=0, dscp=0x0

out id=0xcd345110, priority=70, domain=encrypt, deny=false

hits=162, user_data=0xc587584, cs_id=0xcadd6da0, reverse, flags=0x0, protocol=0

src ip=2.2.2.16, mask=255.255.255.240, port=0

dst ip=1.1.1.192, mask=255.255.255.248, port=0, dscp=0x0

out id=0xcadd6f68, priority=70, domain=encrypt, deny=false

hits=1, user_data=0x0, cs_id=0xcadd6da0, reverse, flags=0x0, protocol=0

src ip=2.2.2.16, mask=255.255.255.240, port=0

dst ip=1.1.1.192, mask=255.255.255.248, port=0, dscp=0x0

out id=0xcadd7540, priority=70, domain=encrypt, deny=false

hits=0, user_data=0x0, cs_id=0xcadd6da0, reverse, flags=0x0, protocol=0

src ip=2.2.2.26, mask=255.255.255.255, port=0

dst ip=1.1.1.192, mask=255.255.255.248, port=0, dscp=0x0

**********************************************************************************

sh asp drop

Frame drop:

Invalid encapsulation (invalid-encap) 15

No valid adjacency (no-adjacency) 12

No route to host (no-route) 211943

Flow is denied by configured rule (acl-drop) 47316687

Invalid SPI (np-sp-invalid-spi) 6497

First TCP packet not SYN (tcp-not-syn) 8623

TCP failed 3 way handshake (tcp-3whs-failed) 34428

TCP RST/FIN out of order (tcp-rstfin-ooo) 45003

TCP packet SEQ past window (tcp-seq-past-win) 2392

TCP invalid ACK (tcp-invalid-ack) 2

TCP replicated flow pak drop (tcp-fo-drop) 6

TCP Out-of-Order packet buffer full (tcp-buffer-full) 5219

TCP Out-of-Order packet buffer timeout (tcp-buffer-timeout) 368

TCP RST/SYN in window (tcp-rst-syn-in-win) 225

TCP dup of packet in Out-of-Order queue (tcp-dup-in-queue) 1460

TCP packet failed PAWS test (tcp-paws-fail) 20

IPSEC tunnel is down (ipsec-tun-down) 855

Slowpath security checks failed (sp-security-failed) 60412870

ICMP Inspect seq num not matched (inspect-icmp-seq-num-not-matched) 15

DNS Inspect packet too long (inspect-dns-pak-too-long) 101

DNS Inspect id not matched (inspect-dns-id-not-matched) 109245

Interface is down (interface-down) 1007

Last clearing: Never

Flow drop:

NAT reverse path failed (nat-rpf-failed) 1132

Need to start IKE negotiation (need-ike) 425706

Inspection failure (inspect-fail) 442

IPsec spoof packet detected (ipsec-spoof-detect) 4042

Last clearing: Never

koen.beuselinck · ‎11-21-2013

Hello,

I'm having the same issue with an ipsec vpn tunnel. In my situation natting is used. @ciscosysadm01 , did you manage to solve it ?

Koen

Peter Koltl · ‎11-21-2013

Please check if Juniper blocks the ICMP echo reply.

koen.beuselinck · ‎11-22-2013

Hello,

It was certainly not the remote side blocking the traffic.

This is a change of firewall ( from an old to a new one) , remote side stays the same. Before it worked.

The packet trace as show above helped me out to find the problem. I had the same Phase 9 drop. So it had to be an any to any rule who's blocking the icmp traffic.

And yes there was a rule (outside_cyrpto_map_X) in the ACL manager(site2site vpn/advanced) that I once added to test the traceroute (never worked ) that was still there.

I cleaned out all the rules so every outside crypto has one line "ip permit" , and suddenly the ping WORKS :-)

Koen