ASA 5510: Allowing ICMP unreachable on outside interface

[jimmcwoftam]

I am having some issues with my ASA 5510 (running ASA 8.2) dropping ICMP unreachable-fragmentation-required-but-df-bit-set type messages coming in on the outside interface. I have the following entry in the ACL for the outside interface:

access-list outside_acl extended permit icmp any interface outside

and there are no other entries in that list that should take precedence and drop the packet. Pings from outside to the ASA work when this ACE is present and do no when it is absent so it is clearly taking effect.

I see the following entries in the debug log when sending a large non-fragmentable packet (that would cause an intermediate router to send back this ICMP response) out to the internet through the ASA:

Denied ICMP type=3, code=4 from y.y.y.y on interface outside

No matching connection for ICMP error message: icmp src outside:y.y.y.y dst identity:x.x.x.x (type 3, code 4) on outside interface. Original IP payload: protocol 50 src x.x.x.x dst z.z.z.z.

where x.x.x.x is my outside interface address, y.y.y.y is some router on the way to the destination (that wants to fragment the packet) and z.z.z.z is the destination.

As far as I can tell I am not running ICMP inspection; I don't want it to do any stateful magic here since the outgoing traffic would have been ordinary data from another protocol and would not have caused an outgoing ICMP connection to be built to match against.

Worth mentioning I guess is that this is IPsec VPN traffic: x.x.x.x and z.z.z.z above are the two ends of the tunnel. It's done with a crypto map, not a GRE tunnel interface. As you may guess I'm having MTU issues through the VPN but have tracked this down to the ASA blocking the ICMP unreachables as above such that path MTU discovery doesn't work.

Also I have tried without success:

• Permitting ICMP from any to any on that outside interface
  
• A more specific entry permitting only ICMP unreachables on the outside interface
  
• icmp permit any outside
  
• Turning on and off ICMP and ICMP error inspect policies
  
• Increasing the icmp unreachable rate-limit
  

Parts of my config relevant to ICMP (I'm not going to post a whole config without good reason, it will take a while to sanitize):

access-list outside_acl extended permit icmp any interface outside

...

icmp unreachable rate-limit 1 burst-size 1

...

timeout conn 1:00:00 half-closed 0:10:00 udp 0:02:00 icmp 0:00:02

Any ideas? I haven't been able to find much information about this problem online - thought it would be more common.

[cadet alain]

Hi'

the syntax is wrong.

you must configure the ACL then apply it with the acces-group command.

Regards.

Alain.

[jimmcwoftam]

Apologies - I did not post that part of the config because it was obviously present and correct to me; I mentioned that I could see the ACE was taking effect through a ping test. Anyway, be assured that elsewhere in my config I have:

access-group outside_acl in interface outside

[cadet alain]

Hi,

access-list outside_acl extended permit icmp any interface outside

this is an incorrect syntax, it should be access-list outside_acl extended permit icmp any any

Regards.

Alain.

[jimmcwoftam]

Well, since this is running config it must have been accepted by the ASA so I would disagree that the syntax is incorrect. In any case I have tried what you suggest (that was in the also-tried section of my post - sorry if the exact config I tried wasn't clear) and the behaviour of rejecting the packets persists.

[cadet alain]

Hi,

ok I just saw you said this was  VPN traffic, can you post all the vpn config part.

Alain.

[jimmcwoftam]

I know I said I wouldn't, but in hindsight it is easier to post the whole config. I have removed passwords and global IP addresses but for IPs have tried to use the same convention of different letters for different addresses as above to help follow them through the config.

There are various internal and remote LANs isolated from each other to varying degress. The ASA terminates two site-to-site VPNs and also remote access VPNs (authenticating via RADIUS). Hopefully debugging the ICMP issue should not need too much of an in depth understanding of that part of the setup as it primarily concerns the outside interface. This config was generated mostly by the ASDM so sorry for the mess.

ASA Version 8.2(1)

!

hostname asa

domain-name nug.local

enable password xxxxxxxx encrypted

passwd xxxxxxxx encrypted

names

name 172.16.1.5 exchange description Exchange server

name x.x.x.x isp-gw description ISP router

name 10.1.2.2 core description Core router

name 172.16.0.0 lan description LAN Clients

name 10.2.1.0 lan2 description Remote site LAN

name 10.1.4.0 lan3 description Another isolated LAN

name 10.1.6.0 vpnclients description VPN Clients

name 172.16.1.29 extranet description Extranet web server

name 10.3.1.0 cc3-lan description Other LAN Clients

name 10.3.5.0 cc3-voice description Other LAN Phones

name 10.1.5.0 voice description Voice VLAN

!

interface Ethernet0/0

nameif outside

security-level 0

ip address x.x.x.x 255.255.255.240

!

interface Ethernet0/1

no nameif

no security-level

no ip address

!

interface Ethernet0/2

no nameif

no security-level

no ip address

!

interface Ethernet0/3

shutdown

no nameif

no security-level

no ip address

!

interface Management0/0

nameif management

security-level 100

ip address 10.1.1.51 255.255.255.0

management-only

!

interface Redundant1

member-interface Ethernet0/1

member-interface Ethernet0/2

no nameif

no security-level

allow-ssc-mgmt

no ip address

!

interface Redundant1.102

description Main internet connection into core

vlan 102

nameif inet-inside

security-level 100

ip address 10.1.2.1 255.255.255.0

!

interface Redundant1.103

description Isolated internet connection for guests

vlan 103

nameif inet-guests

security-level 50

ip address 10.1.3.1 255.255.255.0

!

ftp mode passive

clock timezone GMT 0

dns server-group DefaultDNS

domain-name management.local

same-security-traffic permit intra-interface

object-group network DM_INLINE_NETWORK_1

network-object lan3 255.255.255.0

network-object lan 255.255.248.0

network-object vpnclients 255.255.255.0

network-object cc3-lan 255.255.255.0

object-group network DM_INLINE_NETWORK_2

network-object lan3 255.255.255.0

network-object lan 255.255.248.0

network-object vpnclients 255.255.255.0

network-object cc3-lan 255.255.255.0

object-group network DM_INLINE_NETWORK_4

network-object lan2 255.255.255.0

network-object lan 255.255.248.0

network-object vpnclients 255.255.255.0

object-group network DM_INLINE_NETWORK_3

network-object vpnclients 255.255.255.0

network-object lan 255.255.248.0

object-group network DM_INLINE_NETWORK_5

network-object lan2 255.255.255.0

network-object cc3-lan 255.255.255.0

access-list outside_acl extended permit tcp any interface outside eq https

access-list outside_acl extended permit tcp any host e.e.e.e eq https

access-list outside_acl extended permit icmp any interface outside

access-list outside_acl extended permit icmp any host g.g.g.g

access-list outside_nat0_outbound extended permit ip vpnclients 255.255.255.0 object-group DM_INLINE_NETWORK_5

access-list vpn_nonat extended permit ip object-group DM_INLINE_NETWORK_1 lan2 255.255.255.0

access-list vpn_nonat extended permit ip object-group DM_INLINE_NETWORK_3 cc3-lan 255.255.255.0

access-list vpn_nonat extended permit ip voice 255.255.255.0 cc3-voice 255.255.255.0

access-list vpn_nonat extended permit ip any vpnclients 255.255.255.0

access-list inet-guests_access_in extended deny ip any lan 255.255.248.0

access-list inet-guests_access_in extended deny ip any lan3 255.255.255.0

access-list inet-guests_access_in extended permit ip any any

access-list outside_cryptomap extended permit ip object-group DM_INLINE_NETWORK_2 lan2 255.255.255.0

access-list outside_cryptomap_cc3 extended permit ip object-group DM_INLINE_NETWORK_4 cc3-lan 255.255.255.0

access-list outside_cryptomap_cc3 extended permit ip voice 255.255.255.0 cc3-voice 255.255.255.0

pager lines 24

logging enable

logging asdm informational

mtu outside 1500

mtu management 1500

mtu inet-inside 1500

mtu inet-guests 1500

ip local pool vpnclients 10.1.6.5-10.1.6.250 mask 255.255.255.0

no failover

icmp unreachable rate-limit 1 burst-size 1

asdm location core 255.255.255.255 management

asdm location lan 255.255.248.0 management

asdm location lan2 255.255.255.0 management

asdm location lan3 255.255.255.0 management

asdm location vpnclients 255.255.255.0 management

asdm location extranet 255.255.255.255 management

asdm location voice 255.255.255.0 management

asdm location cc3-lan 255.255.255.0 management

asdm location cc3-voice 255.255.255.0 management

no asdm history enable

arp timeout 14400

global (outside) 1 interface

global (outside) 2 g.g.g.g netmask 255.255.255.248

nat (outside) 0 access-list outside_nat0_outbound

nat (outside) 1 vpnclients 255.255.255.0

nat (inet-inside) 0 access-list vpn_nonat

nat (inet-inside) 1 lan3 255.255.255.0

nat (inet-inside) 1 lan 255.255.248.0

nat (inet-guests) 2 10.1.3.0 255.255.255.0

static (inet-inside,outside) tcp interface https exchange https netmask 255.255.255.255

static (inet-inside,inet-guests) tcp x.x.x.x https exchange https netmask 255.255.255.255

static (inet-inside,outside) tcp e.e.e.e https extranet https netmask 255.255.255.255

static (inet-inside,inet-guests) tcp e.e.e.e https extranet https netmask 255.255.255.255

access-group outside_acl in interface outside

access-group inet-guests_access_in in interface inet-guests

route outside 0.0.0.0 0.0.0.0 isp-gw 1

route inet-inside lan3 255.255.255.0 core 1

route inet-inside voice 255.255.255.0 core 1

route inet-inside lan 255.255.248.0 core 1

timeout xlate 3:00:00

timeout conn 1:00:00 half-closed 0:10:00 udp 0:02:00 icmp 0:00:02

timeout sunrpc 0:10:00 h323 0:05:00 h225 1:00:00 mgcp 0:05:00 mgcp-pat 0:05:00

timeout sip 0:30:00 sip_media 0:02:00 sip-invite 0:03:00 sip-disconnect 0:02:00

timeout sip-provisional-media 0:02:00 uauth 0:05:00 absolute

timeout tcp-proxy-reassembly 0:01:00

dynamic-access-policy-record DfltAccessPolicy

aaa-server Vasco protocol radius

aaa-server Vasco (inet-inside) host 172.16.1.24

key XXXXXXXXX

authentication-port 1812

accounting-port 1813

http server enable

http 10.1.1.0 255.255.255.0 management

http lan 255.255.248.0 management

no snmp-server location

no snmp-server contact

snmp-server enable traps snmp authentication linkup linkdown coldstart

crypto ipsec transform-set ESP-AES-256-SHA esp-aes-256 esp-sha-hmac

crypto ipsec transform-set ESP-3DES-SHA esp-3des esp-sha-hmac

crypto ipsec transform-set ESP-AES-256-MD5 esp-aes-256 esp-md5-hmac

crypto ipsec transform-set ESP-DES-SHA esp-des esp-sha-hmac

crypto ipsec transform-set ESP-DES-MD5 esp-des esp-md5-hmac

crypto ipsec transform-set ESP-AES-192-MD5 esp-aes-192 esp-md5-hmac

crypto ipsec transform-set ESP-3DES-MD5 esp-3des esp-md5-hmac

crypto ipsec transform-set ESP-AES-128-SHA esp-aes esp-sha-hmac

crypto ipsec transform-set ESP-AES-192-SHA esp-aes-192 esp-sha-hmac

crypto ipsec transform-set ESP-AES-128-MD5 esp-aes esp-md5-hmac

crypto ipsec security-association lifetime seconds 28800

crypto ipsec security-association lifetime kilobytes 4608000

crypto dynamic-map SYSTEM_DEFAULT_CRYPTO_MAP 65535 set transform-set ESP-AES-128-SHA ESP-AES-128-MD5 ESP-AES-192-SHA ESP-AES-192-MD5 ESP-AES-256-SHA ESP-AES-256-MD5 ESP-3DES-SHA ESP-3DES-MD5 ESP-DES-SHA ESP-DES-MD5

crypto map outside_map2 1 match address outside_cryptomap

crypto map outside_map2 1 set peer q.q.q.q

crypto map outside_map2 1 set transform-set ESP-AES-256-SHA

crypto map outside_map2 1 set security-association lifetime seconds 3600

crypto map outside_map2 2 match address outside_cryptomap_cc3

crypto map outside_map2 2 set peer z.z.z.z

crypto map outside_map2 2 set transform-set ESP-AES-256-SHA

crypto map outside_map2 2 set security-association lifetime seconds 3600

crypto map outside_map2 65535 ipsec-isakmp dynamic SYSTEM_DEFAULT_CRYPTO_MAP

crypto map outside_map2 interface outside

crypto map inet-guests_map 65535 ipsec-isakmp dynamic SYSTEM_DEFAULT_CRYPTO_MAP

crypto map inet-guests_map interface inet-guests

crypto isakmp enable outside

crypto isakmp enable inet-guests

crypto isakmp policy 10

authentication pre-share

encryption aes-256

hash sha

group 5

lifetime 86400

crypto isakmp policy 30

authentication pre-share

encryption aes

hash sha

group 2

lifetime 86400

telnet 10.1.1.0 255.255.255.0 management

telnet lan 255.255.248.0 management

telnet timeout 5

ssh timeout 5

console timeout 0

dhcpd address 10.1.1.200-10.1.1.250 management

dhcpd dns 172.16.1.18 172.16.1.21 interface management

dhcpd enable management

!

dhcpd address 10.1.3.100-10.1.3.200 inet-guests

dhcpd dns 8.8.8.8 4.4.4.4 interface inet-guests

dhcpd option 3 ip 10.1.3.1 interface inet-guests

dhcpd enable inet-guests

!

threat-detection basic-threat

threat-detection statistics access-list

no threat-detection statistics tcp-intercept

webvpn

group-policy TGVPN internal

group-policy TGVPN attributes

dns-server value 172.16.1.18 172.16.1.21

vpn-tunnel-protocol IPSec

default-domain value nug.local

username admin password XXXXXXXXX encrypted privilege 15

tunnel-group q.q.q.q type ipsec-l2l

tunnel-group q.q.q.q ipsec-attributes

pre-shared-key XXXXXXXXXXX

tunnel-group TGVPN type remote-access

tunnel-group TGVPN general-attributes

address-pool vpnclients

authentication-server-group Vasco

default-group-policy TGVPN

tunnel-group TGVPN ipsec-attributes

pre-shared-key XXXXXXXX

tunnel-group z.z.z.z type ipsec-l2l

tunnel-group z.z.z.z ipsec-attributes

pre-shared-key XXXXXXXX

!

class-map inspection_default

match default-inspection-traffic

!

!

policy-map type inspect dns preset_dns_map

parameters

  message-length maximum 512

policy-map global_policy

class inspection_default

  inspect dns preset_dns_map

  inspect ftp

  inspect h323 h225

  inspect h323 ras

  inspect rsh

  inspect rtsp

  inspect esmtp

  inspect sqlnet

  inspect skinny 

  inspect sunrpc

  inspect xdmcp

  inspect sip 

  inspect netbios

  inspect tftp

  inspect pptp

!

service-policy global_policy global

prompt hostname context

Cryptochecksum:7490943c64fde1d9b617e07c81daf4db

: end

[cadet alain]

Hi,

I know I repeat myself a lot but it must be the age surely.

BUT I have never seen this syntax and I've verified in 8.2 config guide and I don't see it anywhere:

access-list outside_acl extended permit icmp any interface outside

can you post sh access-list outside_acl  for my personal info to see how it works

And did you try the syntax I gave and if so was it still the same? is this  big packet sent from the ASA ?

Can you capture traffic on the ASA and save as pcap and post while doing the thing that fails.

Alain.

[jimmcwoftam]

Please see http://www.cisco.com/en/US/docs/security/asa/asa80/command/reference/a1.html#wp1541842 for documentation on using interface addresses in ACEs. Perhaps it is a relatively new syntax. Also:

asa(config)# access-list outside_acl extended permit ip any ?

configure mode commands/options:
  Hostname or A.B.C.D  Destination IP address
  any                  Abbreviation for destination address and mask of 0.0.0.0
                       0.0.0.0
  host                 Use this keyword to configure destination host
  interface            Use interface address as destination address
  object-group         Network object-group for destination address

I have now tried your suggestion anyway and it has not worked; I am still seeing the packets denied in the real-time log.

I am generating the packets from a Windows machine inside the network by pinging a host on the remote end of the VPN like so:

ping -l 1400 -f 10.3.1.3

They then become oversized for a particular router on the path once the VPN encapsulation is added, and we need to be able to receive their ICMP messages saying so.

I will get you a capture now.

[cadet alain]

Hi,

ok that's cool I learned something new but it wasn't present in the configuration guide and I had never used it, I forgot the command reference

I'm waiting for cap files.

Alain.

[jimmcwoftam]

I won't upload a straight PCAP file as that includes the external IP address of the ASA I have just posted the config for. However I have re-exported it as text from Wireshark with the addresses anonymized in the hope that this is useful to you. This is a capture of all ICMP traffic on the outside interface while I am running a ping -l 1400 -f 10.3.1.3 to a machine on the other side of the VPN.

[Garden4ever]

Hi I saw that you have running ASA 8.0.2 and i found this

http://www.cisco.com/en/US/docs/security/asa/asa80/release/notes/arn803n.html

CSCsk68658

Yes

ICMP (type 3 code 4) messages generated against ESP flow dropped by ASA

  I very alike like your issue but, they have resolved with ASA 8.0.3

I Hope you find the solution.

Regards

APS

[mateja.jovanovic]

If you want to allow only UNRECAHABLES  (type 3) ICMP-s, your syntax is wrong.

Try this:

access-list inside_access_in extended permit icmp any any unreachable

Hope this solves it