ASA 8.3+ traceroute/unreachable - inside to internet (WAN ACL private IP's)

[will]

I'm perplexed by a certain problem that has arisen since 8.3 code, where the WAN interfaces (security=0) of the firewall has an ACL (for internal services) that now includes the _inside_ IP's. This is counter-intuitive; and poses a problem. First consider this:

Scenario: ASA 9.2 Code, 1 WAN interface (1.1.1.1), 1 LAN interface (10.1.1.1/24)

1 internal web server (accessible from internet): 10.1.1.100

Many internal hosts on 10.1.1.0/24 doing PAT for internet services.

I apply a Bogon ACL on WAN using a BogonGroup network object, containing RFC1918 IP’s and more. In 8.2 code, I would order the WAN interface ACL as follows:

1. deny Internet->BogonGroup IP (deny all traffic destined to bogon addresses, including 10.X.X.X)

2. allow Internet->1.1.1.1 80/443 (web traffic to the internal web server); also has a 10.1.1.1->10.1.1.100 NAT.

Now fast forward to 8.3+ ASA and I have a screwy NAT situation to deal with because I now have to use the internal IP in the WAN ACL, changing line 2, as follows:

2. allow Internet->10.1.1.100 80/443

Therein lies the problem. My bogon ACE is now going to deny the web server traffic. But I can re-order the ACE's as follows

1. allow Internet->10.1.1.100 80/443 (web traffic to the internal web server); also has a 10.1.1.1->10.1.1.100 NAT.

2. deny Internet->BogonGroup IP (deny all traffic destined to bogon addresses, including 10.X.X.X)

This leads to an unsettling problem when considering ICMP unreachables and Time-exceeded. I typically would create an ACL like this for TE & UR:

1. deny Internet->BogonGroup IP (deny all traffic destined to bogon addresses, including 10.X.X.X)

2. allow Internet->1.1.1.1 ICMP Unreach & TE (allow a response from any host to the WAN for UR and TE)

3. allow Internet->1.1.1.1 80/443 (web traffic to the internal web server); also has a 10.1.1.1->10.1.1.100 NAT.

Line 2 is really just a way of handling the ICMP response to internal hosts that are NAT-ed out to the internet and receive a UR or TE ICMP message. This allows a traceroute to return the valid path, for example. Without this, I cannot NAT out and do a proper traceroute through the internet. So in 8.2, all worked well, but with 8.3+, this doesn't work anymore as I have to change both lines 2 and 3 to internal addresses. 

Since I don't really know the inside host that may need the UR or TE, I need to make the subnet broad on that ACE, as follows. Pay attention to line 2, which now includes the entire internal subnet. Without this, it will fail as I don't know the host.

1. deny Internet->BogonGroup IP (deny all traffic destined to bogon addresses, including 10.X.X.X)

2. allow Internet->10.1.1.0/24 ICMP Unreach & TE (allow a response from any internet host to the WAN interface for UR and TE)

3. allow Internet->10.1.1.100 80/443 (web traffic to the internal web server); also has a 10.1.1.1->10.1.1.100 NAT.

To solve the 8.3+ private IP on WAN ACL problem, I re-order:

1. allow Internet->10.1.1.0/24 ICMP UR & TE (allow a response from any internet host to the WAN interface for UR & TE)

2. allow Internet->10.1.1.100 80/443 (web traffic to the internal web server); also has a 10.1.1.1->10.1.1.100 NAT.

3. deny Internet->BogonGroup IP (deny all traffic destined to bogon addresses, including 10.X.X.X)

The problem line is now 1, as it seems like this will allow a potential hacker to send UR & TE packets destined to my firewall WAN interface with an ultimate destination of any internal address in the 10.1.1.0/24 subnet. There seems to be some sort of intelligence in ASA to not require an ICMP translation for the UR/TE packets.

So I don’t feel too comfortable by line 2, but am more uncomfortable by line 1, being in front of line 3. Does anyone have a good way to deal with this in 8.3+ OS and configuration? What is the exact mechanism for the return traceroute. Isn't there some sort of fixup/policy to handle TE return traffic? the "inspect icmp error" feature doesn't appear to work.

FYI... Here are some other command regarding ICMP/Traceroute for ASA:

Allows the ASA to show as a hop and decrement the TTL:
policy-map global_policy
   class class-default
      set connection decrement-ttl

Allows the ASA to respond with traceroute statistics when it’s on its hop:
icmp unreachable rate-limit 10 burst-size 5 **

** doesn't seem to work for me, but I may have a host firewall blocking rule.

Allows the ASA to inspect ICMP ping through firewall, but not traceroute:

policy-map global_policy
 class inspection_default
  inspect icmp

Allows the ASA to pass traceroute through firewall (essentially an ICMP echo request with varying TTL):

Need inbound LAN ACL allowing ICMP echo, and the inspect ICMP above.

[Collin Clark]

Agreed that counter-intuitive is an understatement, but the sheep of the network world beat out the professionals ;-)

Here's my $0.02. Filter ICMP with your ISP router. There is more flexibility there. If you *have* to use the ASA use the "icmp" command and only use ACL's for downstream hosts. You don't *need* to create the bogon filter. The only time I create it (and I still usually just use deny ip any any) is when you need to log for security policy purposes. Again, IMO, this should be handled by the upstream ISP router. I typically create an object-group in the edge router that allows specific ICMP types; echo, echo-reply, TE. All else gets filtered and never hits the ASA. I look at it like this; use the router to filter out 80% of the junk and let the firewall do it's job (stateful inspection, NAT translations, etc). Don't use it as a filtering host on the internet edge.

Hope it helps.

[will]

hi Collin, thx for reply on this. with some more thought on the matter and a TAC ticket, I convinced myself that the traffic will flow as follows:

1. inside interface: ACL allowing ICMP echo out (with decremented TTL's). this is actually the form of many traceroutes (from windows boxes, for example)

2. one way (dynamic PAT) from inside to outside, translates the ICMP echo to WAN interface IP.

3. the echo goes to internet, returns as a TE (time exceeded) from various hops.

4. This TE hits the ASA outside interface and gets NAT translated back to the internal IP that originated the request.

5. At this point, if I didn't have the outside interface ACL allowing TE inbound, the response would get dropped. (this is the bizarre ACL allowing replies from the internet to _NOT_ the wan interface, but the RFC1918 IP on the inside.)

6. Since I have the bizarre (_stupid_ IMO!) RFC1918 ACL on the outside interface allowing it inbound, the packet then gets processed by the ACL (allowed) and goes through.

the weird (interesting) thing about this is:

1. this only works if you assume the NAT process takes place before the ACL process. I think this is dangerous and didn't think ASA did this. I suspect sloppy code or a quick "oh-crap" bug fix is in that part of the OS code (when they realized their RFC1918 ACL endeavor was too far down the line to revert it back). Wish I could review the source code now!

2. this TE reply works without any ICMP fixup (inspection) in the default global policy, even though its an echo outbound!

3. normal echo outbound to the end host _doesn't_ work without ICMP fixup (inspection) in the default global policy.

I think this is the case because with the standard echo, the endhost replies back with the end host IP, but the TE's reply back from various hops along the way, and are not in the connection state table established by the fixup.

So with all that ICMP Sudoku, everything seems to work and seems to be safe; although I really don't like those RFC1918 addresses in the outside interface! Bad programming choice on cisco's part.

Lastly, regarding block junk at the router, I like that idea and have done that before by creating a bogon on the router in front of the ASA (if you can actually manage that device - IOW, not owned by a colo facility, etc.). The other drawback to that is that you cannot easily see if you are getting bombarded by some sort of RFC1918 hack in the firewall logs. But I do agree that I would probably do it your way if I owned the router. its a lot less junk in the firewall logs!