Tracert Problem ... Unbelievable

[scheikhnajib]

Hi there,

Sorry for being so stupid, but I'm really starting to have doubt in my skills.

I have a PIX 515e with OS 7.0(1), I have an internal machine that is statically NATing to a public IP using a static command. very simple question, how can trace route out from this machine to a any host. I reached the point of opening all IP traffic in and out and I'm still having timouts. When running a "debug icmp trace" I can see the type 11 IMCP messages moving in and out but somehow they are being stopped at the PIX interface ...

It's really driving me through the wall, and I can't get it ... I feel that I'm losing my conceptual thinking skills.

And what's bothring me more is this trace result:

C:\WINDOWS>tracert www.google.com

Tracing route to www.l.google.com [66.102.9.104]

over a maximum of 30 hops:

1 * * * Request timed out.

2 * * * Request timed out.

3 * * * Request timed out.

4 * * * Request timed out.

5 * * * Request timed out.

6 * * * Request timed out.

7 * * * Request timed out.

8 * * * Request timed out.

9 * * * Request timed out.

10 * * * Request timed out.

11 * * * Request timed out.

12 40 ms 39 ms 39 ms 66.102.9.104

Please tell I'm not that stupid ...

Salem.

[subaa]

Hi,

Allow the following traffic BACK the pix (so access-list on the outside):

access-list aclout permit icmp any any unreachable

access-list aclout permit icmp any any time-exceeded

access-list aclout permit icmp any any echo-reply

A_a

[scheikhnajib]

Hi M8,

That was my initial configuration since I knew that this is the exact required configuration but sorrowfully it is not working.

I ended up in the following config and still not working:

access-list aclout permit icmp any any

access-list aclout permit ip any any

I stopped my IDS engine, stopped my IP spoofing mechanisms ... and still no hope ... really confused.

Thanks for the help anyway.

Salem.

[bigchoice75]

can you please send your config minue sensitive information?

[scheikhnajib]

Here you go ...

Thanks in advance.

Salem.

[bigchoice75]

Wow....I haven't seen the 7.0 configs yet...it looks alot more like IOS.

My guess is that your IDS policy is blocking ICMP. I have had this problem in the past. Try removing IDS policy from interface and try trace route:

no ip audit name MY-ATTACK attack action alarm drop

IF it works, you'll need to figure out which signature is blocking icmp and disable it.

[bigchoice75]

ooops......wrong command:

to remove from interface.......

no ip audit interface outside MY-ATTACK

[Patrick Iseli]

Your config look good.

Have you seen this icmp config guide ?

The PIX and the traceroute Command:

http://www.cisco.com/en/US/products/hw/vpndevc/ps2030/products_tech_note09186a00800e9312.shtml

See: Handling ICMP Pings with the PIX Firewall

http://www.cisco.com/en/US/products/hw/vpndevc/ps2030/products_tech_note09186a0080094e8a.shtml

Command reference:

http://www.cisco.com/en/US/products/sw/secursw/ps2120/products_command_reference_chapter09186a008045277d.html#wp1521845

Working example:

Traveroute

Microsoft:

access-group 101 in interface outside

access-list 101 permit icmp any host YourPublicIP unreachable

access-list 101 permit icmp any host YourPublicIP time-exceeded

access-list 101 permit icmp any host YourPublicIP echo-reply

sincerely

Patrick

[scheikhnajib]

Thanks for all of your efforts guys, but it looks like I found something ...

I did a quick RedHat installation using VMWare on my local machine and tracing worked from it ...

Now I'm more upset, all of the local machines here are Windows based and they are not working, and after wasting two days of working on the perimeter devices ... it's just the stupid MS OS ...

Linux is working fine now, but I don't know what's wrong with XP Pro ???

By the way, I already tried all the ideas that you suggested guys, read all documents related to ICMP on Cisco website .... but sorrowfully nothing worked ...

Thanks alot guys ... but I think I need to post this on some Anti-Microsoft forum ... ;-)

Salem.

[Patrick Iseli]

I don't know if PIX OS 7.0 has the same issue, but it might be the problem source.

PIX Firewall earlier than version 6.3(2) block dns query responses if query comes from Server 2003 and the external DNS server supports EDNSO. Microsoft's workaround of disabling DNS-Probes did not solve the problem. Short of updating the IOS is there a way to change the DNS-UDP packet size limit that is currently set on PIX? . . or some other solution ?

In PIX OS 6.3, it is possible to change the maximum DNS packet length using "fixup protocol" command:

fixup protocol dns maximum-length

http://www.cisco.com/en/US/products/sw/secursw/ps2120/products_command_reference_chapter09186a00801727a8.html#wp1067379

Or disabling dns fixup altogether (no fixup protocol dns)

I don't think there is any workaround with OS 6.2 and earlier, since the "DNS Guard" feature is hard-coded with no knobs for tweaking.

These threads are related to the same issue:

http://forum.cisco.com/eforum/servlet/NetProf?page=netprof&forum=Security&topic=Firewalling&CommCmd=MB%3Fcmd%3Ddisplay_location%26location%3D.1dd75208

http://forum.cisco.com/eforum/servlet/NetProf?page=netprof&forum=Security&topic=Firewalling&CommCmd=MB%3Fcmd%3Ddisplay_location%26location%3D.1dd7c059

I had the same problem and resolved it back in November using this procedure. Make sure you do it on BDC also. Stop/Restart DNS service or reboot domain controllers.

For those customers using the Microsoft Windows 2003 Server's DNS

application, you must first install the dnscmd.exe command-line tool

from the Microsft Windows 2003 Server CD-ROM's Support Tools. Once

this is installed, open a DOS command prompt window and type:

dnscmd /config /enableednsprobes 0

This will turn off EDNS.

If at a future time the customer upgrades their firewall application

to support EDNS, running "dnscmd /config /enableednsprobes 1" will

turn EDNS back on.

sincerely

Patrick

[prasadrp]

Hi Salem

Since this was an interesting discussion, I just want to add to the discovery you made regarding trace route working from Linux but nor from Windows. This is due to the different implementation of traceroute in these 2 OS. Linux implements traceroute through UDP probe packets and Windows does it through icmp echo. That's the reason it was working on linux and not working on Windows

[nfox · Accepted solution]

Hi Salem,

Not sure if this is relevant in your case, but recently I was tearing my hair out over traceroute using PIX v7. The annoying thing was that I had upgraded our internal office PIX from 6.3(4) to v7.0(1) and had not seen this issue, until I performed a customer installation... The issue is to do with the default inspection rules that are applied on startup.

With a default config, PIX v7 does not enable 'inspect icmp error'. If this is enabled, you will find that traceroute will then work fine. See example below:

policy-map global_policy

class inspection_default

inspect icmp error

IIRC, you still need to allow the returning icmp traffic through your external ACL.

In my office, I had used the ASDM GUI to enable every inpection option available, but when performing the customer install, I had started from the default rules, and hence my problem with traceroute!

Hope this may help.

[scheikhnajib]

Thanks alot Mate,

You are a real star ... it's solved ...

Salem.