traceroute not complete

karthikeyan gopinathan · ‎10-12-2011

Hi

i have some wierd problem in my network.

from a desktop/workstation/laptop i am able to traceroute to external IP addresses.the tracroute is complete. this desktop is directly connected to a layer switch . when i login to the layer 3 switch and try a traceroute any external IP address the trace is in complete. i am able to ping external IP address from the switch sucessfully but traceroute is incomplete for some reason. can any one give me some suggestion

thanks

Karthik

nkarpysh · ‎10-12-2011

Hello,

SO I understand it in a following way:

ping from host to eternal ip working
trace from host to ext-ip working
ping/trace is going thrugh L3 switch

Next from L3 switch in turn

ping to ext-ip - working
trace - not working

So as trace is doing actual ping just increaseing TTL one by one to reach each hop in turn. This means that we are reaching some node and the ping to the node from switch ip address is blocked. At the same time - passing by packets (ping from switch and host) are handled in HW - so no problem. The reason why trace from host work on that node - possibly host ip is not blocked there - or choosing other way. There is a chance that you do trace from switch by default from different ip subnet (from host).

So I would check this:

try to do ping/trace from switch using same subnet where host is in
Check the device where trace stopped for any ACL or CoPP blocking the switch ip

Nik,

HTH,
Niko

fb_webuser · ‎10-13-2011

Where does the traceroute fail?

---

Posted by WebUser Daryl W. Clark

karthikeyan gopinathan · ‎10-13-2011

Hi

Below is complete traceroute details from the host

U:\>tracert jupitergrades.com

Tracing route to jupitergrades.com

over a maximum of 30 hops:

1 <1 ms <1 ms <1 ms 172.16.0.1

2 9 ms 1 ms 1 ms c-26.oe450.dc0.bhgov.net

3 4 ms 1 ms 1 ms 12.118.130.29

4 36 ms 35 ms 35 ms cr84.la2ca.ip.att.net

5 36 ms 38 ms 36 ms cr2.la2ca.ip.att.net

6 38 ms 37 ms 35 ms cr2.dlstx.ip.att.net

7 36 ms 34 ms 35 ms gar6.dlstx.ip.att.net

8 36 ms 35 ms 34 ms 12.90.228.14

9 35 ms 34 ms 37 ms border2.ge2-1-bbnet2.ext1a.dal.pnap.net [216.52.

191.85]

10 39 ms 36 ms 36 ms rackmanaged-3.border2.ext1a.dal.pnap.net [63.251

.32.50]

11 63 ms 36 ms 40 ms vlan901.core1.dfw1.rackspace.com

12 40 ms 36 ms 39 ms aggr128a.dfw1.rackspace.net

13 38 ms 36 ms 37 ms jupitered.com

Trace complete.

Below is the trace route details from my layer 3 switch where this host is connected to and trace is for the same IP address

CORE-DO-C6509E-1#traceroute 67.192.47.227

Type escape sequence to abort.

Tracing the route to 67.192.47.227

1 12.164.240.129 4 msec 0 msec 40 msec

2 12.118.130.29 8 msec 4 msec 8 msec

3 12.122.129.50 40 msec 36 msec 36 msec

4 12.123.30.250 40 msec 36 msec 40 msec

5 12.122.28.177 40 msec 40 msec 40 msec

6 12.122.100.65 40 msec 36 msec 40 msec

7 12.90.228.14 40 msec 36 msec 36 msec

8 216.52.191.25 40 msec

216.52.191.85 40 msec

216.52.191.25 40 msec

9 63.251.32.50 40 msec 40 msec 40 msec

10 72.3.128.21 40 msec 44 msec 40 msec

11 72.3.129.157 40 msec 40 msec 40 msec

12 * * *

13 * * *

14 * * *

15 * * *

16 * * *

17 * * *

18 * * *

19 * * *

20 * * *

21 * * *

22 * * *

23 * * *

24 * * *

25 * * *

26 * * *

27 * * *

28 * * *

29 * * *

30 * * *

cadet alain · ‎10-13-2011

Hi,

Cisco devices are doing UDP traceroute vs windows hosts which do ICMP traceroute so if the host you are tracerouting to is filtering those UDP packets you'll get the result you have.Usually it's the other way, as hosts filter ICMP but not UDP.Just try a traceroute from a Linux/Unix host(which do UDP traceroute too) and if it fails again then you'll know this was the reason.

Regards.

Alain.

Don't forget to rate helpful posts.

karthikeyan gopinathan · ‎10-13-2011

You are right I logged in to Cisco ASA firewall and I use the command traceroute x.x.x.x use-icmp and the traceroute was complete. In the switch I don’t have any option to use icmp

DistASA5520# traceroute 67.192.47.227 numeric

Type escape sequence to abort.

Tracing the route to 67.192.47.227

1 12.164.240.129 0 msec 10 msec 0 msec

2 12.118.130.29 0 msec 0 msec 0 msec

3 12.122.129.50 40 msec 30 msec 40 msec

4 12.123.30.250 30 msec 40 msec 30 msec

5 12.122.28.177 40 msec 30 msec *

6 12.122.100.65 30 msec 30 msec 40 msec

7 12.90.228.14 30 msec 40 msec 30 msec

8 216.52.191.85 40 msec

216.52.191.25 30 msec 40 msec

9 63.251.32.50 30 msec 40 msec 40 msec

10 72.3.128.21 30 msec 40 msec 30 msec

11 72.3.129.157 40 msec 40 msec 30 msec

12 * * *

13 * * *

14 * *

DistASA5520# traceroute 67.192.47.227 use-icmp

Type escape sequence to abort.

Tracing the route to 67.192.47.227

1 c-26.oe450.dc0.bhgov.net (12.164.240.129) 0 msec 0 msec 10 msec

2 12.118.130.29 0 msec * 0 msec

3 cr84.la2ca.ip.att.net (12.122.129.50) 60 msec 30 msec 40 msec

4 cr2.la2ca.ip.att.net (12.123.30.250) 30 msec 40 msec 30 msec

5 cr2.dlstx.ip.att.net (12.122.28.177) 40 msec 30 msec 40 msec

6 gar6.dlstx.ip.att.net (12.122.100.65) 30 msec 40 msec 30 msec

7 12.90.228.14 40 msec 30 msec 40 msec

8 border2.ge2-1-bbnet2.ext1a.dal.pnap.net (216.52.191.85) 30 msec 40 msec 30 msec

9 rackmanaged-3.border2.ext1a.dal.pnap.net (63.251.32.50) 40 msec 30 msec 40 msec

10 vlan901.core1.dfw1.rackspace.com (72.3.128.21) 30 msec 40 msec 40 msec

11 aggr128a.dfw1.rackspace.net (72.3.129.157) 30 msec 40 msec 30 msec

12 jupitered.com (67.192.47.227) 40 msec 30 msec *

DistASA5520#

Jonathancert_2 · ‎10-13-2011

Try doing an extended traceroute from your router and change the "numeric display" default option. Will prevent router from trying to resolve domain-name for devices. See if this changes your results.

CISCO#traceroute

Protocol [ip]:

Target IP address: 67.192.47.227

Source address: 12.164.240.129

Numeric display [n]: y

Timeout in seconds [3]:

Probe count [3]:

Minimum Time to Live [1]:

Maximum Time to Live [30]:

Port Number [33434]:

Loose, Strict, Record, Timestamp, Verbose[none]:

Another option is to change the Port Number that traceroute uses by default.

Jonathan,

nkarpysh · ‎10-13-2011

But first try to understand one thing. These all said above we can see that particular traffic (source ip, dest ip, source port, dest port, protocol) is blocked on the router of final destination.

See the working trace:

12 40 ms 36 ms 39 ms aggr128a.dfw1.rackspace.net 72.3.129.157

13 38 ms 36 ms 37 ms jupitered.com 67.192.47.227

This is the non-working one:

11 72.3.129.157 40 msec 40 msec 30 msec

12 * * *

So it stops one hop before destination. That means that actual traceroute packet is sent to final hop (the TTL field just increased to reach it). But final hop not answering back. The fact that ICMP traceroute works says that final router knows the route back - it is not a problem. So it blocking the previous trace by protocol/port. So you can check the ACL on the interface with that ip (or any other incoming int) on the router of final destination to see any blocking string.

Nik

HTH,
Niko

CSCO12302959 · ‎07-13-2018

Even if ICPM can be inspected and you can ping to the internet but when you do a trace to the same IP as you ping the firewall will block the returning traffic, I had the same problem until I allow icmp from any to the internal IPs as traffic hit the outside interface then everything worked.