Strange traceroute output

[Marius Gunnerud]

I am getting the following output when I run a traceroute to a specific IP, all other IPs are ok. 10.255.3.252 is the WAN interface on the router.  I am getting the same result when I run a trace on the Standby router.  Any ideas what might be causing this?

EDGE-ROUTER-ACTIVE#traceroute 10.251.165.198

Type escape sequence to abort.
Tracing the route to 10.251.165.198

  1 10.255.3.252 2 msec 2 msec 2 msec
  2 10.255.3.252 6 msec 4 msec 2 msec
  3 10.255.3.252 4 msec 6 msec 4 msec
  4 10.255.3.252 4 msec 6 msec 6 msec
  5 10.255.3.252 8 msec 6 msec 6 msec
  6 10.255.3.252 8 msec 8 msec 6 msec
  7 10.255.3.252 10 msec 10 msec 10 msec
  8 10.255.3.252 12 msec 12 msec 12 msec
  9 10.255.3.252 12 msec 14 msec 8 msec
 10 10.255.3.252 14 msec 14 msec 22 msec
 11 10.255.3.252 24 msec 20 msec 20 msec
 12 10.255.3.252 12 msec 16 msec 8 msec
 13 10.255.3.252 14 msec 16 msec 16 msec
 14 10.255.3.252 20 msec 24 msec 10 msec
 15 10.255.3.252 24 msec 22 msec 18 msec
 16 10.255.3.252 26 msec 22 msec 16 msec
 17 10.255.3.252 32 msec 16 msec 20 msec
 18 10.255.3.252 20 msec 18 msec 20 msec
 19 10.255.3.252 22 msec 24 msec 20 msec
 20 10.255.3.252 24 msec 24 msec 32 msec
 21 10.255.3.252 34 msec 22 msec 36 msec
 22 10.255.3.252 20 msec 30 msec 28 msec
 23 10.255.3.252 26 msec 22 msec 32 msec
 24 10.255.3.252 26 msec 34 msec 26 msec
 25 10.255.3.252 30 msec 30 msec 26 msec
 26 10.255.3.252 32 msec 30 msec 34 msec
 27 10.255.3.252 32 msec 34 msec 32 msec
 28 10.255.3.252 34 msec 40 msec 36 msec
 29 10.255.3.252 40 msec 40 msec 36 msec
 30 10.255.3.252 48 msec 36 msec 36 msec

[Peter Paluch]

Marius,

What kind of link layer technology is the WAN interface? Is it an Ethernet, or a PPP serial link, or a PPPoE client (DSL et al.), or some other?

Then, is there any NAT configured either on the router from which you are performing the traceroute from, or on the 10.255.3.252? If so, we need to know the specifics.

You are saying: "10.255.3.252 is the WAN interface on the router". Do you mean the same router where you're running the traceroute from? That would be a very strange sign as a router never reports itself as a hop in traceroute.

Best regards,
Peter

[Marius Gunnerud]

The WAN interface is Ethernet.  There is no NAT configured on the edge routers.  The results I posted above are from the edge router.  If I traceroute from the core-switch I see the traffic reach the edge router and then a repeat of the results above.  Again, 10.255.3.252 is the IP of the outside interface of the Active edge router.

EDGE-ROUTER-ACTIVE#show ip int bri
Interface                  IP-Address      OK? Method Status                Protocol
GigabitEthernet0/0         10.255.3.252    YES NVRAM  up                    up      
GigabitEthernet0/1         10.254.3.2      YES NVRAM  up                    up 

CORES-SWITCH#traceroute 10.251.165.198

Type escape sequence to abort.
Tracing the route to 10.251.165.198

  1 10.254.3.2 0 msec 0 msec 0 msec
  2 10.255.3.252 9 msec 0 msec 0 msec
  3 10.255.3.252 0 msec 0 msec 8 msec
  4 10.255.3.252 9 msec 8 msec 9 msec
  5 10.255.3.252 8 msec 8 msec 8 msec
  6 10.255.3.252 9 msec 17 msec 8 msec
  7 10.255.3.252 17 msec 17 msec 9 msec
  8 10.255.3.252 8 msec 17 msec 16 msec
  9 10.255.3.252 17 msec 8 msec 17 msec
 10 10.255.3.252 34 msec 17 msec 17 msec
 11 10.255.3.252 17 msec 17 msec 25 msec
 12 10.255.3.252 17 msec 26 msec 16 msec
 13 10.255.3.252 17 msec 17 msec 25 msec
 14 10.255.3.252 34 msec 26 msec 16 msec
 15 10.255.3.252 25 msec 25 msec 26 msec
 16 10.255.3.252 33 msec 25 msec 33 msec
 17 10.255.3.252 25 msec 25 msec 34 msec
 18 10.255.3.252 33 msec 34 msec 25 msec
 19 10.255.3.252 25 msec 34 msec 34 msec
 20 10.255.3.252 25 msec 34 msec 25 msec
 21 10.255.3.252 67 msec 34 msec 25 msec
 22 10.255.3.252 34 msec 51 msec 42 msec
 23 10.255.3.252 42 msec 50 msec 51 msec
 24 10.255.3.252 50 msec 51 msec 50 msec
 25 10.255.3.252 50 msec 59 msec 50 msec
 26 10.255.3.252 50 msec 42 msec 33 msec
 27 10.255.3.252 42 msec 42 msec 33 msec
 28 10.255.3.252 34 msec 33 msec 34 msec
 29 10.255.3.252 42 msec 42 msec 34 msec
 30 10.255.3.252 25 msec 50 msec 50 msec

[Marius Gunnerud]

The odd thing is that I am experiencing this only with this one IP / subnet that I am trying to ping / traceroute to.

[Charles Hill]

Hello Marius,

What is the IOS and router type?

[Marius Gunnerud]

the Active router is an ISR 3925 running 15.1(1)T1

The Standby router is an ISR 3825 running 12.4(15)T10

I am starting to beleive that the ISP is not routing the subnet/s in question so I am now waiting for an update from them.

[Peter Paluch]

Hi Marius,

Can you please post the following outputs from the EDGE-ROUTER-ACTIVE?

show ip route 10.251.165.198
show ip cef 10.251.165.198 detail
show ip route 10.255.3.252
show ip cef 10.255.3.252 detail

Thank you!

Best regards,
Peter

[Marius Gunnerud]

I have asked the ISP to check if they are routing the subnets.  I wonder if the results are because that they are not routing the subnets.  Will let you know when I know more

[Peter Paluch]

Marius,

If the ISP was not routing the subnets then I suppose we would be seeing packets getting lost entirely, or a routing loop would form between two or more routers. The output showing the same IP all over again at each hop does not correspond to a lack of routing information.

Regarding the command outputs I have asked for, will you be able to pull them from the active edge router please?

Best regards,
Peter

[Krishnendu AR]

Hello Marius,

What device is 10.255.3.252. Is it a firewall?

Krishna

[Marius Gunnerud]

That is the edge router as I have mentioned i previous posts.

I have asked the ISP to check if they are advertising the routes into the WAN and if not, to fix it.

[Krishnendu AR]

Are you able to ping the IP 10.251.165.198. If you are able to ping the IP, I don't think its ISP related issue.

What I think is the edge router 10.255.3.252 is doing some kind of NAT so that all IPs are being translated to 10.255.3.252.

Can you please post the output of 'show ip nat translations' and 'sh run | i nat' from the edge router.

Krishna

[Bilal Nawaz]

Krishna, please accept some humble suggestion to read the thread before posting. The questions being asked that have been "asked already" could potentially get irritating to the original poster.

Thank you

Bilal

[Marius Gunnerud]

The issue was that the ISP was not routing the subnet.  This is sorted now.  Thanks for the replies.