Traceroute in a MPLS/VPN with back to back PE

[AJ Schroeder]

Hello group,

I have setup a test lab in GNS3 that will (hopefully) make it into the production network. The goal is to eventually get an Enterprise MPLS/VPN running in the core of the production network and beyond, similar to a service provider.

Anyway, in my test lab, I have a setup like this (sorry for the crude drawing):

CE-A-----PE-----PE-----CE-B

I am using IS-IS as the IGP, MP-BGP to establish the VPN, and both CE-facing interfaces are in the same VRF. I have established BGP peering between CE and PE nodes and I see all "customer" routes with a "sho ip bgp vpnv4 all". So far so good.

Where I am running into an issue is with ICMP, namely traceroutes. I cannot traceroute between CE-A and CE-B in either direction, I get as far as the remote PE interface and then the traceroute times out. Debugs of ICMP reveal that I am getting a TTL expired message on the sending CE router so I tried "no mpls ip propogate-ttl" on the PE routers and that didn't make a difference either. I have made sure that CEF is enabled on all routers. In this particular test lab I am using all T1 interfaces as interconnects, but I wouldn't think that MTU would be an issue with ICMP.

I've read that this particular configuration is possible without P routers so  I take that as I have something messed up somewhere. If someone could please point me in the right direction it would be much appreciated. I have attached my configuration files.

Thank you in advance,

AJ Schroeder

[hbruyere · Accepted solution]

Hello,

That's correct, no need of a P router for mpls vpn to work.

Your traceroute problem comes likely from the 'no ip unreachables' configured on the serial0/0 of your CE-B.

Note that IOS traceroute is not sending icmp packets, but UDP packets with port like 33434. And the last hop router

is supposed to send back an icmp unreachable when receiving this udp packet.

Regards,

Herve

[AJ Schroeder]

Herve,

That was exactly the issue! I removed that from all the interfaces in my lab and I am able to traceroute between CE routers without issue. The routers were behaving exactly as configured.

Thank you very much for the help.

AJ Schroeder

[milan.kulik]

Hi Herve,

yes, you are right, that's the case.

What's interesting is the ICMP-based Windows traceroute is working fine.

I found in the CiscoPress Router Security Strategies book:

"IP packet with expired TTL: Per RFC 792, if an IP router receives a packet with a
TTL value of 1 or 0, the packet must be discarded and an ICMP Time Exceeded
(Type 11) message must be generated and sent to the original source."

And the UDP packet sent via Cisco trace is such a packet, isn't it?

Looking to the RFC 792, I see

"If the gateway processing a datagram finds the time to live field

is zero it must discard the datagram.  The gateway may also notify
      the source host via the time exceeded message."

Not a must, unlike the book says.

So the Cisco implementation is probably OK.

But I'd appreciate a possibility to use an ICMP-based traceroute from a Cisco router console in that case!!

Maybe it's worth configuring
ip icmp rate-limit unreachable

instead of

no ip unreachable

in enterprise networks?

BR,

Milan

[milan.kulik]

Hi,

I'm observing the same behaviour in a live network:

When I do traceroute "through" the remote CE router (i.e., with a destination behind it), it replies OK.

But when I do traceoute "to" the router itself (i.e., with the destination = router interface IP), it does not reply.

It's interesting when I do a traceroute from a Windows machine to the CE router, it does reply.

As Windows is using ICMP packets for traceroute while Cisco is using UDP, I believe there might be some bug in IOS related to PE traceroute feature implementation (as the router does reply to a traceroute while no MPLS is involved) possibly?

Does your remote CE reply to Pings?

BR,

Milan

[AJ Schroeder]

Milan,

In my lab just the traceroutes weren't working properly, the remote CE router responded to pings. In our production network I have seen what you are describing as tracerouting "through" a router and having it work OK. I never thought anything about it since I was always trying to get to devices behind the routers and not necessarily the router itself.

I went back and re-read the Cisco documentation on unreachable messages to make sure I have a clear understanding what I am turning on and off. I understand the security reasons for disabling 'ip unreachables' in a production environment but it really caused me some grief in the lab.

Best regards,

AJ Schroeder