MPLS VRF Routing

[debottym2]

Folks,

I am having an issue with routing to a certain prefix within a vrf. I cannot seem to route to it from devices 'behind' the PE, however the PE seems to know how to get to it.

Let me illustrate with a few traceroutes. Hoping someone can help. Please!

regards,

Deb

Traceroute to a working prefix within the vrf from a device 'behind' the PE in the vrf

[dmukherjee@cpemon1-roc ~]$ traceroute x.x.x.good

traceroute to x.x.x.good (x.x.x.good), 30 hops max, 60 byte packets

1 gi-0-1.mgmt01.rochny01.   6.544 ms 6.520 ms 6.483 ms

2 gi-9-5.ROCHNY01H07PE02. ( ) 6.683 ms 6.901 ms 6.892 ms

3 ge-2-0-2-13.rochny01h07cr01. ( ) 37.213 ms 37.199 ms 37.957 ms

4 so-1-1-0.albynywbh38cr01. ( ) 40.578 ms 40.586 ms 40.563 ms

5 TE0-1-0-0.ANDVMAACH00CR04. ( ) 45.385 ms 45.375 ms 45.695 ms

6 te0-0-0-0.andvmaach00cr03. ( ) 45.597 ms 37.891 ms 37.873 ms

7 te0-2-0-0.nycmny83iy6ig02. ( ) 39.669 ms 40.320 ms 40.284 ms

8 ge-1-0-0-12.nycmny83w68cr02. ( ) 37.203 ms 36.677 ms 36.652 ms

9 so-4-0-1.nycmnybxh71cr02. ( ) 37.769 ms 35.938 ms 35.890 ms

10   ( ) 77.015 ms 73.050 ms 73.023 ms^^

Traceroute to the non working prefix within the vrf from a device 'behind' the PE in the vrf

[dmukherjee@cpemon1-roc ~]$

[dmukherjee@cpemon1-roc ~]$ traceroute x.x.x.bad

traceroute to x.x.x.bad (x.x.x.bad), 30 hops max, 60 byte packets

1 gi-0-1.mgmt01.rochny01.   0.642 ms 0.586 ms 0.553 ms

2 gi-9-5.ROCHNY01H07PE02. ( ) 0.969 ms 0.938 ms 0.935 ms

3 * * *

4 * * *

5 * * *

6 * * *

7 * * *

8 *^^^C

[dmukherjee@cpemon1-roc ~]$ ^C

------------------------------------------------------------------

However, from the PE I can seem to get to both the prefixes

ROCHNY01H07PE02#traceroute vrf MGMT x.x.x.good

Type escape sequence to abort.

Tracing the route to x.x.x.good

1 ge-2-0-0-12.rochny01h07cr01. (66.155.216.178) [MPLS: Labels 4645/710315 Exp 0] 32 msec

   ge-2-0-2-13.rochny01h07cr01. ( ) [MPLS: Labels 4645/710315 Exp 0] 224 msec

   ge-2-0-0-12.rochny01h07cr01. (66.155.216.178) [MPLS: Labels 4645/710315 Exp 0] 40 msec

2 so-1-1-0.albynywbh38cr01. ( ) [MPLS: Labels 16981/710315 Exp 0] 40 msec 36 msec 40 msec

3 TE0-1-0-0.ANDVMAACH00CR04. ( ) [MPLS: Labels 1048307/710315 Exp 0] 36 msec 36 msec 36 msec

4 te0-0-0-0.andvmaach00cr03. ( ) [MPLS: Labels 17120/710315 Exp 0] 32 msec 36 msec 36 msec

5 te0-2-0-0.nycmny83iy6ig02. ( ) [MPLS: Labels 17125/710315 Exp 0] 36 msec 36 msec 36 msec

6 ge-5-0-0-12.nycmny83w68cr01. () [MPLS: Labels 995/710315 Exp 0] 36 msec 36 msec 36 msec

7 so-4-0-1.nycmnybxh71cr01. () [MPLS: Labels 17198/710315 Exp 0] 36 msec 36 msec 36 msec

8 [MPLS: Labels 40/710315 Exp 0] 36 msec 36 msec 36 msec

9 * *

ROCHNY01H07PE02#

ROCHNY01H07PE02#

ROCHNY01H07PE02#

ROCHNY01H07PE02#traceroute vrf MGMT x.x.x.bad

Type escape sequence to abort.

Tracing the route to x.x.x.bad

1 ge-2-0-2-13.rochny01h07cr01. ( ) [MPLS: Labels 4645/709739 Exp 0] 32 msec

   ge-2-0-0-12.rochny01h07cr01. (66.155.216.178) [MPLS: Labels 4645/709739 Exp 0] 32 msec

   ge-2-0-2-13.rochny01h07cr01. ( ) [MPLS: Labels 4645/709739 Exp 0] 32 msec

2 so-1-1-0.albynywbh38cr01. ( ) [MPLS: Labels 16981/709739 Exp 0] 36 msec 36 msec 36 msec

3 TE0-1-0-0.ANDVMAACH00CR04. ( ) [MPLS: Labels 1048307/709739 Exp 0] 36 msec 36 msec 36 msec

4 te0-0-0-0.andvmaach00cr03. ( ) [MPLS: Labels 17120/709739 Exp 0] 32 msec 36 msec 36 msec

5 te0-2-0-0.nycmny83iy6ig02. ( ) [MPLS: Labels 17125/709739 Exp 0] 36 msec 32 msec 36 msec

6 ge-5-0-0-12.nycmny83w68cr01. () [MPLS: Labels 995/709739 Exp 0] 32 msec 32 msec 32 msec

7 so-4-0-1.nycmnybxh71cr01. () [MPLS: Labels 17198/709739 Exp 0] 32 msec 36 msec 36 msec

8 [MPLS: Labels 40/709739 Exp 0] 32 msec 32 msec 36 msec

9 *

ROCHNY01H07PE02#

[Peter Paluch · Accepted solution]

Hello,

This looks like the router that owns the "bad prefix" does not know the route back to your cpemon1-roc. Recall that in MPLS network, when an ICMP TTL Expired message is generated on a router, it is not this router that sends this message back to you directly. Rather, this message gets an MPLS label stack just like the original packet, and is forwarded along the way towards the original destination - in this case, towards the bad prefix. Only the router at the end of this LSP will deal with the plain ICMP message. If the device you are tracerouting to does not know the path back to cpemon1-roc, it cannot forward the ICMP TTL Expired message back.

The reason it works from the PE is that the PE's interface towards the MPLS network has a prefix that is obviously known to all devices. After all, try running this command on the PE, substituting the S.S.S.S for the IP address of the PE interface in the MGMT VRF going back to cpemon1-roc:

traceroute vrf MGMT x.x.x.bad source S.S.S.S

This test should fail if my theory is correct.

Best regards,

Peter

[Mark Combs]

Peter,

Just a thought in regards to this issue.  Let me know your thoughts

All of the routers in the middle shouldn't have the routes normally.  Even though there is a VPNV4 relationship between all the routers in the "cloud or middle"  this is just used a a transit network in which to forward the MPLS packets.  The only routers with the vrf import/export configurations are the PE routers.  This is why the PE routers, as you stated above, have the routes because its known to this device and proper PHP is going on here.  Nothing gets redsitributed into the specific bgp vpnv4 address family per se meaning that at the PE routers their is mtual redistribution going on between the bgp ipv4 vrf family and the routing protocol that is being used in the vrf for your customer.

If we wanted the routers in the middle to see the MPLS prefixes or have reachability we would have to also have import/export statments on these "middle" routers wouldn't we?  I have actually never tried it so don't know how or if it would work.  Not sure that is a really good idea though.  You dont really want the routers learning all of those prefixes which is the whole point of the mpls tags to begin with.

Just my 2c

-Mark

[milan.kulik]

Hi Peter,

I agree with your theory, but I'm afraid the test you are suggesting might not prove it.

Looking at the original traceroute outputs I understand:

The traceroute command was issued on the cpemon1-roc router while  gi-0-1.mgmt01.rochny01 is a CE router and gi-9-5.ROCHNY01H07PE02 is a PE router following it on the way to the destination subnet.

The MPLS network follows.

As you mentioned, the PE router on the MPLS cloud remote side is crucial for the traceroute to work.

(Details described here:

http://www.cisco.com/en/US/tech/tk436/tk428/technologies_tech_note09186a008020a42a.shtml ).

I'm not sure 100% which router is that, ge-2-0-2-13.rochny01h07cr01 possibly?

But it can still know the route to the S.S.S.S IP address of the PE interface in the MGMT VRF going back to cpemon1-roc. As it's within the subnet between the PE and the gi-0-1.mgmt01.rochny01 router, isn't it?

IMHO, what is necessary to check:

To find the proper PE on the MPLS remote side and check in there is a route to the traceroute originate address (cpemon1-roc outgoing interface) available in the proper VRF on it.

BR,

Milan

[debottym2]

Peter,

I 'think' you are spot on and my test did fail with the traceroute. I am going to investigate the 'router owning the bad prefix' further and report  back on this forum.

Also, thank you for jogging my memory on the MPLS traceroute piece!

Thank you for all your help.

Regards,

Deb

[debottym2]

Peter,

Turns out the far end PE was routing my source IP to the CE interface. (go figure!).

Thanks for your assistance again.

Regards,

Deb