MPLS & IP CEF Load Balancing

[Daryl Benson]

Hello,

I have a generic question on how MPLS and IP CEF load balancing will play together and making sure I'm not crazy.  I believe this is impacting my network and before I spend hours reading articles I figured I'd just potentially throw this out there for smarter heads than mine.  Let's assume in this hypothetical scenario that we have four routers (A, B, C, & D).  Let's assume they are all connected to each other with four links, sequentially, all with equal bandwidth on each of the four links, ergo ECMP is in play.

For example,

A <-> four links <-> B <-> four links <-> C <-> four links <-> D

The next logic of the hypothetical scenario is that each of the four routers has a loopback (Lo0) which is where the MPLS is sourced, all four routers are running some misc IGP, let's make it OSPF across all the links.  Assuming OSPF and basic MPLS are configured properly, all connectivity is up and running.

Here's my question:  Assuming you have something like a L3 VPN or your doing some L2 VPN solution, let's do something simple... just a standard pseudowire encapsulated in MPLS from router A's Lo0 to the other end, router D's Lo0.  How will CEF handle this load-balancing?  So let's assume you have 30 customers on Router A all pseudowired to their other end of those thirty customers to Router D.  Since the traffic is all sourced and destined to the same IP addresses (A's Lo0 and D's Lo0) will CEF load balance this traffic at all across the four links or will it only send traffic over the same single path because the universal algorithm hashes based on source/destination and the source/destination is always the same for all thirty pseudowire paths?

[Nagendra Kumar Nainar]

Hi,

Since the traffic is all sourced and destined to the same IP addresses (A's Lo0 and D's Lo0) will CEF load balance this traffic at all across the four links or will it only send traffic over the same single path because the universal algorithm hashes based on source/destination and the source/destination is always the same for all thirty pseudowire paths?

<Nagendra> A pseudowire between loopback of A and D will not have source and destination as those loopback address for the actual data traffic flowing over the PW. Any traffic over PW from A to D will have a unique VC label (bottom label) and same Transport label (top label). Transport label is used to deliver the packet to D while VC label is used by D to identify the PW.

Now when a transit node (assume B or C), receives a labeled packet and if it got ECMP egress path, it uses certain key values from the packet (varies depending on payload, HW capability etc) to perform hashing and identify the egress. Normally for L2VPN traffic, a transit node uses the label stack for this purpose.

Now assume, VC label for PW1 from A to D is 1001 and VC label for PW2 is 1002. The transport label to reach D is 2000. Traffic over PW1 from A to D will have {2000, 1001} as label stack while traffic over PW2 from A to D will have {2000, 1002}. So hashing varies and may result in different egress interface.

It could be noted that if PW1 is carrying 100 flows while PW2 carries 100k flows, all 100 flows of PW1 will flow over 1 path and all 100k flows of PW2 will flow over other path. To handle this situation, FAT and Entropy labels are proposed. Support for these FAT and Entropy label are available on some platforms.

HTH,

Nagendra

[Daryl Benson]

HI Nagendra,

But your saying the outer label is 2000 in your example, that outer label of 2000 will be load balanced over ECMP via CEF?

Alright, let me lab something up really quick, again to make sure I'm not crazy...  So in this lab I'll make it simpler.  A -> B -> C with two links between each.  Just because I have this lab I've essentially created for something else that mirrors this pretty closely.

A#show ip route
O 10.255.255.4/32
 [110/21] via 10.10.253.29, 00:09:40, GigabitEthernet2/0
 [110/21] via 10.10.253.25, 00:09:40, GigabitEthernet1/0

A#show xconnect all
UP pri ac Gi3/0.200:200(Eth VLAN) UP mpls 10.255.255.4:1006 UP
UP pri ac Gi3/0.300:300(Eth VLAN) UP mpls 10.255.255.4:1007 UP

A#show mpls l2transport vc

Gi3/0.200 Eth VLAN 200 10.255.255.4 1006 UP
Gi3/0.300 Eth VLAN 300 10.255.255.4 1007 UP

A#show mpls l2transport vc 1006 detail | s label stack
Output interface: Gi1/0, imposed label stack {20 24}

A#show mpls l2transport vc 1007 detail | s label stack
Output interface: Gi2/0, imposed label stack {20 27}

Interesting, so a little labbing would have successfully answered my question.  Even though both PW encapsulated in MPLS have the same outer label (20 in this case), they load balance over different interfaces, one over gig1/0 and the other over gig2/0 based on the last two router outputs.  

A#show ip cef 10.255.255.4
10.255.255.4/32
nexthop 10.10.253.25 GigabitEthernet1/0 label 20
nexthop 10.10.253.29 GigabitEthernet2/0 label 20

A#show mpls forwarding-table labels 20
Local Outgoing Prefix Bytes Label Outgoing Next Hop
Label Label or Tunnel Id Switched interface
20 Pop Label 10.10.253.16/30 0 Gi1/0 10.10.253.25
   Pop Label 10.10.253.16/30 0 Gi2/0 10.10.253.29

CEF just handles it, it looks like.  That's pretty awesome.  

My current situation the 15 or so PW that are running in the active network are all fairly close to the same allocated bandwidth so I just need to more or less verify they are actually distributed over different physical interfaces, which I need to confirm.  But I hadn't consider the actual logic of having 15 PW and them having grossly different bandwidths per PW, that could pose an issue as well.

Thanks for answering, I guess I could have verified it with a little labing.  Thank you, definitely appreciate the time and response.