CEF per-packet load balancing with GRE tunnel

[Edwin Summers]

I would like to confirm my understanding of CEF per-packet load balancing.

Assume that two GRE tunnel end-points are separated by four equal-cost, parallel paths.  If I understand the CEF load balancing mechanism correctly, setting the load sharing method to "per-packet" will prompt CEF to distribute packets across the four links in a round-robin fashion.  I understand this presents the possible problem of having packets arrive out of order.

Slightly off-subject, my understanding of the difference in "per destination" for CEF vs fast switching is that fast switching only considers the destination, but CEF considers a src-dst pair.

In summary, I wanted to confirm my understanding that packets from GRE tunnel end points could be load shared over multiple links by using CEF per-packet load sharing.

[Joseph W. Doherty · Accepted solution]

Disclaimer

The    Author of this posting offers the information contained within this    posting without consideration and with the reader's understanding that    there's no implied or expressed suitability or fitness for any  purpose.   Information provided is for informational purposes only and  should not   be construed as rendering professional advice of any kind.  Usage of  this  posting's information is solely at reader's own risk.

Liability Disclaimer

In    no event shall Author be liable for any damages whatsoever  (including,   without limitation, damages for loss of use, data or  profit) arising  out  of the use or inability to use the posting's  information even if  Author  has been advised of the possibility of such  damage.

Posting

CEF per-packet should round-robin across the GRE tunnels, but multiple tunnels might transit the same link as the src/dest for the tunnel itself doesn't change.

[Edwin Summers]

Joseph - Thanks for your response.  I need to clarify my original post a bit.

I have attached a (very) crude drawing that hopefully clarifies the topology.  Not looking at multiple tunnels or multiple entry points, but rather multiple paths that tunneled traffic could take between tunnel endpoints.  Just for argument, take the drawing and assume endpoint 1 is the ingress for traffic, and endpoint 2 will be the egress.  As long as the internal stuff (widgets, cloud) represent equal cost paths, I assume per-packet balancing will round-robin.

Just found out that this is now entirely academic, as the original drawing I was working from was incomplete.  Would still like to confirm just for future reference.

[Joseph W. Doherty]

Disclaimer

The Author of this posting offers the information contained within this posting without consideration and with the reader's understanding that there's no implied or expressed suitability or fitness for any purpose. Information provided is for informational purposes only and should not be construed as rendering professional advice of any kind. Usage of this posting's information is solely at reader's own risk.

Liability Disclaimer

In no event shall Author be liable for any damages whatsoever (including, without limitation, damages for loss of use, data or profit) arising out of the use or inability to use the posting's information even if Author has been advised of the possibility of such damage.

Posting

Perhaps a point of confusion is, when you add GRE tunnels, then the question is where the equal costing is "seen" so that CEF will RR the packets.  It could be at the tunnel interface or it could be at the physical interface; which will determine how the packets will actually be RR'ed by CEF packet-by-packet.

[Edwin Summers]

Appreciate your help, Joseph.  I'm going to mark your first post as the correct answer as I believe it satisfies my general question regarding CEF per-packet load balancing.  Once we get the equipment in-house and lab a few scenarios I'll pop back in if anything unusual arises.

[itai-f]

Hi,

did you had the chance to check it ?

i need the same solution but i want to be sure that if packets get equal load balanced over 4 physical links with single gre tunnel (per packet load sharing) and if packets get to the other side out of order, does the remote gre tunnel will make the "in order delivery ?

thanks,

[Edwin Summers]

itai,

Sorry for the delayed reply, and for the thread resurrection.  I never got a chance to verify this.  However, unless someone corrects me, I would hazard to say that any packet re-sequencing would be handled by the end-station only, not a tunnel endpoint in between.  I don't think it's going to look "up the stack" at the sequence numbers in L4, just treat the packets "fire and forget".

[Edwin Summers]

Itai,

Just to close the loop, I did not lab this.  We were actually able to push a redesign to avoid having to do this.

During the course of the redesign I spoke with a Cisco engineer who actually noted that newer hardware (with some possible exceptions...Nexus?) has removed the feature for per-packet load balancing via CEF.  I believe this was because they have improved the granularity of the per-flow balancing so that it can better load balance.  Of course, that wouldn't help if you're talking a single large flow.

Hope that was helpful, though it didn't really confirm the function you needed.  Wanted to be sure this was closed out for history's sake.