Etherchannel hashing-loadbalancing persistence

[Amirmahdi.M]

Hi everyone

lets assume we have 2 cisco 3750x switches with 4 Gig interfaces to each other that are channled into Po10.

3750X

||||

||||

3750X

for this scenario the etherchannel uses either mode ON or LACP and loadbalances based on src-dst-ip.

using the command "test etherchannel load-balance interface port-channel 10 ip 192.168.1.1 172.16.1.*"  i get the below result:

192.168.1.1 172.16.1.4 Gi0/1 of Po10

192.168.1.1 172.16.1.5 Gi0/2 of Po10

192.168.1.1 172.16.1.6 Gi0/3 of Po10

192.168.1.1 172.16.1.7 Gi0/4 of Po10

now here is my question.

if anything happens to the interfaces or the portchannel including links going down and coming back or the switch reloads, does the outcome of the "test etherchannel" change? i mean will always the data for 192.168.1.1 172.16.1.7 flow through Gi0/4 of Po10?

or will it change?

Thanks

Amir

[MHM Cisco World]

if anything happens to the interfaces or the portchannel including links going down and coming back or the switch reloads, does the outcome of the "test etherchannel" change? i mean will always the data for 192.168.1.1 172.16.1.7 flow through Gi0/4 of Po10?

Not always' if g0/4 is down then the load balance recalculate hash and select new port.

Also cisco recommend to use port number power of 2 in PO' i.e. use port number 2'4'6'8 this make SW perfectly calculated hash.

MHM

[Amirmahdi.M]

Hi @MHM Cisco World 

Thanks for your quick reply.

the reason i asked this question is because im selecting 4 IPs which each one of them go through a diffrent Gig interface.

i want to know if the switch reloads and everything goes back to normal (all interfaces UP), is it possible that the outcome changes and 2 IPs flow toward 192.168.1.1 go through one Gig interface?

i want to know if the balancing(hashing) is persistent if for example i shut and no shut on of the Gig interfaces or i reload the switch.

Thanks

Amir

[MHM Cisco World · Accepted solution]

Shut port <- make SW recalculate hash for remaining ports

No shut port <- the SW recalculate hash again and it return to first point before shut the port.

So add remove port member into PO effect calculate always.

And no need reload this calc is done without reload the SW

MHM

[Ramblin Tech · Accepted solution]

By my quick calculation, the probability of 4 flows randomly hashing evenly over 4 links is 9.4%. Meaning: you got lucky and should not bet on that luck holding in the future.

Hashing algorithms for LAG and ECMP are vendor-proprietary and often share circuitry when implemented in forwarding engines/ASICs/NPUs. That circuitry takes user-configurable criteria such as packet header fields (and possibly some static entropy value to mitigate against polarization) to calculate an index into the list of links in the LAG bundle (or ECMP links to the destination).

Unless you are privy to the implementation details of the Cat3750x, you should not count on the hashing being persistent across reloads or new software releases, as you will not know all the factors that are used in calculating hash bucket values and mapping hash buckets to links. Those factors might include some randomness injected in the form of entropy that might change with a reload, or even the order in which links are added to the bundle (which might vary between reloads as all the links might not always come “up/up” in the same order with every reload). 

If you absolutely need each flow to be on a separate link on sunny days, then I would suggest looking at some traffic routing/steering/engineering method that is more deterministic rather than probabilistic (LAG hashing). If perfect hashing is not a sunny day requirement, just let nature (probability) take its course and consider it serendipitous when it does happen. 

[Joseph W. Doherty]

Jim brings up an interesting point about whether flow mapping to Etherchannel ports is always deterministic (assuming all parameters are the same).  As he correctly notes it may not be, although probably is on the lower end devices.

Jim mentions polarization as being a factor why it may not be, also true, but I wanted to add, in theory, we're usually trying to achieve a random like distribution.  As Jim further correctly notes, if you truly want specific flows to use specific paths, there are methods to achieve that, not, though, using Etherchannel.

As an aside, as Jim also again correctly noted even an ideal random like flow distribution generally isn't the optimal way to take advantage of multi path bandwidth.

[liviu.gheorghe · Accepted solution]

Traffic for a certain src-dest-ip will go through the same member interface. If that interface goes down, the traffic will be redistributed to the other members in the port-channel. Like @MHM Cisco World mentioned, load balancing will not be that efficient because the number of links are not a power of 2, but it will work.

Once the link is back up, the traffic for that specific src-dest-ip will return to the original member interface.

[Joseph W. Doherty]

As the others have ready replied, flows to ports is redetermined if number of active links change.

As also mentioned, the best hash distribution is with a power of two number of active links but some devices support a wider hash calculation and better distribute when there's not a power of two number of links.