11-13-2010 12:16 PM - edited 03-06-2019 02:02 PM
A customer currently had two data centers in different locations, connected via dual 50-Mb carrier circuits in a port channel between single switches on each end. (Thus, there is no current routing between sites, and both sites create a single Layer 2 data center fabric.)
DC-A (172.16.10.x/24) --> SwitchA ======== dual port-channeled carrier links ======== SwitchB <--- DC-B (172.16.10.x/24)
The problem is that true load balancing isn't being achieved, so one of the circuits is getting crushed while the other lies idle. We need to implement a solution so that both links can be used closer to full capacity, and the customer can get closer to the full 100-Mbps of available combined circuit bandwidth. Additionally, we need to preserve the ability for hosts on each end to reach the other using a common Layer 2 domain. I need recommendations on how this could be accomplished, and I'll throw out some possible considerations:
1) somehow modify the port channel load-balancing algorithms on the current switches, so that some traffic prefers one link over the other. One problem here is that the majority of the traffic congesting the first link is between two SAN systems (so, one source IP/MAC and one destination IP/MAC).
2) implement a single (or multiple?) routers on each end, and use routing protocols to handle the load balancing. This breaks the L2 data center, though, unless some form of tunneling can be performed to unify the two sites.
The customer is open to purchasing new gear if that provides the optimal (or only) solution.
Thank you!
11-13-2010 11:19 PM
As you have already figured out one particular flow which is being most utilized and is causing 1 link to be more utilized than the other, based on Source IP/MAC and Dest IP/MAC.
Well the etherchannel algorithm works on hash algorithm, which uses information in the packet/frame header to choose the link. So, if the hash is same the traffic would always traverse over the same link, we have no control over it.
We need to find out ways, we could use information in the packet header which changes within the same flow, to create different flow. For example, if L4 port number varies between the identified flow, then we change the load-balancing to use that information to have different flows.
On my device with SXI IOS version, I could see these options:
Switch(config)#port-channel load-balance ?
dst-ip Dst IP Addr
dst-mac Dst Mac Addr
dst-mixed-ip-port Dst IP Addr and TCP/UDP Port
dst-port Dst TCP/UDP Port
mpls Load Balancing for MPLS packets
src-dst-ip Src XOR Dst IP Addr
src-dst-mac Src XOR Dst Mac Addr
src-dst-mixed-ip-port Src XOR Dst IP Addr and TCP/UDP Port
src-dst-port Src XOR Dst TCP/UDP Port
src-ip Src IP Addr
src-mac Src Mac Addr
src-mixed-ip-port Src IP Addr and TCP/UDP Port
src-port Src TCP/UDP Port
So, please consult the configuration guide for the switch in use, and find out which information varies often to utilize the above options to load-balance the traffic.
HTH.
Regards,
Deepak
11-29-2010 08:27 AM
We're looking to do the exact same thing you're talking about. We currently have 2 100 Mbps links in an port channel, but aren't able to effectively load balance with the methods available in the 3750's we're using on both ends.
Have you made any headway or found anything that will do what you need?
11-29-2010 12:25 PM
Hello,
Is one of the two links of the etherchannel 100% idle? That seems unusual to me...I would check to confirm that etherchannel is functioning properly if one of the links was 100% idle.
If you've already confirmed this, I think what Deepak suggested is the way to go. Etherchannel does a math function (XOR) on parts of the source and destination MAC addresses, by default, to decide which port in the channel to use to send a given frame. Given this, traffic between two hosts will always use the same port in the channel.
I'd use a sniffer/protocol analyzer at one of the two data centers to see what the traffic flow between hosts across the link is like. By looking at L2, L3 and L4 traffic summaries you might be able to deduce a better criteria for load balancing (again, per Deepak's post).
Hope this helps!
Jeff
Discover and save your favorite ideas. Come back to expert answers, step-by-step guides, recent topics, and more.
New here? Get started with these tips. How to use Community New member guide