Would someone be so kind and help me understand the definition of these load balancing options for Etherchannel?
Use cases or examples will extremely help.
What would be best in testing an Etherchannel scenario? I setup a pair for Etherchannel (Gig ports so 2Gbps) and ran iPerf however I was not getting the bandwidth results I expect. Average would be 800Mbps.
For a specific flow only one link in the etherchannel bundle will be used to send the packets.
How that flow is specified is based on the load balancing methods eg. if you balanced based on source and destination IP then a flow between the same two devices would use the same link and if the bandwidth exceeded that link's capacity it won't then use another of the links which is often why people are surprised they don't get the throughput they think they should.
You need to work out what gives you the most variation ie. what load balancing method would spread the traffic evenly between the links.
But like I say for any given flow it will always use just one of the links in the bundle.
They define what frame/packet attribute(s) go into the hash calculation.
dst = destination, src=source, mac = L2 address, ip = L3 address
The purpose of a hash algorithm is to change the attribute(s) into a link member number, ideally, given different frame/packet attributes, they will generate pseudo-random link numbers, so that the frames/packets are about equally distributed across the number of member links.
For example, on the same VLAN, given a server connected to one switch with a dual ether-channel link to another switch which has many hosts, and those hosts communicate with that server. If the first switch has src-mac (or src-ip), it will always use just one link sending server traffic to the second switch because the src-mac (or src-ip) doesn't change. However, for the converse, hosts sending data to the server, each will have a different MAC (or IP), so still using src-mac (or src-ip) traffic should be distributed across the two links. (BTW, although the hashing attribute is different from each client host, a "poor" hashing algorithm can negate the difference. For example, consider using one bit of an IP address to chose between just two member links. If you use the first bit of an IP, likely it's the same for all hosts in the same network but if you use the last bit of the IP, it's likely a 50/50 split across all the IPs.)
As Etherchannel doesn't consider actual member link load, consider two server replicating (a lot of) traffic across our dual Etherchannel. As their MACs and IPs would remain constant, all that data will always use one link. (Your iPerf test, is also an example, you'll not see, using any of the attribute choices listed, more than one link's capacity being used.)
On some of the high end chassis switches, they sometimes offer additional hash attribute choices, such as including UDP/TCP port numbers. This for cases of many flows between the same two hosts.
Generally, you should chose an attribute, or attributes, that are most likely to differ between flows. Often a good choice is src-dst-ip.
BTW, many Cisco hashing algorithms have a binary load balancing bias. I.e. a good choice will provide an even distribution across 2, 4 or 8 links, but not an even distribution across 3, 5, 6 or 7 links. Later Cisco hashing algorithms distribute across non-binary number of links much better.