Mac-Pinning load balancing and load sharing

[v.ivanov]

I am trying to create a situation in our lab when multiple VMs on Nexus 1000v switch contending for the bandwidth in port-channel would loadshare.

2 VMs are pinned by Nexus in round-robin method to one of the links in the “mac-pinned” port-channel. That uplink connects to non-stackable upstream switches ( WS-CBS3012-IBM are not VSS or vPC capable either). The second link in the port-channel has idle VMs pinned. 2 active VMs send “packet generated” traffic to different ip destinations. 1 VMs would take up to 900 Mb/s on 1 Gb/s link. When second VM started, it would consume up to 450 Mb/s making the first VM’s conversation to shrink to 450 Mb/s, so both VMs would generate about 900 Mb/s in total and none of the VMs would switch over to idling link. Adding 3^rd VM would push first 2 VMs to shrink further down to 300 Mb/s each and let the 3^rd one to have 300 Mb/s too.  Thus I am back to 900 Mb/s and 90% bandwidth saturation and none of the VMs would be squeezed out to the idling link (it has 200 Kb/s traffic only )

Replacing the default load-balancing “src-mac” algorithm by many others including the “src-dest-ip-port” did not change anything.

Some technical sources would make me think that Nexus’s “mac-pinning” uses VMware vDs API load-based teaming (LBT) to load share. Normally that technology would re-pin one VM to the idling link if the first link is busy at 75% and more for longer than 30 sec?

Could it be because the “Packet generator” traffic is very “flexible”, readily scales down and if it drops to less than 75 % for a millisecond it is never over 75 % busy for 30 sec then.

I am not hopping for load-balancing but would like to see at least some load-sharing.

Or is it just how mac-pinning works and only manual pinning can regulate the load?

Thanks

Vadim

[Robert Burns · Accepted solution]

That's how mac pinning works.  When VMs are pinned to an uplink, they remain there until there's a link change (addition or removal) or until the virtual port is VMotioned or powered down.  The balancing is not dynamic for each flow in this respect.

If you want load balancing then you should be using port channelling.  Though your upstream switches are not clustered, hopefully you can connect multiple uplinks to each switch. Then you can aggregate links together.  Then create manual sub groups on your 1000v side and let the Port Channel hashing best load balance traffic across each set of uplinks (a set of links going to the same upstream switch).

If you're limited to a single uplink per switch, then you're at the mercy of round-robin unless you manually force VMs to certain uplinks using manual sub group pinning.

Robert

[v.ivanov]

Thank you, Robert.

That definitely answers my question.

I knew I saw it working but was not sure about when and how.

You answer takes me back to the configuration with manual sub-grouping.

Thanks again.

Vadim