cancel
Showing results for 
Search instead for 
Did you mean: 
cancel
4158
Views
10
Helpful
3
Replies

vSwitch load balancing with UCS

dreambull
Level 1
Level 1

Hi there,

Can someone tells me is it recommended to use the vSwitch LB option "route based on IP hash" insted of originating port id in a UCS environment.

I have 2 NIC's active active to the vSwitch ( FabA and FabB) and it was set to origination port ID. everything works fine until I have added another vNIC to a VM which is not able to ping to it peer host on another ESXi.

If I vMotion that VM to a different ESXi and migrate back it start ping. If I chnage the LB to IP hash it works well.

Both VM's are in same vlan and its configured on fabric and on port groups.

Anyone tried IP hash with UCS? The link grouping preference is Port channel in UCS

Regards

1 Accepted Solution

Accepted Solutions

Robert Burns
Cisco Employee
Cisco Employee

With the vSwitch you can only use the default LB hashing with UCS (route based on virtual port ID).  Using IP hashing is used only when you have both pnics (vmnics) port channeled, which you don't.  UCS does not port channel the A & B fabric uplinks together, but rather the port channeling only occurs between each IOM and FI respectfully assuming you have 6200 series FI and 2200 series IOMs. 

What's likely happening is you're two VMs having a problem communicating are getting pinned to different fabrics. 

Ex.  If you have two VMs on two different hosts, and the vSwitch hash happens to send both their traffic via the Fabric-A uplink - then traffic will hit the same FI, switch locally and come back down the same path.   If the two VMs get hashed up different uplinks (one to FI-A, and the other to FI-B), then in order to reach each other (even within the same VLAN), they need to go the northbound LAN, switched in the upstream LAN and back down the other side.  This is because there is no interconnection of dataplane traffic between the FI's, only control traffic is clustered.

When you Vmotion your VM back and forth the vSwitch hashing can change each time, so sometimes both VMs might end up hashing to the same fabric, and othertimes not.  This would also explain the strange behavior when you have the LB hashing set to IP hash.  Each flow has a 50/50 chance of working. 

When the two VMs can't communicate, go to your upstream LAN switch and do a "show mac address vlan x" and see if you find both VM's MAC Addresses visible.  If not you've likely either not created that VLAN on the upstream switches, or you haven't allowed it over one of the trunks in the path.  Ensure the LB is set back to the default Virtual Port ID hash also.

Regards,

Robert

View solution in original post

3 Replies 3

Robert Burns
Cisco Employee
Cisco Employee

With the vSwitch you can only use the default LB hashing with UCS (route based on virtual port ID).  Using IP hashing is used only when you have both pnics (vmnics) port channeled, which you don't.  UCS does not port channel the A & B fabric uplinks together, but rather the port channeling only occurs between each IOM and FI respectfully assuming you have 6200 series FI and 2200 series IOMs. 

What's likely happening is you're two VMs having a problem communicating are getting pinned to different fabrics. 

Ex.  If you have two VMs on two different hosts, and the vSwitch hash happens to send both their traffic via the Fabric-A uplink - then traffic will hit the same FI, switch locally and come back down the same path.   If the two VMs get hashed up different uplinks (one to FI-A, and the other to FI-B), then in order to reach each other (even within the same VLAN), they need to go the northbound LAN, switched in the upstream LAN and back down the other side.  This is because there is no interconnection of dataplane traffic between the FI's, only control traffic is clustered.

When you Vmotion your VM back and forth the vSwitch hashing can change each time, so sometimes both VMs might end up hashing to the same fabric, and othertimes not.  This would also explain the strange behavior when you have the LB hashing set to IP hash.  Each flow has a 50/50 chance of working. 

When the two VMs can't communicate, go to your upstream LAN switch and do a "show mac address vlan x" and see if you find both VM's MAC Addresses visible.  If not you've likely either not created that VLAN on the upstream switches, or you haven't allowed it over one of the trunks in the path.  Ensure the LB is set back to the default Virtual Port ID hash also.

Regards,

Robert

Wes Austin
Cisco Employee
Cisco Employee

In case anyone else reads this:

http://www.cisco.com/c/en/us/support/docs/servers-unified-computing/ucs-b-series-blade-servers/200519-UCS-B-series-Teaming-Bonding-Options-wi.html

-Wes

Thanks Wes!  Your note was super helpful to find!!!

 

Best,

Dave Pfau

 

Review Cisco Networking for a $25 gift card