iSCSI and Nexus1000v

Lukas Mazur · ‎01-27-2010

hi all,

i have a standard ESX - iSCSI Setup with 2 dedicated NICs for iSCSI connected to 2 uplink Switches (catalysts). on the Nexus side i'm using vPC-HM.

my questions are:

* what is the best Channel load-balancing method for iSCSI (there are 2 iSCSI targets (2 storages - each with one IP)?

because all requests are sent from the VM-kernel interface specified for iSCSI the default method is not the best one.

* when setting the system jumbomtu 9000 is it also necessary to set the MTU on the eth interfaces (with the mtu command)?

(fyi: in the documentation it's told that you can set the jumbomtu size between 1500 and 9216. The max jumbomtu size you can set is 9000)

thx, Lukas

Robert Burns · ‎01-27-2010

Lukas,

Yes, you should configure the MTU on the Ethernet Interfaces as well as setting the system MTU.

n1000v(config)# system jumbomtu 9000
n1000v(config)# interface ethernet x/x
n1000v(config-if)# switchport
n1000v(config-if)# mtu 9000

This info can be reference in 1000v Interface Config Guide ->

http://www.cisco.com/en/US/partner/docs/switches/datacenter/nexus1000/sw/4_0_4_s_v_1_2/interface/configuration/guide/n1000v_if.html

You'll also need to modify the VMKernel port you're using for iSCSI if you haven't already. This will cause an interuption as the port needs to be re-created with the new MTU.

Type the following command (that’s a lowercase L) to show the current virtual switching configuration:
esxcfg-vswitch -l
At the bottom of the listing you will see the dvPort IDs listed. Make a note of the dvPort ID for your iSCSI VMkernel port
Delete the VMKernel Port for your iSCSI interface.
esxcfg-vmknic -d -s <dvSwitch Name> -v <dvPort ID>
Recreate the VMkernel port and attach it to the very same dvPort ID:
esxcfg-vmknic -a -i <IP addr> -n <Mask> -m 9000 -s <dvSwitch Name> -v <dvPort ID>
Verify the new VMkernel port has been created and attached to the same dvPort ID as the original VMkernel port.

esxcfg-vswitch -l

5. Verify your new MTU on the iSCSI VMkernel Port

esxcfg-vmknic -l

As for the best load balancing hash, the default is usually the adequate. Are your two iSCSI targets on the same subnet or different? You need to consider that the Port channel Load Balancing hasing is global so ensure you're setting one thats best for your entire system not just iSCSI. Normally the default Source-mac is adequate. The only other hashing I would consider would be source-dest-IP hashing. There are caveates if you change this so be aware Scott Lowe has a good write up here with more info: http://blog.scottlowe.org/2006/12/04/esx-server-nic-teaming-and-vlan-trunking/

If I find any further info I'll update this post.

Regards,

Robert