cancel
Showing results for 
Search instead for 
Did you mean: 
cancel
39729
Views
15
Helpful
7
Replies

Enable Jumbo Frame Support On UCS

Russell Teague
Level 1
Level 1

Could someone direct me to the documentation for the proper way to enable jumbo frame support on UCS?  I'm trying to troubleshoot a problem and have configured MTU of 9000 from ESX running on a blade to an NFS server off a Nexus 5k.  I have good documentation for configuring jumbo frame support on the Nexus 5k and on ESX but nothing really specific for UCS.

For Nexus 5k:

http://www.cisco.com/en/US/docs/switches/datacenter/nexus5000/sw/configuration/nxos/Cisco_Nexus_5000_Series_NX-OS_Software_Configuration_Guide_chapter33.html#con_1150612

show run:

policy-map type network-qos jumbo

  class type network-qos class-default

    mtu 9216

system qos

  service-policy type network-qos jumbo

For ESXi:

esxcfg-vswitch -m 9000 vSwitch0

esxcfg-vswitch -A IP_Storage vSwitch0
esxcfg-vswitch -v 31 -p IP_Storage vSwitch0

esxcfg-vmknic -a -i 10.254.31.2 -n 255.255.255.192 -m 9000 "IP_Storage"
Testing:
I have been able to ping (with large payload) the NFS host interface from a temporary vlan interface on the N5k.  I cannot do the same to the ESX hosts or between hosts.
Thanks,
-R

1 Accepted Solution

Accepted Solutions

Manish Tandon
Cisco Employee
Cisco Employee

Russell

What network adapter are you using? M81KR or the M71KR's or any other?

The MTU is set on a per CoS basis in UCS. If you do not have any QoS policy defined for the vNIC that is going to the vSwitch, then the traffic is going to Best-Effort Class. All you need to do to enable jumbo frames is to enter the MTU to be 9000 in the MTU pulldown for whatever class the traffic is going to.

Its also given in the user guide -

http://www.cisco.com/en/US/docs/unified_computing/ucs/sw/gui/config/guide/1.3.1/UCSM_GUI_Configuration_Guide_1_3_1_chapter18.html

MTU drop-down list

The maximum transmission unit for the channel. This can be:

An integer between 1500 and 9216. This value corresponds to the maximum packet size.

fc—A predefined packet size of 2240.

normal—A predefined packet size of 1500.

Note

This field is always set to fc for Fibre Channel.

--Manish

View solution in original post

7 Replies 7

Manish Tandon
Cisco Employee
Cisco Employee

Russell

What network adapter are you using? M81KR or the M71KR's or any other?

The MTU is set on a per CoS basis in UCS. If you do not have any QoS policy defined for the vNIC that is going to the vSwitch, then the traffic is going to Best-Effort Class. All you need to do to enable jumbo frames is to enter the MTU to be 9000 in the MTU pulldown for whatever class the traffic is going to.

Its also given in the user guide -

http://www.cisco.com/en/US/docs/unified_computing/ucs/sw/gui/config/guide/1.3.1/UCSM_GUI_Configuration_Guide_1_3_1_chapter18.html

MTU drop-down list

The maximum transmission unit for the channel. This can be:

An integer between 1500 and 9216. This value corresponds to the maximum packet size.

fc—A predefined packet size of 2240.

normal—A predefined packet size of 1500.

Note

This field is always set to fc for Fibre Channel.

--Manish

Manish, thank you, that did the trick.  We currently do not have any QoS implemented so I was thinking that the QoS policies would not have any effect, but as you pointed out, everything falls into the 'Best Effort' class.  I made the change to 9000 for the MTU on Best Effort and it works.  I had already set the MTU for the vNIC to 9000 which I assume is also required.

Thanks again,

-R

Russell

Yes, config at the OS level is also required which you had already covered as per the show commands you had sent.

In Linux for example one would use ifconfig to set interface MTU etc.

Thanks

--Manish

When I mentioned vNIC, I was referring to the vNIC or vNIC Template in UCS.  In my scenario the configuration of the MTU value 'end-to-end' would be:

  • Virtual interface on the NFS storage
  • System wide both Nexus 5020s
  • 'Best Effort' QoS policy on UCS
  • vNIC Templates for the Service Profiles in UCS
  • vSwitch in ESXi (this also sets the MTU of the physical uplink interfaces)
  • vmknic for IP storage in ESXi

Cheers,

-R

Hi Guys,

Just a quick one, great post by the way.
I also need to to change my Best Effort traffic to 9212 but all my traffic does indeed fall into this CoS.
I'm in full production so does anyone know the impact it may have on the FI / vNICs or the current traffic passing through the interfaces.

Regards
Steve.


 

Manish,

I also wanted to add to this post that if you SSH into Fab Interconnect and do a show ethernet 1/x that you may see an MTU size listed here of 1500, even though you may have, say, NFS traffic going over an appliance port and vNIC's configured for jumbo in UCSM.

Your response:

The show output is expected.

The FI’s and Nexus 5000 do CoS based MTU and not interface based.

So if you set the MTU in the System class to 9000, any packet recd on any port on the FI with that CoS value will pass that 9000 byte packet.

Thought would be good info for the blog....

good to know - i guess i'll stop refreshing that sh int output waiting for it to change to 9000 then!

Getting Started

Find answers to your questions by entering keywords or phrases in the Search bar above. New here? Use these resources to familiarize yourself with the community: