Question About Jumbo Frames On Servers In Same VLAN

dave cooper · ‎10-18-2015

I am under the understanding that if you enable jumbo frames in a VLAN then all ports in that VLAN should have jumbo frames enabled.

We have a server farm VLAN which we have enabled jumbo frames on. All the ports in that VLAN on the 6500 are set for 9216.

Some server NICs have 9216 sent on them but a lot of them are using 1518. What happens when a server sends a 9216 frame to a mschines NIC only has 1518 set on it that is in the same VLAN?

The server guys don't think this is a problem but we are starting to have performance problems with our 6509 and I believe this is a large cause of it.

What will happen in this case?

Thanks

Zach S · ‎10-18-2015

Dave,

MTU mismatch can definitely cause you issues. For jumboframes to work properly both ends of communication have to agree upon the size, or you will run into dropped frames or fragmented packets.

When you say you set all the ports on the VLAN to the same MTU, I believe that might be a misunderstanding of the advice to keep all jumbo frames contained in one VLAN; again for the whole reason of keeping MTU mismatches contained. So when you say there is 9216 and 1518 mixed in one VLAN, that doesn't bode well for communication. The whole point of these frames is to pack more payload into one frame, but if the other can't support it, then it defeats the purpose.

dave cooper · ‎10-19-2015

Thanks for the responses.

When I mention this to the server guy he says because of Path discovery this is not an issue. Both clients will negotiate a MTU. In this case it will go down to 1500 Bytes and will not cause any problems.

Any thoughts?

Joseph W. Doherty · ‎10-20-2015

Disclaimer

The Author of this posting offers the information contained within this posting without consideration and with the reader's understanding that there's no implied or expressed suitability or fitness for any purpose. Information provided is for informational purposes only and should not be construed as rendering professional advice of any kind. Usage of this posting's information is solely at reader's own risk.

Liability Disclaimer

In no event shall Author be liable for any damages whatsoever (including, without limitation, damages for loss of use, data or profit) arising out of the use or inability to use the posting's information even if Author has been advised of the possibility of such damage.

Posting

PMTUD is for IP and L3, not L2. (Basically, the sending host will send IP packets with DF set, and it will look for L3 hops to send back an ICMP message the IP packet needs fragmentation when it's too large for the hop.)

BTW, generally PMTUD needs to be enabled to avoid hosts restricting themselves to sending off-local-net packets limited to 576.

Hosts do not (directly) negotiate MTU however, if your hosts are using TCP, TCP will negotiate MSS which usually will be the set to be sized for the host with the smaller MTU. So, your server guy is correct if using TCP but not all other traffic. Also, even for TCP, suppose both host NICs were configured for jumbo; for the same VLAN, any switch interface not supporting the same jumbo will drop a too large frame.

Joseph W. Doherty · ‎10-19-2015

Disclaimer

The Author of this posting offers the information contained within this posting without consideration and with the reader's understanding that there's no implied or expressed suitability or fitness for any purpose. Information provided is for informational purposes only and should not be construed as rendering professional advice of any kind. Usage of this posting's information is solely at reader's own risk.

Liability Disclaimer

In no event shall Author be liable for any damages whatsoever (including, without limitation, damages for loss of use, data or profit) arising out of the use or inability to use the posting's information even if Author has been advised of the possibility of such damage.

Posting

"I am under the understanding that if you enable jumbo frames in a VLAN then all ports in that VLAN should have jumbo frames enabled."

Yes, that should be done, but technically only L2 ports that will carry the jumbo frames need to be so configured.

"Some server NICs have 9216 sent on them but a lot of them are using 1518. What happens when a server sends a 9216 frame to a mschines NIC only has 1518 set on it that is in the same VLAN?"

Receiver will drop frame.