SG350XMP jumbo frames not working

[rand__]

Hi,

Firmware Version (Active Image):

2.5.0.78

I have a SG350X-24MP on which I want to use Jumbo Frames on a 10G interface (XG2) to be precise.

I have globally enabled Jumbo Frames and rebooted the switch.

I test this connection between appropriately configured esxi boxes, one has an Intel X540-T2 NIC, the other a Mellanox CX3 and is connected to a Mellanox SX6036.

Tests:

Intel NIC connected via SG350 XG2 -

vmkping -I vmk0 -d -c 200 -s 1500 10.10.10.132
PING 10.10.10.132 (10.10.10.132): 1500 data bytes

--- 10.10.10.132 ping statistics ---
9 packets transmitted, 0 packets received, 100% packet loss

vmkping -I vmk0 -d -c 200 -s 1200 10.10.10.132
PING 10.10.10.132 (10.10.10.132): 1200 data bytes
1208 bytes from 10.10.10.132: icmp_seq=0 ttl=64 time=0.247 ms
1208 bytes from 10.10.10.132: icmp_seq=1 ttl=64 time=0.297 ms

--- 10.10.10.132 ping statistics ---
2 packets transmitted, 2 packets received, 0% packet loss
round-trip min/avg/max = 0.247/0.272/0.297 ms

Same NIC (second port) connected to the Mellanox SX6036 via QSFP and SFP+ to Ethernet adapter

 vmkping -I vmk0 -d -c 200 -s 8973 10.10.10.132
PING 10.10.10.132 (10.10.10.132): 8973 data bytes
sendto() failed (Message too long)
sendto() failed (Message too long)

--- 10.10.10.132 ping statistics ---
2 packets transmitted, 0 packets received, 100% packet loss

[root@esx37:/vmfs/volumes/5cad032f-82bcdfd8-55a2-24be05b56101/log] vmkping -I vmk0 -d -c 200 -s 8972 10.10.10.132
PING 10.10.10.132 (10.10.10.132): 8972 data bytes
8980 bytes from 10.10.10.132: icmp_seq=0 ttl=64 time=0.441 ms
8980 bytes from 10.10.10.132: icmp_seq=1 ttl=64 time=0.450 ms
...
--- 10.10.10.132 ping statistics ---
96 packets transmitted, 96 packets received, 0% packet loss
round-trip min/avg/max = 0.337/0.463/0.554 ms

I also tested a Mellanox ConnectX-3 attached to the SX6036 which of course works fine, and also I connected the CX3 via the same adapter (reverse way) to the SG350x - not working either.

It clearly is the SG350 which is not working as expected. At first I thought it might be vlan setup, but the fact that is possible to ping the remote box with a package size of <1500 shows that that is ok.

Any idea what this could be the cause?

Thanks

[Georg Pauwen]

Hello,

there is not really anything else on the SG350X that you can configure regarding jumbo frames other than enabling the feature globally.

Your firmware release is a bit outdated, the only thing I could think of is to load the latest release, 2.5.0.82.

[rand__]

I know there is nothing much to do wrong, thats why its so baffling - that should come up in UAT pretty quickly.

Will try upgrading to the latest fw and let you know if it helped.

[rand__]

No change :(

[Georg Pauwen]

Hello,

which ESXI version are you running ?

[rand__]

Hi,

VMware ESXi, 6.7.0, 13473784

Thanks

[Georg Pauwen]

Hello,

on a side note, taking into consideration the below:

9000 (MTU size) – 8 (ICMP header) – 20 (IP header) = 8972 bytes for ICMP payload

Sending packet with a size of 8973 would not go through...8972 is the largest you can send...

[rand__]

So still not resolved ... any other ideas?