why does nexus 93180 interface default not support packet size 1500?

[Herman2022]

Hi, we connect nexus 9k to nexus 7k, use default 1500 MTU at both switch interface, but when I ping with packet size 1500 and df-bit set, then ping failed, only allow to ping 147x. Can someone please advise why it behaves like this? Thanks in advance!

[Pavel Tarakanov]

Because mtu configuration on interface including whole IP packet, and packet size configuration of ping command - ICMP payload only.

When you ping with maximum possible packet size 1472, you get 1514 bytes on wire:

C93180YC-FX-2# ping 192.168.1.1 packet-size 1472 df-bit
PING 192.168.1.1 (192.168.1.1): 1472 data bytes
1480 bytes from 192.168.1.1: icmp_seq=0 ttl=254 time=0.927 ms
1480 bytes from 192.168.1.1: icmp_seq=1 ttl=254 time=0.545 ms
1480 bytes from 192.168.1.1: icmp_seq=2 ttl=254 time=0.51 ms
1480 bytes from 192.168.1.1: icmp_seq=3 ttl=254 time=0.518 ms
1480 bytes from 192.168.1.1: icmp_seq=4 ttl=254 time=0.51 ms

--- 192.168.1.1 ping statistics ---
5 packets transmitted, 5 packets received, 0.00% packet loss
round-trip min/avg/max = 0.51/0.602/0.927 ms
C93180YC-FX-2# ping 192.168.1.1 packet-size 1473 df-bit
PING 192.168.1.1 (192.168.1.1): 1473 data bytes
^C
--- 192.168.1.1 ping statistics ---
1 packets transmitted, 0 packets received, 100.00% packet loss
C93180YC-FX-2#

32 2025-06-03 05:52:25.308728333 192.168.1.2 → 192.168.1.1 ICMP 1514 Echo (ping) request id=0xca6f, seq=0/0, ttl=255
33 2025-06-03 05:52:25.309089819 192.168.1.1 → 192.168.1.2 ICMP 1514 Echo (ping) reply id=0xca6f, seq=0/0, ttl=255 (request in 32)
34 2025-06-03 05:52:25.309555163 192.168.1.2 → 192.168.1.1 ICMP 1514 Echo (ping) request id=0xca6f, seq=256/1, ttl=255
35 2025-06-03 05:52:25.309755468 192.168.1.1 → 192.168.1.2 ICMP 1514 Echo (ping) reply id=0xca6f, seq=256/1, ttl=255 (request in 34)
36 2025-06-03 05:52:25.310107905 192.168.1.2 → 192.168.1.1 ICMP 1514 Echo (ping) request id=0xca6f, seq=512/2, ttl=255
37 2025-06-03 05:52:25.310290426 192.168.1.1 → 192.168.1.2 ICMP 1514 Echo (ping) reply id=0xca6f, seq=512/2, ttl=255 (request in 36)
38 2025-06-03 05:52:25.310651394 192.168.1.2 → 192.168.1.1 ICMP 1514 Echo (ping) request id=0xca6f, seq=768/3, ttl=255
39 2025-06-03 05:52:25.310828347 192.168.1.1 → 192.168.1.2 ICMP 1514 Echo (ping) reply id=0xca6f, seq=768/3, ttl=255 (request in 38)
40 2025-06-03 05:52:25.311183606 192.168.1.2 → 192.168.1.1 ICMP 1514 Echo (ping) request id=0xca6f, seq=1024/4, ttl=255

The difference in 42 bytes because of headers:

14 bytes ethernet header
20 bytes ip header
8 bytes icmp header

[Herman2022]

Thanks @Pavel Tarakanov for your advice!  However, I tested on cisco ISO switch like 3850, 2960, can ping with size 1500 df-bit set. 

Can you advise why C3850 can pass through 1500bye icmp packets? thanks. 

[Pavel Tarakanov]

IOS#ping 1.1.1.1 ?
Extended-data specify extended data pattern
data specify data pattern
df-bit enable do not fragment bit in IP header
dscp Specify DSCP value in ASCII/Numeric
ingress LAN source interface for Ingress
repeat specify repeat count
size specify datagram size
source specify source address or name
timeout specify timeout interval
tos specify type of service value
validate validate reply data
<cr>

In IOS/IOS-XE in ping command we specified datagram size (what can be seen in context help), which means that this size include IP and ICMP headers.

ping packet-size 1472 in NX-OS generates the same size packet as ping size 1500 in IOS.

IOS#ping 192.168.1.1 size 1500 df
Type escape sequence to abort.
Sending 5, 1500-byte ICMP Echos to 192.168.1.1, timeout is 2 seconds:
Packet sent with the DF bit set
!!!!!
Success rate is 100 percent (5/5), round-trip min/avg/max = 1/1/2 ms
19 2025-06-03 07:30:43.790800341 192.168.1.2 → 192.168.1.1 ICMP 1514 Echo (ping) request id=0x0003, seq=0/0, ttl=255
20 2025-06-03 07:30:43.791229871 192.168.1.1 → 192.168.1.2 ICMP 1514 Echo (ping) reply id=0x0003, seq=0/0, ttl=255 (request in 19)
21 2025-06-03 07:30:43.792009944 192.168.1.2 → 192.168.1.1 ICMP 1514 Echo (ping) request id=0x0003, seq=1/256, ttl=255
22 2025-06-03 07:30:43.792922307 192.168.1.1 → 192.168.1.2 ICMP 1514 Echo (ping) reply id=0x0003, seq=1/256, ttl=255 (request in 21)
23 2025-06-03 07:30:43.793603548 192.168.1.2 → 192.168.1.1 ICMP 1514 Echo (ping) request id=0x0003, seq=2/512, ttl=255
24 2025-06-03 07:30:43.793769262 192.168.1.1 → 192.168.1.2 ICMP 1514 Echo (ping) reply id=0x0003, seq=2/512, ttl=255 (request in 23)
25 2025-06-03 07:30:43.794431987 192.168.1.2 → 192.168.1.1 ICMP 1514 Echo (ping) request id=0x0003, seq=3/768, ttl=255
26 2025-06-03 07:30:43.794640542 192.168.1.1 → 192.168.1.2 ICMP 1514 Echo (ping) reply id=0x0003, seq=3/768, ttl=255 (request in 25)
27 2025-06-03 07:30:43.795252145 192.168.1.2 → 192.168.1.1 ICMP 1514 Echo (ping) request id=0x0003, seq=4/1024, ttl=255