cancel
Showing results for 
Search instead for 
Did you mean: 
cancel
789
Views
0
Helpful
5
Replies

L3 out interface with MTU 9216 wont send packets larger the 1500 bytes

doridor
Level 1
Level 1

I am trying to increase the MTU of my L3out routed ports to 9216,

when I increased the MTU on one of the ports and pinged it form the other side with a 9216 byte icmp packet, it received the request and responded with a 1500 byte icmp reply. And once I try to initiate a ping larger then 1500 byes from the interface it doen't send anything.

When running "show interface {interface}" the MTU is set to 9216.

Does anyone have any idea why this is happening?

5 Replies 5

r.heitmann
Level 1
Level 1

is the device (router) you try to reach via ping from ACI is directly connected or is there a switch in between?

  • is this switch enabled for jumbo-MTU, too?

...and -

  • is the device you try to reach itself enabled for jumbo-MTU?

The problem isn’t with the device i am trying to reach. The routed port on the border leaf will not even sent the packets if they are larger then 1500 and the df flag is set

I guess, you're using the "iping"-command beeing connected using SSH to a ACI-Leaf switch.

  • -V = <TENANT>:<VRF>
  • -F = don't fragment
  • -s = packet-size

The packet-size-parameter of "iping" specifies the "data-portion" of the ICMP-Message sent.

  • like a windows-ping would do
  • unlike a cisco-ios-router would do

So, if IP-MTU=9000 should be possible specify 8972 bytes.

  • 20 Bytes for IPv4-Header
  • 8 Bytes for ICMP-Header

get added.

The result is an IP-Packet with size 8972+20+8=9000 Byte.

In my example, an IP-MTU of 9000 is possible, so

  • "-s 8972" => works
  • "-s 8973" => fails

 

 

 

leaf1# iping -V MTUTEST:DEFAULT 172.16.0.202 -F -s 8972
PING 172.16.0.202 (172.16.0.202) from 172.16.0.201: 8972 data bytes
8980 bytes from 172.16.0.202: icmp_seq=0 ttl=254 time=1.003 ms
8980 bytes from 172.16.0.202: icmp_seq=1 ttl=254 time=6.615 ms
8980 bytes from 172.16.0.202: icmp_seq=2 ttl=254 time=0.864 ms
8980 bytes from 172.16.0.202: icmp_seq=3 ttl=254 time=0.893 ms
^C
--- 172.16.0.202 ping statistics ---
4 packets transmitted, 4 packets received, 0.00% packet loss
round-trip min/avg/max = 0.864/2.343/6.615 ms

 

 

 

 

 

leaf1# iping -V MTUTEST:DEFAULT 172.16.0.202 -F -s 8973
PING 172.16.0.202 (172.16.0.202) from 172.16.0.201: 8973 data bytes
Request 0 timed out
Request 1 timed out
Request 2 timed out
^C
--- 172.16.0.202 ping statistics ---
4 packets transmitted, 0 packets received, 100.00% packet loss

 

 

just to prove, that the packet gets lost by MTU too big - I'll repeat the command without the "don't fragment"-option:

  • now the "too big" packet gets fragmented inside the leaf making the test a success 

 

leaf1# iping -V MTUTEST:DEFAULT 172.16.0.202 -s 8973
PING 172.16.0.202 (172.16.0.202) from 172.16.0.201: 8973 data bytes
8981 bytes from 172.16.0.202: icmp_seq=0 ttl=254 time=1.084 ms
8981 bytes from 172.16.0.202: icmp_seq=1 ttl=254 time=0.893 ms
8981 bytes from 172.16.0.202: icmp_seq=2 ttl=254 time=1.014 ms
8981 bytes from 172.16.0.202: icmp_seq=3 ttl=254 time=1.093 ms
8981 bytes from 172.16.0.202: icmp_seq=4 ttl=254 time=1.034 ms
^C
--- 172.16.0.202 ping statistics ---
5 packets transmitted, 5 packets received, 0.00% packet loss
round-trip min/avg/max = 0.893/1.023/1.093 ms

 

Forum Tips: 1. Paste images inline - don't attach. 2. Always mark helpful and correct answers, it helps others find what they need.

if this doesn't help we might need to check your L3Out-setup if your neighbor-device is fine.

...some ideas

  • verify the MTU of the L2-Interface your L3-Out uses (indirectly)
    • 9216 Bytes as in your case
    • this is the upper limit for Ethernet-Frames

 

leaf1# show interface ethernet 1/48
Ethernet1/48 is up
admin state is up, Dedicated Interface
  Hardware: 1000/10000 Ethernet, address: 0000.0000.0000 (bia 7c0e.cecb.cc0a)
  MTU 9216 bytes, BW 1000000 Kbit, DLY 1 usec
...

 

For L3-IP traffic there might be a (lower) limit at the IP-Interface. 

  • find out the IP-Interface of your L3-Out

 

leaf1# show ip int brief vrf MTUTEST:DEFAULT
IP Interface Status for VRF "MTUTEST:DEFAULT"(8)
Interface            Address              Interface Status
eth1/48.29           172.16.0.201/30      protocol-up/link-up/admin-up
vlan19               172.16.100.254/24    protocol-up/link-up/admin-up
lo2                  172.16.255.201/32    protocol-up/link-up/admin-up

 

  • find out the IP-MTU of your L3-Out
    • 9000 Bytes in my case
    • that's the real limit for IP

 

leaf1# show int e1/48.29
Ethernet1/48.29 is up
admin state is up, Dedicated Interface, [parent interface is Ethernet1/48
  Hardware: 1000/10000 Ethernet, address: 0000.0000.0000 (bia 7c0e.cecb.cc0a)
  Internet Address is 172.16.0.201/30
  MTU 9000 bytes, BW 1000000 Kbit, DLY 1 usec
  reliability 255/255, txload 1/255, rxload 1/255
  Encapsulation ARPA, medium is broadcast

 

Forum Tips: 1. Paste images inline - don't attach. 2. Always mark helpful and correct answers, it helps others find what they need.

wrgu
Level 1
Level 1

Not sure if you already know the reason.

Anyway, looks like the iping mtu is configured by the control plane MTU which is configured under the system setting. It took me awhile to figure it out. 

Getting Started

Find answers to your questions by entering keywords or phrases in the Search bar above. New here? Use these resources to familiarize yourself with the community:

Save 25% on Day-2 Operations Add-On License