Re: "ip mtu" vs "mtu" and Giant Frames

mfarrenkopf · ‎10-26-2020

I'm having an issue with an interface incrementing giants, when I don't think it should. The setup is a Catalyst 6807 connected to a Catalyst 9500-X. The 6807 is the one incrementing giants. There are no input or CRC errors.

6807

9500

interface TenGigabitEthernet1/15

mtu 9198

ip address 172.21.20.117 255.255.255.252
ip pim sparse-mode
ip ospf authentication message-digest
ip ospf message-digest-key 1 md5 7 #redacted#
ip ospf network point-to-point
ip ospf 5 area 0
mpls ip

6807#show int te1/15 | i MTU|Ten|giant
TenGigabitEthernet1/15 is up, line protocol is up (connected)
MTU 9198 bytes, BW 10000000 Kbit/sec, DLY 10 usec,
0 runts, 248512693 giants, 0 throttles

6807#show int te1/15 | i MTU|Ten|giant
TenGigabitEthernet1/15 is up, line protocol is up (connected)
MTU 9198 bytes, BW 10000000 Kbit/sec, DLY 10 usec,
0 runts, 248546768 giants, 0 throttles

interface TenGigabitEthernet1/0/35
no switchport
ip address 172.21.20.118 255.255.255.252
ip pim sparse-mode
ip ospf message-digest-key 1 md5 7 #redacted#
ip ospf network point-to-point
ip ospf 5 area 0
logging event link-status

9500#show int te1/0/35 | i MTU|Ten|giant
TenGigabitEthernet1/0/35 is up, line protocol is up (connected)
Hardware is Ten Gigabit Ethernet, address is 4cbc.485c.31c6 (bia 4cbc.485c.31c6)
MTU 9198 bytes, BW 10000000 Kbit/sec, DLY 10 usec,
0 runts, 0 giants, 0 throttles

The 9500 interface is configured for MPLS through the "mpls ldp autoconfig" statement in the OSPF process. Area authentication is similarly configured on the 9500. LDP and OSPF relationships exist and are stable.

I don't have a good explanation for why I should be seeing giants on the 6807. I might be able to use the "ip mtu" command. But it doesn't seem necessary. This is not a .1q trunk. Both sides are correctly configured for MPLS and doing label switching. Any management traffic, like CDP or LLDP, is not going to fill 9198 bytes.

This link is over single mode, LR-S interfaces. Light levels are good. Fiber ends and patch panels have been scoped.

Thoughts on why I'm seeing giants? 6807 is running IOS 15.2(1)SY8. 9500 is running IOS-XE 16.6.6.

Reza Sharifi · ‎10-26-2020

Hi,

0 runts, 248512693 giants, 0 throttles

Is this happening only when you configure Jumbo MTU or standard MTU (1500) as well?

HTH

mfarrenkopf · ‎10-26-2020

Hi Reza,

This is a production device, so I can't change the MTU to experiment.

The 9500 is an MPLS PE; the 6807 is an MPLS P. We set the MPLS uplinks to MTU maximum for the platform as a general practice. We don't have more than two labels in the stack right now, but should we get into TE or other MPLS technologies later, we didn't want to have to worry about MTU and total label stack.

Really, I have no reason to think we're reaching anywhere near maximum size. We use VRF lite from the access layer with a layer 2 trunk and multiple SVIs. The 9500 has the system MTU set to 9198. Each of the VRF lite SVIs is configured with "ip mtu 1500". There are no giant counters incrementing on the access switches.

It's possible it's a cosmetic issue on the 6807, just recording any frames > 1518 bytes or some such. Now that I look further, it appears all of the up interfaces are recording giant frames. A small sampling:

TenGigabitEthernet1/1 is up, line protocol is up (connected)
0 runts, 94638658 giants, 0 throttles
TenGigabitEthernet1/2 is up, line protocol is up (connected)
0 runts, 2298256886 giants, 0 throttles
TenGigabitEthernet1/3 is administratively down, line protocol is down (disabled)
0 runts, 0 giants, 0 throttles
TenGigabitEthernet1/4 is up, line protocol is up (connected)
0 runts, 1899902974 giants, 0 throttles
TenGigabitEthernet1/5 is up, line protocol is up (connected)
0 runts, 75129251 giants, 0 throttles
TenGigabitEthernet1/6 is up, line protocol is up (connected)
0 runts, 1189304992 giants, 0 throttles

And I should've checked this earlier, but our monitoring was also logging errors on the interface without input or CRC errors, so I figured it was complaining about the giants.

Still . . . the 9500 doesn't record giants. In poking around elsewhere, it looks like the 6800-class equipment records giants, while other platforms do not.

This might be a red herring. I'm still interested in your input. Thank you!

Reza Sharifi · ‎10-26-2020

Hi,

I am wondering if you also need to enable "system jumbomtu" on these switches since the interface you use to connect these switches together is layer-3 and not layer-2.

If this is impacting your production, you may want to think about a maintenance window and look at changing it and test.

HTH

ngkin2010 · ‎10-26-2020

I found a similar question in community post : https://community.cisco.com/t5/routing/giants-on-catalyst-6880-and-6500-mpls-ports-when-connected-to-a/td-p/4034503

As written by Giuseppe Larosa:

This means that C9x00 do not account for giants as they can be edge nodes in a campus fabric in an SD Access campus fabric.

The combination of DNAC + ISE + UDP VxVLAN is making the  the C9x00 with x>2 very interesting for a lot of new applications like IoT.

Edge nodes in DNAC campus fabric must have max MTU for the above reasons.

dbreese · ‎01-27-2022

This could be cosmetic with the 6807 counting packets larger than a certain value as giants.

However, I had a similar issue with a 6807 connected to a 9500 running MPLS. Interface MTU was set to 1500 on both ends.
The 6807 was registering giants and was actually discarding packets when they were maximum size (IP packet 1500) and MPLS switched.
According to Cisco TAC, an ASIC issue on the 9500 causes it to not properly honor MPLS MTU, so it will take a 1500 byte IP packet and encapsulate it in MPLS so sends a 1508 MPLS+IP playload in the ethernet frame. The 6807 will accept a maximum of 1500 byte payload in the ethernet frame, so counts it as a giant and discards it. We solved this by setting IP MTU to 1480 on both ends.

In the above scenario, it would have to be a 9198 byte IP packet for a similar thing to occur, so that seems unlikely.

MHM Cisco World · ‎01-29-2022

As Mr.dbreese suggest,
there are two value
IP MTU = MTU = show it in each SW
But after apply MPLS then
MPLS MTU = IP MTU + Label (2x4bytes) >= MTU
for example if the MTU is 1500 "from the respect of receive SW" then you must config the MPLS MTU =1500 OR IP MTU equal to 1492.
please Notice:- SW-ethernet-SW if the frame receive large than the default MTU then the frame is drop.