cancel
Showing results for 
Search instead for 
Did you mean: 
cancel
409
Views
10
Helpful
7
Replies
Highlighted
Beginner

MPLS - MTU and packet loss with payload greater than 1492 bytes

Hello all,

 

The following topology has been deployed using Catalyst 9300 switches at the core and ASR920-12SZ-IM routers for WAN aggregation.  OSPF is the IGP with MP-BGP configured for MPLS L3VPNs to support multiple customers.  All interfaces are L3 routed ports.

Screenshot 2020-06-09 at 20.43.48.png

The system MTU on the C9300s is set at 9168 with the interface MTU on the ASR920 set the same.  IP MTU is configured as 1500.  Usually, the MPLS MTU is derived from the system/interface MTU and indeed, a show run all on the interfaces shows the following:

DC2-C9300-01#show run all | section interface TenGigabitEthernet1/1/2
::::
ipv6 mfib forwarding input
ipv6 mfib forwarding output
ipv6 mfib cef input
ipv6 mfib cef output
mpls mtu 9168
mpls ldp igp autoconfig
mpls ldp igp sync
snmp trap link-status
::::

Additionally, a show mpls interfaces detail on the C9300 also shows the MTU of 9168 on all interfaces.

DC2-C9300-01#show mpls interfaces detail | include MTU
        MTU = 9168
        MTU = 9168
        MTU = 9168
        MTU = 9168
        MTU = 9168
        MTU = 9168

And on the ASR920

DC2-ASR920-01#show mpls interfaces detail | include MTU
        MTU = 9168
        MTU = 9168
        MTU = 9168

Despite this, we are experiencing packet loss (from an end device connected to a CE) when the IP packet size is greater than 1492 bytes with df-bit set.  A consequence of this is that certain applications are failing since the TCP segment size will still negotiate at 1460.  I do not want to use the ip tcp adjust-mss command as this is a fudge!

We are using Cisco's recommended release for all devices and the config guide for MPLS on the C9300 states:

 

Screenshot 2020-06-09 at 20.56.20.png

 

Now this seems to contradict what the show mpls interface detail and the show run all on the interface reveals.  Is there a way to verify the true MPLS MTU size on C9300 switches as there is on the ASR920?

The following output clearly shows that despite the MPLS MTU showing as 9168 in the aforementioned commands, it is in fact, only 1504 bytes.

DC2-ASR920-01#show platform hardware pp active feature mpls mtu-table 
MPLS MTU Table

Index    MTU  Ref-Count   
------------------------
0       1504  1           
1       1604  1           
2          0  0           
3          0  0           
4          0  0           
5          0  0           
6          0  0           
7          0  0  

This is what the config guide for the ASR920 states about MPLS MTU:

 

Screenshot 2020-06-09 at 21.22.48.png

 

If it transpires that the MPLS MTU does indeed track the IP MTU, what are the consequences of raising the IP MTU on all core devices to say 2000 to enable the MPLS MTU to allow larger packet sizes?

 

Many thanks in advance.

 

7 REPLIES 7
Highlighted
VIP Advisor

Re: MPLS - MTU and packet loss with payload greater than 1492 bytes

Hi

On release 16.12, mpls isn’t supported.
These catalyst 9300 are the core devices you’re talking about? Max ip mtu will be 1500.
I would use the path MTU discovery to avoid any performance issues due to fragmentation.

Thanks
Francesco
PS: Please don't forget to rate and select as validated answer if this answered your question
Highlighted
Beginner

Re: MPLS - MTU and packet loss with payload greater than 1492 bytes

Thanks for the reply Francesco.  MPLS is indeed supported on the Catalyst 9300 platform with a few limitations.

Namely the inability to manually configure the MPLS MTU command under an interface.  I've included below, configuration from the switch.

 

!---System MTU value
DC2-C9300-01#show system mtu Global Ethernet MTU is 9168 bytes.

 

!---MPLS enabled interfaces
DC2-C9300-01#show mpls interfaces Interface IP Tunnel BGP Static Operational GigabitEthernet1/0/23 Yes (ldp) No No No Yes Te1/1/1 Yes (ldp) No No No Yes Te1/1/2 Yes (ldp) No No No Yes Te1/1/7 Yes (ldp) No No No Yes Te1/1/8 Yes (ldp) No No No Yes

 

!---MPLS interface detail
DC2-C9300-01#show mpls interfaces tenGigabitEthernet 1/1/2 detail Interface Te1/1/2: Type Unknown IP labeling enabled (ldp) : IGP config LSP Tunnel labeling not enabled IP FRR labeling not enabled BGP labeling not enabled MPLS operational MTU = 9168

 

!---Standard running config for example interface
DC2-C9300-01#show run interface tenGigabitEthernet 1/1/2 Building configuration... Current configuration : 295 bytes ! interface TenGigabitEthernet1/1/2 description *** DC2-ASR920-01 - Te0/0/12 (LSP) *** no switchport ip address 10.88.255.9 255.255.255.252 no ip redirects ip mtu 1500 ip ospf message-digest-key 1 md5 ******* ip ospf network point-to-point ip ospf 64532 area 0 end

 

!---Hidden MPLS MTU config (default value)
DC2-C9300-01#show run all | section interface TenGigabitEthernet1/1/2 :::: ipv6 mfib forwarding input ipv6 mfib forwarding output ipv6 mfib cef input ipv6 mfib cef output mpls mtu 9168 mpls ldp igp autoconfig mpls ldp igp sync snmp trap link-status cts role-based enforcement no mka pre-shared-key mka default-policy autonomic arp arpa arp timeout 14400 spanning-tree port-priority 128 spanning-tree cost 0 ::::

 

Referring back to the excerpt from the configuration guide for 16.12:

Screenshot 2020-06-09 at 20.56.20.png

It does state that MPLS MTU will be the same as the IP MTU so despite what the verification command output displays, it should actually be 1500 since this is clearly configured under the interface config.  I am concerned that Cisco's own "show" commands suggest otherwise.

We simply want to allow end-to-end unfragmented packets over the core so want to avoid PMTUD.  May have to open a case with TAC to establish if there is any specific ASIC verification command that will reveal the true MPLS MTU value used on the interface!

 

Cheers,

Highlighted
Beginner

Re: MPLS - MTU and packet loss with payload greater than 1492 bytes

Just a quick update on this following configuration on the ASR920.  I have also raised a case with TAC to establish exactly how the Catalyst 9300 platform tracks MPLS MTU despite glaring confusion by the show commands.

 

!---MPLS MTU reported by standard command
DC2-ASR920-01#show mpls interfaces tenGigabitEthernet 0/0/12 detail Interface Te0/0/12: Type Unknown IP labeling enabled (ldp) : Interface config IGP config LSP Tunnel labeling not enabled IP FRR labeling not enabled BGP labeling not enabled MPLS operational MTU = 9168

 

!---MPLS MTU reported by the platform
DC2-ASR920-01#show platform hardware pp active feature mpls mtu-table MPLS MTU Table Index MTU Ref-Count ------------------------ 0 1504 1 1 1604 1 2 0 0 3 0 0 4 0 0 5 0 0 6 0 0 7 0 0

 

!---Command to enable the configuration of MPLS MTU (which now tracks the interface MTU if not configured explicitly)
DC2-ASR920-01(config)#platform mpls mtu-enable !---MPLS MTU now reported by the platform
DC2-ASR920-01#show platform hardware pp active feature mpls mtu-table MPLS MTU Table Index MTU Ref-Count ------------------------ 0 1504 1 1 1604 1 2 9172 3 3 0 0 4 0 0 5 0 0 6 0 0 7 0 0

 

Any help would be appreciated!

Highlighted
Hall of Fame Master

Re: MPLS - MTU and packet loss with payload greater than 1492 bytes

Hello @DarrenB_UK ,

increasing the IP MTU size on the core links looks like the only option to avoid MTU issues.

 

Your choice of core devices in Catalyst 9300 look like uncommon, however if these are the available devices you can only try to adapt to their limitations.

 

Hope to help

Giuseppe

 

Highlighted
Beginner

Re: MPLS - MTU and packet loss with payload greater than 1492 bytes

Thanks Giuseppe,

The choice of hardware was both commercial and functional.  Essentially, the switches are PE devices for multi-tenant internal  networks in a private cloud.  We extensively researched the feature support and the only functional stretch was the MPLS requirement using MP-BGP.  Other than providing directly connected networks to be redistributed into MPLS L3VPNs (VRFs), we will need to configure two of the devices as MPLS ASBRs using inter-AS option B, which is supported.

Increasing the IP MTU to 2000 looks like a viable option as we have one requirement for a customer payload of 1600 bytes.  So the MPLS MTU (if it does track the IP MTU), will be correctly sized to accommodate the labelled packets.

Still surprised that the show mpls interfaces detail and the show run all | section interface clearly shows the MPLS MTU as 9168! 

 

Cheers,

Darren

Highlighted
VIP Advisor

Re: MPLS - MTU and packet loss with payload greater than 1492 bytes

On these platforms, MPLS MTU won't work like it should on ASR devices that are more convenient for MPLS core.
Anyways, as @giuseppe said, you have to increase the IP MTU end to end.

Thanks
Francesco
PS: Please don't forget to rate and select as validated answer if this answered your question
Highlighted
Beginner

Re: MPLS - MTU and packet loss with payload greater than 1492 bytes

Thanks Francesco - ASR1000 series would have been the preferred choice but were cost prohibitive.

I modelled the environment using CSR1000v routers and the MPLS MTU tracks the interface MTU as expected.  If this is set at 1500 bytes, packet loss is observed (sending a 1500 byte packet df-bit set) due to the label overhead shrinking the payload capacity accordingly.

As soon as the HW MTU is set above 1508, no packet loss.  It's just a bit confusing why the "show" commands on the 9300 platform don't exactly reveal the truth about MPLS MTU!

 

Cheers,

Darren