Solved: changing mtu-size in MPLS networks

pverstegen · ‎07-28-2004

In what circumstances should I change the default mtu-size of tag-switching interfaces.

mheusinger · ‎07-28-2004

Hi,

the advise is to chose from the technical requirements taking into account the specific hardware you have.

So MTU=1508 is sufficent in case you have MPLS VPN and no other feature increasing the label stack.

MTU=1524 allows for 6 labels, plenty from nowadays perspective. However it will not allow you to use EoMPLS with full sized customer frames, which requires 1526 byte as described earlier.

Finally your traceback tells you that only 1572 Byte frames are allowed to be sent, but you had a 1574 Byte frame, which had to be dropped. I would consider contacting TAC for further detailed analysis, as this requires your exact configuration and hardware.

regards

Martin

View solution in original post

mheusinger · ‎05-12-2006

Hello,

this first question to answer is: what is the max MTU the hardware is supporting?

There are interface ASICS and they might have f.e. an input or output per frame buffer of max. 1572 Bytes. Anything larger is dropped immediately, because the hardware is not capable of handling it. Software and configuration might not be able to overcome this limitation.

So you might have to check your hardware or with TAC, what is supported. Worst case you might have to change the switches. Otherwise set the MPLS MTU to something supported (1572 Byte?)

Hope this helps! Please rate all posts.

Regards, Martin

View solution in original post

mheusinger · ‎07-28-2004

Hi,

The main goal usually is to pass your customer traffic through your backbone (ok, earning money is another task, but lets keep this technical). So in a typical environment you will have a MTU of 1500 required from the customer. If you have a labeled packet in your MPLS backbone, which is larger than the MPLS MTU on the interface it should be passed through, the packet will be dropped. There is no fragmentation in label switched paths. Normally you might get an ICMP reply back to the sender allowing for path MTU discovery to work. This is not always ensured especially in MPLS VPNs leading to TCP session timeouts and all sorts of connected problems.

So make sure you consistently offer at least MTU 1500 to your customer to avoid problems, i.e. account for the additional labels by increasing interface/MPLS mtu.

kind regards

Martin

pverstegen · ‎07-28-2004

Well, a lot of icmp messages show up when I turned on debug ip icmp:

Jul 28 11:05:35: MPLS: ICMP: dst (x.x.x.x) frag. needed and DF set unreachable sent to y.y.y.y

Since we changed the tag-switching mtu to 1504 on all Fastethernet interfaces on the LAN the number of icmp unreachables decreases but there are still some left.

Is the 1504 mtu-size correct or do you suggest an other value ?

mheusinger · ‎07-28-2004

The MPLS MTU in the core should be at least 1508 because there are two labels attached to a packet.

The first - top - label is used to send it to the proper PE, the second label is the "VPN label".

Setting "tag-switching mtu 1508" on all label switching interfaces should reduce the number of ICMP messages and more important the number of dropped packets!

Kind regards

Martin

Harold Ritter · ‎07-28-2004

I agree with Martin that in most cases in a MPLS VPN network, 1508 should be sufficient to carry 1500 bytes frame coming from the CEs.

However, you have to bear in mind that other features deployed in the core might need a larger MTU. This features include TE between core routers and Fast Reroute (FRR). In some scenarios, you might add yet another two labels to the frames traversing the core. The Cisco recommendation is usually to set the "mpls mtu" to 1524. This leaves plenty of room for additional labels.

Hope this helps,

Regards,
Harold Ritter, CCIE #4168 (EI, SP)

mheusinger · ‎07-28-2004

Harold is completely right. Just a few examples:

L3VPN minimum MPLS MTU 1508

MPLS TE adds one label 1512

L2VPN EthernetoMPLS 1526 (1518 + 2 Label)

+ TE + CsC + ...

So to come to a conclusion some providers here in Europe chose 4470 because they have POS and that is as large as it can support.

Last remark: be aware of your hardware limitations.

POS, Ethernet (Eth, FE, GE), ATM all have different MTU which is supported by the ASICs on the different line cards. Especially with Ethernet one should be careful and have a close look to the hardware sepcs of all involved systems.

See for Catalysts:

http://www.cisco.com/en/US/products/hw/switches/ps700/products_configuration_example09186a008010edab.shtml

Very last remark: be careful with choosing between MPLS MTU and normal MTU. I would suggest to change MPLS MTU and leave interface MTU at its default value of 1500 Byte, as some unwanted service disruptions can occur in case interface MTU is not set properly (OSPF adjacency problems to name one).

pverstegen · ‎07-28-2004

So what's the advise ?

Follow up Cisco recommendation (1524) ?

At the moment everything seems to work fine on the 1508 size. There was only one problem when I changed the mpls mtu size on the svi vlan9 and 10 of the Core-switch 6513 with sup.720 I got a traceback:

Jul 28 15:07:56.690 WET: %MSFC2-3-TOOBIG: Attempt to send giant packet on Vlan10 (1574 bytes from C3F1CC2, max allowed 1572)

-Traceback= 402F6800 4023DE84 40503ADC 404F096C 404F19C8 404EFDA8 404EFF94 404F01F8

Jul 28 16:31:16.468 WET: %MSFC2-3-TOOBIG: Attempt to send giant packet on Vlan9 (1574 bytes from C3552C2, max allowed 1572)

-Traceback= 402F6800 4023DE84 40503ADC 404F096C 404F19C8 404EFDA8 404EFF94 404F01F8

mheusinger · ‎07-28-2004

Hi,

the advise is to chose from the technical requirements taking into account the specific hardware you have.

So MTU=1508 is sufficent in case you have MPLS VPN and no other feature increasing the label stack.

MTU=1524 allows for 6 labels, plenty from nowadays perspective. However it will not allow you to use EoMPLS with full sized customer frames, which requires 1526 byte as described earlier.

Finally your traceback tells you that only 1572 Byte frames are allowed to be sent, but you had a 1574 Byte frame, which had to be dropped. I would consider contacting TAC for further detailed analysis, as this requires your exact configuration and hardware.

regards

Martin

Harold Ritter · ‎07-29-2004

I should have been more specific. The 1524 mtu size is a recommendation in an MPLS VPN context. As you pointed out, it might not suit a l2vpn deployment.

Thanks,

Regards,
Harold Ritter, CCIE #4168 (EI, SP)

mazlan.alatif · ‎05-14-2006

Hi,

I just read your reply regarding MTU issue. Does this mean that, to run l2vpn such as AToM would require mtu 1526 configured at P routers?

Cisco 7200 IOS version 12.3.25(S8) only allow mtu 1524 configured at Fe interface, and we have upgraded our 7500 using this IOS because its support IPv6.

So do i will face a problem to run AToM?

thanks,

-mazlan

pverstegen · ‎07-28-2004

Well, a lot of icmp messages show up when I turned on debug ip icmp:

Jul 28 11:05:35: MPLS: ICMP: dst (x.x.x.x) frag. needed and DF set unreachable sent to y.y.y.y

Since we changed the tag-switching mtu to 1504 on all Fastethernet interfaces on the LAN the number of icmp unreachables decreases but there are still some left.

Is the 1504 mtu-size correct or do you suggest an other value ?

pverstegen · ‎05-09-2006

Hi Martin,

You wrote: "If you have a labeled packet in your MPLS backbone, which is larger than the MPLS MTU on the interface it should be passed through, the packet will be dropped."

Does this mean that in case the MPLS MTU is 1508 (2 tag's), the largest supported customer traffic is 1500 bytes ? So all larger packets will be dropped ?

Is there no giant support in MPLS ?

Regards,

Peter

mheusinger · ‎05-09-2006

Hello Peter,

the MTU supported in MPLS is only limited by the interface capabilities. Once you specify "mpls mtu 1508" on an interface, the LSR is instructed to drop anything, which is larger. It is up to you what to configure - within the hardware specific interface limits.

Usually for a customer a MTU of 1500 is sufficient, because the default MTU even for WAN links on Cisco routers is 1500 Byte. So there might be no traffic entering the MPLS network with IP packets larger than 1500 Byte. Then you need only take care, that labeled packets are not larger than any interface MTU in your MPLS cloud.

In case you have GigE supporting 9216 Byte there is no objection to offer f.e. MTU 9000 to customers.

What I wanted to point out with my statement cited above is:

There is no fragmentation defined for labeled packets. Only IP packets (type 0x0800) could be fragmented, labeled packets (type 0x8847) can either be forwarded or dropped.

Hope this helps! Please rate all posts.

Regards, Martin

pverstegen · ‎05-12-2006

Hi Martin, this is verry helpfull. But what I don't understand is that packets above 1526 bytes get dropped whit mpls mtu 1600 configured on an interface....

COREIBVST002#ping vrf PRS-netwerk

Protocol [ip]:

Target IP address: 10.2.32.2

Repeat count [5]:

Datagram size [100]: 1526

Timeout in seconds [2]:

Extended commands [n]:

Sweep range of sizes [n]:

Type escape sequence to abort.

Sending 5, 1526-byte ICMP Echos to 10.2.32.2, timeout is 2 seconds:

030554: May 12 11:20:42.307 WET: %CONTROLLER-3-TOOBIG: An attempt made to send giant packet on GigabitEthernet1/16 (1600 bytes from 82F4FE2, max allowed 1572)