MTU Philosophies (To go big, or not to)

[ksolie]

Hi Guys, (and Gals)

I been reading through some of the posts here, and I just wanted to get you opinions or philosophies on the subject of MTU.

Say you have the standard MPLS network, and has a service provider you basically have almost everykind of link made.

Serical DS2, Oc3, Oc12, FastE, GigE, and TenGE.

Most of the books and guides recommend setting the MTU at the smallest common denomitor to avoid fragmentation. Would you do this? I feel sometimes we stick to these old rules that are in print, but are loosing practicality as network evolve.

It seems with modern networks, we should be moving to Jumbo frames on the switches and setting MTU as high as we can on all wan, and metroE type interfaces. Is the fragmentation really that bad if its kept under control.

Your thoughts???? (and my thanks)

-Karl Solie

PS-The reason for the question is we are doinga large rollout and currently debating it. If you have the time I would love your input.

[rknuttila]

I believe that you should set your MTU as high as you can on your core while still being consistent across all of your core link types. So, the smallest common denominator does come into play. I expect your FastE interfaces will be your limiting factor. If your FastE's are only on your edge then your core can go very high.

If you want to go with a higher MTU one some paths then you'd have to insure that the lower-MTU links can't possibly be in the path that requires the higher MTU (maybe your FastE's aren't passing MPLS VPN and/or MPLS TE paths). It can be tricky, though. If you let too-large packets into the network and you have to drop them, I think it's nicer to know that you'll drop them as close to the ingress as possible.

Reid Knuttila

[ksolie]

Thanks for the input. You raise some good points.

I also like Swaroop's post earlier.

Seems like you can almost look at it like a big tree, keep low MTU on the leaves, and high MTU on the root...

Thanks,

Karl Solie

[swaroop.potdar]

If it wasnt for TCP and if we had all the control on the customers first hop IP device things would have been so much simpler.

But its wishful thinking :-), And as Karl pointed it out Low MTU on the leaves and High MTU on the Roots is the way to go.

At the Leaves - Low MTU limited to 1500-1518 to the maximum.

Towards and At the Roots - High MTU - Most use 1526 (1518 plus 2 labels in the stack) but if TE is factored this value will be less, or for that matter even CSC as a service.

So the High MTU value cap depends on the current and future service offerings within your new rollout.

But a good and safe baseline to start adding the MTU value is the 1518, and then we keep adding 4 bytes more per Service which may consume a Label space in the stack.

Also most Core Class interfaces like Gig, TGig, POS are capable of supporting MTU's higher than the conventional 1500.

HTH-Cheers,

Swaroop

[siljugpillai]

1526 is fine if you are not running protocols like l2tp in your network. But I suggest to keep it to a higher value than 1526. It depends on the switches you use within your network as well (if they are between the routers). For eg 3750 supports a maximum mtu of 1546.

Yes, at the edge it can be a smaller value.

Thanks,

Silju

[mheusinger]

Hi,

besides all the already postedhelpful comments my 2 cents: in TCP/IP networks MTU really is an end-to-end question. There is not too much gain in raising the MTU to maximum in a core, if no IP packet larger than 1500 bytes ever will show up because of PC/server - or in general - edge settings.

The main goal must be avoiding fragmentation or dropping packets with DF-bit set. So make sure your SP network does not require IP fragmentation for any customer packet, if possible. This includes to look at all additional headers potentially added in the SP network - MPLS labels, L2TPv3, GRE, etc.

As already mentioned you still have some restrictions imposed by hardware, which might give an upper limit.

And last, make sure OSPF and/or ISIS MTU is the same for adjacent routers or your adjacency will not come up.

Regards, Martin

[davegibelli]

My 2p...the mtu should be as high as possible for this reason: If two core routers have an mtu of 10,000 say, then they can exchange large routing packets, instead of breaking them into 1500 byte packets...

Your customers will only sent 1500 bytes so they will cross your core ok as well...Cisco set the default mtu's carefully for a good reason!

[pmaffei]

The network I'm designing now is a bit different because I'm the customer.. I mean, this is a tier-2 SP and for P-P and P-PE we are using Ethernet Services from another carrier. I'm already running L3VPNs but now need to add TE+FRR(2 extra labels).

So even in the core in this particular solution I'll stick with 1534.

Any hint on how to test the other provider MTU?

I've found this MTU issue a bit confusing!

[davegibelli]

I test MTU using "ping -c 1 -s 1472 -M do ". This will send one 1500 byte packet from a Linux box.

1472 is the amount of data in the ICMP packet, then add 8 for ICMP header and 20 for the IP header, total 1500. You will get back a message telling you either ping success or a message telling you the MTU size.

But I agree MTU is a complex subject, the actual mtu can change from one second to the next.

[swaroop.potdar]

If your ethernet service provider is able to switch a payload of 1516 with DF bit set then his MTU is right for your services.

You can verify this by extended ping setting the DF bit across the devices connected by the leased Ethernet circuit.

HTH-Cheers,

Swaroop

[ksolie]

Thanks for all your input.

I appreciate your help.

Thanks again,

Karl Solie

[colbedi]

Hi all

When we use l2tpv3 in our network then what are the complications that come with mtu. One of our client wasnt able to ping beyond mtu of 1458 .We removed the command no ip route-cache cef on the interface where l2 tun was terminated and the ping went fine. I m not able to corelate these 2 events

regards

Aasheesh Gupta

[jopez]

hi, after following the conversation...

during Network migration, I have some older routers (fast ethernet on core side links) and switches only allowing 1500 mtu. some test users in that older routers (ATM-DSL connected) are experiencing problems (other's not).

Up to date, i thought that was the usual behavour, depending on the mtu configured on the users.... is that right? My idea is that it has not sense to have MPLS enabled routers with physical media on core using mtu's of 1500

thanks once more

[ksolie]

Hi,

Your may seeing re-transmits or other issues related to certain application using a large MTU or a MTU of 1500 bytes.

You need 4 bytes for every Label in the label stack. You will always have one label for MPLS, and you may have one for VPNs and maybe two more, one for QoS and one for MPLS TE.

Most site's are running at least VPNs, therefore they need an extra 8 bytes. To test this on your network goto a CE router and issue an extended ping from CE to CE, and set the Don't Fragment bit "DF". If you ping times out it more then likely a MTU issue. Either use jumbo frames on the switches in the MPLS core and adjust the MTU to 1508 to 1516. Or if you can't do this, set the MTU down to 1492 or 1484 to accomodate a 8 extra bytes for MPLS/VPNs or 16 bytes for MPLS/VPN/QoS/Te...

Hope that helps,

Karl