Disadvantages of Increasing MTU. Router buffers ?

[from88]

Hello,

In our DC we've 1500MTU, we're thinking of increasing it to ~9000. 

I was wondering do it have any disadvantages ? Especially heard something like it could cause router buffers to overfill ?

But it's hard for me understood why it'd happen ? Maybe you can provide any explanation ? Thank you. Talking mostly about ISR Routers.

[curdubanbogdan]

It depends, you would need equipment at every step in the chain to also support jumbo frames (MTUs over 1500). You should take into consideration if the increase in MTU is accepted by your service provider and the type of encapsulation you are doing (IPsec, GRE, VXLAN, OTV etc).  It would be nice if you were able to increase the MTU size in the core of the network or WAN to avoid the fragmentation and PMTUD issues. Ask your service provider if they support larger frame sizes within their network and on the link between their PE and your CE router. Learning about the benefits of jumbo frames may be beneficial to your network's performance. However, it is important to explore if our network devices support jumbo frames before you turn it on. IN DC MTU increase has benefits, to more efficiently transport large volumes of data.. However, you should be cognizant of the fragmentation that may occur if those large frames try to cross a link that has a smaller MTU size. 

[from88]

thank you,

So as I understood the most problems occurs when the bigger packets enters the device which is not capable of supporting it so the device should fragment the packets. Which is CPU intensive job.

And this is not related to router buffers.

And i'm interested how bigger packets affects router performance when it's need to be fragmented ? 

[curdubanbogdan]

It is difficult to predict. You mast test it first and put also do-not-fragment to see how it operates and then do a show cpu process to check the performance. You can read this article about MTU:

https://www.networkworld.com/article/2224654/mtu-size-issues.html

[from88]

Thank you,

I just want to clarify:

on which case the fragmentation occurs

1) If the router receives a bigger packet than it's interface configured. Do it will fragment or drop the packet ?

2)If the router gets the packet in ingress interface where MTU is OK, but the egress interface MTU is smaller

?

Thank you.

[Joseph W. Doherty]

#1 Receiving interface will either "drop it" as an invalid (too large) frame, or it might correctly receive it. (Depends on the device. If the interface hardware can actually accept the size of the frame, even though not configured for it, it may accept it.)

Remember MTU is maximum transmission unit, not maximum receive unit.

#2 If it's a L3 "hop", if DF set, packet will be dropped and message sent to source. If DF not set, packet will be fragmented and forwarded.

If it's a L2 "hop", I believe frame will be dropped without notification to source.

[curdubanbogdan · Accepted solution]

The router will fragment the packet if it has the option for don-not-fragment off. If the int has 1500 MTU and the packet is 4000 it will fragment it.

There are several issues that make IPv4 fragmentation undesirable. There is a small increase in CPU and memory overhead in order to fragment an IPv4 datagram. This holds true for the sender as well as for a router in the path between a sender and a receiver. The creation of fragments simply involves to create fragment headers and copy the original datagram into the fragments. This can be done fairly efficiently because all the information needed in order to create the fragments is immediately available.

Fragmentation causes more overhead for the receiver when reassembling the fragments because the receiver must allocate memory for the arriving fragments and coalesce them back into one datagram after all of the fragments are received. Reassembly on a host is not considered a problem because the host has the time and memory resources in order to devote to this task.

But, reassembly is very inefficient on a router whose primary job is to forward packets as quickly as possible. A router is not designed to hold on to packets for any length of time. Also, a router that does reassembly chooses the largest buffer available (18K) with which to work because it has no way to know the size of the original IPv4 packet until the last fragment is received.

Another fragmentation issue involves how dropped fragments are handled. If one fragment of an IPv4 datagram is dropped, then the entire original IPv4 datagram must be resent, and it is also fragmented. You see an example of this with Network File System (NFS). NFS, by default, has a read and write block size of 8192, so a NFS IPv4/UDP datagram is  approximately 8500 bytes (which includes NFS, UDP, and IPv4 headers). A sending station connected to an Ethernet (MTU 1500) has to fragment the 8500 byte datagram into six pieces; five 1500 byte fragments and one 1100 byte fragment. If any of the six fragments are dropped because of a congested link, the complete original datagram has to be retransmitted, which means that six more fragments will have to be created. If this link drops one in six packets, then the odds are low that any NFS data can be transferred over this link, since at least one IPv4 fragment would be dropped from each NFS 8500 byte original IPv4 datagram.

Firewalls that filter or manipulate packets based on Layer 4 (L4) through Layer 7 (L7) information in the packet might have trouble processing IPv4 fragments correctly. If the IPv4 fragments are out of order, a firewall might block the non-initial fragments because they do not carry the information that would match the packet filter. This would mean that the original IPv4 datagram could not be reassembled by the receiving host. If the firewall is configured to allow non-initial fragments with insufficient information to properly match the filter, then a non-initial fragment attack through the firewall could occur. Also, some network devices (such as Content Switch Engines) direct packets based on L4 through L7 information, and if a packet spans multiple fragments, then the device might have trouble enforcing its policies.

[Joseph W. Doherty]

"There is a small increase in CPU and memory overhead in order to fragment an IPv4 datagram."

BTW, in my experience in smaller (and older) ISRs, the CPU load might increase by a fairly large percentage.

"But, reassembly is very inefficient on a router whose primary job is to forward packets as quickly as possible."

Also BTW, transit devices don't reassembly. Only if the router, itself, was the destination would it reassembly.