MTU vs tcp adjust-mss

tato386 · ‎03-25-2008

My server is having trouble exchanging mail with another SMTP server. After much troubleshooting I ran accross an article that states that this might be caused by a "black-hole" router and MTU/packet issues. Sounds weird but I want to give it a try. My questions are:

1) Should I use "ip mtu xxx" or "ip tcp adjust-mss"? What are the diferences between these two commads.

2) Should I apply this to the WAN or LAN of my Internet router?

Thanks,

Diego

pciaccio · ‎03-25-2008

The differences between the IP MTU and tcp-adjust-mss is that the MTU expands the IP Packet size to the specific size you specify. The tcp-adjust-mss sizes the segment size of the layer 4 segment to the size you specify. If you adjust the MSS then layer 3 (packet) will add on the standard IP packet header and then the layer 2 will add on the frame header to the packet header from layer 3. You will need to size the segment size to a value that when the packet and frame headers are tagged on then the total value of the frame is 1500. If you have control of your devices end to end then you can get away with setting the MTU size assuming you can control any fragmentation and or firewall parameters that may restrict you. However, If you do not have control or go out on the internet then I would suggest sizing the segment and your traffic will have no issues getting from end to end. If you size the MTU then you may have fragmentation issues and if you pass a device that does not allow fragmentation or congestion is too much then you can tempt faith and possibly lose data. Your best bet is to use MSS...Good Luck...

tato386 · ‎03-26-2008

Thanks for the very detailed explanation!

Diego

paolo bevilacqua · ‎03-26-2008

The only problem with the explanation above is that is not correct. Setting ip mtu will never "expand the IP packet". Neither MSS does that.

What IP mtu does, is to set the maximum size of packets that can be sent on an interface. Packets sized bigger that that, will be fragmented.

TCM mss-adjust does something else, that is a bit complicated to explain in few lines, but basically forces the router to adjust the advertised maximum TCP segment size to a smaller value than the one use by computers by default. This is typically needed in presence of PPPoE links.

tato386 · ‎03-26-2008

Check out this link:

http://technet.microsoft.com/en-us/library/bb878081.aspx

Do these "black-hole" routers/problem really exist? I think I am being affect by this and basically am trying to take care of it at my router rather than on the PCs being affected.

Thanks,

Diego

mbroberson1 · ‎03-27-2008

I the MSS to be set on the inside, or outside interface of the routers ethernet interface?

Thanks

mbroberson1 · ‎03-28-2008

Take a look at this article. It will certainly help in assisting.

http://www.nil.si/ipcorner/IP_Fragmentation/

Joseph W. Doherty · ‎03-26-2008

You might want to try the following to verify whether you have a black hole router:

yourRouter#ping

Protocol [ip]:

Target IP address: IP Addr of far SMTP server

Repeat count [5]: 1

Datagram size [100]:

Timeout in seconds [2]:

Extended commands [n]: y

Source address or interface:

Type of service [0]:

Set DF bit in IP header? [no]:

Validate reply data? [no]:

Data pattern [0xABCD]:

Loose, Strict, Record, Timestamp, Verbose[none]:

Sweep range of sizes [n]: y

Sweep min size [36]: 1450

Sweep max size [18024]: 1550

Sweep interval [1]:

Type escape sequence to abort.

Sending 101, [1450..1550]-byte ICMP Echos to 10.5.1.1, timeout is 1 seconds:

!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!

If you see any "." instead of "!", packets are being dropped, which shouldn't happen, they should be fragmented if larger than MTU (note 1550, above).

From most hosts, you can try various sizes. E.g. a Windows host:

C:\windows>ping 10.5.1.1 -l 1550

Pinging 10.5.1.1 with 1550 bytes of data:

Reply from 10.5.1.1: bytes=1550 time=1ms TTL=255

Ping statistics for 10.5.1.1:

Packets: Sent = 4, Received = 4, Lost = 0 (0% loss),

Approximate round trip times in milli-seconds:

Minimum = 1ms, Maximum = 1ms, Average = 1ms

C:\windows>ping 10.5.1.1 -l 1550 -f

Pinging 10.5.1.1 with 1550 bytes of data:

Packet needs to be fragmented but DF set.

Ping statistics for 10.5.1.1:

Packets: Sent = 4, Received = 0, Lost = 4 (100% loss),

C:\windows>ping 10.5.1.1 -l 1500

Pinging 10.5.1.1 with 1500 bytes of data:

Reply from 10.5.1.1: bytes=1500 time=1ms TTL=255

Ping statistics for 10.5.1.1:

Packets: Sent = 4, Received = 4, Lost = 0 (0% loss),

Approximate round trip times in milli-seconds:

Minimum = 1ms, Maximum = 1ms, Average = 1ms

C:\windows>ping 10.5.1.1 -l 1500 -f

Pinging 10.5.1.1 with 1500 bytes of data:

Packet needs to be fragmented but DF set.

Ping statistics for 10.5.1.1:

Packets: Sent = 4, Received = 0, Lost = 4 (100% loss),

If you find the maximum MTU is smaller than 1500, you can set the MTU on either interface (preferably the WAN interface) to the maximum discovered size and set IP tcp adjust-mss to 40 bytes less. NB: This will also impact all traffic transienting this router.

alan.hunt · ‎03-26-2008

Does your WAN consist of GRE/IPSec tunnels? The additional encapsulation overhead can exceed interface MTU, especially for applications that use large efficient packets, like printing, FTP, mail, remote desktop, active directory synchronization, etc. (my experience anyway). After fiddling over the years, I've settled on TCP adjust-mss 1354, applied on the LAN side of my routers. I don't use any MTU commands on other interfaces (or tunnel interfaces), just the mss command...this has solved my issues regardless of application or WAN technology (ATM, broadband, frame-relay, ppp, etc.) The MSS command actually initiates an ICMP conversation between the router and transmitting devices on the LAN side, so you can't have a firewall blocking ICMP along the path. You'll be better off getting the routers out of the fragmentation business...do it at the source with this method. You may run into a device that does not respond to the MSS ICMP messages, but I've never experienced one.

Joseph W. Doherty · ‎03-26-2008

I agree that ip tcp adjust-mss can avoid TCP fragmentation processing, but like Paolo, my understanding of the function of this command is it overwrites the MSS during TCP handshake, i.e. ICMP isn't involved. Perhaps you're thinking of ICMP interaction while supporting PMTU?

PS:

Also, using ip tcp adjust-mss alone, could risk silent drop of large non-TCP packets.

alan.hunt · ‎03-27-2008

Yeah, you're probably right on both counts there...but I noticed that when putting MTU commands on my tunnel interfaces, I would often get an error saying that the value was greater than the default of 1394, and my rule of thumb for mss was 40 bytes less than MTU, so I came up with 1354 and everything started working. That 40 bytes is for a 20 byte IP and 20 byte TCP header, but admittedly, there's no accounting for the layer-2 encapsulation there...(usually 8 bytes). Life is too short...if anybody knows where this default MTU value of 1394 for tunnels is, lemme know...never found reference to it anywhere.

mbroberson1 · ‎03-27-2008

Alan,

Do you put the "ip tcp adjust-mss" command on the inside or outside interface of the routers ethernet interface?

Thanks

alan.hunt · ‎03-27-2008

Typically on the inside (LAN)...facing the workstations/servers.

mbroberson1 · ‎03-27-2008

Thanks for the reply. I have been eperiencing what I think is the same issue on one os our site to site VPNs...how does this config for my intrfaces look to you?

interface GigabitEthernet0/0

description outside$ETH-LAN$

ip address 210.209.230.5 255.255.255.0

ip nat outside

ip virtual-reassembly

duplex auto

speed auto

media-type rj45

crypto map FMC_CMAP

crypto ipsec df-bit clear

!

interface GigabitEthernet0/1

description inside$ETH-LAN$

ip address 10.10.10.10 255.255.255.0

ip nat inside

ip virtual-reassembly

ip tcp adjust-mss 1300

no ip mroute-cache

duplex auto

speed auto

media-type rj45

Richard Burts · ‎03-28-2008

Brandon

The article by Ivan for which you posted the link was interesting and helpful. As he explains in his article the ip tcp adjust-mss can be placed on either the inside or outside interfaces or on both. Like Alan I tend to place it on the inside interface and have achieved good success with it there.

I believe that your config looks fine as it applies the adjust-mss on the inside interface. The value of 1300 is a safe value and perhaps a bit conservative. I have frequently found that I could set the value up to 1375 or 1380 (for GRE with IPSec tunnels). As Ivan explains in his article the optimum value will vary depending on some of the parameters chosen in the IPSec configuration. So 1300 will pretty much always work. Sometimes a larger value will also work - you might need to experiment a bit to find the value at which you start to experience problems.

HTH

Rick

HTH

Rick