04-29-2012 09:05 AM - edited 03-04-2019 04:11 PM
Hi all,
Please let me know if I am right:
I am using a GRE-IPSEC tunnel configuration + EIGRP routing protocol and I would like to confirm if I am right with the following:
Due to I am configuring the IP MTU = 1400 in the Tunnel interface, I am avoiding additional fragmentation after GRE encapsulation because there is enough room for IPSEC encapsulation (no matter which mode I am using - I am considering the worst case Tunnel Mode). However, I would like to know what happens if I ALSO include in the Tunnel configuration the command ip tcp adjust-mss 1360 which is clear operates in layer 4 during the 3 way handshaking process to establish the TCP connection/session between opposite hosts (in this case the interaction is with the respective end routers). By adding this MSS command, I understand that I could also eliminate the initial fragmentation of the 1500 bytes LAN packets (before GRE encap) because the hosts are notified to send 1360 bytes packets to the Router and based on the previous, I would be able to transfer packets without "theorical" fragmentation between both ends.
One more question, how can affect if I include this additional command (TCP ADJUST-MSS) the performance (process + memory) of a router 3845 or 7200 without producing for example a entire crash of the device???. I understand that this TCP MSS negotiation is router process intensive but is less than IPSEC encryption/decryption.
thanks in advance for your comments.
Solved! Go to Solution.
05-01-2012 04:02 PM
Hello Abraham,
Is the IP MTU parameter the responsible for limiting the MSS size during the 3-way tcp handshake to 1360 bytes ((IP MTU configured in the Tunnel Interface = 1400) - (40 bytes headers)) as you see in the next image? How the PMTUD is also participating in this process?
No, the IP MTU parameter configured on a router is not responsible for the initial MSS size as advertised by end hosts. The end hosts initially derive their starting MSS value by looking at their own interface MTU (not the router's MTU - they do not know it at all) and decreasing it by at least 40 bytes. This starting MSS may subsequently be lowered thanks to ip tcp adjust-mss on a router. During the TCP session, the real MSS used to talk to the other party may dynamically change according to the current MTU of the end host's interface, or it may be influenced by PMTUD during the session, but it may never exceed the negotiated MSS.
Quoting from RFC 1122, Section 4.2.2.6 stipulates:
The maximum size of a segment that TCP really sends, the "effective send MSS," MUST be the smaller of the send MSS (which reflects the available reassembly buffer size at the remote host) and the largest size permitted by the IP layer: Eff.snd.MSS = min(SendMSS+20, MMS_S) - TCPhdrsize - IPoptionsize where: * SendMSS is the MSS value received from the remote host, or the default 536 if no MSS option is received. * MMS_S is the maximum size for a transport-layer message that TCP may send. * TCPhdrsize is the size of the TCP header; this is normally 20, but may be larger if TCP options are to be sent. * IPoptionsize is the size of any IP options that TCP will pass to the IP layer with the current message.
The PMTUD - if run by the end host engaged in the TCP communication - may influence the MMS_S element of the equation. Please do not confuse this Effective Send MSS with the value of the MSS option indicated in the TCP header during the 3-way handshake. That MSS is here represented as SendMSS and it may never be exceeded.
Based on my test below, I dont see any significant difference between using IP MTU or IP TCP Adjust-MSS if the IP MTU is already responsible for limiting TCP traffic through the GRE Tunnel connection.
The difference is rather strong. As I just explained, the IP MTU configured on a router is not directly influencing the MSS choice at the end host simply because the end host has no idea about the configuration of the router. In fact, the only way to find out is to use the PMTUD that has to be run by the end host again - not by the router. If the PMTUD is not performed, end hosts may end up negotiating much larger MSS than what would be necessary to prevent fragmentation, and the resulting IP packets sent by end hosts will have to be fragmented by the router.
The ip tcp adjust-mss is therefore indispensable.
Peter, if I am not wrong, the initial MSS value = 1260 sent by my PC depends on the Operating System and Application based on your orientation?
Yes, that is correct. What do you mean by "orientation" here?
Best regards,
Peter
04-29-2012 09:32 AM
Sorry, I forgot to ask that, is there any significant impact over my HSRP operation after adding the ADJUST MSS in both routers (primary/backup) so the TCP session MSS needs to be recalculated after a failure in the primary HSRP Router???.
04-29-2012 11:02 AM
Hello Abraham,
Due to I am configuring the IP MTU = 1400 in the Tunnel interface, I am avoiding additional fragmentation after GRE encapsulation because there is enough room for IPSEC encapsulation (no matter which mode I am using - I am considering the worst case Tunnel Mode).
Yes, you are correct. I assume you are familiar with the following document:
http://www.cisco.com/en/US/tech/tk827/tk369/technologies_white_paper09186a00800d6979.shtml
It directly recommends lowering the IP MTU on tunnel interfaces to 1400 bytes exactly as you did.
By adding this MSS command, I understand that I could also eliminate the initial fragmentation of the 1500 bytes LAN packets (before GRE encap) because the hosts are notified to send 1360 bytes packets to the Router and based on the previous, I would be able to transfer packets without "theorical" fragmentation between both ends.
Yes, this is absolutely correct. In fact, whenever you are lowering the IP MTU to a value less than 1500 bytes, you should always accompany it with the TCP MSS adjustment (decremented by another 40 bytes from the configured IP MTU) to prevent end hosts from sending overly large TCP segments requiring fragmentation.
how can affect if I include this additional command (TCP ADJUST-MSS) the performance (process + memory) of a router 3845 or 7200 without producing for example a entire crash of the device???. I understand that this TCP MSS negotiation is router process intensive but is less than IPSEC encryption/decryption.
This is in general difficult to say. The ip tcp adjust-mss command applies only to the first and second TCP segment exchanged during TCP connection establishment (i.e. segments with SYN flag set). Also, the modification encompasses merely setting the MSS in the TCP segment header to the min(segment-MSS, configured-MSS) and the usual checksum recalc stuff. Other segments are not inspected or modified. Therefore, I would personally believe that the MSS adjustment process is in fact quite negligible - someone please correct me if I am wrong here.
is there any significant impact over my HSRP operation after adding the ADJUST MSS in both routers (primary/backup) so the TCP session MSS needs to be recalculated after a failure in the primary HSRP Router???.
TCP MSS can not be recalculated once negotiated at the beginning of the session. The only stage when MSS is negotiated is the initial TCP three-way handshake. After that, the MSS stays the same during the entire session. Mismatches of IP MTU and TCP MSS in the primary and backup path could therefore be quite unpleasant. But if you set both your HSRP routers identically with regard to IP MTU and MSS, you should have no issues and there should be no impact over your HSRP at all.
Best regards,
Peter
04-29-2012 03:59 PM
Disclaimer
The Author of this posting offers the information contained within this posting without consideration and with the reader's understanding that there's no implied or expressed suitability or fitness for any purpose. Information provided is for informational purposes only and should not be construed as rendering professional advice of any kind. Usage of this posting's information is solely at reader's own risk.
Liability Disclaimer
In no event shall Author be liable for any damages whatsoever (including, without limitation, damages for loss of use, data or profit) arising out of the use or inability to use the posting's information even if Author has been advised of the possibility of such damage.
Posting
Peter Paluch wrote:
This is in general difficult to say. The ip tcp adjust-mss command applies only to the first and second TCP segment exchanged during TCP connection establishment (i.e. segments with SYN flag set). Also, the modification encompasses merely setting the MSS in the TCP segment header to the min(segment-MSS, configured-MSS) and the usual checksum recalc stuff. Other segments are not inspected or modified. Therefore, I would personally believe that the MSS adjustment process is in fact quite negligible - someone please correct me if I am wrong here.
I've used ip tcp adjust-mss since the feature was first offered - never noticed any significant impact to the devices it was configured on. (What I did notice first time I used it, much quicker initial [TCP] transfer rate for GRE tunnels.)
04-30-2012 06:24 AM
Peter / Joseph, thank you so much for your comments.
I would like to add the following, I am planning to use MSS ADJUST because I have both types of traffic (UDP + TCP) and at least the TCP traffic could be controlled in terms of avoiding fragmentation, leaving the MTU IP basically for UDP fragmentation before GRE encap. However, one more consideration (additional comments from both are well-received) that I have is that I am going to check with the application/servers teams the impact of this MSS change because I do not know if any kind of failure/degradation on the service provided by this app/servers could be produced after limiting MSS size.
Peter, I am going to certify MTU/MSS on the backup router as you suggested me in order to avoid additional inconvenients post-implementation of MSS.
Finally, if you have any other recommendation, please let me know.
regards
04-30-2012 08:26 AM
Disclaimer
The Author of this posting offers the information contained within this posting without consideration and with the reader's understanding that there's no implied or expressed suitability or fitness for any purpose. Information provided is for informational purposes only and should not be construed as rendering professional advice of any kind. Usage of this posting's information is solely at reader's own risk.
Liability Disclaimer
In no event shall Author be liable for any damages whatsoever (including, without limitation, damages for loss of use, data or profit) arising out of the use or inability to use the posting's information even if Author has been advised of the possibility of such damage.
Posting
Finally, if you have any other recommendation, please let me know.
Might already be in the document Peter referenced, but enable tunnel path-mtu-discovery on tunnel interface.
04-30-2012 11:35 AM
Joseph / Peter,
Before closing this post I would like to check with you the following:
If I have the following diagram regarding this case: (there is a WCCP module in the router). Do you think that the best place to apply MSS Adjust is in the Ethernet interface connecting to the LAN??. Based on the technical information that I manage, this command should be applied the Tunnel0 interface. Thanks in advance for your final comments
04-30-2012 05:53 PM
Disclaimer
The Author of this posting offers the information contained within this posting without consideration and with the reader's understanding that there's no implied or expressed suitability or fitness for any purpose. Information provided is for informational purposes only and should not be construed as rendering professional advice of any kind. Usage of this posting's information is solely at reader's own risk.
Liability Disclaimer
In no event shall Author be liable for any damages whatsoever (including, without limitation, damages for loss of use, data or profit) arising out of the use or inability to use the posting's information even if Author has been advised of the possibility of such damage.
Posting
TCP adjust should work anywhere along the transit path, where it's supported. I normally prefer placing on tunnel interfaces as that limits MTU reduction to just the traffic that needs it.
04-30-2012 11:52 AM
I found the following in the link:
http://www.cisco.com/en/US/docs/ios/12_2sb/12_2sba/feature/guide/sb_admss.pdf
-At the end of page 7:
If you are configuring the
ip mtu command on the same interface as the ip tcp adjust-mss
command, it
is recommended that you use the following commands and values:
•
ip tcp adjust-mss 1452
•
ip mtu 1492
At the beginning of page 8:
The ip tcp adjust-mss
command does not work on subinterfaces or GRE tunnels.
04-30-2012 05:50 PM
Disclaimer
The Author of this posting offers the information contained within this posting without consideration and with the reader's understanding that there's no implied or expressed suitability or fitness for any purpose. Information provided is for informational purposes only and should not be construed as rendering professional advice of any kind. Usage of this posting's information is solely at reader's own risk.
Liability Disclaimer
In no event shall Author be liable for any damages whatsoever (including, without limitation, damages for loss of use, data or profit) arising out of the use or inability to use the posting's information even if Author has been advised of the possibility of such damage.
Posting
Abraham Camacho wrote:
I found the following in the link:
http://www.cisco.com/en/US/docs/ios/12_2sb/12_2sba/feature/guide/sb_admss.pdf
-At the end of page 7:
If you are configuring the
ip mtu command on the same interface as the ip tcp adjust-mss
command, it
is recommended that you use the following commands and values:
•
ip tcp adjust-mss 1452
•
ip mtu 1492
That's for "native" PPoE.
At the beginning of page 8:
The ip tcp adjust-mss
command does not work on subinterfaces or GRE tunnels.
From the Cisco document Peter referenced:
Use the ip tcp adjust-mss command on the tunnel interfaces so that the router will reduce the TCP MSS value in the TCP SYN packet.
04-30-2012 11:58 AM
I forgot to mention that this command was initially applied by someone else on fastethernet subinterfaces.
04-30-2012 02:15 PM
More about this case:
I configured on the remote branch the following but I am still getting fragmentation (see below)
Remote Router:
interface Tunnel0
description ** Tunnel Central Site **
bandwidth 3000
ip address 10.1.1.1 255.255.255.252
ip accounting output-packets
ip mtu 1400
ip flow egress
ip wccp 62 redirect in
ip tcp adjust-mss 1360
keepalive 10 3
tunnel source 10.1.2.2
tunnel destination 10.1.2.124
Remote-Router#show ip traffic | in fragmented
2062 fragmented, 4124 fragments, 0 couldn't fragment
Remote-Router#show ip traffic | in fragmented
2066 fragmented, 4132 fragments, 0 couldn't fragment
Remote-Router#show ip traffic | in fragmented
2068 fragmented, 4136 fragments, 0 couldn't fragment
Remote-Router#
When I analyzed the IP ROUTE CACHE FLOW for UDP packets (protocol 11-hex) there is an increment which I assume are related with the fragmentation shown above because MTU = 1400 is less than UDP = 1500 packets which are not reduced by MSS Adjust and must be fragmented.
Joseph/Peter, what is your opinion abou this situation?
regards
Abraham
04-30-2012 05:56 PM
Disclaimer
The Author of this posting offers the information contained within this posting without consideration and with the reader's understanding that there's no implied or expressed suitability or fitness for any purpose. Information provided is for informational purposes only and should not be construed as rendering professional advice of any kind. Usage of this posting's information is solely at reader's own risk.
Liability Disclaimer
In no event shall Author be liable for any damages whatsoever (including, without limitation, damages for loss of use, data or profit) arising out of the use or inability to use the posting's information even if Author has been advised of the possibility of such damage.
Posting
Did you try enabling tunnel path-mtu-discovery?
05-01-2012 09:04 AM
Hello Abraham,
I apologize for answering somewhat lately but Joseph has stepped in more than adequately. Joe, thank you!
Regarding the applicability of ip tcp adjust-mss command on Tunnel interfaces, the document you referenced that states it won't work is relevant for an old IOS version. I do not believe that this limitation exists anymore in the IOS version you are running. I had a quick look at the command reference for IOS 12.4:
http://www.cisco.com/en/US/docs/ios/ipapp/command/reference/iap_i2.html#wp1086889
It does not mention any similar limit. So using this command on a GRE tunnel should be fine.
Regarding the remaining fragmentation still reported by the router, I would personally assume that it is the UDP traffic being fragmented. Whether the MSS reduction works can be verified relatively simply by installing Wireshark on a client computer and accessing some TCP services across the GRE tunnel. The MSS observed in the TCP responses coming from across the tunnel must be reduced to 1360 or less.
The tunnel path-mtu-discovery command suggested by Joseph is a good idea. However, it is not going to prevent fragmentation, rather, it is going to dynamically discover the maximum MTU that can be used by a tunnel - provided that ICMP packet-too-big messages are permitted to be delivered to your tunnel endpoint. Sadly, many firewall policies inadvertently filter out ICMP communication and break the Path MTU Discovery process.
Best regards,
Peter
05-01-2012 09:48 AM
Peter / Joseph, I am very grateful with both, thank you so much. Additionally, I agree with you regarding that UDP traffic is the one being fragmented. However, I would like to share with you the last tests that I made using PMTUD + IP MTU = 1400 on both ends (without MSS ADJUST). This are the results that I would like to share with you.
When I made SSH (tcp=22) from a PC in the internal LAN at the Central Site to the IP address of the Router LAN Interface at the Remote Site, passing through the GRE TUNNEL connecting both sites, I got a MSS of 536 bytes (similar to the explanation presented in the following link) but, why this situation if I already activated PMTUD, so the MSS should be 1400 - 40 = 1360???
http://www.cisco.com/en/US/tech/tk331/tk336/technologies_tech_note09186a00801d3a9d.shtml
My final question is: How the MSS for TCP Traffic going through the GRE Tunnel is actually set to the IP MTU configure on both ends of the GRE Tunnel - 40 bytes of headers??.
SHOW TCP command applied at the REMOTE SITE Router
tty516, virtual tty from host 10.1.55.38 (PC LOCATED AT THE CENTRAL SITE INTERNAL LAN)
Connection state is ESTAB, I/O status: 1, unread input bytes: 0
Connection is ECN Disabled, Mininum incoming TTL 0, Outgoing TTL 255
Local host: 10.202.144.2, Local port: 22 IP ADDRESS LAN interface REMOTE SITE
Foreign host: 10.1.55.38, Foreign port: 53256
Connection tableid (VRF): 0
Enqueued packets for retransmit: 0, input: 0 mis-ordered: 0 (0 bytes)
Event Timers (current time is 0x1AEFDCEB4):
Timer Starts Wakeups Next
Retrans 13 0 0x0
TimeWait 0 0 0x0
AckHold 11 1 0x0
SendWnd 0 0 0x0
KeepAlive 16 0 0x1AEFE7DC8
GiveUp 0 0 0x0
PmtuAger 0 0 0x0
DeadWait 0 0 0x0
Linger 0 0 0x0
ProcessQ 0 0 0x0
iss: 3992905895 snduna: 3992907084 sndnxt: 3992907084 sndwnd: 64940
irs: 1480291885 rcvnxt: 1480292290 rcvwnd: 3724 delrcvwnd: 404
SRTT: 271 ms, RTTO: 827 ms, RTV: 556 ms, KRTT: 0 ms
minRTT: 28 ms, maxRTT: 524 ms, ACK hold: 200 ms
Status Flags: passive open, active open
Option Flags: keepalive running, 0x1000000
IP Precedence value : 6
TCB is waiting for TCP Process (220)
MSS FOR THIS TELNET CONNECTION (TCP PORT)!!!! LESS THAN THE MTU 1400 - 40 = 1360
Datagrams (max data segment is 536 bytes):
Rcvd: 17 (out of order: 0), with data: 11, total data bytes: 404
Sent: 15 (retransmit: 0, fastretransmit: 0, partialack: 0, Second Congestion: 0), with data: 13, total data bytes: 1188
Packets received in fast path: 0, fast processed: 0, slow path: 0
fast lock acquisition failures: 0, slow path: 0
FHR-COP-WAN#Discover and save your favorite ideas. Come back to expert answers, step-by-step guides, recent topics, and more.
New here? Get started with these tips. How to use Community New member guide