Why do Cisco IOS devices set the MSS to 536 for remote networks?

[Mitrixsen]

Hello, everyone.

I am covering MSS at the moment and I've read that Cisco IOS devices set the MSS to 536 for TCP connections to remote networks.

From R1 to R2
R1#telnet 10.0.0.2
Trying 10.0.0.2 ... Open
*Jun 16 11:39:18.237: TCP0: Connection to 10.0.0.2:23, advertising MSS 2460
*Jun 16 11:39:18.238: TCP: tcb 10E2A330 connection to 10.0.0.2:23, peer MSS 2460, MSS is 2460

From R1 to R3
R1#telnet 10.0.0.6
Trying 10.0.0.6 ... Open
*Jun 16 11:40:16.232: TCP0: Connection to 10.0.0.6:23, advertising MSS 536
*Jun 16 11:40:16.236: TCP: tcb 10E2A330 connection to 10.0.0.6:23, peer MSS 536, MSS is 536

My question is, why does it work like this? One reason I can think of is that when you initiate a TCP connection to a device on the local network, you know what the MTU of the local segment is, while with remote networks, there could be different MTUs along the path so you play it safe by using a lower MSS value.

However, isn't limiting yourself by so many bytes a little unefficient? If your capable of sending that much more..

Thank you!

David

[Joseph W. Doherty · Accepted solution]

Recall IP requires minimum MTU of 576.  This is used unless PMTUD is enabled.

[Mitrixsen]

Hello.

Right, I completely forgot about this. Once I enable PMTUD by using the ip tcp path-mtu-discovery command, the Cisco router uses an MSS of 1460 (since 1500 is configured on its interface) now instead of 536 so that solves the problem, thanks guys.

[M02@rt37]

Thanks for you feedback !

[Joseph W. Doherty]

Yup.

Another configuration command, that can improve some data reception on Cisco devices is increase TCP RWIN to, at least, 64k.

BTW, although many readers may know I've a keen interest in QoS, my real underlying interest has always been performance, because IMO, "computer systems should wait on people, people should not wait on computer systems".

[MHM Cisco World]

!!!

MHM

[Joseph W. Doherty]

Hmm, haven't tried adjusting MSS to 1400.  Have enabled PMTUD which will start with interface MTU and reduce if notified.

Possible issue with just bumping MSS (besides the obvious it may be too large end-to-end), might still retain 576 for remote networks, using IP fragmentation to contain the larger MSS.  Have you confirmed what's actually transmitted?

[MHM Cisco World]

The "ip tcp adjust-mss" command is effective only for TCP connections that pass through the router. The following example shows how to set ip tcp mss on an interface

NDNA(config)#interface ethernet0/1

NDNA(config)#ip tcp mss 1460

https://www.thenetworkdna.com/2020/09/all-about-ip-mtu-and-ip-tcp-mss.html?m=1

The mss is adjusted by R2 not by R1 not R3 interface which fit what I mention before since traffic (between R1 and R3) is now pass through R2 not originate from R1 nor R3.

Pmtud is not recommend by all vendors since many vendors drop icmp use for it.

Anyway you ask and I reply 

And he select answer 

So 

Goodluck for all

MHM

[Mitrixsen]

I was asking about why do Cisco IOS devices set the MSS to 536 for remote connections and what was the reason behind it which Joseph.W. Doherty answered, that's why I haven't picked your answer as the solution. I know how to configure the MSS and also how the adjustment works,

[MHM Cisco World]

As you like 
MHM

[Joseph W. Doherty]

BTW, the tcp adjust-mss command (I wondered if that's what you had in mind). I recall, can only decrease MSS from what's seen during TCP session setup. So, for example, if one host says it only supports a MSS of 536, it will not change it to 1400.

"Pmtud is not recommend by all vendors since many vendors drop icmp use for it."

Oh, can you provide any references?  It's not a recommendation I would expect (because of issue described in this posting).  However, I could see a recommendation not to use DF (don't fragment) setting, although I wouldn't agree with either "recommendation".

[M02@rt37]

@Mitrixsen 

Cisco IOS devices set different MSS values for local and remote connections due to the varying MTU sizes across network paths. For local networks, such as the connection from R1 to R2, the device can assume a consistent MTU size and therefore advertises a larger MSS (e.g., 2460 bytes), which is more efficient as it reduces overhead and increases throughput by allowing larger segments of data to be sent.

In contrast, for remote networks, like the connection from R1 to R3, the path might traverse multiple network segments with varying MTU sizes. To avoid issues related to fragmentation, which can occur if packets exceed the smallest MTU in the path, a smaller MSS (e.g., 536 bytes) is advertised.

This conservative approach ensures reliable data transmission across diverse network paths, even if it might seem inefficient due to sending smaller segments. This trade-off prioritizes reliability over maximum efficiency, as avoiding fragmentation reduces potential performance issues such as increased latency and packet loss. While it might seem inefficient to limit the MSS by so many bytes when capable of sending more, this method ensures compatibility and reliability across heterogeneous networks, where the exact MTU might be unknown or inconsistent. For optimal performance, MSS values can be manually adjusted when the network path and its MTU are well understood and consistent, balancing the need for efficiency with the necessity of reliability.

[Joseph W. Doherty]

"Cisco IOS devices set different MSS values for local and remote connections due to the varying MTU sizes across network paths."

What any host should do is use interface MTU for interface connected network (as all hosts on local network should be using same MTU).  For non-local networks, MTU should be 576 w/o PMTUD.

From RFC 791:

"All hosts must be prepared to accept datagrams of up to 576 octets (whether they arrive whole or in fragments). It is recommended that hosts only send datagrams larger than 576 octets if they have assurance that the destination is prepared to accept the larger datagrams."

Interestingly, the max MTU is based on destination's capacity, not transit path.  This because IP, by default, can/will use fragments.

So, unless we know what a remote host can accept for MTU, we cannot transmit larger than 576.  Again, a local destination (same L2 domain segment) should support transmitter's MTU.

BTW, on routers, enabling PMTUD can do wonders for BGP.

Oh, it can improve other Cisco device network data transfers too, but to "see" the improvement it needs much data and isn't otherwise bottlenecked.  For example, something like show tech, w/o paging, might go faster.  Something, like FTP copies, to/from flash, tend to be bottlenecked by flash R/W performance.

[M02@rt37]

@Joseph W. Doherty 

Absolutely. Thank you for clarifying the MTU considerations based on RFC 791. Indeed, hosts should adhere to the recommended maximum MTU size of 576 octets unless they have assurance that the destination can accept larger datagrams. This is particularly crucial for non-local networks where PMTUD might not be reliable.

Enabling PMTUD on routers can indeed improve the efficiency and reliability of BGP and other protocols by dynamically adjusting the maximum segment size based on the discovered path MTU. This helps avoid fragmentation and potential issues related to oversized packets.

[MHM Cisco World]

Did you ever try my suggestion?

No I dont think so.

So I will remove my reply 

Goodluck 

MHM