TCP Performance Tuning

[s.wetzel]

I am interested in improving TCP performance on high latency, long WAN circuits. I am researching commands relating to the following TCP areas:

- Maximum-receive-segment-size option

- Window-scale option

- ip tcp path-mtu-discovery

- maximum segment size (MSS) for TCP connections

Questions: 1) Do all of these commands improve the TCP performance of end-systems at sites communicating over WAN links? The command reference seems to imply that some of the commands are only for traffic originating on or destined for a router. 2) Does the end systems at each site need to have any adjustments to the TCP stack (tweaks)? 3) Is there any downside to implementing these commands, e.g., router resources are negatively impacted? 4) Are there any other commands or options to achieve improved performance over long, high-latency WAN links?

Thank you.

[lrian]

Tuning needs to be done on the end stations

A good starting point is

Enabling High Performance Data Transfers

at

http://www.psc.edu/networking/projects/tcptune/

[Richard Burts]

While there are some factors that would favor making any tuning done on the end station, there is a strong factor again tuning on the end station for many of us. And that factor is that it does not scale well. If I support a network with 1,000 end stations and a small number of routers, it is not feasible to make tuning changes on 1,000 end stations while it is feasible to make tuning changes on the routers.

HTH

Rick

[ggatten]

I may be off base, but "tuning the routers" won't help much unless they're doing some sort of proxying. TCP sessions are terminated on the sender and receiver (unless proxy of some sort) and therefore the tuning must be done there.

IMO the primary tweaks revolve around window size, SACK, and MTU. Most newer OS's are tuned pretty well, but it never hurts to double check and validate with a network trace of somekind.

If you have delay sensitive traffic - such as VoIP - mixed in with other traffic, you may not be able to use the largest MTU/MSS possible. But, general rule is always use the largest possible for the media. Various implementations of QoS may fragment larger packets to avoid taking up too much time on the wire.

Anyway, gotta run. HTH

Gary

[Richard Burts]

Gary

There is at least one exception. The ip tcp adjust-mss configured on the router interface will insert the configured value into the mss field of the TCP SYN packet and effectively over-ride the value set by the end station. It is not a proxy, but it effective on the router to affect the session to the end station.

HTH

Rick

[ggatten]

Interesting. It actually affects (rewrites) the handshake packets? Or, does it simply fragment the packets before forwarding them - such as an LFI implementation would?

Gary

[Richard Burts]

Gary

Yes it actually affects the handshake packet.

HTH

Rick

[kbelter]

Actually you will still need to touch the station and server endpoints. In most OS (most notibly Microsoft) the default TCP window size is set to a max of 8K. With that small of a buffer and 200ms latency, your application will be slowed down. If the OS has a max and the TCP session setup reports greater (for the other end), then least common denominator is what is used.

Use this equation (Bandwidth x Latency = Buffer). For a line with 768K port and 250ms latency this would be ((768 Kb/s) x (1000b/Kb) x (1Byte/8bits) x (250s/1000s) = 24KB). You can fix the network but you still have to fix the client/server - unless you also do a proxy.

[ggatten]

FWIW - the default window size actually depends on the media MTU - then * 4, up to a max of 65535 unless window scaling is active. So, on TokenRing, FDDI, etc. - your window size could be much larger than 8K, but in the Ethernet World - less Jumbo Frames on Gig - 8K is the default.

Anyway, RFC 1323 talks about most of this stuff:

http://www.faqs.org/rfcs/rfc1323.html

Also has links to other URLS for other tunings, such as SACK.

Enjoy!

Gary