tcp window exhaustion 9372 and 3172 Nexus

[lecktor10]

Greetings,

We have a small network (think nexus 9372 and 3172 w/ paloalto FW in a connection triangle with VMware 6.7 Gen10 HP servers) our connections are supposed to be 10 Gbps.  we are getting about 70-80 Mbps. Our wireshark captures show that we are seeing tcp window exhaustion. I went through lots of googling and it looks like with my switches there aren't alot options. Anyone else have this problem? should I look to the servers instead of the switches? would changing the MTU help or would that create even more trouble. 

Thank you for reading.

[balaji.bandi]

where do you see this error  "tcp window exhaustion"  Palo side or Switch side ? where is your capture taken ?

is CEF enabled ?  if you like you can change - ip tcp window size ZZZZ

But before changing make sure we understand the problem correctly.

[lecktor10]

Hi,

Capture was taken on the server side on a win 2012 r2 virtual server running wireshark. the capture was of 1 oracle linux box ftp-ing data from another one. 

[balaji.bandi]

what is the MTU Settings on all the way to Server. ? try tweak.

[Reza Sharifi]

Hi,

Have a look at this Wireshark document. It could possibly be an issue on the server-side as well.

The connection gets reset by the Windows server after having exhausted its re-transmission retries trying to get the full size 1448 bytes segments to the Linux client.
This is most probably due to the MTU size available along the route being smaller than 1500, which is what both sides have defined.
Assuming have control over the server you could manually reduce the MTU size to let's say 1400 to circumvent this problem

Link;

https://ask.wireshark.org/question/2690/what-causes-a-connection-reset-after-retransmission-and-tcp-window-update/

HTH

[Joseph W. Doherty]

BTW unless your switch (or router) is the receiving TCP host, its TCP RWIN size is not relevant.  Also, I recall [?] the usual Cisco default for RWIN is only 4 KB.  Some Cisco switches (or routers) support the "ip tcp window-size" configuration command which allows you to change the size of the RWIN.  (Don't recall if Cisco supports scaled TCP RWINs, i.e. RWIN larger than 64 KB.)  However, don't know any option to manage resources to support it.  I presume RWIN buffers are just allocated from free RAM.

Lastly, even if RWIN is "small", each TCP session gets its own, I believe.