Flow-Control and throughput

[jorge.calvo]

Hello,

I have a strange situation on one of my customer´s Data Centers:

There are some servers connected to a C6509 switch via GigabitEthernet links set up at 1000/FD. I have observed the next difference regarding the throughput:

- If flow-control is disabled on both the switch ports and server ports the maximum throughput is 200Mbps.

- If flow-control is enabled on both the switch ports and server ports the maximum throughput is 900Mpbs.

Taking in count there are no capacity/congestion problems on the switch and all the ports are error free and they have no packet drops, is there any reason why this difference exist?

Regards.

[Calin C.]

flow control is the process of managing the rate of data  transmission between two nodes to prevent a fast sender from outrunning a  slow receiver. It provides a mechanism for the receiver to control the  transmission speed, so that the receiving node is not overwhelmed with  data from transmitting node.

If your device have flowcontrol enabled, and you disable it on the switch, than you can run into a mismatch that can cause your slow throughput rate. Depending on the device connected in the switch you have to enable flowcontrol (in your case this is true) or disable it. On your Cisco switch you can also configure receive desired and send desired...so only if the device connected there need flowcontrol, Cisco switch will enable it.

More about configuration here:

http://www.cisco.com/en/US/docs/switches/lan/catalyst5000/catos/4.5/configuration/guide/gigabit.html

if you find this info useful, please rate!

[jorge.calvo]

Thanks for you response.

But what is strange to me is that I have no mismatch on the flow-control configuration. The slow throughput occurs when flow-control is disabled on both the switch and the server.

Regards.

[Calin C.]

It might be that one device is sending at a rate higher than the other device can accept, and then you have this behavior with flowcontrol off. Flowcontrol help those devices realize that they can "push" more data than actually they would. I'm not expert with servers, but I believe this behavior is caused by the server side, as if you connect one router  and one device together without any software limitation, they will send/receive at maximum rate.

Calin

[jorge.calvo]

It makes sense to me because only some type of servers (NAS) are experiencing this issue. Probably flow-control improves the performance for FCP communications. I am chasing the server team to provide me more details about the servers applications and protocols.

Regards.

[Calin C.]

Btw, are you using Jumbo frames (MTU 9216)? If yes, have you configured them on both sides switch and server (if supported). If you have a mismatch there, you can run into low throughput due to fragmentation. Just and idea that might help.

[jorge.calvo]

Jumbo frames are not being used. However the problem resides in the servers NIC itself. The problem was a misconfiguration on the NIC parameter Interrupt Moderation which needs to be taken in count for NAS traffic flows due to the high amount of data.

http://download.intel.com/design/network/applnots/ap450.pdf

Regards.

[Giuseppe Larosa]

Hello Calin,

at ethernet level there is no fragmentation a frame can be accepted or discarded as giant

Hope to help

Giuseppe