what should be the expected performance of a Cisco switch?

[willgart1]

Hi,

My question is simple...

what's the expected performance I can reach between 2 servers connected to the same switch with a 1Gbps card?

here the why this question:

we have installed multiple servers in a blade infrastructure. (we = a team, not me directly)

they are using a cisco catalyst switch & routers etc...

but I'm not able to get more than 30% of throughput between my servers.

spikes to 50%.

because I dont have access to the data center etc... I dont know more than this.

in my past life, when I setup the network myself, I never fall below 80% of usage, specially during benchmark tests.

and when I optimised my network I was able to reached 95%, using dlink switches, not professional Cisco systems.

But I was never able to get less than 50% of thoughput even with a very bad configuration or a lot of traffic from different servers.

Now my problem is the team in charge of the network dont care about this, for them if the connection works that's it, the connection works. They never heard about benchmarking and optimising a network. more than this, they control the network, but its my job to identify what to do!!!! (I never configure a Cisco system, but they are certified for this)

and another more than this, they are using 2 networks: production & management; the management network is able to reach the 80%+ speed, its used for the backups and the admin accesses.

and again, the network team is not able to explain why prod = 30% and management = 80%

When I call them, they just check the number of errors and answer "I dont see any error, there is no problem"

when they send some stats to me with the global usage of the switch, there is no other traffic except the tests I made. so the network is far from behind fully used, and so no traffic sharing.

because we try to implement a datawarehouse solution, we have to move large amount of data between our servers, up to 300GB of data to move. this takes hours while I expect minutes...

so first, I want to know what's the speed of an out of the box config and what we can get if we are able to optimize this.

And what's the admins should monitor to identify the bottlenecks?

any guide or white paper?

which question I have to ask to help me identifying the issue?

thanks for your comments.

[Andras Dosztal]

Did you enable jumbo frames?

It would be greate if you could show a topology and the used equipment.

[willgart1]

activating the jumbo frame was 1 of the small change I was able to convince them.

and here a funny part...

first I do a simple file copy test, and the jumbo frame indicates an improvement of 15%.

but for some reasons, transfering the data to the cube, so real usage scenario, and the performance drops from 30% to 10% !!!

unfortunately I dont had much time to fine tune and test more, the network team decide to disable the jumbo frames as 1 of the servers is outside the current switch and they dont want to activiate the jumbo frames on these switches.

welll... very frustrating when you dont have access and a better control on the activities and config...

but yes, generally I activate the jumbo frames for my servers.

[Joseph W. Doherty]

Disclaimer

The   Author of this posting offers the information contained within this   posting without consideration and with the reader's understanding that   there's no implied or expressed suitability or fitness for any purpose.   Information provided is for informational purposes only and should not   be construed as rendering professional advice of any kind. Usage of  this  posting's information is solely at reader's own risk.

Liability Disclaimer

In   no event shall Author be liable for any damages whatsoever (including,   without limitation, damages for loss of use, data or profit) arising  out  of the use or inability to use the posting's information even if  Author  has been advised of the possibility of such damage.

Posting

Cisco switch performance capabilities can vary greatly depending on actual switch model, and if a chassis, the line card being used.  Some are even sensitive to port usage (i.e. whether nearby ports share the same ASIC or other system resources).  With chassis models, some gig line cards were sold as user edge line cards (high over subscription ratios) and some were sold as server edge line cards (not so high over subscription ratios) and some were sold as line-speed/wire-rate capable.

Beside that, sometimes tuning of parameters is necessary, as one size doesn't always fit all.  This because, especially on older models, there's often insufficient hardware resources to support all ports at full sustained capacity when such resources are allocated equally.  In those case you might reallocate resource from low usage ports to heavily used ports.

PS:

If you're doing high transfer rates between two servers, on the same switch, and your transfers are using TCP, setting TCP receive window to the specific necessary BDP (bandwidth delay product) or even smaller, might actually improve your effective transfer rate. This assuming your poor performance is caused by too many drops.

[willgart1]

thanks for the info.

I tell the network team about this configuration (TCP receive window).

[Joseph W. Doherty]

Disclaimer

The Author of this posting offers the information contained within this posting without consideration and with the reader's understanding that there's no implied or expressed suitability or fitness for any purpose. Information provided is for informational purposes only and should not be construed as rendering professional advice of any kind. Usage of this posting's information is solely at reader's own risk.

Liability Disclaimer

In no event shall Author be liable for any damages whatsoever (including, without limitation, damages for loss of use, data or profit) arising out of the use or inability to use the posting's information even if Author has been advised of the possibility of such damage.

Posting

TCP receive window, would be set on your hosts - not any of the network equipment (actually it can be adjusted on some Cisco network devices, but that's for the network device, itself, acting as a TCP host).