6500 Blades for ISCSI and High Bandwidth Utilization Applications

[thefindjack]

I have recently run into a problem with Oracle RAC and the amount of data that was being generated on the interconnect (between RAC nodes) connections. The interconnect links all happened to lie on the same port ASIC within a WS-X6548-GE-TX (Ports 17-24) so I moved the interconnect links all to different groups of 8 ports (so they each are on a different ASIC) and this resolved the issues I was having. Now my question....Is there a blade that has a single ASIC (or RX/TX queues) per port or is specifically meant for high traffic/bandwidth applications. We are going to start running ISCSI and I want to make sure I dont run into an issue like this again. I have found this document...

http://www.cisco.com/en/US/prod/collateral/switches/ps5718/ps708/product_data_sheet0900aecd8017376e.html

It looks like the WS-X6548-RJ-45 has its own queue per port but it only supports 100Mb traffic. Is there a 10/100/1000 blade that has 1:1 queues???

[Jon Marshall]

David

"Is there a blade that has a single ASIC (or RX/TX queues) per port or is specifically meant for high traffic/bandwidth applications"

I'm not aware of a blade that has 1:1 port to ASIC but there are much better blades than the WS-X6548-GE-TX which is not suited to your needs at all.

The WS-X6548-GE-TX only has a single 8Gbps connection to the switch fabric so that is 48Gbps ports with only an 8Gbps connection to the fabric.

You don't say which supervisor you have but assuming a Supervisor 720 then a good choice would be the WS-X6748-GE-TX which supports 2 X 20Gbps to the switch fabric so there is the possibility of some contention but not much.

Another alternative you may want to look at is to have dedicated switches such as the 4848's purely for the RAC interconnect. These are wire speed switches.

Jon

[thefindjack]

Thanks for your reply. I am using a Sup 720 in both my Core switches. Do you know how the WS-X6748-GE-TX distributes that bandwidth? (i.e every 8 ports have a 1Gb fabric connection in WS-X6548-GE-TX)

[Jon Marshall]

David

This is the distribution on the WS-X6748-GE-TX -

Dual switch-fabric connections

Fabric Channel #1: Ports 25-48

Fabric Channel #2: Ports 1-24

•Number of ports: 48

Number of port groups: 4

Port ranges per port group: 1-12, 13-24, 25-36, 37-48

The key thing to note is that there are 2 fabric channels ie. connections to the switch fabric. On the WS-X6748-GE-TX each fabric channel is 20Gbps.

So each 24 ports share a 20Gbps connection to the switch fabric. So as you can see you could have 20 ports out of each group running at 1Gbps before you get contention. Much better than the 6548.

Jon

[thefindjack]

Thanks, exactly what I was looking for!

[pingwarrior]

Hi Jon,

You have a very great and detailed explanation. May I ask where can I get those informations from as I couldn't find in cisco.com? Thanks

[Joseph W. Doherty]

A couple of other considerations.

If you're truly pushing so much bandwidth, you might also need to insure your line cards have DFCs.

For iSCSI, you might also investigate 10 gig.