cancel
Showing results for 
Search instead for 
Did you mean: 
cancel
1846
Views
0
Helpful
3
Replies

Cisco C3850 stack configuration with Dell Equllogic SAN's

Joel Fox
Level 1
Level 1

Good afternoon! I am currently working on a project to isolate our Dell Equallogic ISCSI SAN traffic to a completely isolated c3850 stack.  Dell recommends enabling jumbo frames on the switch - which I was told that the 3850's can do at the port level, but I haven't found any documentation stating that it is possible. All I've found so far (and have configured) is setting the system MTU to 9160 to support jumbo frames. My first question being is this the best configuration to support jumbo frames, or should I be doing this a different way? My second question is regarding stacking vs not stacking. From what I've read, stacking is the way to go to prevent potentially creating a bottle neck between the switches if configured improperly.

Another question is the use of the management interface. The purpose of isolating the switches is complete segregation of the ISCSI traffic, but I need the ability to remotely manage these switches. From what I'm reading the management interface shouldn't pass network traffic as a traditional switch port would. If that's the case, perfect! 

I am just now gathering all the information I need to plan and implement the change, but I would love some additional input :).  Here's a quick breakdown of what we have.

4 SAN's, all in one group, 6 NIC's each

3+ VM hosts minimum of 2 NIC's designated for ISCSI traffic

2 Cisco c3850 Switches (stacked) with the Ipbase image.

 

Any input is greatly appreciated, and thank you ahead of time!

 


 

3 Replies 3

Leo Laohoo
Hall of Fame
Hall of Fame

6 NICs per SAN and 4 SANs = 24 ports.  

 

Just note Catalyst switches, which the 3850 is one of them, is and was NEVER designed to support high-speed and hitless SAN traffic.  Even with a robust QoS in place there will be severe "penalties" in form of significant amount of packet drops.  

 

For SAN traffic the Nexus family of switches is the solution.

jsurak
Level 1
Level 1

This switch is really not recommended for iSCSI and isn’t included in the tested switches for use with EqualLogic Arrays.  See the Hardware compatibility matrix (http://en.community.dell.com/dell-groups/dtcmedia/m/mediagallery/20438558)

Typically, it will not deliver the expected performance for your SAN simple due to the fact that the C3850 switch is considered a “wiring closet” switch and not “Storage Network”/“Data Center”/”Fabric” designed switch (largely due to the buffer size for each port are small compared to storage network/data center switches).

So in theory, this switch might handle mostly random, small block IO, as long as this IO profile does not change in the future, but it would not be able to handle your iSCSI SAN traffic if you are doing a lot of large block IO (greater than 8K).  You will want to consider a more powerful switch.

The few items to consider;

  • Your SAN switches should be configured for iSCSI traffic exclusively and dedicated to the SAN network.Avoid using any of the other ports for non-iSCSI traffic unless for administration (i.e., its ok to add a port or two on a separate VLAN for any Management network or monitoring software of the switches, etc., but do not pass LAN or other user traffic across the switch).

     

  • Configure a separate VLAN for iSCSI and DO NOT use the default VLAN, the default VLAN is implementation and should be reserved just for administrative traffic (this is true regardless if setting up iSCSI).  It is recommended that all traffic be configured on VLANs other than VLAN 1, primarily to prevent unnecessary broadcast and multicast traffic from being processed by the Network Management Processor (NMP) of the supervisor.

     

  • Plug your Host iSCSI and Array cable connections staggered across two or more ASIC’s to avoid over subscribing the port and ACIS’s buffers.

  • Regarding stacking cable, verses ISL’s (trunk group), both work equally well.Stacking is much easier to implement and has some advantages during maintenance on the switches.Also, stacking typically has a dedicated port so you don’t lose 6x ports for the needed bandwidth required for your ISL’s (a simple calculation for EqualLogic ISL total is:you will need to setup at least 2 ISL ports per member dedicated to the trunking between the switches (note that 2 ports is the minimum ISL’s between switch for a single member group), however the ideal setup would have one ISL port per each active array port, so in your case you have 4 members each with 3 active per member, so 4x3= 12 ISL Ports are needed (or 12Gb’s per ISL), and if your SAN grows, you would lose an additional 12 ports to connect the next switch, as you can see ISL port use adds up quickly).Much better to use a stacking cable if possible.

 

For specific configuration on switches/hosts see:

http://en.community.dell.com/techcenter/storage/w/wiki/3615.rapid-equallogic-configuration-portal-by-sis

 

Use the configuration Guide for general information and setup guidelines:

http://en.community.dell.com/techcenter/storage/w/wiki/2639.equallogic-configuration-guide

 

The Storage Technical Solutions Publications and Resources is useful as well:

http://en.community.dell.com/techcenter/storage/w/wiki/2632.storage-infrastructure-and-solutions-team-publications

 

Regards,

Joe

Joe - thanks for the input. I realize that the Nexxus series are true datacenter switches, and for a typical true "datacenter" they should most certainly be used.  However, in our scenario, I believe these switches are going to be perfectly fine. Our SANS are only 4k, so there won't be any large block traffic. These switches are 100% isolated, and ONLY for ISCSI traffic in a vlan other than 1.

I went the stacking route simply because I didn't want to lose 8-12 ports per switch to support the dataflow.

In a perfect world, I would have Nexxus switches and this wouldn't even be a discussion. However, this is far from a perfect world and I have to work with what I've been provided :). 

Currently our SAN's are on a reachable vlan on our core switch (NOT by my design!) which is a 4506e. I can't imagine processing could be any worse on the 3850's with only ISCSI traffic on it.

Thanks again for your input!