3750X Stack to EMC SAN Design Question

[bbuadministrator]

Hi Folks,

I'm looking to upgrade my single 3560X iSCSI switch to a new pair of 3750-X Stacked. The switch connects my 2 VMware servers to my EMC VNXe3300. This is a very common setup. The EMC VNXe3300 is using all Gig ports on each controller. Here's more information:

VMware Server A/B----------->connect to single switch 3560x (configured 9000 Jumbo Frames globally)------------------->EMC SAN Controller A = 4x1 Gig Ports/Controller B = 4x1 Gig Ports

In a new 3750-x Stacked environment, what would be the best design and configuration to replace a setup like this?

I'm trying to accomplish redundancy, efficiency and added performance. Your guidance is greatly appreciated.

Thanks!

[Liam Kenneally]

Hi bbuadministrator,

With a pair of 3750Xs you would multihome them to get the best redundancy & thru-put... see below

Server A Nic Port (1 - 2) ----->  3750X(A) G0/47 - 8 

Server A Nic Port (3 - 4) -----> 3750X(B) G0/47 - 8

Server B Nic Port (1 - 2) -----> 3750X(A) G0/45 - 6

Server A Nic Port (3 - 4) -----> 3750X(B) G0/45 - 6

You will then create  2 LACP  etherchannels one for server A one for server b

LACP etherchannel 1 will have G0/47 - 8(SWA) & G0/47 - 8(SWB)

LACP etherchannel 2 would have G0/45 - 6(SWA) & G0/45 - 6(SWB)

This will give a 4gb thru-put unless a switch in the stack fails at which point the thru-put will drop to 2Gb.

Ofcourse the VMware infrastructure will have to support LACP link aggregation.

3750X(A) G0/1 - 4 -----> Controller A Ports 1 - 4

3750X(B) G0/1 - 4 -----> Controller B Ports 1 -4

Now I don't know much about EMC SANs but if both controllers are in a  single chassis then you can bundle all 8x 1gb ports into a 3rd LACP  etherchannel and have and 8gb pipe unless one controller/switch fails at  which point the thru-put drops to 4gb. Again this is assuming EMC supports LACP link aggregation.

IF the two controllers work indervidually, you would take the same steps that I have done with the server. And have 4 LACP Etherchannels.

Here is a document of cross-stack Etherchannels.

http://www.cisco.com/en/US/products/hw/switches/ps5023/products_configuration_example09186a00806cb982.shtml

HTH

Kind Regards,

Liam

[bbuadministrator]

Liam,

I appreciate your information! I believe there is a small error in your details. Correct me if I'm wrong but shouldn't it be like this?

Server A Nic Port (1 - 2) -----> 3750X(A) G0/47 - 8

Server A Nic Port (3 - 4) -----> 3750X(B) G0/47 - 8

Server B Nic Port (1 - 2) -----> 3750X(B) G0/45 - 6 *I put a correction of "B: here*

Server B Nic Port (3 - 4) -----> 3750X(A) G0/45 - 6 *I put a correction of "B" and "A" here*

The EMC SAN has two independant controllers so they can't be grouped together. Are connections to SANs usually LACP etherchannels? How would you setup in this place?

[Liam Kenneally]

Hi bbuadministrator,

Yes you are correct that is what happens when you copy and paste

As stated above if their independant you would multihome them similar to what I did with the servers... I am by no means a storage expert but am pretty sure EMC will support LACP...

My configuration would be something like this ...

interface GigabitEthernetx/x/x(interface(s) for 3750XA)

description EMC_CONTROLLER_A

switchport trunk encapsulation dot1q

switchport trunk native vlan X

switchport mode trunk

speed 1000

duplex full

channel-protocol lacp

channel-group 1 mode active

!

interface GigabitEthernetx/x/x(interface(s) for 3750XB)

description EMC_CONTROLLER_A

switchport trunk encapsulation dot1q

switchport trunk native vlan X

switchport mode trunk

speed 1000

duplex full

channel-protocol lacp

channel-group 1 mode active

HTH

Kind Regards,

Liam

[bbuadministrator]

Liam,

Let me clarify what you are saying to do on the SAN:

LACP #3

3750X(A) - G0/43 - 4 -------> Controller A Gi0/1 - 2

3750X(B) - G0/43 - 4 -------> Controller A Gi0/3 - 4

LACP#4

3750X(A) - G0/41 - 2 -------> Controller B Gi0/1 - 2

3750X(B) - G0/41 - 2 -------> Controller B Gi0/3 - 4

Does this look correct? If so, if switch #A fails, that means I have 2 GB throughput correct?

[Liam Kenneally]

Hi,

Yea you got it :) ...

Both switches up abd running you will have 4Gb thruput... In the event switch A or B fail you will have 2Gb thruput.

The great thing about LACP is also it has error loggin. So if you have a syslog server LACP will log that it has lost Switch A or Bs interfaces from the portchannel.

Kind Regards,
Liam


Sent from Cisco Technical Support Android App

[bbuadministrator]

Liam,

Thanks for your response. Why wouldn't I plug in all SPA links to Switch A and all SPB links to Switch B?

What' the downside to that if any?

Please let me know!

Thanks!

[Liam Kenneally]

Hi bbuadministrator,

What is SPA and SPB?

Kind Regards,

Liam

[bbuadministrator]

SPA is Storage Processor A or Controller A (regarding SAN) and vice versa for B.

-Thanks

[Liam Kenneally]

Hi bbuadministrator,

Oh got you   ...

To be honest the reason I configured 2ports of each controller to each switch was as followers...

I was not sure if controller A & B were redundant controllers for a single Chassis... Or if Controller A was to one SAN and Controller B was to another SAN...

As stated limited knowledge of EMC ...

You are correct in saying SPA ---> SwA  |  SPB ----> SWB.... this is assuming SPA & SPB are redundant fabric for a single chassis does this make sense? This is in my opinion the most straightforward way to do things

But if SPA & SPB are two controllers for a single chassis, I don't see why you cant bundle all 8interfaces? Or am I missing something here?

Kind Regards,

Liam

[bbuadministrator]

Yes Controller A and Controller B are on the same SAN and they are configured to be redundant in case one fails the other automatically picks up.

I'll look into bundling all 8 interfaces. But for now, it is safe to say that the prefferred way of setting up the links between 3750-X stacked and the VNXe3300 SAN is to point all iSCSI SPA Ports to SwA and all iSCSI SPB Ports to SwB with no LACP or with LACP?

[Liam Kenneally]

Hi bbuadministrator,

Oh ok so the controllers are Active/Standby one kicks in when one fails...

Now this makes sence! I was under the impression both Controller A & B were Active(simultaneously passing traffic) Hence bundling all eight interfaces...

You will want to setup as show in first topology with LACP!

Kind Regards,

Liam

[Liam Kenneally]

Hi bbuadministrator,

First of all please ignore the terrible diagrams no Visio on my laptop!

This is the topology we have agreeded on...

However I believe you could configure it like this and save yourself the configuration of 4 etherchannels

Let me know what you think.

Kind Regards,

Liam

[bbuadministrator]

Your design looks good. The only thing is that those EMC Controllers look funny from a logical point of view! This is the visio I created..no offense to yours! Let me know if this is OK.

[Liam Kenneally]

Hi bbuadministrator,

No offence taken, its shocking!   I was using some free online topology creator(as this laptop doesn't have visio)

You topology looks spot on!

This is exactly how I would configure it.Of-cource drawing it is once thing implementing is another... I would pre-configure the stack with all the settings and just swap out the 3560Xs in a maintenance window... If something is seriously wrong just swap them back over.

Do keep me up to date with the project, would be nice to hear the outcome!

Kind Regards,

Liam