san redundancy,zoning configured in 1 switch only

[kmong]

I have 2 separate mds 9148 switches (separate datacenters) connected by port-channel (sw1 and sw2) passing through Optera using dwdm technology. ISL was established succesfully. As seen on the topology in Fabric Manager, the switches are now connected. We have configured zonesets and zones on for all devices of the paired switches,but configuration was done through FM in switch 1 only. My question is, if sw1 fails, what happens to the zones? Will it also fail?what should be the approach on this? We want it to be redundant, so that when sw1 fails, sw2 will become active and zoning would still be operational. Thanks for the help

[dynamoxxx]

can you provide an image that shows your topology, it would be much easier to make recommendation.

[Kris Vandecruys]

Hi,

If I understand you correctly,  you have a topology a bit like this

DC1                  DC2

sw1 ----isl--------sw2

Your question is: If I make zoning changes, how can I be sure that they are propagated through the entire fabric, and in case of an outage, will come up again with the correct zone database.  Let me tell you first how zoning works, then we will go over the different enhancements a Cisco NX-OS switch offers to make it less of a pain.

First, suppose you make a zone change to the sw1:

sw1#config terminal

sw1(config)# zone name One vsan 10

sw1(config-zone)# member pwwn 11:11:11:11:11:11:11:11

sw1(config-zone)# member pwwn 22:22:22:22:22:22:22:22

sw1(config-zone)# exit

This creates the following:

- a zone named One in the zone database for vsan 10

- an entry in the running-config of sw1

To actually make this zone do something, you will need to add it into a zoneset, then activate that zoneset:

sw1(config)# zoneset name MyActiveZoneset vsan 10
sw1(config-zoneset)# member One

sw1(config-zoneset)# exit

This creates the following:

- a zoneset entry in the zone database for vsan10, with member zone One

- an entry in the running-config of sw1

The activation happens with this command:

sw1(config)# zoneset activate name MyActiveZoneset vsan 10

Activation will update the zone databases of all the switches in vsan 1 and make this zoneset the one that is actually working on the fabric.

What this will not do is update the running-config on switches other than the one you executed the commands on! That means that, if this is the config you want, and you would like to persist it between reloads, which you would do with a 'copy run start', will only work for the switch you originally executed the commands on.  In the case of a new fabric, *only sw1* will have a copy of the zones stored in it's nvram.  it will not show up in the running-config of any other switch. You would have to enter the same config commands on all the switches to have them recreate a zone db like this one.

If you would not do this, and all your switches went down, this would not even be an issue: sw1 has the zoning config saved into it's nvram with the 'copy run start' and the other switches have no zone config whatsoever, what they had was cleared with a reboot.  When sw1 comes up again, it will merge its zone config with that of the other switches and eventually you would end up with the same config.

It's pretty easy to conceive how errors could be introduced in the scenario above:  Switches could have different conflicting configs, and activations on other switches than sw1 can cause overwrites of prior config and make zones disappear, interrupting service to end nodes.

The solution to this is easy: Configure all your vsans on all your switches with the following command:

'zoneset distribute (full) vsan 10'

The distribute parameter makes it so that when a switch receives an update of the active zoneset (eg someone called zoneset activate on a switch in the fabric) the updates will be stored into the running-config besides the zone database.  if you specify the full keyword, also zones that are not part of the active zoneset will be sent around.  *you will still need to do a 'copy run start' on all switches to make this config persist reloads*

The distribute command has very little to do with the distribute button in the zoning panel in fabric Manager.  Using fabric manager to create and activate zonesets makes sure that all zoning config is distributed (when you click the distribute button) on all switches in that vsan and that the config is stored in nvram.

A further enhancement of this mechanism is available through 'enhanced zoning', but that is beyond the scope of the original question.

If you want to have a more indepth explanation of the above, please visit the link:

http://www.cisco.com/en/US/products/hw/ps4159/ps4358/products_tech_note09186a00802205ff.shtml

Message was edited by: Kris Vandecruys: VSAN1 changed to VSAN10 as per best practice not to use VSAN1

[Steven.langdale_2]

I'd add two small things to that very thorugh answer.

1. You are getting recoverability with that setup, not redundancy.  If you loose a switch (assuming you have 2 because one DC needs to talk to another) you are down.  I'm guessing this is just wording though.

2.  It is best paractice NOT to use VSAN1, pick another and the examples will be perfect.

Steven

[Kris Vandecruys]

Steven,

thanks for your feedback, of course you are correct concerning these points.  I'll change the original message to use a different vsan.

for me it was not clear from the original question what redundancy exists.  In the scenario described, it's of course best to have dual fabrics for redundancy.  Zones will not magically sync across fabrics, simply because they connect different pwwns with each other it would not make sense.

In a simple 2 switch setup, zoning will continue to work on sw2 if sw1 fails, but of course only for targets and initiators connected to sw2.

My points was more to illustrate when and how zonesets will be consistent across switches.

[Kris Vandecruys]

just a quick addendum.  Once you hit activate, any switch in the vsan will be able to zone if sw1 fails. they have the active zoneset running after successful activation.  The trouble starts when you try to bring switches back up with delta's between what's running and what's saved.

[kmong]

Hi everyone thanks for all the replies. were already done with the configuration. So just for you ro verify, I am attaching some images to show the topology.

With this kind of physical connection, and same zones & zonesets configured on each pair of switch (PEAK1 same with PLAZA1, PEAK2 same with PLAZA 2), is the san already redundant?

[Kris Vandecruys]

Hi Kmong,

The topology is certainly redundant, if you configure zones correctly on both fabrics and run multipathing software on all your hosts.

2 small remarks tho:

1. Is the Trunk (EISL) in your visio diagram not supposed to be between sites, as it is in the screenshots of FM. I assume PEAK and PLAZA are 2 different datacenters.  In the FM screenshots it's shown that PEAK-1 connects to PLAZA-1, but in the Unionbank_SAN_Topology, it seems like PEAK-1 and PEAK-2 connect

2. Is there a reason why your storage connects unevenly to your fabrics?  I see 4 links going to PEAK-1 and 2 to PEAK-2, and vice versa in your other site. It's a best practice to scale your fabric at 50% max so that when one of both fabrics fails, the survivng one can easily handle traffic of both.

[kmong]

Hi kvandecr,

please see my answers:

1. Is the Trunk (EISL) in your visio diagram not supposed to be between sites, as it is in the screenshots of FM. I assume PEAK and PLAZA are 2 different datacenters.  In the FM screenshots it's shown that PEAK-1 connects to PLAZA-1, but in the Unionbank_SAN_Topology, it seems like PEAK-1 and PEAK-2 connect

-- forgive me for the labels, EISL is between sites PEAK and PLAZA over DWDM (using Optera). there is not trunk between switches located in the same data center

-- in Unionbank_SAN_topology, labels show that PEAK-1 is connected to PLAZA-2.

When I enabled the portchannel (EISL) between PEAK-1 and PLAZA-1, FM was refreshed and then the 2 switches merged with PEAK-1 as its main switch.same happened with PEAK-2 and PLAZA-2 with PLAZA-2 as its main switch. The FM only sees 2 fabrics (PEAK-PLAZA-1 and PLAZA-PEAK-2), but this is what its supposed to happen, am I correct?

2. Is there a reason why your storage connects unevenly to your fabrics?  I see 4 links going to PEAK-1 and 2 to PEAK-2, and vice versa in your other site. It's a best practice to scale your fabric at 50% max so that when one of both fabrics fails, the survivng one can easily handle traffic of both.

Zoning was configured by our client. What sould be done here. Zones and zonesets were configured in just one switch per pair and then we just clicked copy full zone database for the other switch.

[Kris Vandecruys]

Hi kmong,

the fabric names are just names, you can rename them if you like.

my last remark has little to do with zoning.  There's 4 physical links going to one fabric and 2 to the other.  Usually fabrics are built completely mirrored ie identical connections to both sides.  I was wondering if there was a reasoning to this setup.

[dynamoxxx]

kmong@trends wrote:
 
When I enabled the portchannel (EISL) between PEAK-1 and PLAZA-1, FM was refreshed and then the 2 switches merged with PEAK-1 as its main switch.same happened with PEAK-2 and PLAZA-2 with PLAZA-2 as its main switch. The FM only sees 2 fabrics (PEAK-PLAZA-1 and PLAZA-PEAK-2), but this is what its supposed to happen, am I correct?

yes, that's normal as you merged two fabrics