New ISL - Zone Merge vs Segmented

[rondebbs1]

We have to locations about 1/2 mile apart. They are Pasteur and Alton. Pastuer has 2 9506 switches - pasmds1 and pasmds2. Alton has 2 9506 switches altmds1 and altmds2. Pasmds1 has a new ISL to altmds1. Pasmds2 has a new ISL with altmds2. These are the only ISL's ie: pas1 is not connected to pas2.

For the most part the switches in Pasteur are only connected to hosts and storage in Pasteur. The switches in Alton are mostly connected to hosts and storage in Alton. We do have one small vsan101 on pasmds1 and altmds1 that has 2 hosts in alton and one DMX4 port in pasteur. We have a similar situation where vsan201 has the second hba for the 2 hosts and a different DMX4 port. The ISL between the switches allows the 2 hosts in Alton to talk to the 2 DMX4 ports in Pasteur.

When we implemented the ISLs between pasmds1 and altmds1 and the pasmds2 and altmds2 we got different results and we are not sure why. The pasmds1 and altmds1 seemed to do a fabric merge and created one large fabric where all the hosts and stroage from both locations now exist.

The pasmds2 and altmds2 ISL did not seem to create a merge but created new "segmented" vsans under a new Fabric Manager folder called "Segmented VSANs". The segmented vsan for pasteur only has pasteur stuff and the segmented vsan for alton has only alton stuff.

This is our first experience with createing an ISL between MDS switches and we are not sure what is going on. Any ideas?

[Michael Brown]

Can you please paste in the configuration from the interfaces you are using for the ISL between pasmds2 and altmds2.

On the command line 'show run interface fc x/y'

Also the output from 'show interface fc x/y' from each side. This should show us what is causing the issue.

One point to look at, could there be a duplicate zone name that exists in altmds2 and pasmds2 that contains different members?

If so, that would cause a zone merge failure.

- Mike

[rondebbs1]

ALT-MDS-2# show run interface fc3/1

version 3.2(3a)

interface fc3/1

switchport description ISL to Pasteur MDS2

switchport speed 2000

switchport rate-mode dedicated

no shutdown

switchport mode auto

switchport trunk allowed vsan 201

ALT-MDS-2# show run interface fc3/45

version 3.2(3a)

interface fc3/45

switchport description ISL to Pasteur MDS2

switchport speed 2000

switchport rate-mode dedicated

no shutdown

switchport mode auto

ALT-MDS-2#

_____________________________________

PAS-MDS-2# show run interface fc3/1

version 3.2(3a)

interface fc3/1

switchport description ISL to Alton MDS2

switchport speed 2000

switchport rate-mode dedicated

no shutdown

switchport mode auto

switchport trunk allowed vsan 201

PAS-MDS-2# show run interface fc3/45

version 3.2(3a)

interface fc3/45

switchport description ISL to Alton

switchport speed 2000

switchport rate-mode dedicated

no shutdown

switchport mode auto

PAS-MDS-2#

[rondebbs1]

Now that I look at the ISL interfaces above I notice an issue. The 3/1 interfaces only allows vsan 201. This is what I want. But interface 3/45 lets all VSANs through - this seems like a problem because I only want 201 traffic to pass through the ISL.

Except for the devices in 201 I don't want devices in Pas mixing with devices in Alt.

[Michael Brown]

If both ISLs are to carry only VSAN 201, then the configuration on the interfaces should match. Please add the switchport trunk allowed command to the 3/45 interfaces. This is non-disruptive since nothing should be connecting except devices on VSAN 201.

Please post the results from 'show interface fc 3/45' once the change is made.

- Mike

[rondebbs1]

Mike thanks for the info. Hopefully the steps below will get me on the right track.

1. Shutdown TE port fc3/45.

2. Modify the trunk allowed list to not allow vsan 200 traffic, only allow 201 traffic.

3. Bring the TE port up

Now only VSAN 201 traffic will pass through both mds2 switches. The mds1 switches were set correctly so that only VASN 101 traffic will pass through but the MDS2 switches were not set correctly.

Once the change is made, I have to correct the VSAN merge that occurred when the ISL was created between the pasmds2 switch and the altmds2 switch. Originally the switches had no relationship and are in two sperate locations .5 miles apart.

Each switch had it's own seperate vsan200. Pasmds2 had it's local hosts and storage while altmds2 had it's own seperate hosts and storage. Since the "trunk allow" was set wrong the two separate vsan200s merged into one (probably because they were both vsan200s) when we set up the ISL. This "merged" zoneset is currently the active zoneset called wfs_fabric2. In FM I can also see the old pas zoneset but it is not active.

I would like to get to a segmented pas vsan and a segmented alt vsan. It looks like the inactive pas zoneset is correct and just needs to be activated at some point.

The current active wfs_Fabric2 zoneset has merged devices from pas and alt. I want to remove all the the pas devices and rename this to alt_fabric2.

Hopefully this will give me two segmented vsans on these two switches. They would be vsan200altmds2 and vsan200pasmds2. They would both be up and active at the same time.

Does this make sense?

[Michael Brown]

Yes it makes sense. When you get the trunk to carry only VSAN 201, VSAN 200 will appear to be segmented since it is not permitted across the trunk. What I mean is you will have 2 separate VSAN 200s, one in ALT and one in PAS. Clean up the zonesets in each VSAN 200 separately since they no longer have connectivity to the other VSAN 200.

From the Fabric Manager perspective it still may show VSAN 200 as segmented because from it's point of view the same VSAN number should not be used on 2 switches without an active ISL between them.

Hope this helps,

Mike

[sidbartle]

Hi,

Just some extra info as I think you look to be on the right track already.

Depending on how the ISL was added can sometimes affect the way the switches work out the merges. IE by command line or by Fabric manager. Configuring by FM can sometimes run a bunch of commands very quickly and the fabric doesn't always work it's self out straight away (causes segmented VSAN's), and in some cases I need to bounce the ISL to redo the merge or re-import the zoneset “zoneset import interface fcx/x vsan vsan xx”. I usually only use cli to modify the fabric as at least I know what changed in what order and can back out easily.

You may also want to think about changing the VSAN's id's for separate VSAN's so in the future when you connect all the switches up for redundancy or other reasons the VSAN's will stay apart and not merge. I personally would of added a new VSAN and moved one of the VSAN200's to there before creating the ISL to prevent the merge, but that may have required a outage of the hosts so not always doable.

Just for curiosity sake may I ask why pasmds1, pasmds2, altmds1 and altmds2 are not all connected together to form one fabric and logically separated by vsans and use the inbuilt traffic management if that is a concern?

[rondebbs1]

It sounds like I would be better off not using vsan id 200 on each connected switch since there is no ISL on vsan200. Maybe I'll change one of them to vsan220.

The pasmds1 is connected to altmds1 and pasmds2 is connected to altmds2. We use the vsan101 and vsan201 to allow severs in Alton to talk to a DMX in Pasteur.

I don't think we really have a need to connect pasmds1 to pasmds2 or altmds1 to altmds2. Every single host or storage device in Pasteur connects connects to both pasmds1 and pasmds2 in the same redundant way. The same is true for every device in Alton with altmds1 and altmds2. So if we ever lose an hba or switch or storage port or cable we will keep on running on the other switch.

I don't want to use the ports to ISL pasmds1 to pasmds2 unless we really need to. Would there be a benefit to ISL all four switches together?

[Michael Brown]

Re-numbering a VSAN means you have to create the new VSAN, move the interfaces over, then create new zones and a new zoneset on the new VSAN (and activate it), then delete the original VSAN. This is obviously disruptive to any traffic on the original VSAN.

I see no problem with keeping the 2 fabrics isolated. They can be ISLd together and then isolated via software configuration, but most large SAN shops stay with dual redundant fabrics, which is what you currently have in place.

Renumbering one of the VSAN 200s to 220, will make Fabric Manager look much nicer. There will be no segmentation.

- Mike