Cisco 9513 and Cisco 9148 - Fabric Merger

thabiso.mathabathe1 · ‎03-14-2016

Hi All,

I am currently in the process of merging fabrics;

The main SAN switch is the MDS9513 with an active zone configuration since 2011, the client has furthermore went ahead and bought a 9148, which currently doesn't have any zone config.

I have prepared the ports as follows (lets focus on one port to simplify matters):

Cisco 9148

MTA-9148-FA# show interface fc1/45 br

-------------------------------------------------------------------------------
Interface Vsan   Admin Admin   Status       SFP    Oper Oper   Port
                  Mode   Trunk                       Mode Speed Channel
                         Mode                              (Gbps)
-------------------------------------------------------------------------------
fc1/45     20     E      on      down         swl    --    --     --

Cisco 9513

TECFABA# show interface fc11/31 br

-------------------------------------------------------------------------------
Interface Vsan   Admin Admin   Status       SFP    Oper Oper   Port
                  Mode   Trunk                       Mode Speed Channel
                         Mode                              (Gbps)
-------------------------------------------------------------------------------
fc11/31    20     E      on      down         swl    --    --     --

No physical link is connected as yet because I am afraid the Main SAN Switch will go into segmentation.

Walter Dey · ‎03-14-2016

I assume you have multiple vsan's; therefore your isl's between the 2 MDS would be a trunking E port, allowing a range of vsan tagged frames over the ISL.

Regarding fabric merge, see eg. the recent discussion

https://supportforums.cisco.com/discussion/12916986/merging-two-fabrics

thabiso.mathabathe1 · ‎03-15-2016

Hi Walter,

Thank you for the prompt feedback.

My fear is the main SAN switch that is currently in production going into a segmented mode, meaning that the Fabric will lock, and the small 9148 taking over , How can I avoid that.

Walter Dey · ‎03-15-2016

Hi

Due to the fact, that the new 9148 doesn't have a active zoneset, you should not have a problem.

The zonesets for any vsan's which exists on both switches will be copied from the 9513 to the 9148.

What do you mean by fabric lock ? do you run enhanced zoning ?

Walter.

thabiso.mathabathe1 · ‎03-15-2016

Thank you Walter,

I have the change logged for tonight, I will update you sometime tomorrow:

Just one more question, This is the first time I will be doing as such, what are the results?

How will I complete the zoning for devices connected on the Subordinate Switch (The new 9148)?

Walter Dey · ‎03-15-2016

Please do a backup of the config on the 9513 (show running or show startup configuration) before you change anything.

And as I said, the ISL ports should TE not E !

I assume you do pwwn based zoning ? therefore most likely you just have to add additional zones for devices connected to the 9148; and them to the zoneset; finally activate this zoneset.

see eg.

http://www.cisco.com/en/US/docs/storage/san_switches/mds9000/sw/san-os/quick/guide/qcg_zones.html

Good luck !

Walter.

thabiso.mathabathe1 · ‎03-15-2016

Hi Walter,

These are are the options I have:

TECFABB(config-if)# switchport mode
E F FL Fx NP SD ST auto

As shown above there's no TE.

I have set the ports to E and activated trunking on all ports.

Walter Dey · ‎03-15-2016

Yes, that's it !

I think in the show interface fc ... you will see it.

thabiso.mathabathe1 · ‎03-16-2016

Hi Walter,

The change was completed successfully, however I encountered a few errors which I did not understand.

The plan was to establish 4* ISLs per Fabric, however yesterday I only managed to create only 1* ISL per fabric, please see interfaces as of today:

MDS9500 (The Principal Switch)

TECFABA# show interface br | Grep E
fc1/7      1      E      on      trunking     lwcr TE     4      --
fc1/31     1      E      on      trunking     lwcr TE     4      --
fc1/37     1      E      on      trunking     lwcr TE     4      --
fc1/43     1      E      on      trunking     lwcr TE     4      --
fc5/1      2      E      on      trunking     lwcr TE     4      --
fc5/7      2      E      on      trunking     lwcr TE     4      --
fc5/34     2      E      on      trunking     lwcr TE     4      --
fc6/1      2      E      on      trunking     lwcr TE     4      --
fc6/13     20     E      on      errDisabled swl    --    --     --
fc11/31    20     E      on      trunking     swl   TE     8      --

The port in question above is fc6/13, it gave me the errDisabled after we connected the cables.

9148 (The Subordinate Switch)

MTA-9148-FA# show interface br | grep E
fc1/45     1      E      on      down         swl    --    --     --
fc1/46     1      E      on      down         swl    --    --     --
fc1/47     1      E      on      down         swl    --    --     --
fc1/48     1      E      on      trunking     swl   TE     8      --

Please help I still need to establish 3* more per fabric.

Walter Dey · ‎03-16-2016

Can you tell me which 4 physical interfaces on 9513 and 9148 are used for ISL;

On your initial post: fc 1/45 (9148) and fc 11/31 (9513) both are in VSAN 20;

In your latest post, fc 1/45 is in VSAN 1 ???

Are you sure that the fc interfaces on both switches are in the same port vsan ?

thabiso.mathabathe1 · ‎03-17-2016

Hi Walter,

I am sorry for the confusion,

The ports are on VSAN 1 on the MDS 9148 and on VSAN 20 on the 9513.

Please find the 4 * ISL ports below (TECFABA = 9513):

MTA-9148-FA# show interface br | grep E
fc1/45     1      E      on      down         swl    --    --     --
fc1/46     1      E      on      down         swl    --    --     --
fc1/47     1      E      on      down         swl    --    --     --
fc1/48     1      E      on      trunking     swl   TE     8      --

TECFABA# show interface br | grep E

fc5/39 20 E on errDisabled swl -- -- --

fc6/13     20     E      on      errDisabled swl    --    --     --
fc6/37     20     E      on      errDisabled swl    --    --     --
fc11/31    20     E      on      trunking     swl   TE     8      --

As you can see from the dumps above. I keep on getting the same error on the ISL ports... I have suspended VSAN 20 on the 9148.

Furthermore, should I enable VSAN 20 on the 9148 and restart the fcdomain so that is can be active ? and then re-configure the ISL to VSAN 20?

Walter Dey · ‎03-17-2016

Which vsan's do you want on both MDS ?

You have to create vsan 20 on the 9148, and put all then interfaces fc1/45-48 into vsan 20; otherwise the ISL will not come up ?

thabiso.mathabathe1 · ‎03-17-2016

Hi Walter,

As suggested, I have activated VSAN 20 on the 9148s, please see below:

MTA-9148-FB# show interface br | grep E
fc1/45     20     E      on      down         swl    --    --     --
fc1/46     20     E      on      down         swl    --    --     --
fc1/47     20     E      on      down         swl    --    --     --
fc1/48     20     E      on      trunking     swl   TE     8      --

MTA-9148-FA# show interface br | grep E
fc1/45     20     E      on      down         swl    --    --     --
fc1/46     20     E      on      down         swl    --    --     --
fc1/47     20     E      on      down         swl    --    --     --
fc1/48     20     E      on      trunking     swl   TE     8      --

I still see the same error on some ports on TECFABA and TECFABB (9513s)

TECFABA# show interface br | grep E

fc5/39 20 E on errDisabled swl -- -- --

fc6/13     20     E      on      errDisabled swl    --    --     --
fc6/37     20     E      on      errDisabled swl    --    --     --
fc11/31    20     E      on      trunking     swl   TE     8      --

TECFABB# show interface br | grep E

fc1/1 20 E on trunking swl TE 8 --

fc5/39 20 E on errDisabled swl -- -- --

fc6/13 20 E on errDisabled swl -- -- --

fc11/31 20 E on notConnected swl -- -- --

However I see that fc11/31 on TECFABA has changed to not connected meaning that the cable is having issues. I will replace the cables and monitor.

The issue at hand is the errDisabled port statuses I see, How can I resolve that?

Walter Dey · ‎03-17-2016

Hi

Do I understand this correctly: one of the 4 ISL is up ?

Regarding error disable: Please post

show interface fc ....

show port internal event-history interface fc ...

show logging logfile

http://www.cisco.com/c/en/us/td/docs/storage/san_switches/mds9000/sw/rel_2_x/troubleshooting/guide/ts_guide/ts_port.html#wp32458

thabiso.mathabathe1 · ‎03-17-2016

Hi Walter,

2* ISL are up on each Fabric.

Using the "show port internal event-history interface fc5/39" and "show logging logfile" I could see that the error was caused by the transceiver not being 8GB. I am working on the Client getting more 8GB SFPs.