Connecting Brocade in Access Gateway mode to MDS 9513

[dynamoxxx]

Hello guys,

i need to connect a couple of brocade blade enclosure switches (4024) to MDS 9513. I was going to go ahead and enable Access Gateway mode on the brocade but then i read a note about issues with hosts using Qlogic HBAs that connect through these Access Gateway switches.  The solution that i have seen so far is to do this:

switch#

switch# config terminal

switch(config)# no fcdomain fcid persistent vsan <1-4094>

switch(config)# fcinterop fcid-allocation flat

switch(config)# vsan database

switch(config-vsan-db)# vsan <1-4094> suspend

switch(config-vsan-db)# no vsan <1-4094> suspend

switch(config-vsan-db)# exit

obviously this is very disruptive, even though i could do one fabric at a time. What are my other options (other then using interop mode) ?

Thank you

[Michael Brown]

Those command are straight out of the Brocade manual for attaching an Access Gateway to a Cisco MDS.  The only thing missing is that you must also enable NPIV on the Cisco MDS.  I don't know of anohter way to get the Access Gateway attached.

The only VSAN that you would need to suspend and then no suspend, is the VSAN that the Access Gateway is attaching to.

Actually, I'm not sure that you need to do that...I was able to disable the persistent FCID allocation on the VSAN without suspending the VSAN.

I would try it without doing the suspend/no suspend and see if the Qlogic HBAs work.

HTH,

Mike

[dynamoxxx]

Michael,

would you please glance at this pdf ?

http://community.brocade.com/servlet/JiveServlet/download/2006-2335-4771-1353/Brocade_AG-to-Cisco_GA-TN-083-00.pdf

My 9513 has arrays connected to it so according to this paper i can either edit company id list (not sure what that actually does) or manually assign persistent fcid. I don't believe i have qlogic HBAs with those ID but i need to be prepared just in case.

What is your recommendation? Are you familiar with this issue ?

PS: i have already enabled NPIV as i have MDS 9124e connected in NPV mode, works great. Now just need to get Brocade AG mode working properly.

Thank you

[Michael Brown]

The issue with the Qlogic HBA stems from the fact that the older Qlogic HBAs will not PLOGI into storage devices that have the same area in the FCID.

FCID is like DDAANN

DD is the domain ID where the device is FLOGI'd into

AA is the area assigned by the switch

NN is the port number.

So if Stroage array logs into MDS and is assigned FCID 330000, and then Qlogic HBA logs in and is assigned FCID 330001, the HBA will not PLOGI into the storge array because both have 00 in the area portion of the FCID.  To bypass this, MDS has a list of OUI that have this issue, and if it sees those OUI at FLOGI time (which is part of the PWWN of the attaching device), the MDS will issue an unused area.  The result would be that the storage array was given FCID 330000, and the next device that logged in had an OUI in the list, the MDS would assign it 330100.  Now with the HBA having area 01, and the storage unit haveing 00, the issue is not seen.

The company list maintained in the code...and what this is a list of OUI (Organizationally Unique Identifier) that are assigned to a specific company.  The company that then manufactures fibre channel devices, would use this OUI in the PWWN of any device they manufacture.  The MDS uses this OUI to tell if the device that is logging in is one that needs a unique area number in the FCID.  One company can have many OUI assigned to them.

The command 'show fcid-allocation area' will show the current list of OUI that the MDS would assign unique area for.  If you happen to discover a new OUI that requires a unique area, and it is not in the list, you can add it to the config to make the MDS assign it a unique area.  Likewise, you can remove OUIs from the default list if needed.  If you need to check the OUI in a PWWN, you can issue this command on the MDS.

'show fcid-allocation company-id-from-wwn xx:xx:xx:xx:xx:xx:xx'  and the MDS will show you the 3 byte OUI.  For reference, if the PWWN starts with 2 the OUI is differnt that if it starts with 5.  I have underlined the OUI for 2 PWWN below.

50:00:40:21:03:f4:62:b6

21:01:00:1b:32:31:8c:79

What the goal is to have any Qlogic HBA that need to work with Storage array on MDS where the BAG is attached, end up with different area numbers in their assigned FCIDs.  In order to do this, I do not think you would need to suspend the VSAN.  I would recommend making the attachments, and then determine what FCIDs need to be zoned together in your environment, and then make a decision.  Remember there is also a limitation on the BAG that the last byte of the FCID not be the same on any 2 devices.  I would recommend option D in that document you posted the link to.  It would be the most none disruptive.  The FCIDs you assign would have to be inline with the current domain ID of the VSAN you have the BAG attached to.

IE: if BAG is attached to MDS VSAN 100, and the domain for VSAN 100 is 0x33

- use 'show flogi data vsan 100' to see currently assigned FCIDs for VSAN 100

- configure static entries for any new pwwn that will reside behind the BAG confirming to:

  - using FCIDs that are not currently in use

  - assign unique area

  - assign unique port number

You can always attach the BAG and the servers and see what FCIDs they end up with.  If the connectiviy issue surfaces, it will only be once the devices are zoned together, and would not imact any other devices on the SAN.  Once the FCIDs are determined, you will then know if any issues 'might' arise.

HTH,

Mike

[dynamoxxx]

Michael,

thank you for taking the time to explain this. So if MDS has the list of "affected" devices and it automatically assigns unique area to those ..why do i have to create FCIDs manually ?

[Michael Brown]

Because MDS knows to assign unique area...but may assign same port number in the FCID ...which would break the routing in the BAG.

[dynamoxxx]

Ohh ..so if i were to connect one of this funky HBAs directly into MDS ..MDS itself would "fix" it automatically and i would not have to do anything, but this "auto-fix" is not compatible with how Brocade operates in AG mode when connected to MDS ?

Thank you

[Michael Brown]

You got it!

[dynamoxxx]

Thank you so much Michael. We are migrating a couple of HP C-class blade enclosures so i wanted to have a script ready to create persist FCIDs in case we run into issues. Here is my setup

c9513# sh fcdomain address-allocation vsan 114

Free FCIDs: 0x060031 to 0x0600ff
            0x060300 to 0x06fffe

Assigned FCIDs: 0x060000 to 0x060030
                0x060100 to 0x0602ff
                0x06ffff

Reserved FCIDs: 0x06ffff


Number of free FCIDs: 64974
Number of assigned FCIDs: 562
Number of reserved FCIDs: 1

c9513# sh flogi database vsan 114
--------------------------------------------------------------------------------
INTERFACE        VSAN    FCID           PORT NAME               NODE NAME
--------------------------------------------------------------------------------
fc1/4            114   0x060000  50:06:04:84:52:a5:4e:27 50:06:04:84:52:a5:4e:27
fc1/7            114   0x060001  50:06:04:84:52:a5:4e:03 50:06:04:84:52:a5:4e:03
fc1/13           114   0x060010  50:01:10:a0:00:85:a4:84 50:01:10:a0:00:85:a4:85
fc1/13           114   0x060014  20:0a:00:0d:ec:6b:5f:40 20:72:00:0d:ec:6b:5f:41
fc1/13           114   0x060015  50:01:43:80:03:c0:0a:ac 50:01:43:80:03:c0:0a:ad
fc1/13           114   0x060016  50:01:43:80:03:c0:0e:04 50:01:43:80:03:c0:0e:05

let's say that in port fc1/4 and fc1/7 is my storage array and in port fc1/13 is one of those funky Qlogic HBAs. So i am going to shut port fc1/13 and then run this command:

switch# conf t
switch(config)# fcdomain fcid database
switch(config-fcid-db)# vsan 114 wwn 50:01:43:80:03:c0:0e:04 fcid 0x060316 area

then i will "no shut" the port and it will get the new fcid.  Does that sound correct ? You also mentioned assigning new port id ..not sure what you mean by that ?

Thanks again

[Michael Brown]

Your PWWNs are not in the list of devices that require a unique area, so they were not each assigned their own area.  When you hard code the FCID for unique area, the last byte (which is the port number) must be 00.  Becaus of this, if you configured eachg HBA with a hard coded unique area, this would cause your BAG to have issues.  In order to hard code an FCID, you must remove the exising entry from the FCID database or the confguration will fail since the WWN is already in the database.

You would have to do this for each entry.

fcdomain fcid database

no vsan 114 wwn 50:01:43:80:03:c0:0e:04 fcid 0x060016

vsan 114 wwn 50:01:43:80:03:c0:0e:04 fcid 0x060316

-Mike

[dynamoxxx]

Michael,

i am a little confused, sorry for extra quesions

mikbrown wrote:
Your PWWNs are not in the list of devices that require a unique area, so they were not each assigned their own area.

correct, i just picked a WWN for example.  Let's say my HBA WWN is this: 50:01:00:14:5e:c0:0e:04 . This WWN in the OUI list so when it logins into the switch instead of getting a FCID of 0x060016 , it will get something like 0x060316 ?  This is MDS applying "auto-fix" ?  So the port ID is 16 ?

So the fact that now my area id is 03 ..it will not work with Brocade AG and i have to remove FCID 0x060316 and change it to something else ?

Thank you

[Michael Brown]

Sorry for the delay in getting back on the community...but yes..you understand the issue.  The port number portion of the FCID must be 00, and you must also have a unique area if this is one of the affected HBAs.

-Mike