Cisco UCS connection to CX4-120

[joeharb]

We are starting our UCS deployment on the SAN. We have a CX4-120 that is going to be used. This san will only be used for the UCS environment. We also have a pair of MDS 9216i switches that can be used but they only will do 2 Gb/s FC. The CX4-120 has 2 4 Gb/s connections per storage processor. From what I can tell we have a few options:

• •1. Connect the UCS FI to the MDS and connect the MDS to the CX4-120 and do the zoning with the MDS
  
• •2. Connect the UCS FI directly to the CX4-120 and not be able to do zoning but utilize LUN masking
  
• •3. Connect the UCS directly the CX4-120 as well as the MDS and do the zoning from the MDS but the data path is directly from the FI and the SAN.
  

I am concerned that since we only have 2 interfaces per SP that the 2 Gb/s limitation of the MDS could be an issue. Also do you think we even need the MDS or will LUN masking be acceptable for our environment, we will be using the UCS for our UC infrastructure as well as some management servers.

Has anyone deployed in a similar manner?

Thanks,

Joe

[joeharb]

Update, with the release of UCS 2.1 the FI can do the zones so I would assume I could bypass the use of the MDS switches all together, is this correct?

Thanks,

Joe

[Robert Burns]

Correct.

Robert

[joeharb]

With the direct attach fiber channel would I want to connect the fabric interconnects one storage processor or will each FI have a connection to both storage processors?

Thanks,

Joe

[Robert Burns]

Each port on the Storage processor provides access to that single storage processor.

If you want your hosts to have redundant access, you'll need at least a connection from each FI to a different SP.

FI-A  <----> SP-A

FIB  <----> SP-B

In the setup above each FI will only have access to it's connected SP.

Regards,

Robert

[joeharb]

To clarify, our Storage processors have 2 ports each, will both of those ports be connected to one fabric or with availablity of 2 ports we can "cross connect". 

Thanks,

Joe

[Jeremy Waldrop]

You would connect the CX4 to the Fabric Interconnects the same way you connect the CX4 to an MDS fabric. I have done this a few times and it works great.

FIA<--->SPA0/SPB1

FIB<--->SPA1/SPB0

[joeharb]

Thanks for the response, that makes complete sense.  I would assume with this configuration that the vsan's will need to be common/global, would the be a need to have more than 2 vsan?  Would this a correct/valid setup?

FIA SPA0 ---VSAN3000

FIA SPB1 ---VSAN3001

FIB SPA1 ---VSAN3000

FIB SPB0 ---VSAN3001

I have found several videos and documents but I a still confused on the configuration, I have successfully booted from ISCI via the CX4-120, but now we are able to utilize the fc ports.

If anyone has any goood documents please share?

Thanks,

Joe

[Robert Burns]

General SAN Practice is to keep the fabrics separated.  Normally A/B fabrics are accessible from both FI devices, but there's nothing to say it can't be done and some customers do this without issues.  The concern was simply to keep separation between fabrics to avoid any cross-contamination issues arising.  With the criticality of storage communication you probably understand why.

Jeremy's suggestion is another.  This would allow each FI to access to each VSAN, so yes your VSAN's would now need to be global since they're reachable from either FI. 

All this means for your profiles is that your vHBAs can access either VSAN, and you may want to "balance" the primary boot targets across the different paths available.  When you have separated fabrics, each vHBA would be able to be assigned only VSAN 3000 or 3001 depending on which fabric its assigned to.  With the cross-connect topology, each vHBA will have the option of accessing VSAN 3000 or 3001. 

Just be mindful of the balancing I mentioned above.  You can easily overlook your storage paths and create an  asymetrically balance of storage traffic.  Ex vHBA-1 (assigned to VSAN 3000) and vHBA-2 (assigned to VSAN 3000 also).

Robert

[joeharb]

I have connectivity with the CX4-120 to both FI's, should the ports be configured as Uplink Ports or FC Storage Ports?  If they are configured as uplink the links will not come up, if they are set to FC Storage Ports the links come up but I can' select the proper Vsan, only vsan default (1) is available?  What am I missing?

Thanks,

Joe

[Robert Burns · Accepted solution]

Joe,

Yes the ports would need to be configured as FC storage ports - because they're directly connected and not uplinks. 

Did you create the VSANs in the SAN tab under the "Storage Cloud" also?

You'll need them create there also.

Regards,

Robert

[Jeremy Waldrop]

No the VSANs would be separate just like if there was an MDS. If you had two MDS switches and you connect an EMC CX/VNX to it you would connecte it in this fashion

MDS1-VSAN 3000-SPA0

MDS1-VSAN 3000-SPB1

MDS2-VSAN 3001-SPA1

MDS2-VSAN 3001-SPB0

Your UCS FIs would then connecto the MDS this way

FIA-VSAN 3000-MDS1

FIB-VSAN 3001-MDS2

Both FIs have a path to both EMC SAN front end storage processors.

When going direct with no MDS you still need both FIs to see both storage processors for full multipathing to work. For your setup this is what you want

FIA-VSAN 3000-SPA0

FIA-VSAN 3000-SPB1

FIB-VSAN 3001-SPA0

FIB-VSAN 3001-SPB1

[joeharb]

Thanks for the reply Jeremy but now I am confused...

Per your post:

"When going direct with no MDS you still need both FIs to see both storage processors for full multipathing to work. For your setup this is what you want

FIA-VSAN 3000-SPA0

FIA-VSAN 3000-SPB1

FIB-VSAN 3001-SPA0

FIB-VSAN 3001-SPB1"

What about ports SPA1 and SPB0, or is this a typo and that whould be what is connected to FIB should be:

FIB-VSAN 3001-SPA1

FIB-VSAN 3001-SPB0

Please clarify,

Thanks,

Joe

[Jeremy Waldrop]

Sorry, typo. Should be like this

FIA-VSAN 3000-SPA0

FIA-VSAN 3000-SPB1

FIB-VSAN 3001-SPA1

FIB-VSAN 3001-SPB0

[joeharb]

Thanks for the response on that, and since vsan 3000 only resides on FIA and 3001 is on FIB, would there be a reason for them to be assigned globally or should they only be on their respective fabric?

Thanks,

Joe