PIN GROUP, Port Channel

[K P SIM]

Hi, Cisco Gurus:

Need your authoritative take on the following: Ethernet and FC is all End-Host Mode.

1. If client is using FC Port Channel on all of the FC Uplinks for the 2 Fabric Interconnects with a SINGLE VSAN on each FI, MUST THEY USE SAN PIN GROUP? Currently, they are pinning the vHBA template to this SAN Pin Group. vHBA-Fabric A pins to FC-PIN-GROUP-A and vice-versa,

2.

If FC Port Channel is being used as in 1, from WHICH UCSM option should the VSAN be created?

Should VSAN be STILL created under the Individual Fabric A and B as before?

Or

It have to be Global VSAN or Dual VSAN outside the Fabric A or Fabric B?

3.If client is using Ethernet Port Channel for each Fabric Interconnet or each fabric, must they use LAN PIN GROUP?

Currently, they are pinning the vNIC template to this Pin Group (Fabric A & B). vNICA-Fabric A pins to LAN-PIN-GROUP-A and vice-versa.

Something which I have SELDOM SEE before.

As usual, I know it is easy meat for Cisco gurus. Thanks, PAL.

So Long.

SiM

[Jeremy Waldrop]

No, Pin Groups are not required. When all FC or LAN uplinks are in a single port channel there is only 1 logical uplink (1 for LAN and 1 for SAN) so all vNIC/vHBAs are pinned to that 1 uplink.

I personnally do not like pin groups as they add complexity and do not allow for efficient bandwidth utilization like a singe port channel does.

If you are not using MDS/Nexus for FC SAN uplinks you cannot create a SAN Port channel, you will need to rely on the dynamic round-robin pinning of the vHBAs. This works very well by the way.

If you have MDS FC switches you will need to create a VSAN on each Fabric Interconnect that matches the VSAN on each FC switch fabric. If the MDS switches were setup in a non-best practice configuration where everthing is in VSAN 1 you can use the default. If using non-MDS FC switches you can use the default VSAN 1 but make sure you change th FCoE VLAN to something other than 1.

Best practice for MDS/Nexus FC switching is to have a different VSAN in each fabric, for example Fabric A would be VSAN 10 and Fabric B VSAN 20.

[K P SIM]

Hi, Jeremy:

Firstly, thank for the reply.

1.

I am STILL puzzled by the way VSAN was being created. The 2 VSANs were not created under the traditional Fabric A and B usual option on the left, BUT CREATED via the VSANs option

on the left under Stoage Cloud.

In their case, they have 2 VSANs says VSAN777 (Fabric A) and VSAN888 (Fabric B) created under VSANs option of Storage Cloud. 2 FC Port Channels-FC-1 (Fabric A) and FC-2 (Fabric B) and the following:

                                                                           VSAN (Doubt)

-----------------------------------------------------------------------------------------------------------------------------

FC-1 (makes up of 4 FC ports from FI-A to MDS-1) Fabric dual/vsan Fabric A(777)

FC-2 (makes up of 4 FC ports from FI-B to MDS-2) Fabric dual/vsan Fabric B(888)

Shoudn't the VSAN be created under Fabric A and B as per the usual way

2.

With FC Port Channel and Ethenet Port Channel configued in UCS, WHAT OTHER CONFIGURATIONS needed to be done on MDS-A and MDS-B as well as Nexus 7010-A & Nexus 7010-B?

Are you able to provide some insights?

Really appreciate your helps as usual, one of the most authoritative Cisco gurus for sure.

SiM

[Jeremy Waldrop]

Yes, the VSANs should be created under their respective fabric unless the vsan ID is the same on both fabrics. In your case there are different VSANs in each fabric so you should create the VSAN in that fabric and not as a global VSAN.

MDS needs to be on NX-OS 5.x or better and enable these 2 features

feature npiv

feature fport-channel-trunk

create an active mode port-channel

interface port-channel 11

channel mode active

add interfaces to the port channel

interface fc1/1-4

channel-group 11 force

no shut

On the LAN side it is recommended to create a vPC where you have 1 or more links from each fabric interconnected connected to each N7K. On the N7K create an LACP port channel on both and then configure it as a vPC.

[K P SIM]

Jeremy:

I am getting confusion here.

For client's setup, I think they have the VSAN setup as in the attachment.

Please let me know what kind of VSAN is it? It looks like it is Global VSAN, but configured differently and they pin the respective VSAN on each fabric to the respective FC Port Channel for A and B too.

On the FC Port Channel for Fabric A, the VSAN is shown as below when I took a look at the FC Port Channel for Fabric A:

Fabric Dual/VSAN Fabric-A (10)

Thank again, Jeremy.

SiM

[Jeremy Waldrop]

that setup will work but I would recommend creating the VSANs under each Fabric so that it is less confusing and easier to determine which VSAN belongs to which fabric.

After you create the respective VSAN under each fabric you will need to make sure that VSAN is selected under each port channel and on each vHBA template/vHBA.

[kg6itcraig]

Is the Customer a Dual Fabric or single?

If single that a Global VSAN is fine.

If Dual than you need 2 vHBA's bound to a non-global VSAN on one FI (like to PIN always, and would PIN each vHBA to respective port), or the FC nameservers will see both vHBA WWPN's on their fabric. That is a no-no on Dual Fabrics and I have made this mistake. This will actually work but can cause trouble and might not failover correctly or support proper traffic flow.

vHBA (WWPN/WWNN Pool)-> VSAN -> PIN -> FI -> Switch -> SAN Device.

Don't forget us Brocade types who don't care about VSANs so much. We just PIN.

Craig

[K P SIM]

hi, Criag & Rob:

I would believe it is a DUAL Fabric Setup. I was totally lost when I first saw it.

Based on the attachment, it MAY SEEM to be a Global SAN too. Are you able to confitm from the attachment?

Craig, they CLAIMED it worked but I have concern.

Can you guys help me up?

Truly appreciate it.

SiM

[Robert Burns]

All VSANs in the screenshot above are Global.  Notice they're not created under the Fabric A/ B section.

Unlike "VLANs", which are almost always Global and normally accessible from each Fabric Interconnect, VSANs are different.  Best practice for storage has keeping each SAN Fabric separate.  Keeping this in mind, does it make more sense to create VSANs as global or under their respective Fabric interconnects? (Under the separate Fabric Inteconnects of course.  The only use case for using Global VSANs is as Craig pointed out - when there is a single Storage switch northbound that both Fabric Interconnects uplink to.

Where this gets confusing is when you're assigning vHBA's to a VSAN. 

Ex. 

Let's say, as your customer has done, they've created their two VSAN 10 and 20 as "Global".  For comparison sake I've also created VSAN 100 just under Fabric-A and VSAN 200 just under Fabric-B. Their VSANs would looks like:

Now, when creating an vHBA under your service profile, you need to assign it to a VSAN.  Take a look:

So depending which Fabric (A/B) they want the vHBA connected to the options will change.  You can see that by creating the VSANs 10 and 20 as "Global" they appear when either Fabric is selected - Conversly you can see the two 100 and 200 I created under each Fabric, only appear when that fabric is selected.

So what?  Well if your upstream SAN switches only use one of the respective VSANs configured and only connect to one Fabric Interconnect, then you can potentially "blackhole" your storage traffic by assigning it to the wrong Fabric Interconnect.  Creating the VSANs only under the appropriate Fabric Interconnect prevents this possibility.

SAN Port-Channels have no bearing on this.  All a SAN Port Channel does is make mulitple FC Uplinks appear as one - which requires SAN Port Channels configured upstream also.  Leaving the FC Uplinks as individual just lets UCS dynamically pin virtual interfaces across the FC uplinks in a round-robin fashion.

In your case you have no need for Pin-Groups either.  As Jeremy pointed out, unless you have a specific need for them, it's safer to not use them.

Regards,

Robert