Please tell me how to connect with FCoE UCS-mini and Nexus N5k (N5K-C5548UP-B-S32)

[ghost-337]

I have UCS 5108 Blade Server Chassis (with two B200 M3 Blade Servers inside and two FI 6324) connected to Nexus N5k switch through 4x10GE links.

I want to configure the two aggregated channels FCoE (vFC) from chassi to switch.

All instructions I've been seen refer to the fact that UCS-mini must be in the FC end-host mode, but UCSM 3.0 version does not support this mode, what shoud I do?

I set up following the instructions www.cisco.com/c/en/us/support/docs/switches/nexus-5000-series-switches/116248-configure-fcoe-00.html

But, after setup I have a fault on fabric: FCoE or FC uplink is down on Vsan 500;

And in Nexus: Vsan 500 is down (waiting for flogi)

When I do "show interface vfc 1" from Nexus CLI I see no 'Trunk vsans (up)', but

'Trunk vsans (initializing)             (500)'

Unfortunately, I can't find technical notes for UCS mini (with UCSM v. 3.0), so may be You can promt any guides or suggestion for this?

Thank You in advance!

[Walter Dey]

The release notes of 3.0.1e, which is for UCS-mini,

http://www.cisco.com/c/en/us/td/docs/unified_computing/ucs/release/notes/ucs_3_0_rn.html

says:

Unsupported Features

The following features are not supported in Cisco UCS Manager, Release 3.0:

• Features, defect fixes, and platforms introduced in Cisco UCS Manager, Release 2.2(2c) and newer that are not explicitly called out in New Features or Resolved Caveats sections of this document.
• Some features introduced in Cisco UCS Manager, Release 2.2(1b) and earlier releases:

– Ethernet Switching Mode

– FC End Host Mode

– Private VLANs

– Port Security

– KVM Virtualization is not tested (although not explicitly disabled)

[ghost-337]

Thanks for answer, but I've read this in release notes earlier..

On the basis of this, I asked my question - what exactly I should do for setup fcoe between switch and UCS-mini in FC switching mode (because of all manuals says that it must be in FC end host mode).

I having hard seeing because of this my Trunk vsans forever in initializing status... And flogi fails..

Maybe you can advise me some others manuals or another where are described required settings for UCS-mini or UCS in switching mode.

Thank you.

[Leon Kull · Accepted solution]

Same question here.

TIA

[Walter Dey]

- FCoE is essentially native FC (e.g. flogi, zoning); exception is transport.

- UCS fabric interconnect can be in switch mode (FC, Ethernet) as well as End Host Mode (FC, Ethernet); default: FC and Ethernet are End Host mode

- FC switch mode is used for directly connecting a FC and/or FCoE storage subsystem to UCS FI; FC zoning is done on UCS FI.

- you would like to do multihop FCoE; Host - FCoE - UCS FI - FCoE - Nexus 5k - FCoE - Storage

- this is done with UCS FI in NPV (End Host mode); UCS FI port to Nexus 5K is Uplink unified port; Nexus 5k is NPIV.

see eg.

http://www.cisco.com/c/en/us/support/docs/servers-unified-computing/ucs-manager/116188-configure-fcoe-00.html

above configuration was done with UCS FI FC end host mode (NPV).

I have not tested with FC switch mode, maybe it works as well.

I think / hope, FC end host mode (NPV) will become possible with 3.0.2 release.

[ghost-337]

Do I understand correctly, that my current architecture (attached figure) cann't be implemented?
Because of UCS-MINI is not in NPV mode (FC switching mode instead of this).

Is it possible to realize aggregated FCoE uplinks from UCS-mini to N5k with UCSM version 3.0(1d)??
If this can't be realized, then what types of connected links (From UCS-mini to switch) should I choose? Not FCoE, but unified?

Thanks to all for assistance!

[Walter Dey]

Your situation is tricky and I have to revise my previous posts.

What I have tested before and works is this

- UCS in FC switch mode

- VSAN per fabric, with FC zoning disabled

- Create native FC ports on fabric interconnect (menu : unified ports)

- connect UCS FC native port to Nexus 5k FC native port; N5k is anyway configured as a FC switch.

- do the zoning on N5k; zoneset activation will push back zoneset to UCS FC switch.

- this requires additional uplinks to N5k, for IP. therefore no unified uplinks.

-------------------------------------------------------------------------------------------------------------------

in your picture, you probably want unified uplink, not FCoE uplink. I have not tested this with FC switch mode, only with FC NPV mode.

[frank.han]

Dear walter

i have connect N5K by FC, but can't work. if you have any config detail?

[Walter Dey]

Hi Frank

This will not work.

Recap. at this moment, mini UCS only supports with 3.0.1 FC switch mode (not FC NPV mode);

This FC switch mode doesn't support interop mode, connecting a Brocade FC switch; this configuration will however become possible, when mini UCS supports FC NPV mode. I believe this will happen with 3.02.

Hope that clarifies the situation.

Walter.

PS. If you would instead have a MDS and/or Nexus 5k, it would work today; no interop issue.

[frank.han]

So, you mean we have to connect a storage equipement to FC port on UCS mini.

UCS mini can't connect brocade300 switch always, even connect N5K or MDS right?

[Walter Dey]

Hi Frank

With Fabric Interconnect of mini UCS in FC switch mode (this is the current software release limitation):

you can directly connect FC storage, and/or connect FI to a MDS/Nexus 5k, and connect the storage there. No support for Brocade !

With fortcoming firmware release, mini UCS will also support FC NPV mode, and then you can connect with Brocade.

Walter.
 

[Walter Dey]

UCS mini firmware version 3.0.2 should become available in March 2015, and will support FC NPIV mode !

[lbacchi]

Ok, but i should want to be sure:

Now with 3.0.1 what i can do ?

- A) MiniUcs to MDS9148  and storage V7000-Storewize ?

-B) MiniUcs direct conneted to V7000-Storewize ?

So i i have understood that with 3.0.2 i can be working A) solution , right ? 

[Walter Dey]

Now with 3.0.1

- A) MiniUcs to MDS9148  and storage V7000-Storewize ?

MiniUCS 6324 is a FC switch, connected to MDS 1948 FC switch; so you have a SAN with 2 switches. This only works with MDS and/or N5k switches, NOT Brocade or any other non Cisco switches.

http://www.cisco.com/c/en/us/td/docs/switches/datacenter/mds9000/interoperability/matrix/intmatrx.pdf

-B) MiniUcs direct conneted to V7000-Storewize ?

Could technically work, but is not supported

http://www.cisco.com/c/en/us/td/docs/switches/datacenter/mds9000/interoperability/matrix/intmatrx/Matrix8.html

with 3.0.2

- A) MiniUcs to MDS9148  and storage V7000-Storewize ?

MiniUCS 6324 in FC NPV mode, connected to MDS 1948 FC switch which has NPIV enabled; This works with MDS and/or N5k switches and any other non Cisco switches eg. Brocade. This is much simpler than above, no FC SAN configuration; probably 95 % of all UCS FC configurations are like that.

http://www.cisco.com/c/en/us/td/docs/switches/datacenter/mds9000/interoperability/matrix/intmatrx.pdf

[Jelena Mitrovic]

Hi everyone,

We are thinking of connecting UCS mini with Nexus 5672UP that will be connected directly to the FCoE EMC storage. From the discussion above I am under the impression that this should work (FI will be in the end-host mode for the LAN and in the FC switch mode for the SAN traffic). We are having the dilemma about right way to connect our UCS mini. I have attached two pictures and my question is is our setup going to work and which of two connectivity options shown in the pictures is recommended?   

Regards,

Jelena