What is the maximum number of FC connections supported by the UCS Mini (FI 6324)? 3 or 4?

[melepruma]

Hi All,

I recently tried to do a UCS Mini BOM on CCW and the it gave me a an error when i tired to configure 8 FC SFPs to the two FI6324 saying "UCSB-5108-AC2 requires a minimum of 0 and allows maximum of 6 from Fibre Channel Options"

On the official Cisco documentation 'Cisco UCS 6324 Configuration Limits for Cisco UCS Manager, Release 3.0', under 'Fibre Channel Environment Configuration Limits' it clearly says that each FI6324 supports up to a maximum of 4 Native FC ports.

The same is affirmed on the recently concluded Google Hangout video done by the USC Data Center team.

So why then is this issue of not being able to configure 8 FC SFPs for a UCS mini with two FI6324?

Please refer to the BOM output attached (without pricing).

[Walter Dey]

Cisco UCS Mini Scalability Ports
The Cisco UCS 6324 Fabric Interconnect contains a scalability port as well as four unified ports. The scalability port is a 40GB QSFP+ breakout port that, with proper cabling, can support four 1 or 10G SFP+ ports. The scalability ports can be used as a licensed server port for supported Cisco UCS rack servers, an appliance port, or a FCoE port.

In the Cisco UCS Manager GUI, the scalability port is displayed as Scalability Port 5 below the Ethernet
Ports node. The individual breakout ports are displayed as Port 1 through Port 4.
In the Cisco UCS Manager CLI, the scalability port is not displayed, but the individual breakout ports as
displayed as Br-Eth1/5/1 through Br-Eth1/5/4.

You can have only a even number of FC ports, 0,2,4; as there are a max of 4 unified ports, and you most likely also need Ethernet uplink's, you end up with 2 FC per FI;

[melepruma]

Hi  wdey,

I do understand how the 40GB scalability port works and it's license requirement, etc.

But I do have to disagree with your last statement on the "even number FC ports" configuration on 6324 with UCSM 3.0. The Google Hangout video done by the USC Data Center team clearly says that unlike the previous gen FI (6200), the new FI6324 allows configuration of native FC on a per port basis  for the 4 10G ports stating from the low number ports. They clearly say that it supports 1, 2, 3, 4 FC port configurations and the 40GB scalability port only supports Ethernet and FCoE configurations.

I also understand the requirement for up-link ports and that is the reason why i have included the 40GB license and the 4 x 10GB breakout kit in the BOM.

What I need to know is why CCW is not letting me configure four 8G FCs (4 for each FI6324) on the chassis and says it should be in the range of 0 to 6?

[Walter Dey]

I don't have a miniUCS which I can test, the documentation says

Port Mode Placement
Because the Cisco UCS Manager GUI interface uses a slider to configure the port mode for unified ports , it automatically enforces the following restrictions which limits how port modes can be assigned to unified ports. When using the Cisco UCS Manager CLI interface, these restrictions are enforced when you commit the transaction to the system configuration. If the port mode configuration violates any of the following restrictions, the Cisco UCS Manager CLI displays an error:
• Ethernet ports must be grouped together in a block.
• Fibre Channel ports must be grouped together in a block.
• Alternating Ethernet and Fibre Channel ports is not supported.

[melepruma]

I am aware of the slider configuration on the classic UCS FI6200 with UCSM 2.1/2.2

In the video i shared with you they demonstrate the slider config and it allows for individual port configuration for native FC starting from the left side port. pls check.

I think the dual port or block limitation applies to the classical FI6200 is because there are two rows of ports (on on top of the other) over the slider. But in the FI6324 there is only one row of 10G ports above the slider. hence the individual port assignment capability?

But the specific question here is why the limitation on the CCW? can any Sales Engineer help me out here.

[Walter Dey]

You are 100% correct ! my statement about even number of FC ports is wrong.

Why 6 ?

I could only guess, that 3 are FC, and 1 is Ethernet Uplink per FI.

If you need 8, just order the 2 additional ones as spare.
 

[melepruma]

Yeah initially I too thought the same; 3 for FC and 1 reserved for uplink. But in the CCW config I added the 40G breackout kit and the 40G scalability license as well. Hence I should be able to use it for uplinks.

There is no point in ordering spare FC SFPs if I cannot connect it and use on the FI6324. That's why I'm looking for a solid answer.

thanx for your thoughts anyway wdey :)

[Walter Dey]

The datasheet clearly says 4 FC ports

http://www.cisco.com/c/en/us/products/collateral/servers-unified-computing/ucs-6300-series-fabric-interconnects/datasheet-c78-732207.html

Maybe its a caveat of the configurator :-)

Number of 1 and 10 Gigabit Ethernet, FCoE, or Fibre Channel Enhanced Small Form-Factor Pluggable (SFP+) external ports

4

[Brian Taylor · Accepted solution]

The Scalability Port on the 6324 Fabric Interconnect can only be used for downlink purposes, which include Ethernet Appliance, FCoE Storage, or Server port types.  That means that at least one of the 10Gb Unified Ports on the 6324 must be used for the Ethernet Uplinks, leaving a maximum of three per 6324 FI that can be used for Fibre Channel, hence the rule in CCW about a maximum of 6 FC transceivers configurable per chassis.

See the UCS Manager GUI Configuration Guide at http://www.cisco.com/c/en/us/td/docs/unified_computing/ucs/sw/gui/config/guide/3-0/b_UCSM_GUI_User_Guide_3_0/b_UCSM_GUI_User_Guide_3_0_chapter_0110.html#concept_8B14549B9F6B417B8756061CC26735E2, where it states: "The scalability ports can be used as a licensed server port for supported Cisco UCS rack servers, an appliance port, or a FCoE port."

As to why you can't use one of the breakout ports as an uplink, that likely is a result of testing and certification with various upstream switches using 4x10Gb breakout.  Perhaps future releases will support Uplink port type on these breakout interfaces, but I can't promise anything.

BT