C-Series with FI, external Switching.

[DD-valski]

Hi,

Just have a bit of sideways questions in regards to cabling up C-Series and FI. The idea or question is, would it be possible to connect the C-Series to the FI via the LOM ports for management/profiles,ect and connect the CNAs to external(5k,7k,9k) switches for traffic.

All the docos I've seen around this/ Dual Wire say nothing explicitly that it's not possible to connect the servers CNAs to an external switch. It's just an idea as this would give all the sexy benefits UCSM but the traffic will bypass the FIs.

Does anyone have idea's or links to doco to say that this would not work.

[Walter Dey · Accepted solution]

Hi

The LOM ports on Cisco UCS C-Series Rack-Mount Servers integrated with Cisco UCS Manager can not be used for data traffic. The LOM ports are disabled when the C-Series Rack-Mount Servers are in Cisco UCS Manager mode.

If you want to use the SingleConnect or direct connect modes you must have the following:

• Server CIMC release 1.4(6) or later

• Cisco UCS VIC 1225 Virtual Interface Card

• Cisco UCS VIC 1227 (MLOM)

• Cisco UCS VIC 1225T Virtual Interface Card

• Cisco UCS VIC 1227T (MLOM)

• Cisco UCS VIC 1387 (MLOM)

• Cisco UCS VIC 1385

see

http://www.cisco.com/c/en/us/td/docs/unified_computing/ucs/c-series_integration/ucsm3-1/b_C-Series-Integration_UCSM3-1/b_C-Series-Integration_UCSM3-1_chapter_01.html

Walter.

[DD-valski]

I think you missed he question. I am not trying to use the LOM ports for data traffic only management.

As per the dual wire config, you can have both the LOM and the CNA cards connected to a FEX, and the FEX to an FI. This separates the management traffic from data down to the sever. I'm just trying to take that idea and kinda blow it up.
Dual Wire Config.

http://www.cisco.com/c/en/us/td/docs/unified_computing/ucs/c-series_integration/ucsm3-1/b_C-Series-Integration_UCSM3-1/b_C-Series-Integration_UCSM3-1_chapter_0100.html

As per my idea,

LOM------>FEX(Addapeter Switch)----->FI(Managment Traffic Only)

CNA----->NX Switch.(Data Traffic, Eth,FCOE)

[Walter Dey]

I don't think that this is a supported configuration; see eg. Fig 1 in

http://www.cisco.com/c/en/us/td/docs/unified_computing/ucs/c-series_integration/ucsm3-1/b_C-Series-Integration_UCSM3-1/b_C-Series-Integration_UCSM3-1_chapter_0100.html#reference_C1C82463EE4A4325BF9FDD63602B77C7

How can you configure vnic/vhba, if the CNA is not connected to FI ?

[DD-valski]

Well the server and the CNA can function in standalone/without an FI as the bulk of the needed intelligence is on the VICs. With that and the fact that you can manage the servers via the LOM ports(via FEX), it should be possible from a technological point of view as all the needed tech is already implemented/exists.

Thank you for confirming that it's not possible.

It was just an idea if it was possible in some unofficial/publicized way.

This would greatly increase the number of servers we can have connected/managed by the FI, whilst removing the traffic bottleneck that is the FIs, and increasing network/storage throughput.

Just an idea.

[Kirk J]

Greetings.

Your options for being managed by the FI and having some kind of connectivity outside the FIs, would be limited to fc HBAs or add-in PCI-E Nics.  As those are not on the manageable CNA list, they will not be provisioned/configured by UCSM service profile, and can be manually configured at the OS level.

We occasionally see customers set this up to get SPAN type traffic down to an integrated rack server via add-in PCI non-cna ethernet NICs as the FIs in end host mode won't pass unknown unicast traffic.

Thanks,

Kirk...

[DD-valski]

Hi Kirk,

So that would exclude things like P81E, vic 1225 ect.? If I'm not mistaken cant these be managed locally via IMC. So why would this not work, does cabling it in this way disable the ability to manage the vNICs, vHBAs on the card?

Grated the traffic between the CNA and a FI is somewhat specialized, however it would be cool to give at least basic profile management over the CNA to the configuration ability offered by the IMC.

Sorry for all the questions, I'm just really curious.

Thanks.

[Kirk J]

While the stand alone CIMC has some overlapping functionality as the UCSM, the CIMC is either stand alone or is fully controlled by the UCSM integration... There is no in-between.  The CIMCs capabilities are utilized by the UCSM in helping provision the various manageable devices like VIC cards (vnics, vhbas) and storage.

The UCSM, in the form of a service profile, can configure everything you could try to manually configure in the CIMC in standalone mode, and in a much more programatic, repeatible way.

If you don't really want the integration, but want some management and monitoring, perhaps you should consider the IMC supervisor.

Thanks,

Kirk