UCS C-Series DirectConnect

[Eric Geer]

Just to confirm, this wonderful doc http://www.cisco.com/c/en/us/td/docs/unified_computing/ucs/c-series_integration/ucsm2-2/b_C-Series-Integration_UCSM2-2/b_C-Series-Integration_UCSM2-2_chapter_0110.html#reference_EF9772524CF3442EBA65813C2140EBE6 says NOTHING about storage.  It does state that it supports only SideBand connectivity, which leads me to believe MLOM, FC, and Ethernet all converge over the VIC 1227 adapter directly to the FI.  It would be good to get a better understanding of what the limitations are.  We have a C220 server with no onboard storage of any kind and are looking to see if Direct Connect will allow us to SAN boot from FC while using the other local integrated LAN adapter connect to our DMZ.  For best practice scenarios we want to separate WAN traffic on our DMZ from co-mingling with internal UCS traffic so as to avert VLAN hopping vulnerabilities.

-GEER

[Walter Dey]

Hi Eric

Are you aware of disjoint vlan configuration ? this is what many customers are using for DMZ, in case they use one UCS domain for internal and external resources.

http://www.cisco.com/c/en/us/solutions/collateral/data-center-virtualization/unified-computing/white_paper_c11-692008.html

I don't understand your other question: ....SAN boot from FC ....local integrated LAN adaptor....

Do you mean eg. Emulex or Qlogic CNA ?

As far as I remember, boot from SAN is only supported with VIC adaptors (which are managed by UCS Manager, including SP support), you might have other adaptors, which however are not managed by UCSM.

Please see Tab 2

http://www.cisco.com/c/en/us/td/docs/unified_computing/ucs/c-series_integration/ucsm2-2/b_C-Series-Integration_UCSM2-2/b_C-Series-Integration_UCSM2-2_chapter_0100.html

for supported CNA's (VIC and others)

Walter.

[Eric Geer]

We have VIC 1227 in this C-series unit that came with this bundle, and no onboard storage. So the idea is to boot from SAN, which we've got other blades within a chassis configured to use already with FC. The other method would be to configure our Nexus switches with FCoE, but I want to avoid additional configuration if I can.

[Walter Dey]

We run FCoE with the VIC in the C-series to the UCS FI; your OS sees however classical FC; therefore anything you are used with B-series (SP, SP-Templates, Pools, boot from SAN) is available for the C-Series server.

[Kirk J]

Hi Eric.

As the UCSM (service profile and boot policy) define the devices you will boot to, trying to utilize an unmanaged device (3rd party hba such as emulex, qlogic, etc) will not work for a boot to device.

I have seen customers utilize unmanaged supported addin PCI-E Ethernet cards such as 4 port Intel or Broadcom cards and hook those to different upstream switches.  The nics obviously would need to be managed and configured by the OS.  The onboard LOM ports are disabled in UCSM integration direct connect mode.

As for the unmanaged supported 3rd party HBAs, as long as you are not trying to boot via them, you can configured those at the OS level and still use them for FC connectivity where you have them connected to upstream san switch.

Like Walter mentioned, you can connect your FI to multiple upstream switches, and create a disjointed layer 2 network, which we see configured all the time for DMZ scenarios.

Hope this helps,

Kirk...

[Eric Geer]

Thanks for the quick response.  So as it sits, integrating the C-Series with VIC 1227 will force CIMC to disable the onboard NIC?  I just need to activate this from the OS level as suggested.  So I assume I'd need to place a third-party add-in PCI board.

As far as disjointed networks go, we utilize VLAN 1, but are not using this VLAN 1 network for this particular server cluster, so I assume there wouldn't be any conflict there?  Other servers in the UCS chassis do utilize VLAN 1, so I want to be cautious how this is implemented.

[Kirk J]

CIMC does automatically disable the LOM ports in direct connect integration mode.

Picking one of the supported 1Gb based Ethernet cards from the server's spec sheet, should provide you with a non -integrated set of 1Gb ports you can run to a non-UCSM destination.

VLAN1, or native VLAN is used for some control plan stuff like CDP, LACP, etc

With the disjointed design, you are selectively allowing just certain DMZ vlans out your DMZ uplinks (excluding all the rest), while excluding your DMZ vlans from your standard uplinks.

The VNICs you define for your DMZ based service profiles you would obviously only allow DMZ vlans.

Thanks,

Kirk...

[Walter Dey]

Hi Eric

- we preach since tens of years: don't use Vlan 1 (and Vsan 1), and its done again and again; I would fix this asap

- disjoint requires vlan 1 to be present for all vlan domains, therefore if cannot be used for user data traffic.

Walter.