Traffic between different UCS fabric interconnect

[Jerome BERTHIER]

Hello

I'd like to confirm how traffic is handled between different UCS fabric interconnect.

By reading documentation, it seems that there is no traffic between FI switches :

If servers are bound to two different FI then the traffic is switched on LAN DC. Isn't it ?

Référence: https://www.cisco.com/c/en/us/solutions/collateral/data-center-virtualization/unified-computing/whitepaper_c11-701962.html

I'm not in charge of computing servers system. So what I should exactly asked to be verified if I want to confirm the state on a specific DC UCS fabric ? The way that vnic are attached to IOM/IFM then to FI ?

Regards

[BrianSekleckiGE]

In the diagram above, the scenario would be extraordinarily unlikely as the standard is to have servers dual-homed to both FIs, or their connected FEX, in which case, the FEX or FI simply switches frames between ports; but, in the case of above --  two single-homed servers/hosts/blades -- without a FEX intermediate --  then yes, I suppose:

* (one of the two) external LAN switches would switch the frames first

* Resulting in the frame being forwarded it onto the adjacent FI.

* FI then switches it to the destination server.

[Jerome BERTHIER]

I'm a novice in UCS world.

If I'm right, one IOM is connected only on one FI (no cross connection to FI).

So what about vNics ? As server is dual homed using two IOM on two differents FI, are vNic mapped virtually to both IOM using load balancing or fail over ? Or are they pinned on a single IOM ? 

[Steven Tardy]

Yes. UCS IOMs should only connect to one FI.
UCS server vNICs pin to either FI-A or FI-B.
UCS server vNICS with fabric failover have a shadow vNIC, which the OS does not see, pinned to the "other" FI.

Only server vNICs (in the same VLAN) pinned to the SAME Fabric Interconnect (via an IOM) switch-by/hairpin-at/stay-within the singular Fabric Interconnect.
Servers with vNICs pinned to DIFFERENT Fabric Interconnect (even within the same chassis) go upstream of the Fabric Interconnect.

Always always always configure any VLAN configured on UCS on the upstream network also.
So many people (incorrectly) think "oh, it's all _within_ UCS, then I should not need to add this VLAN upstream."

There are two rules of thumb for vNICs which I use:

1. If a vNIC will only ever have one endpoint/IP/MAC (management IP, NFS connection, iSCSI storage, DB back-end, etc), then create a singular vNIC and use UCS Fabric Failover to provide redundancy.
2. If a vNIC has multiple endpoints/IP/MAC (VMs, etc), then create two vNICs (one to FI-A and one to FI-B) and have OS NIC do (switch independent) teaming to use both uplinks (more available bandwidth) and provide redundancy.