Intersight UCSX Fabric Interconnect east-west traffic

[Weiwen_17]

Hi Everyone,

I would like to know if east-west traffic on same UCS domain but blade servers on different Chassis traverse north bound to uplink switches?

Currently I have a setup where host vnics on vmware vcenter are set to active-active. Intersight server profile has LAN connectivity policies with vnics configured for fabric A and B. Assuming all the VLANs required are configured on Domain profiles VLAN policies and server profile vnics and ethernet network group.

I noticed I have to define all the VLANs on the uplink switch in order for east-west traffic between servers to work smoothly. 

I will experience packet drop intermittently if I did not define the VLANs on the uplink switch.

Thank you. 

[Brian Morrissey · Accepted solution]

If the vSwitch is comprised of a vnic on fabric A and a vnic on fabric B with both active it could potentially need to traverse the upstream switches.

For example:
VM1 on host 1 is mapped by the vSwitch to vnicA on VLAN 10
VM2 on host 2 is mapped by the vSwitch to vnicB on VLAN 10

In this scenario, since the VMs are pinned to different fabric interconnects it would need to go upstream for the communication to succeed.

Another example:
VM1 on host 1 is mapped by the vSwitch to vnicA on VLAN 10
VM2 on host 2 is mapped by the vSwitch to vnicA on VLAN 10

In this scenario, it would be switched at the FI level as both VM's are on fabric A.

[Steven Tardy · Accepted solution]

ALWAYS create the VLAN upstream of the UCS Fabric Interconnect.
Never assume "Oh this is _within_ UCS, the VLAN isn't needed upstream."
This is a very common misconception which turns into a P1 outage TAC cases on countless occasions.

Even if all servers are pinned to FI-A (or FI-B) and can communicate just fine, there are failure scenarios (failed cable, failed port, FI reboot, UCS infra upgrade, etc) where some portion of the UCS servers MUST communicate upstream to reach all of the other UCS servers.

[Weiwen_17 · Accepted solution]

Thanks everyone for the response. The solutions provided help me have a better understanding of east west traffic on Fabric Interconnect. I will advise on the network team to define the required VLANs on the upstream switches.