UCS and VMWare-VNIC-VMNIC active-active/active-standby

[JO BO]

Dear Team,

We have 3 ESXi hosts each with vSwitch1 and portgroup with name TEST with VLAN 50
We have 3 VMs setup with 19.168.0.X series and are assigned to VLAN TEST

Each vSwitch1 is assigned with 2 vmnics on same VLAN 50(vnics created from UCS,service profile--where in 1 vnic is from From Fabric A and other from Fabric B)

1)Inter VM communication will work if we make one Vmnic active and other standby

2)Inter VM communication may work sometime or may not work if we make both vmnics active

The above behaviour is same for VMs in  combination of ESXihosts,expect if these vms are on same ESxi host the interVm communication works fine..

What setting to be made if we need to make vmnic active-active and still inter VM communication to happen

NB:

As mentioned above these vnics are created from UCS Service profile, where in 1 vnic is from From Fabric A and other from Fabric B, and when we try the active -standy mode the interVm comunication works and even when we make the FI A down, the standy adapter is coming up automatically and traffic is getting routed correctly

Thanks and Regards

Jose

[sprasath]

Jose,

From what it looks like, communication does not work when the upstream network is involved. With active-active, there is a chance that traffic from one VM (on say ESX 1) would go to FI A and traffic from another VM (say on ESX 2) would go to FI B.

This then has to traverse the upstream switch or switches, that are connected to the UCS FIs, and get switched. So chances are that the VLAN 50 is not allowed on the upstream switches or is blocked. Can you please check that?

Thanks,

Shankar

[JO BO]

Hi Shankar,

Thank you.. for replying...

This is an internal VLAN inside UCS and it has no connectivity to the uplink switch (used for mgmt vlan)

VLAN 50 is an internal vlan that has access within FIA FIB and the vm port group with vSwitch1( which has 2 vmnics assigned frm FIA and FIB with Vlan 50) (vSwicth's port group is also in VLAN 50,which is assigned to VMs nic)

and as mentioned before if we keep either vmnic0 from A as active and vmnic1 from B as standby it will communicate.. even keeping vice versa also it communicates.. but keeping both active it did work on one host but when moved to a different host it did not.

Thanks and Regards

Jose

[sprasath · Accepted solution]

Same state still - the link between FI A and FI B is not used for any data traffic. So if your source is using FI A and your destination is on FI B, then traffic has to go through the upstream switch. If you're still unsure, have vmnic0 active on one host and vmnic1 as active on the other host - communication will not happen (assuming these are going to different FIs)

So the solution to your problem is to define VLAN 50 in the upstream network. If you do not want to do this, then you need to restrict all traffic within the same FI (definitely not advisable).

Thanks,

Shankar

[JO BO]

Hi Shankar,

Thank you for replying..

Ok if that is th case the 4 uplink ports 2 from FIA and 2 from FI B are trunk and these 4 are conncted to 2 5k Switches-2 each with (cross for redundancy) and on these 5k switches there is no VLAN restrictions,kind trunk.

So what you are telling is the 'communication circle'  to the outside n/w should complete for these internal network VMs to communicate with active-active n/w adapters? as th communication happns via th uplink switch from FI A and FI B?

Thanks and Regards

Jose

[sprasath · Accepted solution]

Hello Jose,

It's not specific to just active-active - in your setup, you are seeing it with active-active. Like I said, the rule is that the link between the two FIs does not pass data traffic. So if your source machine is on FI A and your destination machine is on FI B, then it has to go up and complete the circle.

The FIs are capable of switching in the same VLAN. So if your source and destination are in the same VLAN and go to the same FI (what you are doing with your active-standby), then packet goes only till the FI and not to the upstream. The FI does the switching in this case.

Thanks,

Shankar

[JO BO]

Hi Shankar,

Thank you for replying....

Got your Point so if we create a VM with VNIc from FI-A and another VM with vnic from FI-B for these two VMs to communicate it has to travel via the upstream switch( data to reach from FI-A to FI-B)

Scenario1:-

So in our case (assuming that VLANs are not allowed on the upstream  switch) when we keep active-active and if the cards getting used at that point are from same FI (for 2 VMs) then communication happens.

And

(assuming that VLANs are allowed on the upstream  switch) when we keep active-active if the cards getting used at that point are from different or same FI (for 2 VMs) then communication will happen as they complete the communication circle in both the ways(via upstream or same FIs)

Also,

Scenario2:-

when we keep active standby (assuming that VLANs are not allowed on the upstream  switch) the communication will happen only if all the active cards are from same FI correct?

And

when we keep active standby (assuming that VLANs are allowed on the upstream  switch) the communication will happen by any combination of vnic from any FI correct?

please..correct me if am wrong

Thanks and Regards

Jose

[sprasath]

Yes, you got all of that correct.

Regards,

Shankar