Use of Vlan 1 work arround for issues

morphotek · ‎07-14-2018

So in my setup I have a disjointed network setup Vlan 1 is my core business and vlan 902 is for my iSCSI I have no issus connecting to vlan 902 and the blades themselves I have been able to have connect to the Vlan 1 uplink properly. I have Vmware ESXi 6.5 installed on the blades and they can connect to both Vlans. My issue occurs with the guest OS's by Vlan tagging Vlan 1 on the vswitch the guests can communicate with other devices on the network on Vlan 1 however when trying to communicate with guests on other hosts within the UCS environment it sometimes works and sometimes doesn't. I can't seem to find the cause of that or how to resolve it. currently using the standard switch on vmware. Any help is appreciated.

Kirk J · ‎07-15-2018

Use of vlan 1 for actual data is the intro line of a 1000 bad IT jokes... Your network vlan use needs to be redesigned.....

As to the hit/miss connectivity, I would check to see if hosts/guests that are pinned to the same FI are the working ones, and hosts/guest pinned to different FIs are the ones with issues.

The ones pinned to same FIs are switched at that FI level, and don't need to go to the northbound switches. The ones pinned to different FIs have to egress your FI uplinks and traverse your upstream switch and come back down.

If this is the case, then your upstream switch config needs to be evaluated as well, and you will need to do a MAC walk, to see where you are/aren't learning a sample MAC between your two intended endpoints.

Thanks,

Kirk...

morphotek · ‎07-16-2018

Well redesigning the network isn't really an option right now since this is a production environment and the vlan 1 was defined for it a long time ago by someone else.

I seemed to have solved my issue it wasn't quite at the Cisco level in the vm host configuration I had two nics active that are pinned one to FI1 and the other to FI2 as soon as I moved one of them to standby in the vswitch the servers that couldn't talk could

Steven Tardy · ‎08-22-2023

Just helped solve a Cisco TAC case with this issue.

The UCS configuration was:

VLAN 1 (tagged/non-native)
VLAN 1234 (untagged/native)

The upstream Nexus configuration was what I would call "trunk only" (no native VLAN explicitly configured).

interface Po10
switchport mode trunk
switchport trunk allowed vlan 1,1234

ARPs from the VM were seen on the upstream gateway and replies went out from the gateway but the VM did NOT see the ARP reply.

This was isolated to be an upstream Nexus issue using command:

Nexus# show interface trunk
---------------------------------------------------
Port        Native VLAN     Status     Port Channel
---------------------------------------------------
Eth1/1      1               trnk-bndl  Po10

This means that the Nexus will egress traffic in VLAN 1 as untagged.

UCS will in turn see this untagged packet and put that packet into VLAN 1234 (not the intended VLAN).

The fix (for this customer) was simple and detailed in almost every Cisco UCS CVD:

int po 10
switchport trunk native vlan 1234

After the configuration change the VM could `ping` the upstream gateway. The Nexus then showed:

Nexus# show interface trunk
---------------------------------------------------
Port        Native VLAN     Status     Port Channel
---------------------------------------------------
Eth1/1      1234            trnk-bndl  Po10

Hope that helps others fighting this not-a-UCS UCS issue.