(Edit:  Updated info and answered some of my own questions after hitting the lab)   Hello all. I had an interesting request come across my messy desk yesterday, and I wanted to get some thoughts on this.   Background One of our server teams has chosen a hyper converged VxRail appliance, and they are doing the right thing by adhering to the vendor recommended design, planning the network design, and trying to think ahead, under the assumption that there will be additional appliances in the future.   The specific question revolves around the vSAN network. The appliance has two or more network connections (active/standby, NOT LACP) specifically for the purpose of communication among nodes within the vSAN cluster. The typical number of nodes is apparently four, with a hard cap of 64. In reality, you would add additional clusters before you would add that many nodes to a cluster.   The vSAN network is completely isolated. No routing, nothing special at all. The only network requirement is that it be a trunk port. In other words, the VID is tagged in the 802.1q header. No problem.   The Request Here's how the request was proposed to me. I like the way he worded his request.    Let's assume that we have a cluster of four nodes. We'll use VID 10 and we have a subnet of 10.10.10.0/24 to KISS and to allow for future expansion. Non routed, so no biggie either way.   Node vSAN1-n1 has 10.10.10.1 Node vSAN1-n2 has 10.10.10.2 Node vSAN1-n3 has 10.10.10.3 Node vSAN1-n4 has 10.10.10.4   Assume now that we have our own switch dedicated to this network. All ports are configured as switchport mode trunk and switchport trunk allowed vlan 10. No uplinks, no mess. Everybody is happy and sings Kumbaya. The best part is that nobody else has to hear it.   A year goes by and they purchase another appliance. Assume that they again buy a switch specifically for vSAN. All ports have vlan 10 trunked to them, and the nodes look like this:   Node vSAN2-n1 has 10.10.10.1 Node vSAN2-n2 has 10.10.10.2 Node vSAN2-n3 has 10.10.10.3 Node vSAN2-n4 has 10.10.10.4   No problem. As time goes on, they have a drop-in solution for vSAN. And I have to operate under the assumption that they are correct when they say that there is no chance that these will ever need to route.   That works great if you have a dedicated switch for every vSAN cluster. But that's not realistic.   The request was that they be able to do exactly that with our production data center switches.   Ideas My first thought was to use port-local VLANs, but that pretty much means a 1:1:1:1 VLAN Pool : Domain : EPG : BD which feels... wasteful.  However, it does, as some quick lab work just proved out, satisfy the requirements outlined in the request.  (See below)   I was also thinking that useg may be an option? I don't know enough about how useg works, but if I understand it correctly, I can keep it simple with one bridge domain and a separate community useg EPG per cluster? I figured I would get some thoughts before I hit the lab with this.   Thanks in advance for your thoughts.   Details (Per-Port VLAN Lab) In my lab, I created three VLAN static VLAN pools with a single encap:  VID 100.  I created three physical domains and tied each to its respective VLAN pool.  I tied all three to a common AAEP.  I then configured my locally-scoped interface policy groups and created profiles for these.  I labeled them as follows:     L201# show int e1/17-18,e1/20-21,e1/27-28 descr ------------------------------------------------------------------------------------- Port Type Speed Description ------------------------------------------------------------------------------------- Eth1/17 eth inherit vSAN-Cluster1_n1 Eth1/18 eth inherit vSAN-Cluster1_n2 Eth1/20 eth inherit vSAN-Cluster2_n1 Eth1/21 eth inherit vSAN-Cluster2_n2 Eth1/27 eth inherit vSAN-Cluster3_n1 Eth1/28 eth inherit vSAN-Cluster3_n2     I in my tenant, I created three bridge domains with no unicast routing enabled.  They have the following VNIDs:   moquery -c fvBD -x query-target-filter='and(wcard(fvBD.name,"vSAN-Cluster"))' | grep 'name\ \|seg\' name : vSAN-Cluster1 seg : 16482210 name : vSAN-Cluster2 seg : 16383903 name : vSAN-Cluster3 seg : 16121798     I created three EPGs, one per bridge domain, applied the respective physical domain to the EPG, and statically-assigned the respective EPGs to the respective ports:   L201# show int e1/17-18,e1/20-21,e1/27-28 trunk | grep -A7 -B1 Allowed ----------------------------------------------------------------------------------- Port Vlans Allowed on Trunk ----------------------------------------------------------------------------------- Eth1/17 171-172 Eth1/18 171-172 Eth1/20 173,175 Eth1/21 173,175 Eth1/27 176-177 Eth1/28 176-177 And note the vlan-100 encap on all ports, plus the vnid of the bridge domains (see above): L201# show vlan extended | grep 17[1-7]\ *enet 171 enet CE vxlan-16482210 172 enet CE vlan-100 173 enet CE vxlan-16383903 175 enet CE vlan-100 176 enet CE vxlan-16121798 177 enet CE vlan-100 ... View more