cancel
Showing results for 
Search instead for 
Did you mean: 
cancel
342
Views
5
Helpful
3
Replies
Highlighted
Beginner

WWYD? - Turnkey solution for an isolated L2 BD

(Edit:  Updated info and answered some of my own questions after hitting the lab)

 

Hello all. I had an interesting request come across my messy desk yesterday, and I wanted to get some thoughts on this.

 

Background

One of our server teams has chosen a hyper converged VxRail appliance, and they are doing the right thing by adhering to the vendor recommended design, planning the network design, and trying to think ahead, under the assumption that there will be additional appliances in the future.

 

The specific question revolves around the vSAN network. The appliance has two or more network connections (active/standby, NOT LACP) specifically for the purpose of communication among nodes within the vSAN cluster. The typical number of nodes is apparently four, with a hard cap of 64. In reality, you would add additional clusters before you would add that many nodes to a cluster.

 

The vSAN network is completely isolated. No routing, nothing special at all. The only network requirement is that it be a trunk port. In other words, the VID is tagged in the 802.1q header. No problem.

 

The Request

Here's how the request was proposed to me. I like the way he worded his request. 

 

Let's assume that we have a cluster of four nodes. We'll use VID 10 and we have a subnet of 10.10.10.0/24 to KISS and to allow for future expansion. Non routed, so no biggie either way.

 

Node vSAN1-n1 has 10.10.10.1

Node vSAN1-n2 has 10.10.10.2

Node vSAN1-n3 has 10.10.10.3

Node vSAN1-n4 has 10.10.10.4

 

Assume now that we have our own switch dedicated to this network. All ports are configured as switchport mode trunk and switchport trunk allowed vlan 10. No uplinks, no mess. Everybody is happy and sings Kumbaya. The best part is that nobody else has to hear it.

 

A year goes by and they purchase another appliance. Assume that they again buy a switch specifically for vSAN. All ports have vlan 10 trunked to them, and the nodes look like this:

 

Node vSAN2-n1 has 10.10.10.1

Node vSAN2-n2 has 10.10.10.2

Node vSAN2-n3 has 10.10.10.3

Node vSAN2-n4 has 10.10.10.4

 

No problem. As time goes on, they have a drop-in solution for vSAN. And I have to operate under the assumption that they are correct when they say that there is no chance that these will ever need to route.

 

That works great if you have a dedicated switch for every vSAN cluster. But that's not realistic.

 

The request was that they be able to do exactly that with our production data center switches.

 

Ideas

My first thought was to use port-local VLANs, but that pretty much means a 1:1:1:1 VLAN Pool : Domain : EPG : BD which feels... wasteful.  However, it does, as some quick lab work just proved out, satisfy the requirements outlined in the request.  (See below)

 

I was also thinking that useg may be an option? I don't know enough about how useg works, but if I understand it correctly, I can keep it simple with one bridge domain and a separate community useg EPG per cluster? I figured I would get some thoughts before I hit the lab with this.

 

Thanks in advance for your thoughts.

 

Details (Per-Port VLAN Lab)

In my lab, I created three VLAN static VLAN pools with a single encap:  VID 100.  I created three physical domains and tied each to its respective VLAN pool.  I tied all three to a common AAEP.  I then configured my locally-scoped interface policy groups and created profiles for these.  I labeled them as follows:

 

 

L201# show int e1/17-18,e1/20-21,e1/27-28 descr
-------------------------------------------------------------------------------------
 Port           Type    Speed    Description
-------------------------------------------------------------------------------------
 Eth1/17        eth     inherit  vSAN-Cluster1_n1
 Eth1/18        eth     inherit  vSAN-Cluster1_n2
 Eth1/20        eth     inherit  vSAN-Cluster2_n1
 Eth1/21        eth     inherit  vSAN-Cluster2_n2
 Eth1/27        eth     inherit  vSAN-Cluster3_n1
 Eth1/28        eth     inherit  vSAN-Cluster3_n2

 

 

I in my tenant, I created three bridge domains with no unicast routing enabled.  They have the following VNIDs:

 

moquery -c fvBD -x query-target-filter='and(wcard(fvBD.name,"vSAN-Cluster"))' | grep 'name\ \|seg\'
name                     : vSAN-Cluster1
seg                      : 16482210
name                     : vSAN-Cluster2
seg                      : 16383903
name                     : vSAN-Cluster3
seg                      : 16121798

 

 

I created three EPGs, one per bridge domain, applied the respective physical domain to the EPG, and statically-assigned the respective EPGs to the respective ports:

 

L201# show int e1/17-18,e1/20-21,e1/27-28 trunk | grep -A7 -B1 Allowed
-----------------------------------------------------------------------------------
 Port           Vlans Allowed on Trunk
-----------------------------------------------------------------------------------
 Eth1/17        171-172
 Eth1/18        171-172
 Eth1/20        173,175
 Eth1/21        173,175
 Eth1/27        176-177
 Eth1/28        176-177

And note the vlan-100 encap on all ports, plus the vnid of the bridge domains (see above):

L201# show vlan extended | grep 17[1-7]\ *enet
 171  enet  CE         vxlan-16482210
 172  enet  CE         vlan-100
 173  enet  CE         vxlan-16383903
 175  enet  CE         vlan-100
 176  enet  CE         vxlan-16121798
 177  enet  CE         vlan-100
1 ACCEPTED SOLUTION

Accepted Solutions
Highlighted
Collaborator

Re: WWYD? - Turnkey solution for an isolated L2 BD

Hi Kelley,

I think your design is exactly how I would do it. I can't see how a microsegmented (µseg) design would work if you wish to use the same VLAN ID each time, unless you still went ahead and used per-port vlans (as you did in your design) and separate bridge domains (as you did in your design).  In which case, all you've done by adding microsgemention is another level of complexity and chewed up a few more outer VLAN IDs

 

RedNectar
aka Chris Welsh


Don't forget to mark answers as correct if it solves your problem. This helps others find the correct answer if they search for the same problem

View solution in original post

3 REPLIES 3
Highlighted
Collaborator

Re: WWYD? - Turnkey solution for an isolated L2 BD

Hi Kelley,

I think your design is exactly how I would do it. I can't see how a microsegmented (µseg) design would work if you wish to use the same VLAN ID each time, unless you still went ahead and used per-port vlans (as you did in your design) and separate bridge domains (as you did in your design).  In which case, all you've done by adding microsgemention is another level of complexity and chewed up a few more outer VLAN IDs

 

RedNectar
aka Chris Welsh


Don't forget to mark answers as correct if it solves your problem. This helps others find the correct answer if they search for the same problem

View solution in original post

Highlighted
Beginner

Re: WWYD? - Turnkey solution for an isolated L2 BD

Thanks, Chris. Your response is very much appreciated. I think it showed me that I need to spend some time in the lab futzing with useg just to get a better understanding. But I have the freedom of doing so at my leisure, as I now have a definitive answer for the server folks, who are no longer waiting on me for an answer.

 

Add a theoretical second beer to my tab. The first theoretical beer I owe you was from a few months ago when I was asked to come up with a solution to ensure traffic from a VDI would traverse an inline IPS before hitting its default gateway. Your blog post / tutorial on per-port VLANs was the basis for my being able to better understand how PPV worked, and was, with very little modification, the basis for the VDI / IPS solution. Like I said, your post was what helped solidify in my mind the nature of PPV, and how it is configured in ACI. So double kudos.

Highlighted
Collaborator

Re: WWYD? - Turnkey solution for an isolated L2 BD

Hi Kelley,

Glad you found my answers here and eslewhere useful. And if I recall, I believe there is a theoretical beer on a tab for you as well for the assistance you gave me in labbing 2nd gen swithes for my ACI ARP Delgue post!

RedNectar
aka Chris Welsh


Don't forget to mark answers as correct if it solves your problem. This helps others find the correct answer if they search for the same problem

CreatePlease to create content
Content for Community-Ad

Cisco COVID-19 Survey