VXLAN VNI and VLANs

[CiscoPurpleBelt]

Are you able to assign the same VLAN number to a VNI as long as it is in a different VRF in regards to Spine Leaf VXLAN architecture? Is it best to keep all VLANS/VNIs in same VRF for simplicity?

[Andersjul]

Hi,

First off, we will have to look at the different VNI's we are using in VXLAN. Two types of VNI's are used which is one for L2 operations and one for L3 operations. The VNI tag is a 24bit field which gives us a massive 16 million unique tags, compared to 4096 in standard dot1q. The VNI tag is kept inside VXLAN header while the packet is moving in the fabric - this gives you segmentation.

The two types of VNI: 

L2VNI

L3VNI

L2VNI is a direct link between laver 2 VNI and VLAN's (dot1q) and it's recommended to keep this one-to-one relation between L2VNI and VLAN's.

L3VNI is a routed VNI which is used when traffic is moving between two different L3VNI (subnets), so this will have an IP associated which is used for routing purposes. 

Often L3VNI are member of a VRF and depending on your setup/lab, you will have different VRF's for different purposes, guest, administration, INFRA (wireless).

[CiscoPurpleBelt]

Thanks this is awesome!

So how do you assign hosts in a L2VNI an IP or how do you associate a subnet to a L2VNI? I know how it is done by creating L3VNI (and associated SVI) but what is actual purpose you would use L2VNI instead of L3VNI?

[Andersjul]

Glad to help you ;)

The L2VNI is in charge of forwarding traffic within the same VNI ("VLAN") between two switches. This is part of the overlay, and from a user's perspective the network is behaving as one big switch.

The L2VNI do not have an IP associated because we are not doing any routing in L2VNI - when we are talking about the overlay.

If you don't make use of the L2VNI, you wouldn't be able to stretch your L2 network consistently across different switches.

/Anders

[CiscoPurpleBelt]

Thanks!

So if only a L2VNI is created for let's say VLAN100, how do you determine what IP to assign a host in VLAN100?

Also, do you know some considerations to take into account for a VXLAN fabric that will need to communicate across geo boundaries such as to another fabric in another location?

[Andersjul]

Okay, here is a little configuration example, where you can see how the configuration is associated.

So the VLAN100 is associated to a L2VNI:

vlan 100

 name VXLAN-L2-VNI-100

 vn-segment 30100

and the L3VNI:

vrf context EVPN-L3-VNI-VLAN-100
 vni 50100
 rd auto
 address-family ipv4 unicast
 route-target both auto
 route-target both auto evpn

interface Vlan100
 no shutdown
 vrf member EVPN-L3-VNI-VLAN-100
 ip address 172.19.0.0/16

So here you have the two VNI's - L2 and L3 which has VNI number 30100 and 50100 respectively.

Hope it make sense ;)

[CiscoPurpleBelt]

Awesome this is great! 

So basically, in order for a VLAN/Subnet to route, I need the L2VNI, L3VNI(with associated VRF), and the SVI with the IP conifgured under it correct as opposed to tradional ethernet (layer2 VLAN and an SVI or sub-int)?

[Andersjul]

Hi,

Correct. Well it's only routing for that particular VTEP. In order have a working solution with multiple VTEP's which maybe is spread across different data centers, we have a lot of design questions to consider.

How is the underlay built to ensure reachability between VTEP's with ECMP?

How is BUM traffic handled in the underlay?

Which protocol is used to signal host reachability between VTEP's? BGP evpn or flood and learn?

Datacenter interconnect ("DCI")?

Are there any dual homed devices which requires VIP addresses?

And the list continues....

So, what I’m trying to say is that VXLAN is really cool but at the same time complex.

[CiscoPurpleBelt]

In regards to stretching vlans out can you give me an example?

[andcole]

I ran into this very issue with VLAN/VNI mapping in an L2 topology yesterday in a VIRL lab. I had 2 VTEP's (A & B) with VLAN/VNI combinations of 34/34 56/56 and 100/100. Traffic in each VLAN to communicate with other peers in the same, as expected.

I decided to change a mapping on VTEP B, say vlan 56 to 560, but retain VNI 56. No communication, and when I checked with wireshark, I saw VTEP B sending frames encapsulated with VLAN 56, even though this was not configured on VTEP B. I thought BUG, and was going to look at this again today.

Your post explains this behavior in an L2 only topology to me, I had totally missed this point when studying VXLAN! 

Really shows the value in these communities.

Andy

[CiscoPurpleBelt]

What does 560 represent if your vni is still 56? Also, you must put each vlan/vni on each leaf that will have hosts to be in that vlan correct?