VxLAN VNI to VLAN Mapping

[visitor68]

Folks - this may seem like a pretty fundamental question, but the implementation details still escape me. While it is the case that VxLAN offers up to 16 million VNIs in principle, every time I see an implementation of VxLAN, there is a 1:1 mapping with VLANs. In that case, you're not getting more than 4,096 VNIs. So, how does one take advantage of the 16 million VNI capability? 

Before seeing these ubiquitous implementations, I always assumed a hierarchical model in which a VNI would map to multiple VLANs - up to 4096. But that is not the case it seems.

[balaji.bandi]

VLAN use a 12 bit for  id, VXLAN uses a 24bit ID or VNI with 16 million possible segments.

VLAN  id is mapped to a VNI to extend a vlan across a layer 3 infrastructure, it encapsulating layer 2 into an IP packet and routing it across the network. So a VLAN ID is associated with a VNI in a VXLAN n environment and is a method to transport a VLAN  across an underlying L3 infrastructure.

Suggestion to read : (good cisco document to understand VXLAN)

https://www.cisco.com/c/en/us/products/collateral/switches/nexus-9000-series-switches/white-paper-c11-729383.html

BB

[visitor68]

I know everything you said already. You haven't answered my question. Thanks anyway. 

[Vinit Jain]

Hi

What is said is true for an individual VTEP that there is a 1:1 mapping between the VLAN and the VNI but in a VXLAN fabric, you can have different VTEPS with different VLANs be mapped to a same VNI. For instance, VLAN 100 on VTEP-1 can be mapped to VNI 10000 and VLAN 200 on VTEP-2 may be mapped to the same VNI 10000. thats how you can scale within the fabric. 

Also, there might be future enhancements that might allow more VNI's on a single VTEP.

[visitor68]

Thanks. I know this. But you still have ONE VLAN mapped to ONE VxLAN. The fabric will have 4,096 VLANs and so it will have 4,096 VxLANs. 

[visitor68]

Hmmm. this board aint what it used to be. A few years ago, Giuseppe, Jon Marshall, Edison Ortiz, et al, would have been all over this question...

[Rajeshkumar Gatti]

The leaf switch imposes a limit on the number of vlans (4k). The fabric could allow you to have 16 million VNI segments depending on the scale of the deployment.
If each leaf switch is configured with the same number of vlans say 1000 vlans and each is assigned to an unique VNI then in theory the number of VNI segments(in the fabric) = 1k x number of leaf switches. So we could very well scale beyond the 4k limit that traditional L2 design imposes and that is what can be achieved through the VXLAN design.

Typical customer deployment tend to use the 1:1 mapping and stick to using the 4k limit at the fabric level but it can always and easily be exceeded as needed.

Hope this helps.

[CiscoBrownBelt]

So are you saying create different vlans/vni on each switch? Doesn't matter if you put vlan2/vni10002 on leaf 1 you can assign a host port to vlan 2 on leaf 4 for example?

[Rajeshkumar Gatti]

Hi,

Apologies but I don't want my previous answer to be taken literally wherein each leaf has different VNI mapping for the same vlan and expect it to communicate seamlessly. The point I was trying to make is how scalable the fabric can be without having an absolute dependency of a 4k vlan limit at the leaf. The use of different VNI will give you the boundaries/separation/segments as you desire from your design.

This bog can be a good starter

https://blogs.cisco.com/datacenter/digging-deeper-into-vxlan

-Raj

[Theodossis Mantzouratos]

The same VLAN-ID (VLAN1-to-VNI1) can be mapped to a different VNI (VLAN1-to-VNI2) on another VTEP in the fabric, and then the new VLAN-to-VNI combination can carry a whole different kind of traffic and it can also be assigned a different IP address space.

e.g.

VNI 123456 --> VLAN 456 --> 10.1.1.0/24 --> VOICE

VNI 321456 --> VLAN 456 --> 10.2.2.0/24 --> VIDEO

VxLANs do not carry VLAN IDs (not by default). VLAN IDs are locally significant.

Within the fabric, the VNI is the significant ID, not the VLAN ID. Check the VxLAN header .

Truth, the VLAN limitation still stands locally on the device (VTEP), but not in the fabric.

Hope this helps you.

Theo

[CiscoBrownBelt]

Sorry been out of mix a bit. Awesome thanks!

[GangadharHariharan83171]

How do we differentiate what VNI should be applied to a packet coming in on a port belonging to VLAN 456, based on the following example?

VNI 123456 --> VLAN 456 --> 10.1.1.0/24 --> VOICE

VNI 321456 --> VLAN 456 --> 10.2.2.0/24 --> VIDEO

[Theodossis Mantzouratos]

You won't see any VNI on any switchport (either physical or virtual) other than the NVE interface. Only the VTEP interface can encapsulate/decapsulate VxLAN packets on a VxLAN/EVPN enabled device.

Keep always in your mind that all VxLAN/EVPN traffic is L3 (routed), NOT L2 (switched), and that's the whole idea behind network overlays like VxLAN, TRILL, Fabric path etc, to use a regular L3 transport so to pass L2 traffic across.

Switching actually happens after the VxLAN packet has been received, decapsulated and pushed to the switching engine as a regular ethernet frame.

Again, the VLAN-to-VNI mapping is locally significant (local on each device/VTEP) and has to be unique within the device, meaning a VLAN-ID can be mapped only to one VN-Segment. BUT... each device within the fabric can have different VLAN-to-VNI mapping configured.

In general, you have to deal with two worlds within a VTEP, the classical ethernet which applies to the switchports and follows all switching rules that you know and the overlay world which applies to the NVE and allows the classical ethernet to span across your network.

It might get even more complex but I hope that was a bit helpful for you.

Theo.

[JohnJoseph]

https://www.cisco.com/c/en/us/products/collateral/switches/nexus-9000-series-switches/white-paper-c11-732453.html

VXLAN bridging: VXLAN bridging is the function provided by VTEP devices to extend a VLAN or VXLAN VNI over the Layer 3 infrastructure. Figure 3 shows VLAN-to-VLAN and VXLAN-to-VXLAN bridging.

VXLAN routing: VXLAN routing is also referred to as inter-VXLAN routing. It provides IP routing service between two VXLAN VNIs in the overlay network in a way similar to inter-VLAN routing. Figure 4 shows the logical concept of VXLAN routing.

https://www.cisco.com/c/en/us/td/docs/switches/datacenter/nexus5600/sw/layer2/7x/b_5600_Layer2_Config_7x/config_vxlans.html

The classical Ethernet (CE) packet on an edge interface is mapped to a Virtual Network Identifier (VNI) based on the VLAN to which it is associated. The VLAN to VNI mapping is created under the VLAN configuration, which limits the number of supported VNIs on a switch to 4000.

Even though 16 million VNIs are possible, is there a use case for at least 1 million VNIs ? You have to use 250+ times each of those 4000 vlans, even with 2 hosts in each vni we are looking at 2 million mac table entries. 

[Patrick Colbeck]

I dont think anyone would ever use the number of VNIs that are theoretically possible but that number just comes from the number of bits in the relevant field in the VXLAN header. Having enough bits for a large number is a better design for the protocol than having to revisit it later because there weren't enough bits if someone does come up with a use case for that many VNIs.

As has already been pointed out the approx 4K limit of VLANs is a per switch not per fabric limit. I can envisage a large multi-tenant fabric where more than 4K segments are required across the whole fabric which would translate into >4K VNIs. The fact that these map to <4K VLANs per VTEP/Switch is not an issue as the VLANs only have local significance on that switch.