VXLAN How does VNI solve the limit of 4096 VLANs on each device?

[Ali.Raeesi]

Given the 1:1 mapping operation, it does not make sense for only 4096 VLANs on each switch to map to 4096 VNIs, while VXLAN has provided us with 16 million VNIs to solve this problem.!!!

[Christopher Hart]

Hi Ali,

You are correct that VXLAN alone does not allow you to utilize more than 4,096 VLANs (technically speaking, less than that, once you factor in VLANs that are reserved for purposes internal to each device).

However, realistically speaking for most environments, a single leaf/access switch in a data center will not have ~4,000 VLANs configured and operational on it. Even if there are more than ~4,000 VLANs present in your data center network, only the VLANs that you need to be forwarded on an individual switch will be configured and operational. In this design, the advantage of VXLAN is not in the leaf/access switches, but the spine/core switches connecting all leaf switches together. These spine/core switches are simply routers forwarding Layer 3 traffic - they do not need knowledge of every single VLAN configured across the entire data center network.

Let's analyze a practical example. Let's say you have three leaf switches and one spine switch in a VXLAN BGP EVPN network. Each leaf switch has 3,000 VLANs configured. On Leaf-1, VLAN 10 may represent a different broadcast domain than VLAN 10 on Leaf-2. Speaking in VXLAN terms, this means VLAN 10 on Leaf-1 is mapped to a different VNI (e.g. 10010) from VLAN 10 on Leaf-2 (e.g. 20010). When this is done, VLAN 10 on Leaf-1 and VLAN 10 on Leaf-2 are effectively two separate broadcast domains/subnets.

This means that this network has a total of 9,000 unique VLANs/broadcast domains/subnets in it. As we know, in a traditional collapsed core network, this would not be possible because the core switches (which would have all VLANs configured) can only support a maximum of 4,096 VLANs. However, in a VXLAN BGP EVPN network, this is possible because the spine/core switch does not have knowledge of each unique VLAN present in the network. It is simply forwarding Layer 3 packets between VTEPs/leaf switches.

If you have a unique use case where you need to have more than 4,096 VLANs/broadcast domains and all broadcast domains need to be configured on all leaf switches, then you will encounter the 4,096 VLAN limitation you've described. However, this is a niche use case, and when you are working at that kind of scale, you would typically use another solution that does not use IEEE 802.1Q tags to accomplish the same task (such as routing directly to the host and having hosts participate in your routing protocol).

I hope this helps answer your question - thank you!

-Christopher

[Zakaria BEKADDOUR]

Hello,

If we follow the logic that the broadcast domain is relevant only within the VNIs and not within the VLANs, can't we say that the VLANs are completely irrelevant? In the case of a VxLAN fabric, we suppose that we can map up to 16 million VNI to only VLAN 10 and still have 16 million different broadcast domains?

I hope you can enlighten me on this one.

[sp2720401]

Network devices swap VLAN and VXLAN headers in an L3 packet because that is what they do. Nowhere does it say a VNI requires a VLAN.

If you move the VXLAN switching to the host, the acceleration on modern NICs can do VXLAN headers natively at wire speed but you will still need network devices that can do routing, multicast, BGP EVPN networking, and gateway functions at wire speed. 

For most deployments that aren't scaling with OVS/OVSDB or NSX, 2000 vlans per switch is plenty.

They say 4094 vlans but cisco nexus only supports VLANs up to 3967 (3968-4094 are system reserved) and most chassis have a platform limitation of 2000 VLANs.