cancel
Showing results for 
Search instead for 
Did you mean: 
cancel
2001
Views
5
Helpful
3
Replies

Understanding VXLAN packet flow

Thiyagarajan K
Level 1
Level 1

Hello All,

I'm trying to get a better picture of VXLAN packet flow and could you please help me with the following questions on Nexus 9K platform switches:

How are VXLAN ID's allocated? If it is dynamic and what would be the formula for a new VXLAN?

 

how VXLAN identifies the  host on other VXLAN?

 

In the VxLAN to VXLAN traffic when will the broadcast, multicast and unicast are used?

 

Regards,

Thiyagu

3 Replies 3

jkilleda
Level 3
Level 3

Hi Thiyagarajan,

 

VXLAN is part of L2 switching an L3 routing across ACI fabric . VXLAN is used in MPBGP-EVPN mode inside ACI . Each leaf / spine switch will be assigned VTEP address from the pool of VTEP assigned during APIC initialization.

Hey jkilleda

 

I think you missed the point in Thiyagu's question.  And the MPBGP part is a real distraction - the core operation of ACI VXLAN doesn't invlove MPBGP until an external router is introduced, and I don't think Thiyagu's question had anything to do with routing.  Anyway, between us let's hope we can help - I just wanted to take the MPBGP part out of the already complex question until there is some routing involved. :) Perhaps you meant to say VRF-lite?

 

RedNectar aka Chris Welsh.
Forum Tips: 1. Paste images inline - don't attach. 2. Always mark helpful and correct answers, it helps others find what they need.

RedNectar
VIP
VIP

Hi Thiyagu,

 

I have some advice about trying to understanding VXLAN packet flow in ACI.

 

Tip#1: Don't try to understanding VXLAN packet flow in ACI - ACI just takes care of everything!  It is no more imprtant than trying to understand how packets are encapsulated on the backplane of a Catalyst 6500.  If you are a Catalyst 6500 engineer designing Line cards for the device, it's very important. If all you want to do is push packets through the device - not very important at all.

 

OK. I understand you probably don't like Tip#1, so I'll take your questions and add some.

 


How are VXLAN ID's allocated? If it is dynamic and what would be the formula for a new VXLAN?

VXLAN ID's are allocated to every VRF, BD and EPG. Depending how the packet traverses the fabric, it will be sent with the VNID of the VRF or BD or EPG.  As far as I know, it is dynamic, but I'm afraid I don't know the formula for a new VXLAN. But why do you need to know?  It would be a very unusual case that you'd need to be able to predict what the VNID would be.

 

 how VXLAN identifies the  host on other VXLAN?

Now this question indicates to me that you are NOT talking about the VXLAN used by ACI across the fabric, but by the VXLAN tag assigned to a port group by a vSwitch running on a Hypervisor.  I *THINK* what you are asking is:

 

"If a VXLAN encapsulated packet arrives at an ACI Leaf switch, and later an ACI Leaf switch transmits a packet in VXLAN encapsulation using a different VNID, then how does ACI translate from on VNID to the other?"

 

And the answer to that lies in the key understanding that ACI doesn't do any translation at all. What it DOES do, is see the incoming packet and says "Oh, this packet is encapsulated with VXLAN VNID 1234, therefore the source EPG is EPG666". It then throws the VXLAN encapsulation away (this is called packet normalisation) and is left with the innner payload.

 

It looks at the inner payload and determines the destination using its Local Station Table and its Global Station Table. (And if it can't find the destination, it sends it to the Proxy - but that is another story).  Having determined the destination switch, the ACI leaf re-encasulates the inner original frame in a VXLAN (let's call it aVXLAN - a for ACI) ... in a aVXLAN encapsulation and sends it to the target leaf.  Remember, the original VXLAN encapsulation is long lost.

When the frame arrives at the destination leaf, the destination leaf removes the aVXLAN encapsulation, and looks at the inner packet adn determines the exit port AND exit encapsulation.  It MAY be that the exit encapsulation is VLAN 222 or perhaps VXLAN 333 or even VXLAN 1234 just like the original frame. The exit leaf has no knowledge of what the encapsulation was when it arrived at the ACI edge, and doesn't care. It simple wraps it inthe appropriate encapsulation and sends it on the appropriate port. 

 

No translations necessary.

 

In the VxLAN to VXLAN traffic when will the broadcast, multicast and unicast are used?

Unicast was discussed above.

When a broadcast or multicast arrives at a switch, the process is a little different but similar and depends on the configuration of the Bridge Domain, but in general, you can consider the Bridge Domain as a Broadcast Domain, with an exception for ARP broadcasts, which are treated like IP packets and sent directly to the switch that has the target IP attached (by default - not so if ARP flooding is enabled)

 

So the simple answer is that broadcasts and multicasts are flooded within the Bridge Domain, which potentially involves multiple VLANs and multiple VXLANs - so there is an option in your BD configuration to restrict broadcasts to Encapsulation only - which means that a broadcast/multicast will only be flooded within the same VLAN/VXLAN.

 

I hope this helps

 



Don't forget to mark answers as correct if it solves your problem. This helps others find the correct answer if they search for the same problem


RedNectar aka Chris Welsh.
Forum Tips: 1. Paste images inline - don't attach. 2. Always mark helpful and correct answers, it helps others find what they need.

Save 25% on Day-2 Operations Add-On License