How Traffic Forwarding Within in ACI Fabric

[williammanurung]

I am newbie to ACI technology and still cannot get the fundamental of ACI.

Anyone can explain detail, how the forwarding traffic within in ACI (between two leaf and if the endpoint on same leaf), how the packet process like encapsulate and decapsulate (tagging vxlan, vtep,etc).

Please give me too datasheet or link to support the theory.

Thanks Before,

William

[micgarc2]

This is kind of a broad question but behavior changes based on if the source leaf knows the destination or not and if the source and dest are on different leafs (have to be vxlan encapsulated).

Watch this and download PPT slides

https://www.ciscolive.com/global/on-demand-library.html?search=3545#/session/1532112832248001txlP

Thank you for participating in the Cisco Support Forum for ACI! If you have other questions related to this post, please let us know. If this response answers your questions, please mark this post "answered" and assign a rating to the response(s) provided. This will help notify other viewers that your question(s) is answered and this helps us provide better responses for this and future questions.

Regards,

Michael G.

[RedNectar]

Hi William,

Firstly the datasheet. If you can find a copy of the Cisco Live presentation BRKACK-3101, then I think you'll get all the information  you want there.  Another option is the DCAC9K courseware - there are some flowcharts in there that are not too bad.

And if given about 90 mins with a whiteboard, I can also explain how traffic is forwarded.  I don't have 90 mins, and I type slower than I talk, so here goes.

A frame arrives at a leaf from an EndPoint.

The leaf examines the encapsulation (VLAN/VXLAN) of the arriving frame to determine the Source EPG, then removes the encapsulation.

The leaf examines the destination of the arriving frame and looks at it Station Tables to see if it know the destination MAC

IF the destination MAC is the leaf MAC, then route the packet - see explanation for routed packets later

IF the destination MAC is found, determine the destination EPG from the Station Table.

IF the source and destination MAC are in the same EPG,  then forward the frame:

1. If the destination MAC is local to the leaf, encapsualte it in the appropriate encapsualtion (VLAN/VXLAN) and forward it.

2. If the destination MAC is remote, encapsulate the entire frame (minus the original VLAN/VXLAN header) in an iVXLAN encapsulation packet addressed to the VTEP of the destination leaf and the VNID of the Bridge Domain.

ELSEIF source and destination MAC are in different EPGs

Check to see if a contract exists that allows the frame.

IF the frame is allowed, then:

1. If the destination MAC is local to the leaf, encapsualte it in the appropriate encapsualtion (VLAN/VXLAN) and forward it.

2. If the destination MAC is remote, encapsulate the entire frame (minus the original VLAN/VXLAN header) in an iVXLAN encapsulation packet addressed to the VTEP of the destination leaf and the VNID of the Bridge Domain.

ELSE (ie there is no contract or the frame is forbidden), drop the frame.

ELSEIF the destination MAC is unknown, forward according to the L2 Unknown Unicast setting - by default it will encapsulate the entire frame (minus the original VLAN/VXLAN header) in an iVXLAN encapsulation packet addressed to the VTEP L2 UNKNOWN MAC anycast address of the PROXY and the VNID of the Bridge Domain.

That pretty much takes care of L2 traffic. But you also need to understand how iVXLAN encapulation works - that a look at https://tools.ietf.org/html/draft-smith-vxlan-group-policy-05 for some clues - it is not the whole story, but the most importnat part is desribed here.

Now L3 traffic.  I don't have time to include External L3 traffic in this story, but for IP packets destined to subnets that are part of the known subnets for a particular VRF, the story is very simlar to layer 2.  REcall I started with:

A frame arrives at a leaf from an EndPoint.

The leaf examines the encapsulation (VLAN/VXLAN) of the arriving frame to determine the Source EPG, then removes the encapsulation.

The leaf examines the destination of the arriving frame and looks at it Station Tables to see if it know the destination MAC

IF the destination MAC is the leaf MAC, then route the packet

Here is the "Route the packet" logic

IF the destination IP is found, determine the destination EPG from the Station Table.

IF the source and destination IP are in the same EPG,  then forward the packet:

1. If the destination IP is local to the leaf, encapsualte it in the appropriate encapsualtion (VLAN/VXLAN) and forward it.  This may invlove having to ARP for the destination MAC address first.

2. If the destination IP is remote, encapsulate the entire frame (minus the original VLAN/VXLAN header) in an iVXLAN encapsulation packet addressed to the VTEP of the destination leaf and the VNID of the VRF.   

ELSEIF source and destination IPs are in different EPGs

Check to see if a contract exists that allows the packet.

IF the packet is allowed, then:

1. If the destination IP is local to the leaf, encapsualte it in the appropriate encapsualtion (VLAN/VXLAN) and forward it.

2. If the destination IP is remote, encapsulate the entire frame (minus the original VLAN/VXLAN header) in an iVXLAN encapsulation packet addressed to the VTEP of the destination leaf and the VNID of the VRF.

ELSE (ie there is no contract or the frame is forbidden), drop the packet.

ELSEIF the destination IP is unknown, encapsulate the entire frame (minus the original VLAN/VXLAN header) in an iVXLAN encapsulation packet addressed to the VTEP L3 UNKNOWN IP anycast address of the PROXY and the VNID of the VRF.

My flight is boarding. That's all for now

I hope this helps

---

Don't forget to mark answers as correct if it solves your problem. This helps others find the correct answer if they search for the same problem

---

[Marcel Zehnder]

DELETED (redundant information)