What table ACI check when traffic enter leaf

[Leftz]

When traffic enters leaf switch, what tables aci system will check in order to reach certain destinations? below are several tables. If  the below is not that correct, can you provide your suggestions? Thanks

1. MAC Address Table:

   • When a packet arrives, the leaf switch first checks if the destination MAC address is on the local leaf switch.
   • If the destination MAC is known locally, the switch forwards the packet to the local port.

2. Endpoint Table:

   • If the destination MAC is not on the local leaf, the switch checks its endpoint table (which is part of the ACI's COOP database) to see if it knows the destination MAC.
   • The COOP database (Council Of Oracles Protocol) keeps track of all endpoint locations across the fabric. If the destination MAC is known in the COOP database, the packet is forwarded appropriately (either to a remote leaf or via a spine proxy).

3. Bridge Domain (BD) Configuration:

   • If the destination MAC is not known, the switch refers to the BD configuration to determine the action for unknown unicast (L2 Unknown Unicast setting), which might be to flood within the BD or forward to a hardware proxy for resolution.

4. Routing Table (RIB - Routing Information Base):

   • For L3 traffic, the leaf switch checks its routing table to see if the destination IP is associated with a subnet known to the local leaf.
   • If the destination IP is known as a local endpoint or part of a local BD subnet, the packet is routed locally.

5. Proxy ARP Table:

   • If ARP flooding is off and the destination IP is not known, the leaf switch may use a Proxy ARP if it is configured to respond to ARP requests for the destination IP on behalf of the actual endpoint.

6. Ternary Content Addressable Memory (TCAM):

   • TCAM is used for high-speed packet forwarding decisions. It stores ACLs, QoS information, and other policy details that might need to be applied to the packet as per the contracts.

7. L3Out Routes:

   • If the destination IP is part of an external network (outside of the ACI fabric), the leaf switch checks for L3Out configurations that determine how to route the packet to external networks.

[balaji.bandi · Accepted solution]

check below some information traffic flow :

https://www.cisco.com/c/en/us/td/docs/switches/datacenter/aci/apic/sw/4-x/aci-fundamentals/Cisco-ACI-Fundamentals-42x/Cisco-ACI-Fundamentals-41X_chapter_0101.html#id_51037

check  Live presentation : (Mastering ACI Forwarding Behavior)

https://www.ciscolive.com/c/dam/r/ciscolive/emea/docs/2020/pdf/BRKACI-3545.pdf

[RedNectar · Accepted solution]

Hi @Leftz ,

Let me start by saying I'm probably not going to add any more information than what you'll find in those excellent references supplied by @balaji.bandi .  But I'll try and relate that same information to your question by adding comments/edits in red to your summary

---

1. MAC Address Table: Let me add a picture
   
   

   • When a packet arrives, the leaf switch first checks to see if the MAC address is its own - i.e. the endpoint has sent a packet to the Leaf SVI default gateway 
   • If not, it then checks if the destination MAC address is on the local leaf switch.
   • If the destination MAC is known locally, the switch forwards the frame packet to the local port. [Pedantic change I know, but in this case it makes sense to be clear when talking about Layer2 vs layer3]
   • If the destination MAC is known on a tunnel interface, the switch encapsulates the frame and forwards the VXLAN encapsulated packet to the tunnel destination IP using the VNID of the BD
   • [I've moved this point from the Endpoint Table section because it's still about MAC address resolution] 
     If the destination MAC is not on the local leaf, the switch checks the settings for L2 Unknown unicast for the BD
     • If L2 Unknown unicast for the BD is set to flood then the switch encapsulates the frame and forwards the VXLAN encapsulated packet to the BD Group Outer IP multicast address using the VNID of the BD
     • If L2 Unknown unicast for the BD is set to proxy then the switch encapsulates the frame and forwards the VXLAN encapsulated packet to the L2 Proxy IP anycast address using the VNID of the BD

2. Endpoint Table:

   • [The Endpoint table is really just a combined view of the MAC address table and the (enhanced) ARP table - it is not really part of the COOP database, the COOP database is maintained ONLY on the Spines, but created from the information that the leaves send about their endpoints]
     If the destination MAC is not on the local leaf, the switch checks its endpoint table [Not quite true - see above]
     (which is part of the ACI's COOP database -[Not really - see above]) to see if it knows the destination MAC.
   • The COOP database (Council Of Oracles Protocol) keeps track of all endpoint locations across the fabric.[Correct] If the destination MAC is known in the COOP database, the packet is forwarded appropriately (either flooded to a remote leaf or via a spine proxy). [As you explain later, the action for unknown MAC is determined by the L2 Unknown unicast setting for the BD]

3. Bridge Domain (BD) Configuration:

   • If the destination MAC is not known, the switch refers to the BD configuration to determine the action for unknown unicast (L2 Unknown Unicast setting), which might be to flood within the BD or forward to a hardware proxy for resolution. [Correct] 

4. Routing Table (RIB - Routing Information Base): Let me add a picture
   

   • NOTE: If ARP flooding is turned off, ARP packets are treated as L3 traffic. Here's a picture of how ARP is handles
     
     
   • For L3 traffic, the leaf switch checks its Endpoint table to see if the destination IP is known
     • If the destination IP is known locally, the switch forwards the packet to the local port.
     • If the destination IP is known on a tunnel interface, the switch encapsulates the packet and forwards the VXLAN encapsulated packet to the tunnel destination IP using the VNID of the VRF
   • If the endpoint is NOT known in the Endpoint Table then the switch checks the routing table to see if the destination IP is associated with a subnet known to the local leaf. [Correct - although just because the subnet is known to the local leaf, it does not necessarily mean that there are any local endpoints on that subnet] 
     • In this case, (endpoint subnet known) the routing table for the leaf will show either the TEP of a border leaf (for external subnets - see later) or the TEP of the L3 Proxy IP anycast address.
       • In the case that the IP is unknown in the COOP database, the spine sends a special ARP Glean request to every switch in the BD using the BD Group IP outer multicast address
       • Every  switch in the BD will then send an ARP request for the unknown IP address on all ports that are part of that BD 
   • If the destination IP is known as a local endpoint or part of a local BD subnet, the packet is routed locally. [As detailed above]

5. Proxy ARP Table:

   • If ARP flooding is off Intra EPG Isolation is Enforced for an EPG and the destination IP is not known, the leaf switch may use a Proxy ARP if it is configured to respond to ARP requests for the destination IP on behalf of the actual endpoint. [Proxy ARP is a strange beast in ACI, it's not normally needed, but it has nothing to do as to whether ARP flooding is enabled or not. You can read more about this feature here]

6. Ternary Content Addressable Memory (TCAM):

   • TCAM is used for high-speed packet forwarding decisions. It stores ACLs, QoS information, and other policy details that might need to be applied to the packet as per the contracts. [Correct] 

7. L3Out Routes:

   • If the destination IP is part of an external network (outside of the ACI fabric), the leaf switch checks for L3Out configurations that determine how to route the packet to external networks. [Correct. As mentioned above, when a leaf has to forward an IP packet that is NOT in the endpoint table, it looks at it's routing table, just like any other router.  Whatever is the next-hop address will determine where the packet goes.  If the destination subnet is external, then the next-hop IP will be the TEP of a boarder leaf.] 

There is much more fantastic information in BRKACI-3545 as suggested by @balaji.bandi 

[Leftz]

Thank you very much for your nice reply!