Contracts with ACI

[dhr.tech1]

Hi All,

while reading through many Cisco articles, I am slightly confused on when and where we need to use the contracts.

1. Scenario 1: Host A (EPG1/BD1) -Leaf 1 is trying to connect to Host B (EPG2/BD2) - Leaf 2 with pervasive gateways. 

In this scenario we create a contract to allow the communication and hence the Leaf 1 knows the route for host B, as the routes are imported into leaf 1. Finally leaf01 send the traffic to spine proxy, which later sends the traffic to host B.

2. Scenario 2: Host A (EPG1/BD1) -Leaf 1 is trying to connect to Host B (EPG2/BD2) - Leaf 2 with pervasive gateways. 

In this scenario if we don't create a contract. Leaf 1 don't know the destination but can still send traffic to spine proxy ? 

I am not able to understand the difference between above 2 scenarios, as in both case Spine receive the traffic. Only scenario, spine don't receive the traffic when both leaf knows source and destinations in LST/GST.

[RedNectar · Accepted solution]

Hi @dhr.tech1 ,

Let me analyse your thinking

---

1. Scenario 1: Host A (EPG1/BD1) -Leaf 1 is trying to connect to Host B (EPG2/BD2) - Leaf 2 with pervasive gateways. 

In this scenario we create a contract to allow the communication

CORRECT

and hence the Leaf 1 knows the route for host B, as the routes are imported into leaf 1.

CORRECT - when the contract is applied to both EPG1 and EPG2, the APIC will install the route FOR HOST B SUBNET into Leaf 1 - this route will say "Host B's subnet is reachable via the Spine Proxy"

Finally leaf01 send the traffic to spine proxy, which later sends the traffic to host B.

NOT QUITE RIGHT. Leaf 1 sends the traffic to the spine proxy. Next come several possibilities

1. The Spine Proxy knows that Host B is located on Leaf 2 because Leaf 2 reported this information tot he Spine via COOP
   1. In this case, the spine sends the packet to Leaf 2 (NOT to host B as you said)
   2. When the leaf receives that frame, it forwards it to Host B
2. The Spine Proxy doesn't know where Host B is
   1. If ARP flooding is DISABLED for the BD, the spine initiates an ARP glean process which is:
      1. The spine sends a special frame to each leaf switch that participates in BD2 (in your scenario)
      2. Each Leaf switch in BD 2 tries to find Host B by sending ARP requests sourced from the pervasive gateway IP for DB 2
      3. Presumably, Host B on Leaf 2 replies to the ARP request
      4. Leaf 2 reports that Host B is local to Leaf2
      5. The Spine now carries on with the original frame as described in 1.1 and 1.2 above
   2. If ARP flooding is ENABLED (and the spine doesn't know about Host ) the frame is dropped

2. Scenario 2: Host A (EPG1/BD1) -Leaf 1 is trying to connect to Host B (EPG2/BD2) - Leaf 2 with pervasive gateways. 

In this scenario if we don't create a contract. Leaf 1 don't know the destination but can still send traffic to spine proxy ? 

NO. (Probably) In this case The APIC has no reason to install the route for Host B's subnet in Leaf 1. Host 1 will send the frame (intended for Host B) to the pervasive GW address, and the leaf will drop the frame because it has no route. UNFORTUNATELY ACI Leaf switches do NOT send ICMP destination unreachable or ICMP destination administratively prohibited packets which would be a help when troubleshooting.

Now let me explain the reason I added "probably" above.

It could be (quite likely in fact) that Leaf 1 does indeed have the route to Host B's subnet via the proxy installed because another EPG in the same VRF on Leaf 1 has a contact with EPG2, or even because there are instances of BD2 on Leaf 1.

In this case, it comes down to the contract. If Leaf 1 knows that IP B is in EPG2 and there is no contract, it will drop the frame. But if Leaf 1 does NOT know what EPG Host B is in, it will send the frame to the Proxy BUT SET A FLAG WITHIN THE iVXLAN header indicating that policy has NOT been applied.

The process will follow as above until this frame reaches Leaf 2, at which point Leaf 2 will see the flag and apply policy - i.e. drop the frame

I am not able to understand the difference between above 2 scenarios, as in both case Spine receive the traffic.

Hopefully the above has made it clear.  You may also find this blogpost I wrote about ARP Gleaning in 2018 useful

[Disclaimer, the link above is to my personal blog]

[AshSe]

In addition to what @RedNectar mentioned, below is my one cent towards understanding ARP Gleaning:

When an ACI leaf fails to learn a local endpoint, Cisco ACI has several mechanisms to detect such silent hosts. These mechanisms are configured under the Bridge Domain (BD).

Mechanisms to Detect Silent Hosts:

1. Layer 2 Unknown Unicast:
1. Floods Layer 2 switched traffic to an unknown MAC address.
2. ARP Flooding:
1. Floods ARP requests with a broadcast destination MAC address.

In addition, Cisco ACI uses:

1. ARP Gleaning:
• Sends ARP requests to resolve the IP address of an endpoint that is yet to be learned (silent host detection).
• Applies to Layer 3 routed traffic regardless of configuration, such as ARP flooding, as long as the traffic is routed to an unknown IP.

If the spine doesn't know where the ARP request's destination is (the target IP isn't in the COOP database), the fabric sends an ARP request from the bridge domain SVI (pervasive gateway) IP address. This ARP request is sent out to all the leaf nodes' edge interfaces that are part of the bridge domain.

ARP Gleaning Requirements:

1. The IP address is used for forwarding, which means:
1. ARP requests are sent even if ARP flooding is disabled, or
2. Traffic is sent across subnets with the BD SVI as the gateway.
2. Unicast routing is enabled.
3. A subnet is created under the bridge domain.

[dhr.tech1]

Thanks Ash, but in a nutshell if there is no contract for inter-EPG communication, traffic will be dropped ? right

[AshSe]

Yes, @dhr.tech1  you are right. In the absence of a Contract; End Points in two different EPGs can't talk to each other. Below is the diagram to understand and answer your query: