ACI Contract Inter EPG

[netbeginner]

Hi All,

This is related to cisco ACI contract. 

ACI Version 4.1

We have a VRF (Eg : VRF-A) with multiple EPGs.

There is a contract applied on VRF-A with ANY-ANY rule, which obviously reflect for EPGs as well.

We want to apply a Contract restriction (allow specific port only) between 02 EPGs in VRF-A   (Eg :  EPG-1 and EPG-2)

We have applied a standard contract / rule set (Bi-Directional) with PROVIDED and CONSUMED on both EPG-1 and EPG-2 , still All things are working as like before (i.e. no restriction been applied and effective).

We also checked by applying Taboos contract (Created with specific port allow), but this time result is something different .....Everything stopped working between EPG-1 and EPG-2.

Pls note that Taboos contract by-default applied with PROVIDED only. it's not available with CONSUMED

Read somewhere that taboos contract will take priority always and taboos contract will be applied where there is ANY-ANY contarct applied on respective VRF (just like our case). 

please guide what we are missing and why our contract for allowing specific port is not taking effect.  

[Sergiu.Daniluk]

Hi @netbeginner 

The vzAny-to-vzAny contract is the one allowing the traffic. Basically, the traffic is matched against the zoning rules, in the following order:

1. contracts between specific EPGs
2. contracts defined for vzAny-to-vzAny
3. Preferred group
4. vzAny configured to provide and consume a contract with a filter such as common/default
5. The implicit deny

In other words, if your flow will not match on the EPG-to-EPG contract, but it will match on the vzAny-to-vzAny contract, thus will be allowed.Note: the flow does not match because of the specific ports configured in the EPG-to-EPG contract. Zoning rules table, works basically like an ACL.  The reason why it works like this is for example to have vzAny with a contract to redirect all traffic to an external firewall and then to use specific EPG-to-EPG contracts for specific traffic for direct communication, without being redirected to firewall.

What I believe you can do is to add another filter into the existing EPG-to-EPG contract, filtering on IP (so basically any traffic) with action "deny" and priority "medium" or "lowest".

Stay safe,

Sergiu

[Robert Burns]

Rather than using vzAny with a permit-any rule, you might be better served using Preferred Groups and/or standard Contracts if you need more granular policy.  You could keep all EPGs other than these two in a Preferred Group, then apply your standard contract between EPG-1 and EPG-2.  Your problem is that you're never going to hit the implicit Deny with a vzAny contract allowing any:any.  Deny rules (standard contracts) do take precedence over vzAny, but its a deny:specific type of rule.  You want an "only-allow" type of rule which you can only achieve with a discrete contract and implicit deny catching everything else. 

• vzAny - Defines communication between all EPGs within a VRF (in your case any any:any rule)
• Preferred Groups - Allows unrestricted communication between select EPGs within a VRF
• Contract - Allows specific communication between EPGs
• Taboo - Denys specific communication coming into an EPG.

Security policies within ACI follow this precedence:

1. Inter-EPG Contract
2. Inter-EPG Isolation
3. Taboo Contract
4. EPG-to-EPG (specific filters)
5. EPG-to-EPG (default filters)
6. EPG-to-vzAny
7. vzAny-to-vzAny (specific filters)
8. vzAny-to-vzAny (default filters)
9. any-to-any Implicit Deny

Robert

[RedNectar · Accepted solution]

Hi @netbeginner ,

I went to reply to this question when I saw that you have two excellent answers already, so I hesitated. Then decided I'd throw in some personal ideas for ACI best practice. I'll call them...

RedNectar's Rules for ACI contracts

1. Never apply a contract that is both consumed and provided in each direction.  Doing so just shows that the implementer can't figure out which EPG contains the servers and which contains the clients.
   • Exception: When using first generation switches, there was a legitimate use for using a contract that allowed TCP Established traffic to be consumed and provided by vzAny to conserve TCAM, but it was a kludge to overcome a design weakness of too little TCAM to do the job properly.
   • OK. There may be other exceptions too, such as if one EPG's servers need to SSH to another EPG's servers, and vice-versa. But even then, I might be tempted to create two identical contracts (EPG.A.SSH.Service_Ct and EPG.B.SSH.Service_Ct) to save confusion.
2. Never have vzAny provide and consume the default contract or any other contract that allows all traffic. I realise this rule is redundant given Rule#1 above, but I really want to emphasise that this is a VERY BAD IDEA and I am ASTOUNDED at the number of sites that have implemented such a contract, and even more astounded that some so-called experts have suggested using such a design. Doing so just shows that the implementer has no idea that the same result can be much more easily achieved by setting the VRF Policy Control Enforcement Direction to UNENFORCED and is setting the environment up to run into the exact problem you have in the future.
   • If the design calls for all traffic to be permitted between all EPGs in a VRF, achieve this using Preferred Groups as my esteemed colleagues have pointed out.
3. Never use Taboo Contracts. And avoid using deny statements in all contracts, life will be simpler if you go to the trouble of sorting out what services each EPG needs to provide, and which services each EPG needs to consume, and creating appropriate contracts for each service.

So now comes the tricky question of how to apply these rules to your situation.  These are the steps I'd recommend, assuming the EPGs you wish to isolate are EPG.A and EPG.B:

Part 1: Preferred Groups

1. Enable Preferred Groups in the VRF: Tenants > YourTenant > Networking > VRFs > YourVRF >| [Policy] tab
2. Make each EPG except EPG.A and EPG.B members of the preferred group: Tenants > YourTenant > Application Profiles > YourAppProf > Application EPGs > Each_EPG_in_turn >| [Policy] >| [General]

Part 2: Filters and Contracts:
This is the tricky bit, and has several sub-stages:

Individual EPGs stage

1. Identify what services need to be PROVIDED by EPG.A. In other words, examine all the listening TCP/UDP ports in all the endpoints in EPG.A
2. Create a filter based on each service, typically named after the service, such as SQL.Service_Fltr and add the appropriate TCP (or UDP) port(s) for that service as the destination port.
   • Tip: Create all filters in the common tenant
   • If you have services that negotiate TCP/UDP ports dynamically, such as FTP, then you are screwed - ACI is NOT able to to handle traffic at the Application layer. (I've always wondered why ACI is actually called Application Centric Infrastructure). If this is the case forget all the above and work out how to use a firewall.
3. Create a Contract to accumulate all the different services for EPG.A that are to be made available to EPG.B. Perhaps call this EPG.A.to.EPG.B.Services_Ct.
4. Have EPG.A provide this contract
5. Have EPG.B consume this contract
6. Repeat steps 1-5 above swapping EPG.B with EPG.A (if only ONE EPG is actually providing any services, this step may not be necessary)

EPG.A and EPG.B services stage. In this case, I'm assuming that the remaining EPGs that are communicating happily with each other using Preferred Groups as described in Part 1 will be allowed to consume all services from both EPG.A and EPG.B, and both EPG.A and EPG.B will be allowed to consume all services from all other EPGs

1. Create a Contract to accumulate all the different services for EPG.A that are to be made available to all the other EPGs.  Recall you created filters for ALL services for EPG.A in step 2 above. Perhaps call this EPG.A.All.Services_Ct.
2. Have EPG.A provide this contract
3. Have each other EPG consume this contract
4. Repeat steps i-iii above swapping EPG.B with EPG.A 

Other EPG Services stage. In this case, I'm assuming that both EPG.A and EPG.B will be allowed to consume all services from all other EPGs.

1. Create a filter that allows all traffic - there is one in the common tenant called default, but I prefer to spell it out and create an All.IP.Traffic_Fltr and restrict it to all IP traffic rather than the default ALL traffic.
2. Have every EPG EXCEPT EPG.A and EPG.B provide this contract
3. Have EPG.A and EPG.B consume this contract

I hope this helps.

---

Don't forget to mark answers as correct if it solves your problem. This helps others find the correct answer if they search for the same problem

---

[Sergiu.Daniluk]

"RedNectar's Rules for ACI contracts" - love it!

[netbeginner]

Fantastic ! Explanation RedNectar.... 

@ Sergiu : You mean to add one more rule in Contract Rule Group with Deny all and then apply . is it...

[Sergiu.Daniluk]

Yes, but with lower priority, because by default deny filters have higher preference.

[weylin.piegorsch]

Might PBR to a 2-arm FW be another example of a contract that can be both provided and consumed by an EPG?

• EPG1: protected by FW
• EPG2: protected by FW
• EPG3: not protected by FW but must traverse the FW when going to EPG1 and EPG2
• EPG4: not protected by FW but must traverse the FW when going to EPG1 and EPG2

1. EPG1: not member of preferred group, provides contract1_2 that PBR's to FW "outside" interface, consumes contract1_2 and also contract3_4 that PBR's to FW "inside" interface.
2. EPG2: not member of preferred group, provides contract1_2, consumes contract1_2 and contract3_4
3. EPG3: member of preferred group, provides contract3_4, consumes contract1_2
4. EPG4: member of preferred group, provides contract3_4, consumes contract1_2
5. L3out External EPG: member of preferred group, provides contract3_4, consumes contract1_2

The real rationale I'm considering is to extend one step further, which is to make EPG1 a contract master for FW'd EPGs, and EPG3 a contract master for non-FW'd EPGs.

[RedNectar]

Hi @weylin.piegorsch ,

First of all, you have not given enough information.  When dealing with contracts, you MUST think about

1. which servers are providing service(s)
   • And precisely WHICH service(s) are being provided
2. which EPGs are consuming which services

Note that in 1 above I said SERVERS - to emphasise that servers provide services, consumer end points don't (or if they do, that should be considered in a separate contract)

So my first question about you example is:

What service is EPG1 providing?

And if it is the same service as EPG2 is providing, then why are they in different EPGs?

And one final point regarding 'EPG1: not member of preferred group, provides contract1_2 that PBR's to FW "outside" interface'

PBR is placed in a contract between two EPGs - NOT to a FW "outside" interface - that sounds more like a normal contract that has nothing to do with PBR.

---

Other Points

1. I know a lot of people find it too hard to figure out what services are being provided by what, and defer to a permit any style contract - and IF you are relying on a FW to filter ALL traffic, then doing something like using vzAny to provide and consume a permit-any type contract gives you the flexibility to selectively peel off certain flows (say between the DB servers and the Backup servers of specific port numbers) via a direct EPG-to-EPG contract.
   • Perhaps it is something like this that you are trying to do, but when I tried to draw a diagram of your scenario I got a lot of lines that didn't seem to make sense.
2. Contract Masters are a whole new discussion and worth a much deeper discussion that I've got time for today. But Contract Masters can be a great way of rationalising contracts between a lot of EPGs

[weylin.piegorsch]

Thanks @RedNectar. Very well-stated reply - good point that the question isn't sufficiently formed.

I currently have a FabricPath network that looks generally like this (i've chosen convenient names, though backward from above (oops)), that Im trying to migrate to ACI:

FW is in vWire mode (L1, not even VLAN translation).  Also, no BD has more than one EPG (if that's even pertient here).

Hopefully I can streamline the discussion - security team is not allowing me to optimize or get creative with what is and what isn't directed to the FW, regardless what devices are protected or what services those devices are providing/consuming.  If a TCP SYN or ECHO_REQUEST is going to a protected EPG, it gets directed to the FW's outside interface, even if it's originated from a protected EPG. (I think we can leave other traffic patterns aside for now, this likely is sufficient to debate the merits of the question.)

Given this is a L1 Palo Alto FW, we're evaluating L1/L2 PBR service graphs.  I understand those require a contract? (We first tried selective BD stiching, that didn't seem to scale well or be a good fit for my operations team.)

In this case, does it make sense for EPG3 and EPG4 to provide said contract, and all EPGs (including EPG3 and EPG4) consume them?  It cant be vzAny to provide since there are non-FW'd VLANs, and vzAny cant consume a contract for L1/L2 PBR.

I acknowledge you're point here: "But even then, I might be tempted to create two identical contracts (EPG.A.SSH.Service_Ct and EPG.B.SSH.Service_Ct) to save confusion."  Unfortunately, there's going to be a LOT of protected and non-protected EPGs (70 protected at current count, I havent yet properly counted non-protect) that require any-to-any communication with FW'ing of proteced EPGs. a full mesh of contracts would be Just Too Hard.

--------------

Good point about contract masters. separete discussion... if needed (they "seem" like a straight-forward enough macro).