IP Addresses and Subnets in Contracts

[weylin.piegorsch]

I feel like I'm missing something that should be obvious. I'm at a loss for identifying where to define a host or a subnet as a source or destination within the contract policy heirarchy.

If I have a contract, I can specificy allowed/denied TCP and UDP port ranges between two EPGs.  But I can't define subnets and IPs?

If I'm doing a brownfield migration, and I'm starting with endpoints within a VLAN/subnet that need different policies to replicate their existing ACL (there's no firewall in the particular set of cases I'm considering), do they need to be in different EPGs with different contracts?  That will NOT scale at all for what I need - I'll end up with hundreds or possibly thousands of EPGs, with something approaching a full mesh of several thousands of contracts, for just a handful of ACI subnets, compared to IOS's max of #ACL's = 2x #VLANs.

weylin

[Ali Aghababaei]

@weylin.piegorsch 

In Cisco ACI, you cannot directly specify individual subnets or IPs within a contract. Contracts in ACI apply to the entire Endpoint Groups (EPGs) they are associated with. To achieve more granular control similar to traditional ACLs, you need to:

1. Use Micro-Segmentation: Use micro-segmentation with Endpoint Security Groups (ESGs) or use attributes like IP addresses to create smaller EPGs.
2. Use Application Profiles: Define specific EPGs for different sets of hosts or subnets requiring unique policies.
3. Leverage Filters: Use filters within contracts to allow or deny specific types of traffic (TCP/UDP port ranges).
4. To handle more granular policies without creating numerous EPGs, you can leverage Cisco ACI's Layer 4 to Layer 7 (L4-L7) service insertion. This approach allows you to use service devices, like firewalls or load balancers, to enforce more specific rules for traffic within the same EPG or between EPGs.

[RedNectar]

Hi @weylin.piegorsch ,

Part of the problem with applying policy based on IP addressing is that IP addresses often indicate both location and identity, and traditional methods of applying policy using access control lists forces you to apply apply policy based on IP addresses, you can't apply policy based purely on location or purely on identity.

For instance, to apply policy between two hosts with IP addresses of 10.10.10.10 and 10.10.10.11 using ACLs is impossible unless you arduously resort to using MAC addresses to indicate identity.

ACI gets around that problem by introducing a concept that is independent of IP Addressing - the EPG.

Typically, endpoints are assigned to EPGs based on the VLAN ID. So if your design has one subnet per EPG, go ahead and design it that way.

But a better approach is to look at which servers/endpoints are permitted to communicate with each other without restriction, and have the same policies applied when communicating with other groups of servers.  This should be the basis for your EPGs rather than purely based on subnets.  Another approach is to group EPGs with similar policies into Application Profiles (you need at least ONE Application Profile, and EPGs get a little hard to manage if you lump all of the into one Application Profile as is often the case, so whichever approach you take, it is a good idea to think about grouping EPGs into Application Profiles right from the get-go)

Having said that, very few places implement ACI like this - it is much more common to stick with the "one VLAN=one subnet=one EPG" philosophy, sometimes referred to as "Network Centric Design"

So let's look at your concern of the number of contracts needed.

Firstly, understand that you can use the same contract over and over between different EPGs.  And ACI provides several ways of building contracts in a scaleable way. The Cisco ACI Contract Guide White Paper describes these, but is woefully lacking in practical examples, and some examples are close to useless. I've summarised the items I think you should explore first when designing a solution

• Contract Scope
  • If you set that scope of a contract to Application Profile, the same contract can be used between similar pairings of EPGs without traffic leaking between Application Profiles
• Contract Inheritance and Master EPGs 
  • a feature that allows an EPG to inherit the contract relationships of another EPG (known as the Master EPG)
• Preferred Group
  • allows you to group a collection of EPGs (not necessarily in the same Application Profile, but linked to the same VRF) to have full connectivity between themselves
    • Unfortunately, you can have only ONE preferred group per VRF, but a similar result can be achieved using Contract Scope and Application Profiles as described above, or using ESGs (Endpoint Security Groups)
• ESGs (Endpoint Security Groups)
  • Unlike EPGs which are identified (mostly) by 802.1q tags, ESGs are Security zones linked to a VRF and can be based on more flexible criteria such as IP addressing (ACI v5+)
    • You pretty much use ESGs OR EPGs - you can't have a contract between an ESG and an EPG

I haven't explored the possibility of using VRFs to group similar groups of servers, but that is another possibility, but applying contracts gets tricky because the Provider EPG(s) need to have subnets defined on the EPGs

[weylin.piegorsch]

Thanks @Ali Aghababaei and @RedNectar. I had a proper responses written about why some of those won;t work in my environment, but it became very "woe is me" sounding so I'll save you from that.

Summary:
I can't really crack up the EPGs (I need to be Network-Centric on that point);
Contract Inheritance, Master Contracts, and contract labels aren't really something I can do; and
I've to-date found insufficient commonality from one legacy ACL to another to be able to take advantage of code re-use.

But....

Is there a meaningful difference to Preferred Groups vs a contract that is the equivalent of "permit IP any any", applied to vzAny as both consumer and provider?

I had a quick thought on ESGs but dismissed it quickly, recognizing there were issues with mixing EPG and ESG contracts, and separately there were issues with using ESG contract with the L4L7 PBR we're developing (one fabric will have this (we hope), the other two fabrics will have no L4L7 devices).  Am I mistaken to dismiss ESGs so casually?

I was hoping to avoid microsegmentation (TAC always gets antsy whenever the topic comes up, then audibly gets relieved when I say we don't have any yet, plus I'm running into issues already with the number of VLAN IDs I'm already using even without that) by hearing about potential other approaches.  I'll keep my fingers crossed for finding an alternative solution?

I do spread EPGs across various APs.  How do I leverage that to aid in redefining legacy ACLs in an ACI security model, with recognizing that (for example) "ZenossProd_EPG" won't be split into Web/App/DB/etc EPGs? (In practice, we static path bind to VMware ports that, for now, don't do VMM Integration, and the VMware team would have their own challenges in their operations team supporting multiple port-groups / VLAN IDs for the same subnet).

[RedNectar]

Hi @weylin.piegorsch ,

OK. Looks like you have a difficult situation and have already made some good choices regarding Application Profiles and EPGs anyway. And I heartily endorse your excellent naming scheme.

I may not have a great answer for you, but I can address a couple of things.

---

Is there a meaningful difference to Preferred Groups vs a contract that is the equivalent of "permit IP any any", applied to vzAny as both consumer and provider?

Yes. With the Preferred Group (NOTE: Preferred Group NOT Preferred Groups - you only get one Preferred Group per VRF) you could have say 5 EPGs in the Preferred Group that don't need any contract to communicate, and maybe another 5 EPGs linked to the same VRF that DO require a contract to communicate with another EPG.  If you used a contract with a Permit IP filter that was provided and consumed by vzAny for the VRF, then all 10 EPGs for that VRF would have full IP communication.

I had a quick thought on ESGs but dismissed it quickly, recognizing there were issues with mixing EPG and ESG contracts, and separately there were issues with using ESG contract with the L4L7 PBR we're developing (one fabric will have this (we hope), the other two fabrics will have no L4L7 devices).  Am I mistaken to dismiss ESGs so casually?

No, you are not mistaken, ESGs always seem like a good idea until you try and implement them. Although you can still do PBR with ESGs. It might be worth taking the time to digest Cisco's ACI Endpoint Security Group (ESG) Design Guide if you haven't already. There MAY be a solution lurking there somewhere.

I was hoping to avoid microsegmentation (TAC always gets antsy whenever the topic comes up, then audibly gets relieved when I say we don't have any yet, plus I'm running into issues already with the number of VLAN IDs I'm already using even without that) by hearing about potential other approaches.  I'll keep my fingers crossed for finding an alternative solution?

Avoiding micro-segmentation is always a good idea. My theory is that ACI only created the ability to do micro-segmentation as a marketing gimmick because vmWare supported micro-segmentation. With the intention to convince customers to use regular EPGs once they'd bought the product. (This is purely the personal opinion of an old cynic who has absolutely no evidence to back it up.)

I do spread EPGs across various APs.  How do I leverage that to aid in redefining legacy ACLs in an ACI security model, with recognizing that (for example) "ZenossProd_EPG" won't be split into Web/App/DB/etc EPGs? (In practice, we static path bind to VMware ports that, for now, don't do VMM Integration, and the VMware team would have their own challenges in their operations team supporting multiple port-groups / VLAN IDs for the same subnet).

If you want endpoints within an EPG to have restricted access to each other, and given the restrictions you have regarding not being able to split EPGs, then your choices really are:

1. Use intra-EPG isolation on the EPG, along with Intra-EPG contracts to allow selected communication within the EPG 
2. Use micro-segmentation

Not much more I can add at the moment. I guess we'd have to get down to specific details to go much further.

[weylin.piegorsch]

---

And I heartily endorse your excellent naming scheme.and IPs?

yeah, well, I had good guidance.

...Avoiding micro-segmentation is always a good idea...
...intra-EPG isolation on the EPG...
...Intra-EPG contracts...

---

I think I should clarify.  Example below (IPs changed).

Ultimately, I think this is getting to the point now that I need to engage my SE, start doing a lot of reading through the ESG whitepaper, and re-read the EPG and PBR whitepapers for their ESG content.  I've toyed with ESG a little bit, their construct is far more complicated and requires a lot more work to setup than a simple ACL, especially when there's not a lot of re-use I can do.

interface Vlan120
  description TEST-NET-1
  ip address 192.0.2.1/24
  ip access-group Vlan120:in in
  ip access-group Vlan120:out out
!
ip access-list Vlan120:in
  1 remark 198.51.100.0/24 is TEST-NET-2
  9 remark port 88 = Kerberos
  10 permit tcp 198.51.100.8/29 192.0.2.10/32 eq 88 
  20 permit udp 198.51.100.8/29 192.0.2.10/32 eq 88  
  30 permit tcp 198.51.100.16/29 192.0.2.11/32 eq 88  
  30 permit udp 198.51.100.16/29 192.0.2.11/32 eq 88  
  40 permit tcp 198.51.100.24/29 192.0.2.12/32 eq 88  
  50 permit udp 198.51.100.24/29 192.0.2.12/32 eq 88  
  60 permit tcp 198.51.100.32/29 192.0.2.13/32 eq 88  
  70 permit udp 198.51.100.32/29 192.0.2.13/32 eq 88  
  80 deny tcp any any eq 88
  90 deny udp any any eq 88
  99 remark port 389 = LDAP
  100 permit tcp 198.51.100.8/29 192.0.2.10/32 eq 389
.
.
.
I think you get the idea

[RedNectar]

Hi @weylin.piegorsch ,

Looks like you are caught between a rock and a hard place if you can't isolate those Kerberos/LDAP/... servers into their own EPG.

And ESGs may get you out of trouble, but to be honest I've never really got into ESGs. For a long time I figured, "What's the point if ESGs are not supported on MSO/NDO" however that hurdle is now crossed so maybe it is time I started thinking about ESGs. Then again ... until I have a use case, why bother?

Getting your SE to earn his/her keep is probably a good idea. Hope it goes well.  If I get bored and have spare time over the next few days I might have a go at modelling ESGs for fun.