EPGs

[john.fredrik.olstorn]

Hi all.

I have only played with ACI on dCloud and have read whatever I can find from resources online. We don't have a live ACI-infrastructure (yet). There are some questions I cannot seem to find answers to, and hope that some of you can enlighten me. 

As far as I understand, external (outside to the fabric) EPGs are built in conjuction with L3-Outs.

Let's say I have one customer connected with eBGP to a L3-Out, and the customer advertises two /16's to the fabric: 10.1.0.0/16 and 10.2.0.0/16. I can then build two external EGPs - one for 10.1/16 and one for 10.2/16 (if I need to separate them, or else I can just create one that covers both). Lets call them cust1_clients and cust1_servers just for the fun of it.

Then, onto my questions:

• So now I can make EPG cust1_client consume contracts from my datacenter, and even provision rules on the firewalls accordingly. Nice. But the next day the customer calls and says: "I want my local on-premise biztalk-servers in 10.2.3.0/26 (and only those) to access your db EPG inside your datacenter."  Is it possible to create an external EPG for 10.2.3.0/26? It will overlap with my cust1_servers EPG. And they don't want to change their routing-policy and advertise more prefixes than the two /16. Is this even possible? What have I missed?
  
  

• In most examples I read, they map one external EPG to 0.0.0.0/0. How will this affect the communication between other EPGs; Lets say my db-EPG consumes a contract provided by the external EPG defined by network 0.0.0.0/0, permitting all http-traffic. How will the fabric decide whether or not this EGP can talk other EPGs (ie my web-EPG). Is destination-epg matched purely based on a longest-prefix match? This is often also the case in traditional design with traditional firewalls: if badly designed, a "permit http to 0.0.0.0/0" might cover more than expected if "expected" is "only Internet, not all servers within the DC".
  
  
• Can an End Point be part of several EPGs (for instance if a server provides multiple services)? Or can I at least group EPGs into "EPG-groups" to ease management of contracts? Today we build a large hierarchy of objects and group of objects within our firewall-management system and assign rules to them accordingly. How is this typically managed with ACI?
  
  I really like the application-perspective ACI have, but my concern is that today we have thousands of external objects, groups of objects and groups of groups from our customers that we assign access to the different services in our datacenters. I cannot see how this separation can be replicated with ACI (if we must replicate it - I hope not). The external EPGs seems too static and limited. What are your views on this?

Thanks

[Jason Williams · Accepted solution]

John, 

See below:

"So now I can make EPG cust1_client consume contracts from my datacenter, and even provision rules on the firewalls accordingly. Nice. But the next day the customer calls and says: "I want my local on-premise biztalk-servers in 10.2.3.0/26 (and only those) to access your db EPG inside your datacenter."  Is it possible to create an external EPG for 10.2.3.0/26? It will overlap with my cust1_servers EPG. And they don't want to change their routing-policy and advertise more prefixes than the two /16. Is this even possible? What have I missed?"

For starters, if the prefix on the external EPG is only set to "External Subnet for External EPG" then it has nothing to do with routing or route advertisement. It only defines an IP prefix scope for which contracts/policy will be applied to...

ACI uses LPM for external IP prefixes. Assuming that cust1_client is the external EPG which represents the 10.2.0.0/16 external IP prefix, then you can overlap 10.2.3.0/26. The only time we cannot overlap IP prefixes is if one of the IP subnets is a bridge domain subnet. Example of external EPGs could look like this: 

L3-out-1 / Ext-EPG-1 :: External prefix = 10.1.0.0/16

L3-out-1 / Ext-EPG-2 :: External prefix = 10.2.0.0/16

L3-out-1 / Ext-EPG-3 :: External prefix = 10.2.3.0/26

This is ok to have. All traffic sourced from 10.2.3.0/26 will always be classified as as Ext-EPG-3. As long as this configuration is in place then it will never assume the same policies as Ext-EPG-2 due to longest prefix match. 

'In most examples I read, they map one external EPG to 0.0.0.0/0. How will this affect the communication between other EPGs; Lets say my db-EPG consumes a contract provided by the external EPG defined by network 0.0.0.0/0, permitting all http-traffic. How will the fabric decide whether or not this EGP can talk other EPGs (ie my web-EPG). Is destination-epg matched purely based on a longest-prefix match? This is often also the case in traditional design with traditional firewalls: if badly designed, a "permit http to 0.0.0.0/0" might cover more than expected if "expected" is "only Internet, not all servers within the DC".'

In the case that db-EPG and web-EPG are just normal internal EPG contained within a layer 3 bridge domain (BD has unicast routing enabled with a subnet defined), then external IP prefix 0.0.0.0/0 has no impact on this communication. 

Lets assume the configuration below

web-EPG :: web-BD :: Subnet = 20.0.0.0/24

db-EPG :: db-BD :: Subnet = 30.0.0.0/24

L3-out-1 :: ext-EPG-1 :: External Subnet for External EPG = 0.0.0.0/0 

db-EPG neither provides nor consumes a contract to/from web-EPG

web-EPG provides a contract to ext-EPG-1 with an HTTPS filter.

20.0.0.20 is an endpoint learned within the web-EPG and sends an HTTPS packet to 30.0.0.30 IP address. The ingress leaf will need to determine the appropriate EPG association for the destination IP before applying any contracts (meaning determine which EPG or L3-out-EPG the dest IP belongs too). A learned endpoint always takes precedence over IP prefixes and L3-out tables. If the ingress leaf has 30.0.0.30 installed in its endpoint table, then it already knows that 30.0.0.30 belongs to the db-EPG. It will perform a policy look-up in the zoning-rule table to see that web-EPG and db-EPG have no filters between the two, therefore the leaf will just drop the packet. 

If the destination endpoint is not learned, then it will check the routing table. The routing table will have 30.0.0.0/24 pointing to the Spine IPv4 Proxy Anycast address. A random spine will receive the traffic and perform a lookup on the destination IP. If the lookup is successful, then it will forward the traffic to the appropriate leaf (which has the endpoint locally). If the IP is not within the BD, then the spine will drop the packet and perform an ARP glean within the BD. 

If 30.0.0.0/24 subnet did not exists as a BD subnet and there was a default route installed on the fabric (to go to an external router), then the leaf will use LPM to match the IP to the appropriate external prefix. 

'Can an End Point be part of several EPGs (for instance if a server provides multiple services)? Or can I at least group EPGs into "EPG-groups" to ease management of contracts? Today we build a large hierarchy of objects and group of objects within our firewall-management system and assign rules to them accordingly. How is this typically managed with ACI?'

An endpoint can be contained within a single EPG. Usually EPGs are mapped at a per-VLAN basis. web-EPG = VLAN 100 :: app-EPG = VLAN 200 etc... If you're doing external L3 networks, then those are purely based on IP prefix. If you're curious about managing policy/rules efficiently or want to know how to get more granularity, then you should look into the documentation below. 

EPG Preferred Group

http://www.cisco.com/c/en/us/td/docs/switches/datacenter/aci/apic/sw/kb/b_APIC_Contract_Preferred_Group.html

Provide from One EPG to an entire VRF using vzAny Contracts

http://www.cisco.com/c/en/us/td/docs/switches/datacenter/aci/apic/sw/kb/b_KB_Use_vzAny_to_AutomaticallyApplyCommunicationRules_toEPGs.html

Microsegmentation

https://www.cisco.com/c/en/us/td/docs/switches/datacenter/aci/apic/sw/1-x/virtualization/b_ACI_Virtualization_Guide_1_3_x/b_ACI_Virtualization_Guide_1_3_x_chapter_0100.pdf

If you would like to keep a firewall in place, then you could include a firewall using L4-L7 service graph implementation 

http://www.cisco.com/c/en/us/solutions/collateral/data-center-virtualization/application-centric-infrastructure/white-paper-c11-734298.html#_Toc456397468

'I really like the application-perspective ACI have, but my concern is that today we have thousands of external objects, groups of objects and groups of groups from our customers that we assign access to the different services in our datacenters. I cannot see how this separation can be replicated with ACI (if we must replicate it - I hope not). The external EPGs seems too static and limited. What are your views on this?'

Regarding only external IPs, you apply policy based on the source/destination IP using LPM. If you want a /32 to be in its own external EPG, then that is fine. There should not be any limitation holding you back for policy enforcement. If you want more detail on how ACI would fit into your data center then I would recommend that you contact a sales engineer and potentially get in touch with the Advanced Services team. They would be great help for design questions.

Jason 

[john.fredrik.olstorn]

Hi Jason, and thanks for your thorough reply.

This made my day. I don't know where I got the assumption that the external EPGs was tightly coupled with routing. Now it makes sense, and most of my concerns related to policy scaling and flexibility are gone. I look forward to get more familiar with this fabric in real life.

Regards,
John Fredrik