ACI Micro-segmentation with physical domain

[sutha_entc]

Hi All, 

I am trying to put my bare metal servers which is in physical domain in micro EPG using IP address attributes. here is a simple diagram. 

both of my micro EPG belongs to same base EPG and same BD subnet. I can see my endpoints are moved to correct micro EPG but they can still talk each other without any contract applied. is this expected behavior? as per my understanding inter EPG communication need a contract. once I changed the base EPG Intra EPG isolation configuration to Enforced, endpoints stop communicate to each other which is expected behavior. there is no contract. later once I applied the contract also they are not able to communicate which is also expected because I m missing the proxy-arp. once I enable the proxy-arp it started to work again with contract.

I still don't understand the first two behavior.

1. how my two endpoints are able to communicate to each other without contract even though they are in different micro EPG?  is this something related to physical domain and IP attributes?? 

2. why my endpoints stop communicating each other after I enable the intra EPG isolation under Base EPG? base EPG and micro EPGs class ID are different. Intra EPG isolation for the base EPG create deny zoning rule only for the base EPG class ID. how it affect micro EPG??

thanks n regards

Sutha

[RedNectar · Accepted solution]

Hi @sutha_entc ,

Thanks for the great diagram. Always a great idea!

My fist question is - why are you using micro-segmentation EPGs rather than regular Application EPGs?  Are you really making the configuration easier by doing so?

You can probably guess by my questions that I'm not a fan of µSeg EPGs. My cynical explanation of why Cisco has µSeg EPGs is because VMware had this feature, so Cisco said "we can do that too".  And initially, to create µSeg EPGs, you would check a box in your VMM Domain configuration saying [x] Allow micro segmentation - which turned the associated dvPortGroup on the vSwitch into an isolated PVLAN.

Later, when the 2nd generation of hardware came out, Cisco enabled the same feature on regular physical Domains, but to emulate the PVLAN feature of the dvPortGroup  of the vSwitch, the user now had to set up  Intra EPG isolation for the base EPG and allow Proxy-ARP to achieve the same thing.

So your experimentation and (excellent) description of the behaviours you are seeing is showing that you µSeg WPGs are Functioning As Designed.

[sutha_entc]

@RedNectar thank you for details explanation. I have some requirement to isolate the bare metal server from VM. I am planning to use ESG and I am going to test ESG feature also. exploring the simple solution. 

as per your explanation, if we use micro segmentation with physical domain, PVLAN feature not enable by default, to enable the PVLAN we need to do Intra EPG isolation ( or Intra EPG contract). 

thanks n regards

[RedNectar]

Hi @sutha_entc ,

ACI doesn't have PVLANs like regular Nexus or Catalyst switches, but can achieve the same result using Intra EPG isolation, so your statement 

to enable the PVLAN we need to do Intra EPG isolation ( or Intra EPG contract). 

---

is as close to the truth as you are going to get.

Now ESGs are another concept altogether!!  But you may find that ESGs are easier to use than µSeg EPGs.  The uptake of ESGs has been pretty slow in the community, I'm not sure why, but my suspicions are:

1. If you can achieve what you want with EPGs, why complicate things with ESGs?
2. Many sites are looking to use Nexus Dashboard Orchestrator - NDO (previously called Multi-Site Orchestrator) to configure tenants, and configuring ESGs using NDO is still not supported (as of NDO version 4.2(3)), although this blogpost from March 2023 hints that ESG support for NDO may be coming.

As for me, until #2 above is fixed (ESG support in NDO) I'm not putting any effort into learning all that much about ESGs, so my knowledge on this subject is a little thin.

[sutha_entc]

Hi @RedNectar,

 ESG support for NDO is coming soon. I heard that from some sources also. I just dont want to do micro segmentation. I am exploring all the possible options. easy way I am thinking is just put it al the bare metals in different EPG.  thanks for your reply.