ACI Intra EPG Isolation

[cowser1979]

Hello

can someone list a definitive resource for ACI intra epg isolation?  I am trying to figure out how to configure this and how it will work.  I understand the basics, but how do you enable communications between specific hosts?  Does ACI HAVE to be acting as the GW for these nodes? Or can i get away with Layer2 BDs only?

thank you

[liguoriariel]

Hi cowser,

  I really recommend you that u go out with this helpful resource to fully understand: http://www.cisco.com/c/en/us/td/docs/switches/datacenter/aci/apic/sw/1-x/virtualization/b_ACI_Virtualization_Guide_2_0_1x/b_ACI_Virtualization_Guide_2_0_1x_chapter_0101.pdf

No need to be GW since intra epg isolation occurs internally at layer 2 level and its based on PVLAN behaviour. The link that I send you is based on intra epg isolation for VMware VDS, you can find similar to AVS.

HTH,

KR,

AL

[cowser1979]

Hello

Thanks for the document.  

*I should point out that i am an ALL Bare metal environment (i didn't point this out in my original post).

I have read the doc and I was hoping for something a little more detailed.  I was able to figure out how to get this going in my lab...at least somewhat..  Basically I split 2 epgs over 1 BD.  EPG1 has my normal hosts in it, and EPG2 is set to isolated.  I am able to get communication flowing back and forth by using contracts between epgs and still isolating host to host traffic in EPG2.  But unlike the documentation would indicate, I didn't manipulate primary/secondary vlan tag info.  I only set normal port encap.  The only gotcha that i see myself running into so far, is the VLAN tag overlap issue between EPGs on the same switch HW (really wish Cisco would solve this limitation).  As best I can tell, communication seems to be working as expected.

thanks

[liguoriariel]

Gotcha,  I still believe that same method that i mention applies, here is a document: 

http://cisco-images.test.edgekey.net/c/en/us/td/docs/switches/datacenter/aci/apic/sw/1-x/aci-fundamentals/b_ACI-Fundamentals/b_ACI-Fundamentals_chapter_010001.pdf

You need to assign primary/sec encap and enforce isolation under epg.

KR,

AL

[cowser1979]

thanks - i read that doc also, and while it does make mention of the primary Pvlan tag, it doesn't really explain how that works?  I have been able to get this working without specifying a primay VLAN tag...atleast as far as I can see.  

So now this makes me think either the doc is wrong or i am still doing this wrong, but I don't understand how this is supposed to work?

thanks

[liguoriariel]

cowser,

  This works as usual PVLAN, when you set the EPG to isolated the secondary vlan or second encap is the homonymous to Secondary Isolated VLAN, that VLAN is used to send traffic from your EPs to P (promiscous) ports. The primary encap is the Primary VLAN used in PVLAN scenario.

Look at this diagram to get the full picture, it says VMw DVS but is the same to bare metal domains

Here is also a good post related pvlan work: http://blog.ine.com/tag/private-vlan/

HTH,

AL

[cowser1979]

Thanks that helps -

So in this case - the secondary Vlan is the ISOLATED vlan (so vlan tag that isolated hosts send) and the Primary VLAN, is for promiscuous ports (so vlan tag that GWs and other community devices, send)?

thanks

[liguoriariel]

Yes, you got the point :)

KR,

[cowser1979]

Hello

So i tried configuring this 2 different time, but this doesnt seem to work.  When I specify a 2ndary/Primary VLAN (collapsing all hosts into 1 isolated EPG) ALL traffic now fails?

thanks

[liguoriariel]

Hi, by specifying both encaps and isolated, traffic between EPs into EPG should not be allowed, that is the expected behaviour is this what is happening to you?

KR,

AL

[cowser1979]

Hello

No, the point that you were making was that IntraEPG isolation, should function like PVLANS, where you can specify a common/promiscuous host and several isolated and community hosts.  You were saying that by specifying a PVLAN/2ndary VLAN in the static port mapping (for the isolated EPG) you can get traffic to flow like a PVLAN, where you have some hosts that can succesfully send/receive traffic to/from isolated hosts, but isolated hosts cant talk to each other.

I am trying to replicate this behavior in my lab.  I can already lock down ALL host to host traffic in an EPG by just turning on isolation (again, keep in mind my environment is all BM),  and ONLY specifying the primary EPG encapsulation.  I can then use contracts to get traffic in/out of the isolated EPG to normal EPGs.  Your response had given me the impression that I can accomplish the same thing with primary/secondary vlan tags in the same isolated EPG?

My overall goal is to replicate PVLAN like behavior in ACI, as I am migrating legacy hosts from a vlan where I need to lock down east/west host to host traffic INSIDE the vlan.  This EPG is L2 ONLY, the GWs for these VLANs will live on 3rd party devices (FWs and LBs) connected on dotQ host ports (NO device integration).  While I can get the desired result with my original attempt, my problem becomes, that NOW i have to tell my dotQ hosts to retag to a DIFFERENT VLAN tag.  That tag will also be different than the one that their respective GWs use (which i could see causing confusion with other engineers down the road).  Overall, i think that's a terrible solution, one that I would prefer to not put forward, without exhausting all other options. 

thanks

[liguoriariel]

Hi, I got your point and what you understand is correct;  it behaves like pvlan but no need to retag in any point, your gw device or promiscuous device should only use primary vlan while all other hosts on epg should remain with isolated option in secondary. No retag but as you mention you should use primary vlan on gw device and that is different from vlan on eps on epg. 

HTH

[guillerm]

hello,

we had to do PVLANs tests on an ACI environment, and, if this can help, here are some informations we collected based on those tests (tcpdump traces done when possible) and based on CISCO TAC feedback :

1) only Primary vlan and Secondary Isolated vlan are useable on ACI;

no Sec Community vlan can be defined

2) the only device that can be connected to the primary vlan is the ACI itself acting as the EPG Gateway; it can't be an external device such as a router or a FW, since the current ACI PVLAN implementation (V2, V3) does not support inbound frames (entering Leafs) with a vlan tag equal to the Primary vlan;

this seems enough to prevent, for example, the configuration of a PVLAN that would need to be extended between 2 ACI Fabrics thru L2 DCI;

3) with the single EPG option "intra-EPG isolation" set to Enforced, you create a PVLAN where all servers directly connected to the Leaf (bo intermediate switch) via Tag or Untagged mode (802.1p option on EPG) will be automatically isolated from each other, and they will all be able to communicate thru their ACI default GW (which is the only possible Primary host on this PVLAN as mentionned above): You must not define a Primary vlan on EPG to get this config, just the Secondary vlan field; if you define a Primary VLAN, no communication will be possible between devices using vlan tagging and their ACI default GW, simply because, in this case, frames sent by ACI to those devices (such as ARP replies) will be sent to those devices with a tag equal to the primary vlan, which will not be accepted since the devices only know the secondary vlan;

4) in case there is an intermediate swich (real or virtual such as vDS) between the ACI Leafs and the servers, then you must also configure the Primary vlan option on the EPG path to this switch, and not only EPG option "intra-EPG isolation";

and, moreover, those intermediate switches must be PVLAN aware and configured with the same Pri/Sec vlans options