uSEG EPG and Intra-EPG Isolation

[Ezequ!el]

Hi,

I have created an uSEG EPG with Intra-EPG Isolation enabled, as I do not want EPs to talk to each other inside this uSEG. This uSEG is also not part of the Preferred Groups, as I want to define any flow communication over a contract.

I check and it is working as expected, no flows are allowed. Then I create an Intra-EPG contract with one subject, allowing ICMP and ARP. Check again and ping is working as expected. But also any other service, like SSH, Web... etc. So basically everything is allowed.

How can I check where is this traffic flowing and why is being accepted? If I delete the contract again I can check that no traffic is allowed.

I am using ACI 4.2(6d).

Thanks for any help.

[Sergiu.Daniluk · Accepted solution]

Hi @Ezequ!el 

There are multiple ways to verify the traffic flow and why is accepted, but the easiest way in my opinion is to use ELAM assistant:

1. Download ELAM Assistent: https://dcappcenter.cisco.com/elam-assistant.html

2. Install the app: Apps -> Installed Apps -> Add Application

3. Configure the filter on source IP and destination IP. Make 2x captures:

3.1 Generate a ping, capture the first ICMP packet and verify the results

3.2 Generate ssh traffic, capture the first packet and verify the results

Based on the outputs you can verify if the same contract is allowing the traffic or not.

Second option is to manually perform the zoning-rules troubleshooting. For this you will need to get the VRF VNID and EPG pcTAG (sclass).

Once you have those, you can use "show zoning-rules scope <VRF VNID>" on the compute leafs and verify what contracts are matching.

You can also use contract parser (built-in script). Command is "contract_parser.py vrf <VRF name>". Don't forget, teh vrf name has <tenant_name>:<vrf_name> format.

Cheers,

Sergiu

[Ezequ!el]

Hi @Sergiu.Daniluk ,

Thanks for your help. I took the ELAM app way.

It is interesting to see that if I do not apply any Intra uSEG contract, traffic is flowing with Source EPG pcTag (sclass) from the uSEG, therefore not working. As expected.

As soon as I configure the Intra uSEG contract, I perform a Ping or SSH, I can check on ELAM Source EPG pcTag is no longer the uSEG value, but the Base EPG where this uSEG is coming from. This Base EPG does not have Intra EPG Isolation enabled, therefore all traffic is being accepted.

Any ideas why is this happening? Must the Base EPG and its uSEG EPG have always same configuration regarding Intra Isolation?

Thanks in advance.

[Robert Burns]

Need some additional info:

1. Version of ACI

2. Virtual or Baremetal endpoints?

3. How are the endpoints connected - directly to a Leaf or behind intermediate switch/blade switch?

4. When you apply the intra-EPG contract, do you still see the endpoints learned in the uSeg EPG or the base EPG?

Robert

[Robert Burns]

So I just ran a few quick tests in my lab, here are the results (as expected).

Lab Setup

=========

VRF1

 BD1

  base_EPG1 (allow for uSeg)

        EP1: rob-vm1 (hosted on ESX01)

       Contract Prov: allow_all

  base_EPG2 (allow for uSeg)

      Contract Cons: allow_all

      EP2: rob-vm2 (hosted on ESX02)

 uSeg_EPG3

Test 1: no Match attribute defined on uSeg EPG

Result: both endpoints appear in respective (separate) base EPGs and can communicate due to allow_all contract

Test2: Add match 'vm name' attribute 'starts with rob' to uSeg_EPG3

Result: Both endpoints show up under uSeg_EPG3 and can communicate due to intra-EPG being allowed (default)

Test3: Enable 'intra-EPG isolation' on uSeg_EPG3

Result: Both endpoints show up under uSeg_EPG3 still, but can't communicate with each other

Test4: Add 'allow_all' contract as intra-EPG contract to uSeg_EPG3

Result:  Both endpoints show up under uSeg_EPG3 and communicate with each other

Robert

[Ezequ!el]

Hi @Robert Burns ,

1. Version of ACI: 4.2(6d)

2. Virtual or Baremetal endpoints?: Both, one is Baremetal, the other is a VMware machine

3. How are the endpoints connected - directly to a Leaf or behind intermediate switch/blade switch? Directly

4. When you apply the intra-EPG contract, do you still see the endpoints learned in the uSeg EPG or the base EPG? Yes, the endpoints are always assigned to the uSeg over IP address.

Your test scenario is the correct one, but please could you at Test4 instead of "allow_all" contract set a allow ARP and ICMP only. And then check if SSH or Web access is possible, should not be the case. But in my deployment, it is allowed as well

On some Cisco documentation I've read about Proxy ARP feature. This is something I cannot activate at uSeg level (option is not available even when selecting Intra-EPG Isolation). This option is only available at Base EPG when selecting Intra-EPG isolation. Is this something I have to worry about?

Thanks again.

[Robert Burns]

Proxy ARP is enabled on the uSeg EPG automatically, nothing to configure.  The HW does need to support this, which Leaf model are you using?  (1st Gen Leafs don't support proxy arp).

Robert

[Ezequ!el]

We are using -FX models, which are supported for Proxy ARP

[Robert Burns]

I ran your requested test - 

Test5: Removed 'allow_all' contract as intra-EPG contract on uSeg_EPG3

(connectivity/icmp stops)

Added 'icmp_arp' intra-EPG contract (with common:icmp & common:arp filters) to uSegEPG3

Result:  Both endpoints can ping each other, but unable to reach each other using SSH/HTTP/HTTPS

Robert

[Ezequ!el]

Very strange behaviour in my deployment. Any other ideas about why when the contract is configured the EPs are evaluated as a part of the Base EGP and no longer as a part of the uSEG?

[Sergiu.Daniluk]

Few more questions for clarifification:

What attributes do you use, for both vmm and phy domains?

What is the contract scope?

THanks,

Sergiu

[Ezequ!el]

Hi,

Both EPs are selected via exact match of IP address. And everything is under the same VRF

[Sergiu.Daniluk · Accepted solution]

If the EPs are part of the same subnet, you also need to add the MAC address as attribute.

[Robert Burns · Accepted solution]

Sergiu is exactly correct - nice catch.  This is likely your issue.  In my test, my endpoints were in different IP subnets (as well as using VM attributes). IP-based microsegmented EPGs are supported only when traffic requires Layer 3 routing. If the traffic is bridged, the microsegmentation policy cannot be enforced.
Ref: https://www.cisco.com/c/en/us/td/docs/switches/datacenter/aci/apic/sw/4-x/virtualization/Cisco-ACI-Virtualization-Guide-42x/Cisco-ACI-Virtualization-Guide-421_chapter_0100.html?bookSearch=true#concept_D527CF389D4440E69A288093516B3643

Robert

[Ezequ!el]

Many thanks you both @Sergiu.Daniluk , @Robert Burns 

At least we have found the explanation.

So as far as I understand, if I use MAC address attributes both policies will be enforced; the one which apply for traffic within the same subnet and the one for traffic out of the subnet.

And this is applicable for all traffic from this uSEG, not only for the Intra uSEG traffic but also for the one leaving the uSEG (I ask this because my Base EPG and uSEG shares same IP subnet).