How to get EPGs in separate Application Profiles / Application EPGs to ping?

[ckronk]

I'm having some troubles here and would greatly appreciate the help, even if it's being pointed towards reading material. I'm having a hard time finding resource on how this should be setup.

The topology:

2 nx-os switches. One has VLAN 1 and is connected to the fabric. One has VLAN 2 and is connected to the fabric.

There is one VRF and two Bridge Domains. Each Bridge Domain has a gateway IP. The is one Application Profile with two Application EPGs, App EPG VLAN 1 and App EPG VLAN 2. Each Application EPG is setup with a static end point connecting as a trunk to the switches.

The problem:

Switch 1 with VLAN 1 can ping it's gateway and Switch 2 with VLAN 2 can ping it's gateway. When I put VLAN 1 on both switches and put them in the same EPG, they can ping. Neither VLAN can ping the other's gateway or SVI on the switches. I've tried putting a contract to all all icmp under both of the Application EPGs' contract folder as both a consumer and provider at the same time. I have been told that this should be all that's needed to get this to work. There is no routing setup in the ACI fabric at this time. 

Normally an L3 switch would have to have a routing instance. Is this not the case when trying to ping within the fabric within the same VRF? How do I get two EPGs on separate bridge domains to ping each other?

[Robert Burns]

Without a topology, I'm assuming that the GWs for each VLAN reside on the ACI fabric (BD subnet address).  I imagine your goal is to have 2 x EPGs, 2 x BDs, and you want these resources to be able to communicate.  

To do this easily, ensure both BDs belong to the same VRF and that both BDs have unicast routing enabled.  If they do then the only other item you'd need to configure is a contract between the respective EPGs (You can attach the common/default contract which is the equivalent of an ip any:any ACL).   Alternately, if you want to remove contracts completely from the equation you can even change the VRF "Enforcement Policy" to unenforced.  I would only change VRF enforcement as a testing measure.  If it works unenforced, then this means basic routing works, and you need to attach an appropriate contract to re-enforce the VRF. 

Also, did you configure all the necessary Access Policies?  (VLAN Pool, Domain, Interface Policies etc)?

This should allow your endpoints to talk.  BTW - if everything is configured correctly, endpoints in either VLAN should be able to reach the GW SVI of each BD.

Robert

[Robert Burns]

BTW -  here's a good reference cookbook to get you started: https://www.cisco.com/c/en/us/td/docs/switches/datacenter/aci/apic/white_papers/Cisco-ACI-Initial-Deployment-Cookbook.html

Robert

[ckronk]

The gateways are in the fabric assigned under the bridge domains.

Unicast routing is on for both bridge domains.

Both bridge domains are in the same VRF.

Policy enforcement is off in the VRF.

The VRF shows both EPGs under Associated EPGs.

Still can't ping.

[Robert Burns]

If you click on the EPG, do you have faults?
Did you setup the Access Policies?

Robert

[ckronk]

There are no faults. The access policies and EPGs are setup correctly as I can put SVI's with the same VLAN on both switches, created the static ports in the same EPG and can ping from one switch to the next across the fabric. It's when it comes to pinging from one VLAN to the next, it doesn't work.

[Robert Burns]

Do you see your endpoints learned under the EPG > Operational > Client Endpoint?

Robert

[ckronk]

No. They are set as static EPG members though. Does that matter?

The nx-os switches can ping the bd gateway in the fabric and when the same vlan is on both switches, they can ping across the fabric, so there is connectivity to and through the fabric.

[Robert Burns]

Yes, ACI needs to "learn" the endpoint (MAC/IP) under the EPG in order for communication to work.  There may be a "Static Path Binding" under the EPG - which dictates the VLAN and Path (interface) which the EPG is programmed.   If you don't see the Learned Endpoint, you have something misconfigured.  
Can you Export your Tenant Config (right click - Save As - XML > Only Config + Subtree) and post here or DM it to me.  Also post a running config of one of your switches in question.

Robert

[ckronk]

Never mind, they are showing up in the as Client End-Points now. This was after logging into the switches and doing ping tests to the gateways which hadn't been done in a while.