Help the total Noob,intra EPG with VLAN based EPG's

[daz_efx_2000]

Please can someone help with this LAB, I have the following topology shown in the Left hand side I am trying to set up the Tenant as shown in the top right, Once setup as shown EPG-b to EPG-c can communicate but not EPG-a to EPG-b. I believe this topology should be possible. After many mouse clicks I Found the only way I could get the topology to work is by setting it up as per the topology in the bottom right.

How can I get the first topology in the top right to work?

Also please can someone confirm or deny the following statement:

"In ACI you cannot base an EPG on more than one VLAN i.e If you want to base EPG assignment by defining the VLAN a host resides in, you can only specify one VLAN per EPG."

Thanks

ACI Noob

[RedNectar]

Hi daz_efx_2000@ya...

I fell your pain - getting started with ACI can be frustrating - but to answer your question:

"In ACI you cannot base an EPG on more than one VLAN i.e If you want to base EPG assignment by defining the VLAN a host resides in, you can only specify one VLAN per EPG."

The answer is a resounding DENY.  In fact it is quite common to have multiple VLANs linked to a single EPG - typically one statically mapped VLAN, and another dynamically allocated from a vCenter VVM.  And multiple subnets on a Bridge Domain is quite common too.

So the real question is "Why didn't Config #1 work?"  And to answer that, I need to ask if you saw any errors under the Faults tab for the EPG?  And if you did, what did it say.  Here are a couple of likely faults:

1. Invalid VLAN - this error occur if the VLAN you assigned in the physical mapping of the Static Port to the EPG is
1. not part of the VLAN pool that is linked to the Physical Domain that you linked to the EPG under the Domains option.
2. or not configured as a port in an interface profile that is linked both to the appropriate Leaf Profile and to an Access Port Policy Group that is linked to an Attachable Access Entity Profile that is linked to the Physical Domain that is linked to the VLAN Pool that contains the VLAN
2. Invalid Path - probably means you haven't linked the Physical Domain to your EPG.

If you find all that too confusing, perhaps you should read my tutorial.

Let me know how you go after looking for faults.

HTH

RedNectar
aka Chris Welsh

[daz_efx_2000]

OMG your rednectar !, Iv spent may hours on your website. the visualisations of the links between objects is very helpful.

Will check my ACI config and report back.

[Robert Burns]

daz,

Are we assuming your routing is occurring on the 4500? From ACI's perspective, if they're in the same EPG, they can communicate - but if the endpoints are in different IP subnets, routing has to occur.  ACI can do this if you define the SVIs on each BD subnet, otherwise its up to the external device to route between subnets/vlans - which is the responsibility of your 4500 in this case.

Do you have inter-vlan / vrf routing correctly configured on the 4500?

Robert

[daz_efx_2000]

Thanks Robert

Yes so on the 4500 there are 3 VRF's and 3 VLAN's configured, each VLAN is VRF forwarding for its retrospective VRF (Please see the diagram at the top of page with the 4500 representation).

• Each VRF has a default route to the ACI Bridge domain gateway IP.
• There is only one physical link between the 4500 and the ACI Leaf.
• This link is a dot1q trunk. All VLANs are active on the 4500 trunk .

Thanks

[Robert Burns]

Ok, so Routing is occuring on the ACI fabric.  That's simplifies things.  How many VRFs are you using on the ACI Tenant side?  Typically you would mirror the VRF/subnets from your legacy network (4500) into ACI.   You have three external VRFs defined, but on ACI you're only using at most 2 (noted by two unique bridge domains).  If you wanted to mirror what you have done on the legacy side, you'd need 3 x VRFs each with a single BD on ACI side.

Can you attach a screenshot of the BD-a and BD-b subnets.

Robert

[daz_efx_2000]

ah yes, if you look at the last screen shot above, I have left the BD sections expanded with all the subnets listed.

[Robert Burns]

One issue I noticed is your VLAN pool is "Dynamic".  It needs to be "Static" type since you're doing static path bindings.  Dynamic is only used for VMM integration.

Fix that first, then tell me how many VRFs (unique) you're using on ACI.

Robert

[daz_efx_2000]

Thanks Robert

Ok so VLANs changed

Also there is only one VRF, Im just using VRFs on the 4500 to simulate an individual host.

Thanks

[Robert Burns]

I'm assuming you've retested and still having issues.  From your Leaf CLI, provide the following:

-show endpoint detail

-show vlan extended

Robert

[daz_efx_2000]

oo CLI trouble shooting, not gone down to that level yet!

Sorry ,yes the issue is still there

Ok so I did a - Show endpoint detail, I have done screen shots as the text does not format very well.

first result

I did not see VLAN 30 so thought let me do a ping from the 4500 to its default gateway (ACI BD IP) thats all I did. Did not change the ACI in any way. After this the below is the result of the "show endpoint detail"

And finally below is the output of the "show vlan extended"

Thanks

[Robert Burns · Accepted solution]

The problem is the Source MAC used by the 4500.  If you look at the Endpoint detail, you see the same MAC address is learned for the "endpoint" in the Gamma_Corp_EPG as well as Beta_Customer_EPG.  We'd essentially be trying to learn the same endpoint in two different EPGs - which would cause it to bounce.  All 3 of your EPG endpoints will share the same MAC.

Anytime you try to use a L3 Switch as a "test host" issues like this will come up.  ACI knows its not an endpoint due to CDP/LLDP information as well (and will treat the port differently).  Ex. When using an L3 Routed Interface as a test "endpoint" often we'll learn the MAC of the CDP packets and use that for endpoint learning, rather than the interface MAC.  It requires that CDP/LLDP be turned off when trying to use a routed interfaces in place of an actual test host endpoint.

On your 4500 take a look at "show int vlan XX | i address" and you should see duplicate MACs.  On some Catalyst platforms you can manually assign a unique MAC to an SVI with:
switch(config)#interface vlan X
switch(config-if)#mac-address 0007.0001.0001

EDIT - Doesn't look like that platform supports custom SVI MAC.  http://www.cisco.com/c/en/us/support/docs/switches/catalyst-6000-series-switches/41263-catmac-41263.html

Robert

[daz_efx_2000]

Well cover me in sugar and call me a doughnut !!

That looks like it could be it.

Was just looking at the same Cisco page you sent in your link. And yes is not possable.

However for completeness what I will do is connect up 2 more physical ports, so 3 in total, make the bare min changes on ACI to bring up the new links, test and report back. This may have to be done tomorrow as I will have to get someone to connect up the ports.

Thanks for your help and your time you make the world a better place to live in.

[Robert Burns]

Let us know how it goes.  We're here to help! 

Robert

[daz_efx_2000]

Ok so just managed to get the new ports connected up. The results however are still the same, in that EPG-alpha can ping EPG-beta (they sit in different BD's). However EPG-alpha cant not get to EPG-gamma ( both sit in the same BD).

So now each EPG now has its own port with its own VLAN.

when do a "show endpoint detail" I was still getting 3 same MAC's. I came to the assumption that ACI can handle 2 same MAC's but not 3. As I can ping from EPG-alpha to EPG-beta. To get the MAC to change for the 3rd interface on the 4500 I configured HSRP coz you know that's just how HSRP rolls...But it still does not work. Below is the output on the leaf of a "show endpoint detail"

EDIT- Forgot to add - I have also turned off CDP and LLDP under the interface leaf policy group