Multiple encap VLANs in the same EPG?

[tuanquangnguyen]

Hi community,

Can multiple VLANs be used within the same EPG? I have a design where ACI needs to integrate with vCenter, which is to be used dynamic VLAN IDs within a dynamic VLAN Pool.

I'm asking this because in the future, I might need to associate static path binding for physical appliances within this very same EPG. However, if I were to use the same VLAN Pool with static VLAN blocks (for the AEP on those physical ports), then I don't think it is possible to use the same VLAN ID as the port group pushed to VDS.

I'm coming across a topic where basically ACI would match FD_VLAN with a BD_VLAN, then from that BD_VLAN to the VNI when egressing the uplinks. So I'm wondering if two different encap VLANs can be used that will be matched to the same BD_VLAN?

Thanks in advance.

P/s: on the ACI Simulator 4.x I don't think it raised any warnings or errors when I tried configuring this. But there's no data path to verify

[joezersk · Accepted solution]

Hello.  There is no problem using multiple VLAN encaps in the same EPG.  In fact, you can even mix and match VLAN and VxLAN encaps (with AVE) in the same EPG. 

[tuanquangnguyen]

Hi @joezersk,

Thanks for your answer. I open this discussion since I got a little confused with another discussion (on the same topic), dating back from 2017: https://community.cisco.com/t5/application-networking/epg-and-vlans/td-p/3218110

Do the VLANs have to be in different pools or different AEP, or can they be in the same? And what would be the best practice, because I'm trying not to touch the Enforce EPG VLAN Validation - I find it irritating while not really contribute much to the performance of the fabric.

In another scenarios where 2 physical endpoints are attached to different ports - They can use different VLANs but put into the same EPG, yes?

Thanks again for your input.

[Sergiu.Daniluk]

@tuanquangnguyen wrote:

I'm trying not to touch the Enforce EPG VLAN Validation - I find it irritating while not really contribute much to the performance of the fabric.

---

The Enforce EPG VLAN Validation is doing a check against overlapping vlan pools associated to an EPG. This is probably one of the best knob in ACI to avoid getting into issues which are 'hidden' from the human eye, especially since majority of the problem with overlapping vlans are not noticed at the configuration, but only after a reload. This knob is really really good for bad designs or for large multi-tenant ACI environments. Trust me on this :-) 

Continuing on this topic, as long as you have different non-overlapping vlan pools, it's safe to assign them to different domains and associate the domains to EPG. Meaning you can have the EPG vlans from same vlan pool or different vlan pools, as long as the mapped vlan pools are not overlapping.

In another scenarios where 2 physical endpoints are attached to different ports - They can use different VLANs but put into the same EPG, yes?

---

Yes. Same logic applies.

Stay safe,

Sergiu

[tuanquangnguyen]

Hi @Sergiu.Daniluk, @RedNectar and @joezersk,

Thanks for all of your input.

About the overlapping VLAN pool, my idea is to provision a dynamic pool for each customer to define their blocks. The pool will later be referenced by different domain profiles (be it physical or VMM), then associated with each customer's respective AEPs and EPGs (so, each customer use their own VLAN pool, Domain Profiles, AEPs, etc.)

If one customer is moving from the traditional network model to ACI, they would think that they could use the same VLAN ID for both VMM domain and physical domain that is associated to the EPG (for example web VM port group with VLAN 10 tag from the VDS, and then VLAN 10 untagged for different physical appliances like unmanaged F5, Citrix, etc.). But if Enforce EPG VLAN Validation is enabled, then it is not even possible for them to associate VMM and physical domain with overlapping pool to the same EPG, let alone using the same VLAN ID. Hence, I was asking if the EPG could use different encap VLANs.

Do you have any use case or hint for this design?

Thank you all in advance.

[RedNectar · Accepted solution]

---

@tuanquangnguyen wrote:

Hi @Sergiu.Daniluk, @RedNectar and @joezersk,

Thanks for all of your input.

NP - I think Joe's latest answer probably nails it.

About the overlapping VLAN pool, my idea is to provision a dynamic pool for each customer to define their blocks.

Good idea

The pool will later be referenced by different domain profiles (be it physical or VMM),

Two problems:

1. Not a good idea (in general) to have a dynamic VLAN Pool referenced by more than one Domain - although if every Domain services a different Tenant and you make sure you either:
   (i) use the L2 Interface policy to enforce Local Scope, or
   (ii) all tennats use different hardware
   ...then you could get away with it.
2. Not a good idea to link Physical Domains with Dynamic VLAN pools. Not impossible (it used to be) but if I was troubleshooting, it would make me feel nervous every time I saw it - ESPECIALLY if the pool was also being used by e.g. a VMM Domain

then associated with each customer's respective AEPs and EPGs (so, each customer use their own VLAN pool, Domain Profiles, AEPs, etc.)

Good idea

If one customer is moving from the traditional network model to ACI, they would think that they could use the same VLAN ID for both VMM domain and physical domain that is associated to the EPG (for example web VM port group with VLAN 10 tag from the VDS, and then VLAN 10 untagged for different physical appliances like unmanaged F5, Citrix, etc.).

OK. This is where you have to get off your horse and embrase the new Automobile technology. Or in your case, get the customer's mind out of the rut that makes them think that they should "use the same VLAN ID for both VMM domain and physical domain" - THIS IS NOT THE ACI WAY OF THINKING.

Oh - you WILL get push-back, and you may have to give in. BUT the whole idea of moving to a Software Defined Network approach is for YOU to specify the WHAT (policy) and let the software (ACI) define the HOW (the implementation). In this approach, VLAN IDs become irrelevant (yes, I know, you'll need to find out what it is for TS etc... But have you ever had a customer want to define which VNID is used for a particular BD or VRF? No, because they accept the dynamic nature with something that they are not familar with)

Sidenote: I often ask at the beginning of an ACI class "Who understands VLANs?". If someone doesn't rise their hand I tell them they have a great advantage over those that did raise their hand. And tell the others that they should forget whatever they have learned.

But if Enforce EPG VLAN Validation is enabled, then it is not even possible for them to associate VMM and physical domain with overlapping pool to the same EPG, let alone using the same VLAN ID.

I told you that Enforce EPG VLAN Validation would tell you when you made mistakes. :-)

Hence, I was asking if the EPG could use different encap VLANs.

Already answered what the limitations are. If you look at the Distinguished Name of a static mapping eg 

uni/tn-Tenant17/ap-2Tier_AP/epg-AppServers_EPG/rspathAtt-[topology/pod-1/paths-2201/pathep-[eth1/27]]

you'll notice that neither the VLAN ID nor the encapsulation type come into it - hence if you want to use a second VLAN ID  for the same EPG, you'll have to add it from the AAEP "up" to the EPG - but again, looking at the DN

uni/infra/attentp-T17:HostLinks_AAEP/gen-default/rsfuncToEpg-[uni/tn-Tenant17/ap-2Tier_AP/epg-AppServers_EPG]

you'll notice that neither the VLAN ID nor the encapsulation type come into it agin - hence you can only map one VLAN per EPG in this fashion.

Do you have any use case or hint for this design?

Use static mappings for the legacy VLANs, dynamic mappings from a new non-overlapping pool for the new ones.

Thank you all in advance.

---

[RedNectar]

---

@tuanquangnguyen wrote:

Hi @joezersk,

Thanks for your answer.

Joe's answer is indeed a good answer - but he did miss a point

I open this discussion since I got a little confused with another discussion (on the same topic), dating back from 2017: https://community.cisco.com/t5/application-networking/epg-and-vlans/td-p/3218110

And the point he missed is discissed in the document that is linked in the discussion in the link above. I'll explain below.

Do the VLANs have to be in different pools or different AEP,

No

or can they be in the same?

Yes

And what would be the best practice, because I'm trying not to touch the Enforce EPG VLAN Validation - I find it irritating while not really contribute much to the performance of the fabric.

The first thing you should do when you deploy a new fabric is to check that box - what it does for you is that it raises errors for you when you do something stupid.  I'd rather be told that I'd done something stupid rather than have to troubleshoot it with NO error showing.

HOWEVER: Once you have checked this box, every backup you have ever done PRIOR to checking the box is useless. That's why it should be the FIRST thing you do when setting up a new fabric.

But this point is just a distraction...

In another scenarios where 2 physical endpoints are attached to different ports - They can use different VLANs but put into the same EPG, yes?

Correct

Thanks again for your input.

---

Now the real truth about Multiple encap VLANs in the same EPG

1. Yes. You certainly can have have multipe VLANs pointing to the same EPG, the case you mention is the typical scenario.
2. HOWEVER, if you have say VLAN 10 on port 1/1 on leaf 1201 statically mapped to EPG2, YOU CANNOT ADD A MAPPING FOR say VLAN 11 on port 1/1 on leaf 1201 statically mapped to EPG2. If you try it, you will get the error mentioned in this reference.
3. You can get around this a little bit by going to the AAEP and mapping VLAN 11 to EPG2. But again, when you do this "mapping up" method, you are limited to ONE VLAN per EPG that you can "map up" fromt eh AAEP to the EPG.

Now back to your original Q for a tick

I might need to associate static path binding for physical appliances within this very same EPG.

Precisely - not a problem

However, if I were to use the same VLAN Pool with static VLAN blocks (for the AEP on those physical ports), then I don't think it is possible to use the same VLAN ID as the port group pushed to VDS.

Don't get too caught up on VLAN Pools - apart from the fact that you shoudl use a static pool for Physical Domains and a Dynamic Pool for VMM Domains - now with VMM Domains, you CAN (if you want) add static blocks to the VLAN pool and then when you deploy an EPG, you can specify exactly which VLAN from the static block you want to use. This is typically used when trying to integrate an existing vDS into the VMM Domain.

I hope this helps

---

---

[joezersk · Accepted solution]

Hello again.  Two things to mention.  First, forget about that other post from 2017 you shared.  It has to do with a specific situation where they want different VLANs on the same static binding and in the same EPG.  That is not possible in a single EPG. 

But....this is not your case (or at least I don't think it is based on your original post). 

Second, I just built your use case in my lab real quick to show you.  I have VMM integration to vCenter, so there is a VM in this EPG on VLAN-909 (dynamic vlan pool) 192.168.1.11.  Then I have a baremetal server, using a static binding on VLAN-99 at 192.168.1.2.  The vlan comes from the same pool, but I added a static range (this is perfectly fine to do - see below). 

Note the VLAN Pool was built as dynamic, but you can also add more ranges static or dynamic as you wish.

So, I guess I add a third thing....as you asked about vlan pools.  Is it good to have more than one?  I suppose it is your choice.  But honestly, you only need one and you can use it for everything.  Most of my colleagues would say one is perfect and simpler, but you can have as many as you like, with certain care taken not to overlap if you can help it. 

And a forth thing ;)  You can have multiple static bindings in an EPG, using same or different VLANs...as long as they are using different ports.

Hope that helps...