Accepted Solutions
Hi @networkinblood ,
But I am still confused whether I can attach same vlan pool to two
different physical domain in same aci fabric or not.
Yes you can attach the same VLAN pool to two different Physical Domains in the same ACI fabric.
But you'd better be aware of the possible consequences.
In a typical ACI environment, each EPG is assigned a single VLAN on a particular set of ports within a particular Tenant space.
The connection between the EPG and the Physical Domain defines exactly WHICH set of ports MAY exist on, and then (typically) one or more of those ports is statically mapped to the EPG for a particular VLAN.
The connection between the EPG and the Physical Domain ALSO defines exactly WHICH set of VLANs MAY be used to map that EPG.
Now - you may have a different Physical Domain that defines a different set of ports, and still use the same VLAN Pool - BUT you may see some problems - and they pretty much all relate to how ACI treats Spanning Tree BPDUs.
Firstly, you need to understand that not all BPDUs are created the same way. There are tow species of BPDUs that you need to be concerned about.
- Cisco Proprietary BPDUs, which carry a VLAN tag in the BPDU 802.1Q frame
- IEEE standard MST BPDUs which are untagged.
Each has its own story - let's begin with the Cisco Proprietary BPDU story in an environment where I have two Tenants - Tenant1 and Tenant2. Each has been allocated a Physical Domain and those PhysDoms share a VLAN Pool that contains VLANs 10-20. To begin with, we'll say each tenant has a few ports (1-4) allocated on two separate switches. In this arrangement, life will be sweet. No problems, or as we say down under "No worries, mate".
Let's add some EPGs to the picture, and to add some spice, let's say each tenant has mapped VLAN 10 to an EPG on interface Eth1/1 of their respective switches, and that interface connects to each Tenant's own external switches, that are sending Cisco tagged BPDUs on VLAN 10.
And this is where the shared VLAN Pool becomes a (potential) problem.
You see, a BPDU that arrives at interface Eh1/1 on Leaf1 will be flooded throughout VLAN 10 based on the VLAN ID in the VLAN Pool. The fine details of the process are explained in the ACI Design Guide and in particular:
The FD VNID that a VLAN maps to depends on the VLAN number itself and on the VLAN pool object (and because of this, indirectly also the domain, but if two domains use the same VLAN pool, the same VLAN gets the same FD VNID) that it is from. Defining the same encap VLAN range in two VLAN pools does not result in the same FD VNID being assigned to the same VLAN number. In other words the FD VNID is a function of the VLAN encapsulation number and the VLAN pool object. Each VLAN on a given bridge domain has a unique FD_VLAN VNID and this number is identical on all leaf switches where the same bridge domain and VLAN are present.
So potentially, BPDUs from one Tenant could leak into another Tenant. Probably not a good idea.
Separation of the VLAN Pools ensures that both Tenants have their respective (Cisco proprietary) BPDUs flooded in separate VNIDs
But perhaps we shouldn't forget the other part of the story. IEEE standard MST BPDUs.
In the configuration above, the MST BPDUs would be dropped, because they arrive untagged, and unless there is a specific EPG created to catch those untagged BPDUs, there is a real chance of a L2 Loop (I REFUSE to call it a Spanning-Tree loop because by definition no such condition exists). And in that case, if each of the VLAN 10s above are in different MST Regions (they should be), you need to configure a policy for each region. (Fabric > Access Policies > Policies > Switch > Spanning Tree > default >| >+ Region Policies)
So hopefully, I convinced you that sticking with one VLAN Pool per Domain is a good idea. Although another topic that would be interesting to discuss is "What about if Tenant1 and Tenant2 share some VLANs on an external interface?"
And the short answer to that is - they CAN'T share VLAN 10 on the same interface, but if they use discrete VLANs - say Tenant1 uses VLAN 11 and Tenant2 uses VLAN 12, then a new AAEP should be crated to accommodate the shared interface - say a VPC on interface Eth1/5 on leaves 1 & 2.
Hi @networkinblood ,
I'm not sure where you got the example from, but let me begin by saying PLEASE - don't ever use the CLI to configure ACI. IF you were to enter the commands as specified in the example above, you will have created three Domains (one Physical Domain, one External Bridged Domain and one L3 Domain) just with the commands
apic1(config)# vlan-domain dom_exampleCorp
But it gets worse. When you enter the command
apic1(config-leaf-if)# vlan-domain member dom_exampleCorp
a whole bunch of hidden entities get created which are virtually impossible to delete, and likely to cause future configurations to fail.
For fun, I entered the configuration you specified above - and this is what happened:
I then went and deleted the Physical Domain, External Bridged Domain and L3 Domain, and the VLAN pool (the ones created by issuing the CLI config commands), but I was still left with the hidden objects, which I now have to use moquery to even see:
apic1# moquery -c infraAttEntityP | grep __ui name : __ui_l1201_eth1--4 dn : uni/infra/attentp-__ui_l1201_eth1--4 rn : attentp-__ui_l1201_eth1--4 name : __ui_l1201_eth1--2 dn : uni/infra/attentp-__ui_l1201_eth1--2 rn : attentp-__ui_l1201_eth1--2 name : __ui_l1201_eth1--3 dn : uni/infra/attentp-__ui_l1201_eth1--3 rn : attentp-__ui_l1201_eth1--3
And I'm now left with the almost impossible task of removing these weird objects. Which I did using these commands:
apic1# bash
admin@apic1:~> cd /mit/uni/infra
admin@apic1:infra> for i in `find *__ui*`
> do
> modelete $i
> done
moconfig commit
And PATIENTLY ignore the MANY error messages that appear!
So. PLEASE - don't ever use the CLI to configure ACI
But to answer your question (which BTW seems to have no relation to the actual code you posted because there is absolutely NO Bridge Domains mention in the code. I suspect you have ACI Domain objects (Physical Domains, External Bridged Domains and L3 Domains - aka physDomP, l2extDomP and l3extDomP) confused with ACI Bridge Domains (aka fVBD)
- Can we use same vlan on different Bridge domain
the answer is there is absolutely NO relationship between ACI Bridge Domains (aka fVBD) and VLANs. VLANs are associated with EPGs (fvAEPg), which in turn are linked to Bridge Domains.
BUT, given the nature of the CLI code you posted, I suspect you really meant to ask
- Can we assign the same ACI Domains (called vlan-domains in the CLI) to multiple interfaces
And the answer to that is that you MUST assign multiple VLANs to any interface that is a trunk. I'm not quite sure what kind of Kool-Aide @mvankleij was drinking when saying "it's not always a good idea"
@networkinblood PLEASE take my advice and learn a bit more about how the ACI Access Policy Chain works. (google ACI Access Policy Chain). It is a very difficult concept and much of it is hidden from you if you use the CLI. Trust me, you will live to regret ever trying to use the CLI.
Hi @networkinblood ,
But I am still confused whether I can attach same vlan pool to two
different physical domain in same aci fabric or not.
Yes you can attach the same VLAN pool to two different Physical Domains in the same ACI fabric.
But you'd better be aware of the possible consequences.
In a typical ACI environment, each EPG is assigned a single VLAN on a particular set of ports within a particular Tenant space.
The connection between the EPG and the Physical Domain defines exactly WHICH set of ports MAY exist on, and then (typically) one or more of those ports is statically mapped to the EPG for a particular VLAN.
The connection between the EPG and the Physical Domain ALSO defines exactly WHICH set of VLANs MAY be used to map that EPG.
Now - you may have a different Physical Domain that defines a different set of ports, and still use the same VLAN Pool - BUT you may see some problems - and they pretty much all relate to how ACI treats Spanning Tree BPDUs.
Firstly, you need to understand that not all BPDUs are created the same way. There are tow species of BPDUs that you need to be concerned about.
- Cisco Proprietary BPDUs, which carry a VLAN tag in the BPDU 802.1Q frame
- IEEE standard MST BPDUs which are untagged.
Each has its own story - let's begin with the Cisco Proprietary BPDU story in an environment where I have two Tenants - Tenant1 and Tenant2. Each has been allocated a Physical Domain and those PhysDoms share a VLAN Pool that contains VLANs 10-20. To begin with, we'll say each tenant has a few ports (1-4) allocated on two separate switches. In this arrangement, life will be sweet. No problems, or as we say down under "No worries, mate".
Let's add some EPGs to the picture, and to add some spice, let's say each tenant has mapped VLAN 10 to an EPG on interface Eth1/1 of their respective switches, and that interface connects to each Tenant's own external switches, that are sending Cisco tagged BPDUs on VLAN 10.
And this is where the shared VLAN Pool becomes a (potential) problem.
You see, a BPDU that arrives at interface Eh1/1 on Leaf1 will be flooded throughout VLAN 10 based on the VLAN ID in the VLAN Pool. The fine details of the process are explained in the ACI Design Guide and in particular:
The FD VNID that a VLAN maps to depends on the VLAN number itself and on the VLAN pool object (and because of this, indirectly also the domain, but if two domains use the same VLAN pool, the same VLAN gets the same FD VNID) that it is from. Defining the same encap VLAN range in two VLAN pools does not result in the same FD VNID being assigned to the same VLAN number. In other words the FD VNID is a function of the VLAN encapsulation number and the VLAN pool object. Each VLAN on a given bridge domain has a unique FD_VLAN VNID and this number is identical on all leaf switches where the same bridge domain and VLAN are present.
So potentially, BPDUs from one Tenant could leak into another Tenant. Probably not a good idea.
Separation of the VLAN Pools ensures that both Tenants have their respective (Cisco proprietary) BPDUs flooded in separate VNIDs
But perhaps we shouldn't forget the other part of the story. IEEE standard MST BPDUs.
In the configuration above, the MST BPDUs would be dropped, because they arrive untagged, and unless there is a specific EPG created to catch those untagged BPDUs, there is a real chance of a L2 Loop (I REFUSE to call it a Spanning-Tree loop because by definition no such condition exists). And in that case, if each of the VLAN 10s above are in different MST Regions (they should be), you need to configure a policy for each region. (Fabric > Access Policies > Policies > Switch > Spanning Tree > default >| >+ Region Policies)
So hopefully, I convinced you that sticking with one VLAN Pool per Domain is a good idea. Although another topic that would be interesting to discuss is "What about if Tenant1 and Tenant2 share some VLANs on an external interface?"
And the short answer to that is - they CAN'T share VLAN 10 on the same interface, but if they use discrete VLANs - say Tenant1 uses VLAN 11 and Tenant2 uses VLAN 12, then a new AAEP should be crated to accommodate the shared interface - say a VPC on interface Eth1/5 on leaves 1 & 2.
@RedNectar's post is awesome, as always. He went beyond your question and definitely gave some very good insights. My answer only focussed on the title of your question, which was whether you can use the same vlan on multiple bridge domains. I don't know where his Kool-Aide remark comes from as I'm not saying that it's not a good idea to put multiple vlans on a trunk. So either I'm misunderstanding him, or he was misunderstanding me.
As to your second question whether you can use the same vlan pool to two different physical domains in the same fabric, yes you can:
