Hi experts,
According to the design guide, Cisco ACI floods BPDUs only between the ports in the bridge domain that have the same encapsulation.
Please refer to the diagram below
1. As you may see, the incoming traffic to port A & B are untagged, incoming traffic to port C as tagged with VLAN 10 but all 3 ports are configured with port encap VLAN 10 (EPG static binding). Are the ports A, B, C considered as using the same encapsulation? From design perspective, I just want to make sure that:
- BPDU coming from Legacy switch #1 will reach Legacy switch #2 via ACI
- BPDU coming from Legacy switch #1 will reach Legacy switch #3 via ACI
2. If MCP is enabled on port A & B and somehow STP stops working, from my understanding, ACI will detect the loop and block a port (A or B) with MCP. But which one will be blocked? Is there any documentation explaning with details how ACI identifies that port? Let's say if we have port A as primary port and port B as secondary port, is there any way to configure with MCP so that we will always block port B in case of loop?
Thanks for your help.
Accepted Solutions
Hi @suneq ,
Sorry it's taken so long to answer this. Required a bit of research
Firstly, thanks for the great diagram. Let me repeat it here so people reading this on email can see it.
Now to your comments/questions
According to the design guide, Cisco ACI floods BPDUs only between the ports in the bridge domain that have the same encapsulation.
Almost correct - ACI floods BPDUs only between the ports in the bridge domain EPG that have the same VLAN encapsulation (and from the same VLAN Pool).
When a BPDU arrives at a port carrying a VLAN tag, the BPDU is flooded on a multicast tree using the VXLAN VNID of the FD_VLAN, this VNID is actually calculated from the VLAN pool that the VLAN was taken from. So if, say VLAN 5 was used by two different tenants, and each tenant had its own VLAN Pool containing VLAN 5, each instance of VLAN 5 would use a different VNID, so there'd by no leaking of BPDUs from one instance of VLAN 5 to the other. (This reference explains this concept more fully)
In essence, this means that ACI floods BPDUs only between the ports in same EPG. Forget about the corner cases and keep that concept in mind. And in your case, this is true since you are using one VLAN ID and one EPG.
1. As you may see, the incoming traffic to port A & B are untagged, incoming traffic to port C as tagged with VLAN 10 but all 3 ports are configured with port encap VLAN 10 (EPG static binding). Are the ports A, B, C considered as using the same encapsulation? From design perspective, I just want to make sure that:
- BPDU coming from Legacy switch #1 will reach Legacy switch #2 via ACI
- BPDU coming from Legacy switch #1 will reach Legacy switch #3 via ACI
Yep - that will work. But keep in mind
- It's not usual to have switches connected via access ports
- You can't mix access and trunk ports for the same VLAN encap on an ACI switch, in other words, it would be OK for ports A & B to be on the same ACI switch, but port C (the trunk) could NOT be on the same switch as either port A or B. If you DO try, you'll get a fault
F0467 Configuration failed for node xxx due to Different encap modes are not allowed for an encap on a given interface
2. If MCP is enabled on port A & B and somehow STP stops working, from my understanding, ACI will detect the loop and block a port (A or B) with MCP.
Thats not quite how MCP works. MCP is independent of STP and works by sending multicast packets with a unique source MAC address linked to the sending port. If the port receives one of its own MCP packets, MCP operation assumes that a loop exists in the L2 network - which could indeed be the result of mis-configured STP, and shuts down the port. And for the record, STP almost never stops working, its just that humans don't configure their switches to make STP work the way they intended.
Having said that, there is SOME STP consideration with MCP in that it waits a period of time to allow STP to converge before shutting down ports.
But which one will be blocked?
The one that receives its own MCP PDU. If there is a L2 loop in the external network, it is possible multiple ports will be blocked.
Is there any documentation explaning with details how ACI identifies that port?
Yes. A port that receives its own MCP PDU will be the port that blocks.
Let's say if we have port A as primary port and port B as secondary port, is there any way to configure with MCP so that we will always block port B in case of loop?
There is no concept of "primary port" and "secondary port" with MCP, or with spanning tree for that matter. And there is no guarantee that both ports A and B will detect the loop - potentially one of those ports will detect the loop quick enough that the other port does NOT shut down. If you can figure out a way to make sure A detects the loop first, then "MCP ... will always block port B in case of loop". Setting the root bridge in the right place will probably do it (so that the switch connected to port B is an STP Alternate port - i.e. blocking)
And one more thing. Keep in mind that this entire discussion ASSUMES you are using the Cisco Proprietary PVSTP (or some variation of) - not the IEEE standards based MST which sends BPDUs untagged always. If you are using MST, then you need to create a special EPG, then configure all ports that connect to switches as either access ports (surely not) or 802.1p ports with the same dedicated VLAN ID associated (untagged) in each case, and placed in the special EPG you created to catch MST BPDUs.
Hi @suneq ,
Sorry it's taken so long to answer this. Required a bit of research
Firstly, thanks for the great diagram. Let me repeat it here so people reading this on email can see it.
Now to your comments/questions
According to the design guide, Cisco ACI floods BPDUs only between the ports in the bridge domain that have the same encapsulation.
Almost correct - ACI floods BPDUs only between the ports in the bridge domain EPG that have the same VLAN encapsulation (and from the same VLAN Pool).
When a BPDU arrives at a port carrying a VLAN tag, the BPDU is flooded on a multicast tree using the VXLAN VNID of the FD_VLAN, this VNID is actually calculated from the VLAN pool that the VLAN was taken from. So if, say VLAN 5 was used by two different tenants, and each tenant had its own VLAN Pool containing VLAN 5, each instance of VLAN 5 would use a different VNID, so there'd by no leaking of BPDUs from one instance of VLAN 5 to the other. (This reference explains this concept more fully)
In essence, this means that ACI floods BPDUs only between the ports in same EPG. Forget about the corner cases and keep that concept in mind. And in your case, this is true since you are using one VLAN ID and one EPG.
1. As you may see, the incoming traffic to port A & B are untagged, incoming traffic to port C as tagged with VLAN 10 but all 3 ports are configured with port encap VLAN 10 (EPG static binding). Are the ports A, B, C considered as using the same encapsulation? From design perspective, I just want to make sure that:
- BPDU coming from Legacy switch #1 will reach Legacy switch #2 via ACI
- BPDU coming from Legacy switch #1 will reach Legacy switch #3 via ACI
Yep - that will work. But keep in mind
- It's not usual to have switches connected via access ports
- You can't mix access and trunk ports for the same VLAN encap on an ACI switch, in other words, it would be OK for ports A & B to be on the same ACI switch, but port C (the trunk) could NOT be on the same switch as either port A or B. If you DO try, you'll get a fault
F0467 Configuration failed for node xxx due to Different encap modes are not allowed for an encap on a given interface
2. If MCP is enabled on port A & B and somehow STP stops working, from my understanding, ACI will detect the loop and block a port (A or B) with MCP.
Thats not quite how MCP works. MCP is independent of STP and works by sending multicast packets with a unique source MAC address linked to the sending port. If the port receives one of its own MCP packets, MCP operation assumes that a loop exists in the L2 network - which could indeed be the result of mis-configured STP, and shuts down the port. And for the record, STP almost never stops working, its just that humans don't configure their switches to make STP work the way they intended.
Having said that, there is SOME STP consideration with MCP in that it waits a period of time to allow STP to converge before shutting down ports.
But which one will be blocked?
The one that receives its own MCP PDU. If there is a L2 loop in the external network, it is possible multiple ports will be blocked.
Is there any documentation explaning with details how ACI identifies that port?
Yes. A port that receives its own MCP PDU will be the port that blocks.
Let's say if we have port A as primary port and port B as secondary port, is there any way to configure with MCP so that we will always block port B in case of loop?
There is no concept of "primary port" and "secondary port" with MCP, or with spanning tree for that matter. And there is no guarantee that both ports A and B will detect the loop - potentially one of those ports will detect the loop quick enough that the other port does NOT shut down. If you can figure out a way to make sure A detects the loop first, then "MCP ... will always block port B in case of loop". Setting the root bridge in the right place will probably do it (so that the switch connected to port B is an STP Alternate port - i.e. blocking)
And one more thing. Keep in mind that this entire discussion ASSUMES you are using the Cisco Proprietary PVSTP (or some variation of) - not the IEEE standards based MST which sends BPDUs untagged always. If you are using MST, then you need to create a special EPG, then configure all ports that connect to switches as either access ports (surely not) or 802.1p ports with the same dedicated VLAN ID associated (untagged) in each case, and placed in the special EPG you created to catch MST BPDUs.
