Overview
When we enable peer-switch, each nexus will share a virtual bridge ID which allows both of them to act as root for the vlan. For devices with a connection to each N7k in the vPC domain that are not capable of port-channel, L2 topology will rely on STP to block the redundant links. The peer-switch feature allows for pseudo STP configurations to allow non-vPC connections to load balance between the two N7k's. This article discusses in details the reason for the pseudo STP configurations and how they affect non-VPC and vPC links.
MAC address for each switch used within article:
N7K-1: 00:24:98:6f:3b:41
N7K-2: 00:24:98:6f:3b:42
SW-1 : 00:24:98:6f:3b:44
SW-2 : 00:24:98:6f:3b:43
Normal vPC behavior for hybrid-setup (mix of vPC and non-vPC links)
Both nexus switches are configured with a priority of 8192 for all vlans. We can see that N7K-1 wins the bridge election as it has the lower bridge ID (derived from the system MAC). Therefore, we expect SW-1 to block on the link from N7K-2. SW-2, since it is connected via a vPC will be forwarding. SW-2 will receive BPDUs only from the primary switch in the vPC, N7K-1 in this setup.
vPC without peer-switch enabled
SW-1# show span vlan 1
VLAN0001
Spanning tree enabled protocol rstp
Root ID Priority 8193
Address 0024.986f.3b41
Cost 4
Port 295 (Ethernet2/39)
Hello Time 2 sec Max Age 20 sec Forward Delay 15 sec
Bridge ID Priority 32769 (priority 32768 sys-id-ext 1)
Address 0024.986f.3b44
Hello Time 2 sec Max Age 20 sec Forward Delay 15 sec
Interface Role Sts Cost Prio.Nbr Type
---------------- ---- --- --------- -------- --------------------------------
Eth2/39 Root FWD 4 128.295 P2p
Eth2/40 Altn BLK 4 128.296 P2p
SW-1# show span vlan 1 detail
VLAN0001 is executing the rstp compatible Spanning Tree protocol
Bridge Identifier has priority 32768, sysid 1, address 0024.986f.3b44
Configured hello time 2, max age 20, forward delay 15
Current root has priority 8193, address 0024.986f.3b41
Root port is 295 (Ethernet2/39), cost of root path is 4
Topology change flag not set, detected flag not set
Number of topology changes 4 last change occurred 0:29:13 ago
from Ethernet2/39
Times: hold 1, topology change 35, notification 2
hello 2, max age 20, forward delay 15
Timers: hello 0, topology change 0, notification 0
Port 295 (Ethernet2/39) of VLAN0001 is root forwarding
Port path cost 4, Port priority 128, Port Identifier 128.295
Designated root has priority 8193, address 0024.986f.3b41
Designated bridge has priority 8193, address 0024.986f.3b41
Designated port id is 128.260, designated path cost 0, Topology change is set
Timers: message age 16, forward delay 0, hold 0
Number of transitions to forwarding state: 1
Link type is point-to-point by default
BPDU: sent 4, received 898
Port 296 (Ethernet2/40) of VLAN0001 is alternate blocking
Port path cost 4, Port priority 128, Port Identifier 128.296
Designated root has priority 8193, address 0024.986f.3b41
Designated bridge has priority 8193, address 0024.986f.3b42 <-- Although same priority,
Designated port id is 128.272, designated path cost 2 advertising Bridge-ID is lower
Timers: message age 16, forward delay 0, hold 0 and therefore this link will be BLK
Number of transitions to forwarding state: 2
Link type is point-to-point by default
BPDU: sent 6, received 895
Enable Peer-switch on both Nexus switches
When we enable peer-switch each nexus will share a virtual bridge ID which allows both of them to act as root for the vlan. The vPC peer-link will always be in a forwarding status and it runs Layer 2 Gateway Interconnection Protocol (L2GIP) to prevent bridging loops. Each nexus switch will send BPDUs with root bridge identified by virtual bridge-ID. On vPC links, the designated bridge ID will also have the virtual bridge-ID. For non-vPC links, the designated bridge ID will be the physical bridge-ID of the corresponding Nexus switch. This allows for the non-vPC switch (SW-1) to make a root decision based on the BPDU advertisements instead of depending on port-priority to break the tie. Note that peer-switch requires the vlan priority on each N7k to be identical for all vPC vlans (http://www.cisco.com/en/US/partner/docs/switches/datacenter/sw/6_x/nx-os/interfaces/configuration/guide/if_vPC.html#wp1830749).
vPC with peer-switch enabled
Non-vPC connection
SW-1# show span vlan 1
VLAN0001
Spanning tree enabled protocol rstp
Root ID Priority 8193
Address 0023.04ee.be01
Cost 4
Port 295 (Ethernet2/39)
Hello Time 2 sec Max Age 20 sec Forward Delay 15 sec
Bridge ID Priority 32769 (priority 32768 sys-id-ext 1)
Address 0024.986f.3b44
Hello Time 2 sec Max Age 20 sec Forward Delay 15 sec
Interface Role Sts Cost Prio.Nbr Type
---------------- ---- --- --------- -------- --------------------------------
Eth2/39 Root FWD 4 128.295 P2p
Eth2/40 Altn BLK 4 128.296 P2p
SW-1# show span vlan 1 detail
VLAN0001 is executing the rstp compatible Spanning Tree protocol
Bridge Identifier has priority 32768, sysid 1, address 0024.986f.3b44
Configured hello time 2, max age 20, forward delay 15
Current root has priority 8193, address 0023.04ee.be01
Root port is 295 (Ethernet2/39), cost of root path is 4
Topology change flag not set, detected flag not set
Number of topology changes 6 last change occurred 0:25:38 ago
from Ethernet2/39
Times: hold 1, topology change 35, notification 2
hello 2, max age 20, forward delay 15
Timers: hello 0, topology change 0, notification 0
Port 295 (Ethernet2/39) of VLAN0001 is root forwarding
Port path cost 4, Port priority 128, Port Identifier 128.295
Designated root has priority 8193, address 0023.04ee.be01 <---Root Bridge = virtual ID
Designated bridge has priority 8193, address 0024.986f.3b41 <---Designated Bridge ID = N7k1
Designated port id is 128.260, designated path cost 0, Topology change is set
Timers: message age 16, forward delay 0, hold 0
Number of transitions to forwarding state: 1
Link type is point-to-point by default
BPDU: sent 4, received 2280
Port 296 (Ethernet2/40) of VLAN0001 is alternate blocking
Port path cost 4, Port priority 128, Port Identifier 128.296
Designated root has priority 8193, address 0023.04ee.be01 <---Root Bridge = virtual ID
Designated bridge has priority 8193, address 0024.986f.3b42 <---Designated Bridge ID = N7k1
Designated port id is 128.272, designated path cost 0
Timers: message age 15, forward delay 0, hold 0
Number of transitions to forwarding state: 2
Link type is point-to-point by default
BPDU: sent 7, received 2278
The root bridge ID is still the virtual bridge ID. Additionally, the designated bridge is that of N7k-1 as the MAC of N7k-1 is lower than the MAC of n7k-2. In this scenario, all non-vPC connections will allows forward on the links directly connected to N7k-1 and all links connected to N7k-2 will be in a blocking status.
vPC connection
SW-2# show span vlan 1
VLAN0001
Spanning tree enabled protocol rstp
Root ID Priority 8193
Address 0023.04ee.be01
Cost 3
Port 4105 (port-channel10)
Hello Time 2 sec Max Age 20 sec Forward Delay 15 sec
Bridge ID Priority 32769 (priority 32768 sys-id-ext 1)
Address 0024.986f.3b43
Hello Time 2 sec Max Age 20 sec Forward Delay 15 sec
Interface Role Sts Cost Prio.Nbr Type
---------------- ---- --- --------- -------- --------------------------------
Po10 Root FWD 3 128.4105 P2p
SW-2# show span vlan 1 detail
VLAN0001 is executing the rstp compatible Spanning Tree protocol
Bridge Identifier has priority 32768, sysid 1, address 0024.986f.3b43
Configured hello time 2, max age 20, forward delay 15
Current root has priority 8193, address 0023.04ee.be01
Root port is 4105 (port-channel10), cost of root path is 3
Topology change flag not set, detected flag not set
Number of topology changes 5 last change occurred 0:21:40 ago
from port-channel10
Times: hold 1, topology change 35, notification 2
hello 2, max age 20, forward delay 15
Timers: hello 0, topology change 0, notification 0
Port 4105 (port-channel10) of VLAN0001 is root forwarding
Port path cost 3, Port priority 128, Port Identifier 128.4105
Designated root has priority 8193, address 0023.04ee.be01 <--- Virtual Bridge-ID
Designated bridge has priority 8193, address 0023.04ee.be01 <--- Virtual Bridge-ID
Designated port id is 128.4105, designated path cost 0, Topology change is set
Timers: message age 15, forward delay 0, hold 0
Number of transitions to forwarding state: 2
Link type is point-to-point by default
BPDU: sent 96, received 2804
For vPC connections, both the root bridge and designated bridge use the virtual Bridge-ID.
Enable load-balancing between vlans on non-vPC links
Under the default peer-switch configuration, all vlans on the non-vPC switch will be forwarding on a single link. To load-balance between vlans, the designated and root priorities advertised can be manually set via spanning-tree pseduo-information configurations. It is recommended that the root priority under the pseduo-information be lower than the global spanning-tree priority to prevent TCNs under failover conditions. The designated priorities can be load-balanced between the two nexus switches in the vPC domain.
In this example, the global spanning-tree priorities on both nexus switches have been set to 8192. Under the pseudo-information, the root priority has been configured as 4096 which is lower than the "best" priority of 8192. Finally, we are load-balancing between the two N7k's by alternating the designated priorities for vlan 9 and vlan 10. For the non-vPC connections to SW-1, vlan 9 will be forwarded on the link to N7K-1 and vlan 10 will be forwarded on the link to N7K-2.
N7K-1
spanning-tree vlan 1-10 priority 8192
spanning-tree pseudo-information
vlan 9-10 root priority 4096
vlan 9 designated priority 8192
vlan 10 designated priority 12288
vpc domain 1
peer-switch
N7K-2
spanning-tree vlan 1-10 priority 8192
spanning-tree pseudo-information
vlan 9-10 root priority 4096
vlan 9 designated priority 12288
vlan 10 designated priority 8192
vpc domain 1
peer-switch
vPC with peer-switch enabled and load-balancing between vlans on non-vPC links
In this example the dark yellow comments represents vlan 9 and the teal comments represents vlan 10.
non-vPC Connection
SW-1# show span vlan 9
VLAN0009
Spanning tree enabled protocol rstp
Root ID Priority 4105
Address 0023.04ee.be01
Cost 4
Port 295 (Ethernet2/39)
Hello Time 2 sec Max Age 20 sec Forward Delay 15 sec
Bridge ID Priority 32777 (priority 32768 sys-id-ext 9)
Address 0024.986f.3b44
Hello Time 2 sec Max Age 20 sec Forward Delay 15 sec
Interface Role Sts Cost Prio.Nbr Type
---------------- ---- --- --------- -------- --------------------------------
Eth2/39 Root FWD 4 128.295 P2p
Eth2/40 Altn BLK 4 128.296 P2p
SW-1# show span vlan 10
VLAN0010
Spanning tree enabled protocol rstp
Root ID Priority 4106
Address 0023.04ee.be01
Cost 4
Port 296 (Ethernet2/40)
Hello Time 2 sec Max Age 20 sec Forward Delay 15 sec
Bridge ID Priority 32778 (priority 32768 sys-id-ext 10)
Address 0024.986f.3b44
Hello Time 2 sec Max Age 20 sec Forward Delay 15 sec
Interface Role Sts Cost Prio.Nbr Type
---------------- ---- --- --------- -------- --------------------------------
Eth2/39 Altn BLK 4 128.295 P2p
Eth2/40 Root FWD 4 128.296 P2p
SW-1# show span vlan 9 detail
VLAN0009 is executing the rstp compatible Spanning Tree protocol
Bridge Identifier has priority 32768, sysid 9, address 0024.986f.3b44
Configured hello time 2, max age 20, forward delay 15
Current root has priority 4105, address 0023.04ee.be01
Root port is 295 (Ethernet2/39), cost of root path is 4
Topology change flag not set, detected flag not set
Number of topology changes 16 last change occurred 0:06:56 ago
from Ethernet2/39
Times: hold 1, topology change 35, notification 2
hello 2, max age 20, forward delay 15
Timers: hello 0, topology change 0, notification 0
Port 295 (Ethernet2/39) of VLAN0009 is root forwarding
Port path cost 4, Port priority 128, Port Identifier 128.295
Designated root has priority 4105, address 0023.04ee.be01 <--- Root Virtual Bridge-ID
Designated bridge has priority 8201, address 0024.986f.3b41 <--- Designated N7k-1, 8201
Designated port id is 128.260, designated path cost 0
Timers: message age 15, forward delay 0, hold 0
Number of transitions to forwarding state: 3
Link type is point-to-point by default
BPDU: sent 31, received 3486
Port 296 (Ethernet2/40) of VLAN0009 is alternate blocking
Port path cost 4, Port priority 128, Port Identifier 128.296
Designated root has priority 4105, address 0023.04ee.be01 <--- Root Virtual Bridge-ID
Designated bridge has priority 12297, address 0024.986f.3b42 <--- Designated is N7k-2, 12297
Designated port id is 128.272, designated path cost 0
Timers: message age 15, forward delay 0, hold 0
Number of transitions to forwarding state: 4
Link type is point-to-point by default
BPDU: sent 31, received 3496
For vlan 9, sw-1 see's the pseudo root bridge priority and bridge ID as the same value from both N7k-1 and N7k-2. However, both N7k-1 and N7k-2 send their configured pseudo designated priorities. Therefore, sw-1 sees the designated bridge priority of 8201 (8192+9) from N7k-1 and the designated bridge priority of 12297 (12288+9) from N7k-2 which allows sw-1 to choose the link toward N7k-1 as the forwarding link.
SW-1# show span vlan 10 detail
VLAN0010 is executing the rstp compatible Spanning Tree protocol
Bridge Identifier has priority 32768, sysid 10, address 0024.986f.3b44
Configured hello time 2, max age 20, forward delay 15
Current root has priority 4106, address 0023.04ee.be01
Root port is 296 (Ethernet2/40), cost of root path is 4
Topology change flag not set, detected flag not set
Number of topology changes 7 last change occurred 0:07:13 ago
from Ethernet2/40
Times: hold 1, topology change 35, notification 2
hello 2, max age 20, forward delay 15
Timers: hello 0, topology change 0, notification 0
Port 295 (Ethernet2/39) of VLAN0010 is alternate blocking
Port path cost 4, Port priority 128, Port Identifier 128.295
Designated root has priority 4106, address 0023.04ee.be01 <--- Root Virtual Bridge-ID
Designated bridge has priority 12298, address 0024.986f.3b41 <--- Designated N7k-1, 12298
Designated port id is 128.260, designated path cost 0, Topology change is set
Timers: message age 16, forward delay 0, hold 0
Number of transitions to forwarding state: 1
Link type is point-to-point by default
BPDU: sent 4, received 3497
Port 296 (Ethernet2/40) of VLAN0010 is root forwarding
Port path cost 4, Port priority 128, Port Identifier 128.296
Designated root has priority 4106, address 0023.04ee.be01 <--- Root Virtual Bridge-ID
Designated bridge has priority 8202, address 0024.986f.3b42 <--- Designated N7k-2, 8202
Designated port id is 128.272, designated path cost 0
Timers: message age 16, forward delay 0, hold 0
Number of transitions to forwarding state: 3
Link type is point-to-point by default
BPDU: sent 10, received 3492
Similarly for vlan 10, sw-1 see's the pseudo root bridge priority and bridge ID as the same value from both N7k-1 and N7k-2. Again, both N7k-1 and N7k-2 send their configured pseudo designated priorities. For vlan 10, sw-1 sees the designated bridge priority of 12298 (12288+10) from N7k-1 and the designated bridge priority of 8202 (8192+10) from N7k-2 which allows sw-1 to choose the link toward N7k-2 as the forwarding link. In this way, the non-vPC connected switches can load-balance vlans between N7k-1 and N7k-2.
vPC Connection
SW-2# show span vlan 9
VLAN0009
Spanning tree enabled protocol rstp
Root ID Priority 4105
Address 0023.04ee.be01
Cost 3
Port 4105 (port-channel10)
Hello Time 2 sec Max Age 20 sec Forward Delay 15 sec
Bridge ID Priority 32777 (priority 32768 sys-id-ext 9)
Address 0024.986f.3b43
Hello Time 2 sec Max Age 20 sec Forward Delay 15 sec
Interface Role Sts Cost Prio.Nbr Type
---------------- ---- --- --------- -------- --------------------------------
Po10 Root FWD 3 128.4105 P2p
SW-2# show span vlan 10
VLAN0010
Spanning tree enabled protocol rstp
Root ID Priority 4106
Address 0023.04ee.be01
Cost 3
Port 4105 (port-channel10)
Hello Time 2 sec Max Age 20 sec Forward Delay 15 sec
Bridge ID Priority 32778 (priority 32768 sys-id-ext 10)
Address 0024.986f.3b43
Hello Time 2 sec Max Age 20 sec Forward Delay 15 sec
Interface Role Sts Cost Prio.Nbr Type
---------------- ---- --- --------- -------- --------------------------------
Po10 Root FWD 3 128.4105 P2p
SW-2#show span vlan 9 detail
VLAN0009 is executing the rstp compatible Spanning Tree protocol
Bridge Identifier has priority 32768, sysid 9, address 0024.986f.3b43
Configured hello time 2, max age 20, forward delay 15
Current root has priority 4105, address 0023.04ee.be01
Root port is 4105 (port-channel10), cost of root path is 3
Topology change flag not set, detected flag not set
Number of topology changes 12 last change occurred 0:04:29 ago
from port-channel10
Times: hold 1, topology change 35, notification 2
hello 2, max age 20, forward delay 15
Timers: hello 0, topology change 0, notification 0
Port 4105 (port-channel10) of VLAN0009 is root forwarding
Port path cost 3, Port priority 128, Port Identifier 128.4105
Designated root has priority 4105, address 0023.04ee.be01 <--- Root Virtual Bridge-ID
Designated bridge has priority 4105, address 0023.04ee.be01 <--- Root Virtual Bridge-ID
Designated port id is 128.4105, designated path cost 0, Topology change is set
Timers: message age 15, forward delay 0, hold 0
Number of transitions to forwarding state: 2
Link type is point-to-point by default
BPDU: sent 119, received 4867
SW-2# show span vlan 10 detail
VLAN0010 is executing the rstp compatible Spanning Tree protocol
Bridge Identifier has priority 32768, sysid 10, address 0024.986f.3b43
Configured hello time 2, max age 20, forward delay 15
Current root has priority 4106, address 0023.04ee.be01
Root port is 4105 (port-channel10), cost of root path is 3
Topology change flag not set, detected flag not set
Number of topology changes 6 last change occurred 0:04:36 ago
from port-channel10
Times: hold 1, topology change 35, notification 2
hello 2, max age 20, forward delay 15
Timers: hello 0, topology change 0, notification 0
Port 4105 (port-channel10) of VLAN0010 is root forwarding
Port path cost 3, Port priority 128, Port Identifier 128.4105
Designated root has priority 4106, address 0023.04ee.be01 <--- Root Virtual Bridge-ID
Designated bridge has priority 4106, address 0023.04ee.be01 <--- Root Virtual Bridge-ID
Designated port id is 128.4105, designated path cost 0, Topology change is set
Timers: message age 17, forward delay 0, hold 0
Number of transitions to forwarding state: 2
Link type is point-to-point by default
BPDU: sent 96, received 5179
Note that the pseudo designated priorities have no affect on vPC links. Both switches will advertise the designated root and the designated bridge with the values of the virtual root.