Re: Spanning-tree blocks VPC port-Channel

Kaneki007 · ‎07-19-2024

Hello,

I got an issue with a VPC link after upgrading my two VPC peers to 10.3.5(M) NXOS, below i little demonstration on how my my connections are:

My ports configuration is like this:

N9K1#interface ETH1/9

lacp rate fast
switchport
switchport mode trunk
switchport trunk allowed vlan 4008
spanning-tree port type normal
channel-group 106 mode active
no shutdown

N9K1#int Po106

switchport
switchport mode trunk
switchport trunk allowed vlan 4008
spanning-tree port type normal
vpc 106

======================================

N9K2#

N9K2#interface ETH1/9

lacp rate fast
switchport
switchport mode trunk
switchport trunk allowed vlan 4008
spanning-tree port type normal
channel-group 106 mode active
no shutdown

N9K2#int Po106

switchport
switchport mode trunk
switchport trunk allowed vlan 4008
spanning-tree port type normal
vpc 106

=================================================

When i let the two ports UP both of them start flapping and then they go down, but when i shutdown the second port it works, now i working with just the port channel of the N9K1, i tried to understand the issue but i don't find why would spanning three blocks my port-channel since i have vpc he should consider the two nexus as to be one switch

this is the result when i let two port-channels UP:

On the N9K1:

N9K1#Show spanning-tree vlan 4008

VLAN4008
Spanning tree enabled protocol rstp
Root ID Priority 36776
Address 0023.04ee.be65
This bridge is the root
Hello Time 2 sec Max Age 20 sec Forward Delay 15 sec

Bridge ID Priority 36776 (priority 32768 sys-id-ext 4008)
Address 0023.04ee.be65
Hello Time 2 sec Max Age 20 sec Forward Delay 15 sec

Interface Role Sts Cost Prio.Nbr Type
---------------- ---- --- --------- -------- --------------------------------
Po100 Desg FWD 1 128.4195 (vPC peer-link) Network P2p
Po107 Desg FWD 1 128.4202 (vPC) Edge P2p
Eth1/57 Desg FWD 4 128.225 Edge P2p
Eth1/58 Desg FWD 4 128.229 Edge P2p

N9K1#show port-channel sum | inc 106

106 Po106(SD) Eth LACP Eth1/9(D)

N9K1#show interface po106 brief

------------------------------------------------------------------------------------------
Port-channel VLAN Type Mode Status Reason Speed Protocol
Interface
------------------------------------------------------------------------------------------
Po106 1 eth trunk down No operational members auto(D) lacp

N9K1#show interface ETH1/9 br

--------------------------------------------------------------------------------
Ethernet VLAN Type Mode Status Reason Speed Port
Interface Ch #
--------------------------------------------------------------------------------
Eth1/9 1 eth trunk down initializing auto(D) 106

======================================================

On the N9K2

N9K2#show spanning-tree vlan 4008

VLAN4008
Spanning tree enabled protocol rstp
Root ID Priority 36776
Address 0023.04ee.be65
This bridge is the root
Hello Time 2 sec Max Age 20 sec Forward Delay 15 sec

Bridge ID Priority 36776 (priority 32768 sys-id-ext 4008)
Address 0023.04ee.be65
Hello Time 2 sec Max Age 20 sec Forward Delay 15 sec

Interface Role Sts Cost Prio.Nbr Type
---------------- ---- --- --------- -------- --------------------------------
Po100 Root FWD 1 128.4195 (vPC peer-link) Network P2p
Po107 Desg FWD 1 128.4202 (vPC) Edge P2p
Eth1/57 Desg FWD 4 128.225 Edge P2p
Eth1/58 Desg FWD 4 128.229 Edge P2p

#show int Po106 brief

------------------------------------------------------------------------------------------
Port-channel VLAN Type Mode Status Reason Speed Protocol
Interface
------------------------------------------------------------------------------------------
Po106 1 eth trunk down No operational members auto(D) lacp

show int Eth1/9 brief

--------------------------------------------------------------------------------
Ethernet VLAN Type Mode Status Reason Speed Port
Interface Ch #
--------------------------------------------------------------------------------
Eth1/9 1 eth trunk down initializing auto(D) 106

And then when i shutdown the port on the second switch

N9K1#show spanning-tree vlan 4008

Po106 Desg FWD 1 128.4201 (vPC) P2p

I also checked vpc

show vpc consistency-parameters vpc 106

Legend:
Type 1 : vPC will be suspended in case of mismatch

Name Type Local Value Peer Value
------------- ---- ---------------------- -----------------------
delayed-lacp 1 disabled disabled
lacp suspend disable 1 enabled enabled
mode 1 active active
Switchport Isolated 1 0 0
Interface type 1 port-channel port-channel
LACP Mode 1 on on
Virtual-ethernet-bridge 1 Disabled Disabled
Speed 1 10 Gb/s 10 Gb/s
Duplex 1 full full
MTU 1 1500 1500
Port Mode 1 trunk trunk
Native Vlan 1 1 1
Admin port mode 1 trunk trunk
Port-type External 1 Disabled Disabled
STP Port Guard 1 Default Default
STP Port Type 1 Normal Port Normal Port
STP MST Simulate PVST 1 Default Default
lag-id 1 [(7f9b, [(7f9b,
0-23-4-ee-be-65, 806a, 0-23-4-ee-be-65, 806a,
0, 0), (8000, 0, 0), (8000,
98-d8-8c-94-6c-0, 98-d8-8c-94-6c-0,
43e8, 0, 0)] 43e8, 0, 0)]
Allow-Multi-Tag 1 Disabled Disabled
Vlan xlt mapping 1 Disabled Disabled
L2PT Vxlan Vlans 2
vPC card type 1 N9K TOR N9K TOR
Allowed VLANs - 4008 4008
Local suspended VLANs - - -

Can you please help to resolve the issue? Thanks a lot

Flavio Miranda · ‎07-19-2024

Hi @Kaneki007

It was working before the upgrade?
Who is putting the port in block the Nexus or the nortel?

Kaneki007 · ‎07-19-2024

Yes it was working till i did the upgrade, its the nexus who is blocking the port

MHM Cisco World · ‎07-19-2024

SPT dont show PO 106?
so first
1- shut/no shut the PO106 in both NSK
2- show vpc brief if above step not work

MHM

Kaneki007 · ‎07-19-2024

Hello @MHM Cisco World , thanks for your reply

1- no when i let the two portchannels up i cannot see then on spanning three

2 - also VPC on the two N9K goes down when both portchannels are UP, when i let only one portchannel UP, vpc also terns up

MHM Cisco World · ‎07-19-2024

1- When it appear in STP can I see shwo spanning tree

2- what you meaning both PO' there is only one PO which is 106

MHM

MHM Cisco World · ‎07-20-2024

also one more point to check

show vpc role <<- check if both NSK is primary i.e. you have split brain

MHM Cisco World · ‎07-25-2024

show vpc role <<- this need for both NSK to check if both is primary or not
show vpc consistency-parameters global/vlan/interface e/interface PO/vpc <> <<- there is different in config between two NSK

MHM

Kaneki007 · ‎07-25-2024

@MHM Cisco World thanks for your replay

Primary switch:

show vpc role

vPC Role status
----------------------------------------------------
vPC role : primary
Dual Active Detection Status : 0
vPC system-mac : 00:23:04:ee:be:65
vPC system-priority : 32667
vPC local system-mac : c0:14:fe:b0:d6:13
vPC local role-priority : 100
vPC local config role-priority : 100
vPC peer system-mac : c0:14:fe:b0:be:8f
vPC peer role-priority : 200
vPC peer config role-priority : 200

Secondary switch:

show vpc role

vPC Role status
----------------------------------------------------
vPC role : secondary
Dual Active Detection Status : 0
vPC system-mac : 00:23:04:ee:be:65
vPC system-priority : 32667
vPC local system-mac : c0:14:fe:b0:be:8f
vPC local role-priority : 200
vPC local config role-priority : 200
vPC peer system-mac : c0:14:fe:b0:d6:13
vPC peer role-priority : 100
vPC peer config role-priority : 100

====================================================

show vpc role

vPC Role status
----------------------------------------------------
vPC role : primary
Dual Active Detection Status : 0
vPC system-mac : 00:23:04:ee:be:65
vPC system-priority : 32667
vPC local system-mac : c0:14:fe:b0:d6:13
vPC local role-priority : 100
vPC local config role-priority : 100
vPC peer system-mac : c0:14:fe:b0:be:8f
vPC peer role-priority : 200
vPC peer config role-priority : 200

#show vpc consistency-parameters global

Legend:
Type 1 : vPC will be suspended in case of mismatch

Name Type Local Value Peer Value
------------- ---- ---------------------- -----------------------
STP MST Simulate PVST 1 Enabled Enabled
STP Port Type, Edge 1 Normal, Disabled, Normal, Disabled,
BPDUFilter, Edge BPDUGuard Disabled Disabled
STP MST Region Name 1 "" ""
STP Disabled 1 None None
STP Mode 1 Rapid-PVST Rapid-PVST
STP Bridge Assurance 1 Enabled Enabled
STP Loopguard 1 Disabled Disabled
STP MST Region Instance to 1
VLAN Mapping
STP MST Region Revision 1 0 0
Interface-vlan admin up 2
Interface-vlan routing 2 3956 3956
capability
Xconnect Vlans 1
QoS (Cos) 2 ([0-7], [], [], [], ([0-7], [], [], [],
[], [], [], []) [], [], [], [])
Network QoS (MTU) 2 (1500, 1500, 1500, (1500, 1500, 1500,
1500, 1500, 1500, 1500, 1500, 1500,
1500, 1500) 1500, 1500)
Network Qos (Pause: 2 (F, F, F, F, F, F, F, (F, F, F, F, F, F, F,
T->Enabled, F->Disabled) F) F)
Input Queuing (Bandwidth) 2 (0, 0, 0, 0, 0, 0, 0, (0, 0, 0, 0, 0, 0, 0,
0) 0)
Input Queuing (Absolute 2 (F, F, F, F, F, F, F, (F, F, F, F, F, F, F,
Priority: T->Enabled, F) F)
F->Disabled)
Output Queuing (Bandwidth 2 (100, 0, 0, 0, 0, 0, (100, 0, 0, 0, 0, 0,
Remaining) 0, 0) 0, 0)
Output Queuing (Absolute 2 (F, F, F, F, F, F, F, (F, F, F, F, F, F, F,
Priority: T->Enabled, T) T)
F->Disabled)
Allowed VLANs - 40,3398,3401-3409, 40,3398,3401-3409,
3493-3496,3498-3500, 3493-3496,3498-3500,
3505,3525-3528, 3505,3525-3528,
3533-3536,3539-3542, 3533-3536,3539-3542,
3546-3547,3549-3550, 3546-3547,3549-3550,
3601-3603,3612-3619, 3601-3603,3612-3619,
3761-3762,3956, 3761-3762,3956,
4008-4009 4008-4009
Local suspended VLANs - - -

# show vpc consistency-parameters vlans

Name Type Reason Code Pass Vlans
------------- ---- ---------------------- -----------------------
STP MST Simulate PVST 1 SUCCESS 0-4095
STP Port Type, Edge 1 SUCCESS 0-4095
BPDUFilter, Edge BPDUGuard
STP MST Region Name 1 SUCCESS 0-4095
STP Disabled 1 SUCCESS 0-4095
STP Mode 1 SUCCESS 0-4095
STP Bridge Assurance 1 SUCCESS 0-4095
STP Loopguard 1 SUCCESS 0-4095
Xconnect Vlans 1 SUCCESS 0-4095
STP MST Region Instance to 1 SUCCESS 0-4095
VLAN Mapping
STP MST Region Revision 1 SUCCESS 0-4095
Pass Vlans - 0-4095

show vpc consistency-parameters interface port-channel 106

Legend:
Type 1 : vPC will be suspended in case of mismatch

Name Type Local Value Peer Value
------------- ---- ---------------------- -----------------------
delayed-lacp 1 disabled disabled
lacp suspend disable 1 enabled enabled
mode 1 active active
Switchport Isolated 1 0 0
Interface type 1 port-channel port-channel
LACP Mode 1 on on
Virtual-ethernet-bridge 1 Disabled Disabled
Speed 1 10 Gb/s 10 Gb/s
Duplex 1 full full
MTU 1 1500 1500
Port Mode 1 trunk trunk
Native Vlan 1 1 1
Admin port mode 1 trunk trunk
Port-type External 1 Disabled Disabled
STP Port Guard 1 Default Default
STP Port Type 1 Normal Port Normal Port
STP MST Simulate PVST 1 Default Default
lag-id 1 [(7f9b, [(7f9b,
0-23-4-ee-be-65, 806a, 0-23-4-ee-be-65, 806a,
0, 0), (8000, 0, 0), (8000,
98-d8-8c-94-6c-0, 98-d8-8c-94-6c-0,
43e8, 0, 0)] 43e8, 0, 0)]
Allow-Multi-Tag 1 Disabled Disabled
Vlan xlt mapping 1 Disabled Disabled
L2PT Vxlan Vlans 2
vPC card type 1 N9K TOR N9K TOR
Allowed VLANs - 4008 4008
Local suspended VLANs - - -

# show vpc consistency-parameters vpc 106

Legend:
Type 1 : vPC will be suspended in case of mismatch

Name Type Local Value Peer Value
------------- ---- ---------------------- -----------------------
delayed-lacp 1 disabled disabled
lacp suspend disable 1 enabled enabled
mode 1 active active
Switchport Isolated 1 0 0
Interface type 1 port-channel port-channel
LACP Mode 1 on on
Virtual-ethernet-bridge 1 Disabled Disabled
Speed 1 10 Gb/s 10 Gb/s
Duplex 1 full full
MTU 1 1500 1500
Port Mode 1 trunk trunk
Native Vlan 1 1 1
Admin port mode 1 trunk trunk
Port-type External 1 Disabled Disabled
STP Port Guard 1 Default Default
STP Port Type 1 Normal Port Normal Port
STP MST Simulate PVST 1 Default Default
lag-id 1 [(7f9b, [(7f9b,
0-23-4-ee-be-65, 806a, 0-23-4-ee-be-65, 806a,
0, 0), (8000, 0, 0), (8000,
98-d8-8c-94-6c-0, 98-d8-8c-94-6c-0,
43e8, 0, 0)] 43e8, 0, 0)]
Allow-Multi-Tag 1 Disabled Disabled
Vlan xlt mapping 1 Disabled Disabled
L2PT Vxlan Vlans 2
vPC card type 1 N9K TOR N9K TOR
Allowed VLANs - 4008 4008
Local suspended VLANs - - -

Kaneki007 · ‎07-25-2024

Hello @MHM Cisco World

in fact before ugrading the switch i did a write memory so noting is changed when it comes to configurations, i double checked the things you mentionned above:

- Vlan is allowed on the peer link

- we don't use MST

- in the primary switch :

show port-channel summary | inc Po106
106 Po106(SU) Eth LACP Eth1/9(P)

===========================

show spanning-tree vlan 4008

VLAN4008
Spanning tree enabled protocol rstp
Root ID Priority 36776
Address 0023.04ee.be65
This bridge is the root
Hello Time 2 sec Max Age 20 sec Forward Delay 15 sec

Bridge ID Priority 36776 (priority 32768 sys-id-ext 4008)
Address 0023.04ee.be65
Hello Time 2 sec Max Age 20 sec Forward Delay 15 sec

Interface Role Sts Cost Prio.Nbr Type
---------------- ---- --- --------- -------- --------------------------------
Po100 Desg FWD 1 128.4195 (vPC peer-link) Network P2p
Po106 Desg FWD 1 128.4201 (vPC) P2p
Po107 Desg FWD 1 128.4202 (vPC) Edge P2p
Eth1/57 Desg FWD 4 128.225 Edge P2p
Eth1/58 Desg FWD 4 128.229 Edge P2p

in the secondary switch the PO doesn't appear on spanning-tree because i shutdown it :

show port-channel summary | inc Po106
106 Po106(SD) Eth LACP Eth1/9(D)

===================================

show spanning-tree vlan 4008

VLAN4008
Spanning tree enabled protocol rstp
Root ID Priority 36776
Address 0023.04ee.be65
This bridge is the root
Hello Time 2 sec Max Age 20 sec Forward Delay 15 sec

Bridge ID Priority 36776 (priority 32768 sys-id-ext 4008)
Address 0023.04ee.be65
Hello Time 2 sec Max Age 20 sec Forward Delay 15 sec

Interface Role Sts Cost Prio.Nbr Type
---------------- ---- --- --------- -------- --------------------------------
Po100 Desg FWD 1 128.4195 (vPC peer-link) Network P2p
Po106 Desg FWD 1 128.4201 (vPC) P2p
Po107 Desg FWD 1 128.4202 (vPC) Edge P2p
Eth1/57 Desg FWD 4 128.225 Edge P2p
Eth1/58 Desg FWD 4 128.229 Edge P2p

==============================================================

show vpc consistency-parameters interface port-channel 106

Legend:
Type 1 : vPC will be suspended in case of mismatch

Name Type Local Value Peer Value
------------- ---- ---------------------- -----------------------
delayed-lacp 1 disabled disabled
lacp suspend disable 1 enabled enabled
mode 1 active active
Switchport Isolated 1 0 0
Interface type 1 port-channel port-channel
LACP Mode 1 on on
Virtual-ethernet-bridge 1 Disabled Disabled
Speed 1 10 Gb/s 10 Gb/s
Duplex 1 full full
MTU 1 1500 1500
Port Mode 1 trunk trunk
Native Vlan 1 1 1
Admin port mode 1 trunk trunk
Port-type External 1 Disabled Disabled
STP Port Guard 1 Default Default
STP Port Type 1 Normal Port Normal Port
STP MST Simulate PVST 1 Default Default
lag-id 1 [(7f9b, [(7f9b,
0-23-4-ee-be-65, 806a, 0-23-4-ee-be-65, 806a,
0, 0), (8000, 0, 0), (8000,
98-d8-8c-94-6c-0, 98-d8-8c-94-6c-0,
43e8, 0, 0)] 43e8, 0, 0)]
Allow-Multi-Tag 1 Disabled Disabled
Vlan xlt mapping 1 Disabled Disabled
L2PT Vxlan Vlans 2
vPC card type 1 N9K TOR N9K TOR
Allowed VLANs - 4008 4008
Local suspended VLANs - - -

====================================

I checked spanning-tree logs and this is what i found:

2024 Jul 23 10:06:07.301891 stp: RSTP(4008): initializing port port-channel106 vlan VLAN4008
2024 Jul 23 10:06:07.301925 stp: RSTP(4008): port-channel106 (Slave vPC) is selected designated
2024 Jul 23 10:06:07.302066 stp: RSTP(4008): Port port-channel106 instance VLAN4008 moving from DIS to BLK
2024 Jul 23 10:06:07.307305 stp: RSTP(4008): port-channel106 state change to BLK completed, current state is BLK
2024 Jul 23 10:06:08.262261 stp: RSTP(4008): Port port-channel106 instance VLAN4008 moving from BLK to DIS
2024 Jul 23 10:06:13.289861 stp: RSTP(4008): initializing port port-channel106 vlan VLAN4008
2024 Jul 23 10:06:13.289889 stp: RSTP(4008): port-channel106 (Slave vPC) is selected designated
2024 Jul 23 10:06:13.290036 stp: RSTP(4008): Port port-channel106 instance VLAN4008 moving from DIS to BLK
2024 Jul 23 10:06:13.294708 stp: RSTP(4008): port-channel106 state change to BLK completed, current state is BLK
2024 Jul 23 10:06:14.261314 stp: RSTP(4008): Port port-channel106 instance VLAN4008 moving from BLK to DIS
2024 Jul 23 10:06:19.290866 stp: RSTP(4008): initializing port port-channel106 vlan VLAN4008
2024 Jul 23 10:06:19.290899 stp: RSTP(4008): port-channel106 (Slave vPC) is selected designated
2024 Jul 23 10:06:19.291039 stp: RSTP(4008): Port port-channel106 instance VLAN4008 moving from DIS to BLK
2024 Jul 23 10:06:19.295642 stp: RSTP(4008): port-channel106 state change to BLK completed, current state is BLK
2024 Jul 23 10:06:20.260696 stp: RSTP(4008): Port port-channel106 instance VLAN4008 moving from BLK to DIS
2024 Jul 23 10:06:25.293817 stp: RSTP(4008): initializing port port-channel106 vlan VLAN4008
2024 Jul 23 10:06:25.293850 stp: RSTP(4008): port-channel106 (Slave vPC) is selected designated
2024 Jul 23 10:06:25.293997 stp: RSTP(4008): Port port-channel106 instance VLAN4008 moving from DIS to BLK
2024 Jul 23 10:06:25.299527 stp: RSTP(4008): port-channel106 state change to BLK completed, current state is BLK
2024 Jul 23 10:06:26.263814 stp: RSTP(4008): Port port-channel106 instance VLAN4008 moving from BLK to DIS

MHM Cisco World · ‎07-25-2024

NSK-1
VLAN4008
Spanning tree enabled protocol rstp
Root ID Priority 36776
Address 0023.04ee.be65
This bridge is the root
Hello Time 2 sec Max Age 20 sec Forward Delay 15 sec

Bridge ID Priority 36776 (priority 32768 sys-id-ext 4008)
Address 0023.04ee.be65
Hello Time 2 sec Max Age 20 sec Forward Delay 15 sec

Interface Role Sts Cost Prio.Nbr Type
---------------- ---- --- --------- -------- --------------------------------
Po100 Desg FWD 1 128.4195 (vPC peer-link) Network P2p
Po106 Desg FWD 1 128.4201 (vPC) P2p
Po107 Desg FWD 1 128.4202 (vPC) Edge P2p
Eth1/57 Desg FWD 4 128.225 Edge P2p
Eth1/58 Desg FWD 4 128.229 Edge P2p

NSK-2

VLAN4008
Spanning tree enabled protocol rstp
Root ID Priority 36776
Address 0023.04ee.be65
This bridge is the root
Hello Time 2 sec Max Age 20 sec Forward Delay 15 sec

Bridge ID Priority 36776 (priority 32768 sys-id-ext 4008)
Address 0023.04ee.be65
Hello Time 2 sec Max Age 20 sec Forward Delay 15 sec

Interface Role Sts Cost Prio.Nbr Type
---------------- ---- --- --------- -------- --------------------------------
Po100 Desg FWD 1 128.4195 (vPC peer-link) Network P2p
Po106 Desg FWD 1 128.4201 (vPC) P2p
Po107 Desg FWD 1 128.4202 (vPC) Edge P2p
Eth1/57 Desg FWD 4 128.225 Edge P2p
Eth1/58 Desg FWD 4 128.229 Edge P2p

you are run Peer-switch
and you this make bridge ID same in both NSK but still the primary is root of all vpc VLAN
here the primary and secondary have same bridge ID and both is root, in STP even with NSK there is only one root in STP domain
so this can due to
1- check the connect SW see if it also elect as root if Yes change the VLAN4008 priority to make NSK-1 is primary root and NSK-2 secondary root
2- please check show vpc brief <<- are you see primary operation primary and secondary operation secondary or NOT?

MHM