Buy or Renew
Buy or Renew
Cisco + Splunk: It’s a new day for your data. Learn more.
 
cancel
Turn on suggestions
Auto-suggest helps you quickly narrow down your search results by suggesting possible matches as you type.
Showing results for 
Search instead for 
Did you mean: 
cancel
Start a conversation
1448
Views
5
Helpful
5
Replies

Pair of 7K vpc members with different stp states

cristian26
Level 1
Level 1

‎01-11-2018 07:15 AM - edited ‎03-01-2019 08:42 AM

Hi, we had an issue last weekend with a pair of 7k (7710) connected in vpc with a pair of 5k (5672) of 2 links through a DWDM. the thing is that the dwdm provider had a fiber issue, after they solved it and the dwdm link whent back up, the vpc between the 7K and 5K, that consists of 2 members, had diffrent STP states, 7k-1 had its vpc with some vlans in blk state and 7k-2 had all of the forwarding ( correct state)

here's a capture of a vlan in that state:

 

 

7k-1
Port 4113 (port-channel18, vPC) of VLAN1052 is designated blocking 
  Port path cost 1, Port priority 128, Port Identifier 128.4113
  Designated root has priority 9244, address 0023.04ee.be0a
  Designated bridge has priority 9244, address 002a.6adc.9841
  Designated port id is 128.4113, designated path cost 0
  Timers: message age 0, forward delay 0, hold 0
  Number of transitions to forwarding state: 0
  Link type is point-to-point by default
  BPDU: sent 160976, received 0

7k-2
Port 4113 (port-channel18, vPC) of VLAN1052 is designated forwarding 
 Port path cost 1, Port priority 128, Port Identifier 128.4113
 Designated root has priority 9244, address 0023.04ee.be0a
 Designated bridge has priority 9244, address 002a.6adc.9841
 Designated port id is 128.4113, designated path cost 0
 Timers: message age 0, forward delay 0, hold 0
 Number of transitions to forwarding state: 1
 Link type is point-to-point by default
 BPDU: sent 160999, received 118602

 thats an example of 1 of the vlans o a total of 18 :

ACO_N7k_CORE-01_F1R4# sh span int po18 | i BLK
VLAN0014 Root BLK 1 128.4113 (vPC) P2p
VLAN0155 Root BLK 1 128.4113 (vPC) P2p
VLAN0801 Root BLK 1 128.4113 (vPC) P2p
VLAN1052 Desg BLK 1 128.4113 (vPC) P2p
VLAN1059 Root BLK 1 128.4113 (vPC) P2p
VLAN1060 Desg BLK 1 128.4113 (vPC) P2p
VLAN1061 Desg BLK 1 128.4113 (vPC) P2p
VLAN1366 Desg BLK 1 128.4113 (vPC) P2p
VLAN1462 Root BLK 1 128.4113 (vPC) P2p
VLAN1567 Root BLK 1 128.4113 (vPC) P2p
VLAN1571 Root BLK 1 128.4113 (vPC) P2p
VLAN1606 Root BLK 1 128.4113 (vPC) P2p
VLAN1631 Root BLK 1 128.4113 (vPC) P2p
VLAN2020 Root BLK 1 128.4113 (vPC) P2p
VLAN2021 Root BLK 1 128.4113 (vPC) P2p
VLAN2023 Root BLK 1 128.4113 (vPC) P2p
VLAN2024 Root BLK 1 128.4113 (vPC) P2p
VLAN2101 Root BLK 1 128.4113 (vPC) P2p

 

as you can see there are some that even thou they're ROOT PORT are in BLK state even if that 7k is the root of some of that vlans.

 

these are the config that ocurs to me that you'll need to get the picture.

 

Spoiler
7k2
Software
BIOS: version 3.1.0
kickstart: version 6.2(12)
system: version 6.2(12)
BIOS compile time: 02/27/2013
kickstart image file is: bootflash:///n7700-s2-kickstart.6.2.12.bin
kickstart compile time: 12/3/2014 18:00:00 [02/02/2015 23:11:17]
system image file is: bootflash:///n7700-s2-dk9.6.2.12.bin
system compile time: 12/3/2014 18:00:00 [02/03/2015 00:42:21]

vpc domain 10
peer-switch
role priority 16384
peer-keepalive destination 123.123.123.2 source 123.123.123.1 vrf VPC_KEEP_ALIVE
delay restore 120
auto-recovery reload-delay 300
!
interface port-channel10
description 7k2
switchport
switchport mode trunk
switchport trunk native vlan 899
spanning-tree port type network
vpc peer-link
!
interface port-channel18
description N5K
switchport
switchport mode trunk
switchport trunk native vlan 899
switchport trunk allowed vlan 3,5,14,133,155,175,660,665-668,670
switchport trunk allowed vlan add 766,800-801,999,1019,1024-1027,1032
switchport trunk allowed vlan add 1050-1066,1356-1357,1366,1450-1464
switchport trunk allowed vlan add 1466,1550-1558,1560-1561,1563-1569
switchport trunk allowed vlan add 1571-1575,1577-1578,1580,1582-1585
switchport trunk allowed vlan add 1588,1591,1596,1598,1604,1606,1608
switchport trunk allowed vlan add 1610,1614-1615,1618-1619,1623,1631
switchport trunk allowed vlan add 1780,2020-2025,2030,2050-2055,2064-2065
switchport trunk allowed vlan add 2101,2104,2111,2255-2256,2400-2401
switchport trunk allowed vlan add 2403,2405,2414,2417,2422,2607,2615
switchport trunk allowed vlan add 2870,3016-3019,3024,3102
spanning-tree port type normal
speed 10000
vpc 18
--------------
7k1

Software
BIOS: version 3.1.0
kickstart: version 6.2(12)
system: version 6.2(12)
BIOS compile time: 02/27/2013
kickstart image file is: bootflash:///n7700-s2-kickstart.6.2.12.bin
kickstart compile time: 12/3/2014 18:00:00 [02/02/2015 23:11:17]
system image file is: bootflash:///n7700-s2-dk9.6.2.12.bin
system compile time: 12/3/2014 18:00:00 [02/03/2015 00:42:21]

vpc domain 10
peer-switch
role priority 20480
peer-keepalive destination 123.123.123.1 source 123.123.123.2 vrf VPC_KEEP_ALIVE
delay restore 120
auto-recovery reload-delay 300
!
interface port-channel10
description 7k1
switchport
switchport mode trunk
switchport trunk native vlan 899
spanning-tree port type network
vpc peer-link
!
interface port-channel18
description 5k
switchport
switchport mode trunk
switchport trunk native vlan 899
switchport trunk allowed vlan 3,5,14,133,155,175,660,665-668,670
switchport trunk allowed vlan add 766,800-801,999,1019,1024-1027,1032
switchport trunk allowed vlan add 1050-1066,1356-1357,1366,1450-1464
switchport trunk allowed vlan add 1466,1550-1558,1560-1561,1563-1569
switchport trunk allowed vlan add 1571-1575,1577-1578,1580,1582-1585
switchport trunk allowed vlan add 1588,1591,1596,1598,1604,1606,1608
switchport trunk allowed vlan add 1610,1614-1615,1618-1619,1623,1631
switchport trunk allowed vlan add 1780,2020-2025,2030,2050-2055,2064-2065
switchport trunk allowed vlan add 2101,2104,2111,2255-2256,2400-2401
switchport trunk allowed vlan add 2403,2405,2414,2417,2422,2607,2615
switchport trunk allowed vlan add 2870,3016-3019,3024,3102
spanning-tree port type normal
speed 10000
vpc 18
----------------------------------------------------------------------

5k1
vpc domain 1
role priority 16384
peer-keepalive destination 10.8.20.35
delay restore 150
auto-recovery reload-delay 300
!
interface port-channel1
description 5k2
switchport mode trunk
switchport trunk native vlan 899
spanning-tree port type network
speed 10000
vpc peer-link
!
interface port-channel18
description N7K 
switchport mode trunk
switchport trunk native vlan 899
switchport trunk allowed vlan 14,133,155,175,660,665-668,670,766,800-801,899,1019,1024-1027,1032,1050-1066,1356-1357,1366,1450-1464,1466,1550-1558,1560-1561,1563-1569,1571-1575,1577-1578,15
80,1582-1585,1588,1591,1598,1604,1606,1608,1610,1614-1615,1618-1619,1623,1631,1780,2020-2025,2030,2050-2055,2064-2065,2101,2104,2111,2255-2256,2400-2401,2403,2405,2414,2417,2422,2607,2615,301
6-3019,3024,3102
spanning-tree port type normal
speed 10000
vpc 15
------------------
5k2

vpc domain 1
role priority 20480
peer-keepalive destination 10.8.20.34
delay restore 150
auto-recovery reload-delay 300
!
interface port-channel1
switchport mode trunk
switchport trunk native vlan 899
spanning-tree port type network
speed 10000
vpc peer-link
!
interface port-channel18
description N7K
switchport mode trunk
switchport trunk native vlan 899
switchport trunk allowed vlan 14,133,155,175,660,665-668,670,766,800-801,899,1019,1024-1027,1032,1050-1066,1356-1357,1366,1450-1464,1466,1550-1558,1560-1561,1563-1569,1571-1575,1577-1578,15
80,1582-1585,1588,1591,1598,1604,1606,1608,1610,1614-1615,1618-1619,1623,1631,1780,2020-2025,2030,2050-2055,2064-2065,2101,2104,2111,2255-2256,2400-2401,2403,2405,2414,2417,2422,2607,2615,301
6-3019,3024,3102
spanning-tree port type normal
speed 10000
vpc 15

image.png



Heres a simple diagram of the topology where the issue was found
image.png

 

 

 

 

 

 

 

 

Labels:
0 Helpful
5 Replies 5

Rick1776
Level 5
Level 5

‎01-13-2018 09:45 AM

Why do yuh have the spanning tree the as normal? Should be set to type network as a best practice as a back to back VPC.

Also from a spanning tree instance the two core 7ks should be the root ports for all vlans which it clearly isn’t as you can tell from the designated and blocked ports.

This is a good guide that shows you the best practices.

https://www.cisco.com/c/dam/en/us/td/docs/switches/datacenter/sw/design/vpc_design/vpc_best_practices_design_guide.pdf#page62
0 Helpful

cristian26
Level 1
Level 1
In response to Rick1776

‎01-15-2018 05:09 AM

Hi, the pair of 7k ARE the root of the mayority of the vlans ( feature peer-switch enabled), the weird thing is that one member of the vpc was in blk state ( yes ONLY ONE member of the two links of the vpc) even thou they are root, in other cases, I mean in other vlans where the 7Ks aren't the root, the root port gets in blk state, thats another weir thing. The most bothersome and weird thing is that the MAC addresses stil gets forwarded and recieved by that blk port so, the traffic still get switched or "routed"over that link but being it int blk the data get droped so you have data failures depending on the algorithm hash of the loadbalance of the vpc, I mean if you ping from one IP to another it could get to the destination but the reply gets forwarded throu the blk link and gets droped.

 

Answering you question about brigde assurance. Cisco isn't clear about wether you should or shouldn't use bridge assurance in vpc. In this DOC they said it's NOT recommended to use it in VPC

https://www.cisco.com/c/dam/en/us/td/docs/switches/datacenter/sw/design/vpc_design/vpc_best_practices_design_guide.pdf

 

and checking the doc you send I founded this also:

7k-caso.JPG

 

We will open a case with TAC and if there's a solution I'll post it here.

0 Helpful

Rick1776
Level 5
Level 5
In response to cristian26

‎01-15-2018 10:15 AM

Can you bounce the interfaces and have them reconverge? That is strange. For the STP Network I meant from the VPC peer-link (I misread that from my mobile device much easier to see from my desktop.) Can you also send the interface configurations for both sides. Also is VLAN 52 used on other links that are not vpc enabled?



0 Helpful

cristian26
Level 1
Level 1
In response to Rick1776

‎01-15-2018 10:46 AM

Yes we kind of did that to solve it, a collegue "removed" and "added" the vlan of the trunk in order to force it to reconverge as a work arround. We thought that bringing down the entire interface would be too invasive for the company we work for.

The config of both sides is on the "Spoiler"section hidden. And no, that vlan or all the vlans are in vpc olny topology.

Thankyou for your intrest in helping.

Rick1776
Level 5
Level 5
In response to cristian26

‎01-16-2018 09:42 AM

Thanks for sharing. That is interesting that specific vlan was stuck in a blocked state. That might be a bug.
0 Helpful
Getting Started

Find answers to your questions by entering keywords or phrases in the Search bar above. New here? Use these resources to familiarize yourself with the community:

Customers Also Viewed These Support Documents