cancel
Showing results for 
Search instead for 
Did you mean: 
cancel
1484
Views
0
Helpful
6
Replies

Spanning Tree Fails on Vlan when VPC is configured

I'm new to configuring vPC, but ran into a problem with a set of Nexus 5548s.  When I add the vpc peer-link command to my port channel, SPT becomes unavailable for that interface.  Note, I have the exact same configuration running on another set of 5548 (not connected to this set) with no issue.  When SPT becomes unavailable, I'm no longer capable of reaching the mgmt interface of the peer Nexus.  When the command is removed, I can ping across.

nx1

cfs eth distribute

feature lacp

feature vpc

feature lldp

feature ftp

spanning-tree vlan 1-3967 priority 4096

vrf context management

  ip route 0.0.0.0/0 10.244.137.225

vpc domain 100

  peer-keepalive destination 10.244.137.228

interface port-channel1

  switchport mode trunk

  spanning-tree port type network

  speed 1000

  vpc peer-link

interface Ethernet1/1

  description asa-1-pri gi0/0

  switchport access vlan 50

  speed 1000

interface Ethernet1/2

  description asa-1-pri gi0/1

  speed 1000

interface Ethernet1/3

  description asa-1-pri gi0/2

  switchport access vlan 10

  speed 1000

interface Ethernet1/4

  description asa-1-pri gi0/3

  switchport access vlan 99

  speed 1000

interface Ethernet1/5

  description asa-1-pri mgmt

  speed 1000

interface Ethernet1/6

  description ch2-lb-1 eth0/1

  shutdown

  switchport access vlan 10

  speed 1000

interface Ethernet1/7

  description ch2-lb-1 eth0/2

  shutdown

  switchport access vlan 11

  speed 1000

interface Ethernet1/8

  description ch2-lb-1 mgmt

  switchport access vlan 99

  speed 1000

interface Ethernet1/9

  description Internet

  switchport access vlan 50

  speed 1000

interface Ethernet1/10

  description Temp-Switch

  switchport access vlan 50

  speed 1000

interface Ethernet1/11

  description nx2 mgmt port

  switchport access vlan 99

  spanning-tree port type edge

  speed 1000

interface Ethernet1/23

  description temp-pc-access

  switchport access vlan 99

  speed 1000

interface Ethernet1/24

  description ch2-ucs01 mgmt

  switchport access vlan 99

  speed 1000

interface Ethernet1/25

  description ch2-netapp-1 E0E

  speed 1000

interface Ethernet1/26

  description ch2-netapp-1 mgmt

  switchport access vlan 99

  speed 1000

interface Ethernet1/27

  description ch2-netapp-1 E0C

interface Ethernet1/28

  description ch2-netapp-2 E0C

interface Ethernet1/29

  description ch2-ucs-1 Po29

interface Ethernet1/30

  description ch2-ucs-2 Po30

interface Ethernet1/31

  description ch2-nx-2 eth1/31

  switchport mode trunk

  speed 1000

  channel-group 1 mode active

interface Ethernet1/32

  description ch2-nx-2 eth1/32

  switchport mode trunk

  speed 1000

  channel-group 1 mode active

interface mgmt0

  description mgmt int to nx-2 eth 1/11

  vrf member management

  ip address 10.244.137.227/27

nx2

cfs eth distribute

feature lacp

feature vpc

feature lldp

feature vtp

spanning-tree vlan 1-100 priority 28672

vrf context management

  ip route 0.0.0.0/0 10.244.137.225

vpc domain 100

  peer-keepalive destination 10.244.137.227

interface port-channel1

  switchport mode trunk

  spanning-tree port type network

  speed 1000

  vpc peer-link

interface Ethernet1/1

  description asa-1-sec gi 0/0

  switchport access vlan 50

  speed 1000

interface Ethernet1/2

  description asa-1-sec gi 0/1

  speed 1000

interface Ethernet1/3

  description asa-1-sec gi 0/2

  switchport access vlan 10

  speed 1000

interface Ethernet1/4

  description asa-1-sec gi 0/3

  switchport access vlan 99

  speed 1000

interface Ethernet1/5

  description asa-1-sec mgmt

  shutdown

  switchport access vlan 99

  speed 1000

interface Ethernet1/6

  description ch2-lb-2 eth0/1

  shutdown

  switchport access vlan 10

  speed 1000

interface Ethernet1/7

  description ch2-lb-2 eth0/2

  shutdown

  switchport access vlan 11

  speed 1000

interface Ethernet1/8

  description ch2-lb-2 mgmt

  switchport access vlan 99

  speed 1000

interface Ethernet1/11

  description dc5-nx-1 mgmt 0

  switchport access vlan 99

  spanning-tree port type edge

  speed 1000

interface Ethernet1/23

  description temp-pc-link

  switchport access vlan 99

  speed 1000

interface Ethernet1/24

  description ch2-ucs-2 mgmt

  switchport access vlan 99

  speed 1000

interface Ethernet1/25

  description ch2-netapp-2 E0E

  speed 1000

interface Ethernet1/26

  description ch2-netapp-2 mgmt

  switchport access vlan 99

  speed 1000

interface Ethernet1/27

  description ch2-netapp-1 E0D

interface Ethernet1/28

  description ch2-netapp-2 E0D

interface Ethernet1/29

  description ch2-ucs-2 Po29

interface Ethernet1/30

  description ch2-ucs-1 Po30

interface Ethernet1/31

  description ch2-nx-1 eth 1/31

  switchport mode trunk

  speed 1000

  channel-group 1 mode active

interface Ethernet1/32

  description ch2-nx-2 eth 1/32

  switchport mode trunk

  speed 1000

  channel-group 1 mode active

interface mgmt0

  description mgmt int to nx-1 eth 1/11

  vrf member management

  ip address 10.244.137.228/27

nx-1# sho spanning-tree int port-channel 1

No spanning tree information available for port-channel1

nx-1# sho vpc brief

Legend:

                (*) - local vPC is down, forwarding via vPC peer-link

vPC domain id                     : 100 

Peer status                       : peer link is down             

vPC keep-alive status             : peer is not reachable through peer-keepalive

Configuration consistency status  : failed  

Per-vlan consistency status       : success                       

Configuration inconsistency reason: Consistency Check Not Performed

Type-2 consistency status         : failed  

Type-2 inconsistency reason       : QoSMgr type-1 configuration incompatible

vPC role                          : none established              

Number of vPCs configured         : 0   

Peer Gateway                      : Disabled

Dual-active excluded VLANs        : -

Graceful Consistency Check        : Disabled (due to peer configuration)

Auto-recovery status              : Disabled

vPC Peer-link status

---------------------------------------------------------------------

id   Port   Status Active vlans    

--   ----   ------ --------------------------------------------------

1    Po1    up     - 

Once again, I have this same configuration running on another set with no issue.  

Any suggestions would be appreciated.

1 Accepted Solution

Accepted Solutions

Cisco recommendation is below

We recommend that you configure the vPC peer-keepalive link on the Cisco Nexus 5000 Series switch to run in the management VRF using the mgmt 0 interfaces. If you configure the default VRF, ensure that the vPC peer link is not used to carry the vPC peer-keepalive messages.

www.cisco.com/c/en/us/td/docs/switches/datacenter/nexus5000/sw/layer2/b_Cisco_Nexus_5000_Series_NX-OS_/Cisco_Nexus_5000_Series_NX-OS__chapter8.html

Why is it working in your existing setup could be for the following reason

1. When you build the vpc most likely the mgmt0 vlan was not on the peer link. Once the VPC is up then you can move the mgmt0 vlan to the vpc peer link and it will work since all we are doing is exchanging the keepalive messages on the peer link. The peer link does not go down when the peer keepalive is lost once vpc is established.

I would recommend you to change this design on your existing setup as you may see a issue if for any reason both 5k reboots at the same time say during a power outage.

-Raj

View solution in original post

6 Replies 6

Rajeshkumar Gatti
Cisco Employee
Cisco Employee

Couple of things-

1. I don't see the peer keep alive up between the peers

vPC keep-alive status             : peer is not reachable through peer-keepalive

2. Are you also doing the config on your peer by enabling "vpc peer link" on the other side as well? make sure you do that as well.

-Raj

1.  Correct, the keep-alive fails and I can't ping the other side.  If I remove the vpc peer link command from both switches, I'm able to ping across and spanning-tree restores for that plan.

2.  Yes, both switches have vpc peer-link configured on the port channel

Tx

How is your mgmt0 connected to each other between the VPC peers? Are they using a vlan which is a vpc vlan that gets suspended when you bring up the peer link?

If so can you have the mgmt0 connected to a non vpc vlan that stays up keeping the keepalive in play. Once the peer keep alive stays up then we can see if po1 will have the vlan forwarding.

-Raj

Raj,

My mgmt0 interface was on a Vlan that was being trunked via the port channel that is the peer link.  I removed that vlan from the trunk and setup a separate interface to trunk that Vlan.  This did bring up the peer-link for vpc between the switches, but what puzzles me is that I have the above configuration on another set of switches w/ no issues.

Tx

Cisco recommendation is below

We recommend that you configure the vPC peer-keepalive link on the Cisco Nexus 5000 Series switch to run in the management VRF using the mgmt 0 interfaces. If you configure the default VRF, ensure that the vPC peer link is not used to carry the vPC peer-keepalive messages.

www.cisco.com/c/en/us/td/docs/switches/datacenter/nexus5000/sw/layer2/b_Cisco_Nexus_5000_Series_NX-OS_/Cisco_Nexus_5000_Series_NX-OS__chapter8.html

Why is it working in your existing setup could be for the following reason

1. When you build the vpc most likely the mgmt0 vlan was not on the peer link. Once the VPC is up then you can move the mgmt0 vlan to the vpc peer link and it will work since all we are doing is exchanging the keepalive messages on the peer link. The peer link does not go down when the peer keepalive is lost once vpc is established.

I would recommend you to change this design on your existing setup as you may see a issue if for any reason both 5k reboots at the same time say during a power outage.

-Raj

Tx for your assistance!

Review Cisco Networking for a $25 gift card