Nexus VPC - Peer-switch command takes down keepalive

[Puts219]

Hello all,

I have configured VPC on two Nexus 3K switches and everything is good as far as the keepalive and vpc peer status until I introduce the peer-switch command on both units under vpc domain.

When that command is entered the the vlan interface with the peer IP keep alive goes to the down state on the secondary nexus, in order to fix that I need to add the keep alive vlan(51) as allowed on the VPC port channel, but I'm reading that the keep alive vlan should not be a part of the VPC port channel trunk.

Any suggestion or opinions on my config and what I could do better would be greatly appreciated.

 

Here is my relevant config and show commands:

Nexus-primary:
vpc domain 1
role priority 8192
peer-keepalive destination 10.51.51.2 source 10.51.51.1 vrf keepalive
delay restore 150
peer-gateway
ip arp synchronize

interface Vlan51
no shutdown
vrf member keepalive
no ip redirects
ip address 10.51.51.1/29
no ipv6 redirects

interface Ethernet1/37
description UPLINK-N9K2-ETH37
switchport
switchport access vlan 51
no shutdown

interface port-channel51
description VPC-PEER-LINKS
switchport
switchport mode trunk
switchport trunk allowed vlan 10,12-13,15-19,21,28,89,91,105,147,150,412-413
spanning-tree port type network
vpc peer-link

interface Ethernet1/39
description VPC-PEER-LINKS
switchport
switchport mode trunk
switchport trunk allowed vlan 10,12-13,15-19,21,28,89,91,105,147,150,412-413
channel-group 51 mode active
no shutdown

interface Ethernet1/40
description VPC-PEER-LINKS
switchport
switchport mode trunk
switchport trunk allowed vlan 10,12-13,15-19,21,28,89,91,105,147,150,412-413
channel-group 51 mode active
no shutdown

show vpc:

vPC domain id : 1
Peer status : peer adjacency formed ok
vPC keep-alive status : peer is alive
Configuration consistency status : success
Per-vlan consistency status : success
Type-2 consistency status : success
vPC role : primary
Number of vPCs configured : 3
Peer Gateway : Enabled
Dual-active excluded VLANs : -
Graceful Consistency Check : Enabled
Auto-recovery status : Disabled
Delay-restore status : Timer is off.(timeout = 150s)
Delay-restore SVI status : Timer is off.(timeout = 10s)
Operational Layer3 Peer-router : Disabled
Virtual-peerlink mode : Disabled

vPC Peer-link status
---------------------------------------------------------------------
id Port Status Active vlans
-- ---- ------ -------------------------------------------------
1 Po51 up 10,12-13,15-19,21,28,89,91,105,147,412-413

vPC status
----------------------------------------------------------------------------
Id Port Status Consistency Reason Active vlans
-- ------------ ------ ----------- ------ ---------------
20 Po20 up success success 10,12-13,15-19,21,
28,89,91,412-413
30 Po30 up success success 10,12-13,91,412

31 Po31 up success success 10,12

Nexus-secondary:
vpc domain 1
peer-keepalive destination 10.51.51.1 source 10.51.51.2 vrf keepalive
delay restore 150
peer-gateway
ip arp synchronize

interface Vlan51
no shutdown
vrf member keepalive
no ip redirects
ip address 10.51.51.2/29
no ipv6 redirects

interface Ethernet1/37
description UPLINK-N9K1-ETH37
switchport
switchport access vlan 51
no shutdown

interface port-channel51
description VPC-PEER-LINKS
switchport
switchport mode trunk
switchport trunk allowed vlan 10,12-13,15-19,21,28,89,91,105,147,150,412-413
spanning-tree port type network
vpc peer-link

interface Ethernet1/39
description VPC-PEER-LINKS
switchport
switchport mode trunk
switchport trunk allowed vlan 10,12-13,15-19,21,28,89,91,105,147,150,412-413
channel-group 51 mode active
no shutdown

interface Ethernet1/40
description VPC-PEER-LINKS
switchport
switchport mode trunk
switchport trunk allowed vlan 10,12-13,15-19,21,28,89,91,105,147,150,412-413
channel-group 51 mode active
no shutdown

show vpc

vPC domain id : 1
Peer status : peer adjacency formed ok
vPC keep-alive status : peer is alive
Configuration consistency status : success
Per-vlan consistency status : success
Type-2 consistency status : success
vPC role : secondary
Number of vPCs configured : 3
Peer Gateway : Enabled
Dual-active excluded VLANs : -
Graceful Consistency Check : Enabled
Auto-recovery status : Disabled
Delay-restore status : Timer is off.(timeout = 150s)
Delay-restore SVI status : Timer is off.(timeout = 10s)
Operational Layer3 Peer-router : Disabled
Virtual-peerlink mode : Disabled

vPC Peer-link status
---------------------------------------------------------------------
id Port Status Active vlans
-- ---- ------ -------------------------------------------------
1 Po51 up 10,12-13,15-19,21,28,89,91,105,147,412-413

vPC status
----------------------------------------------------------------------------
Id Port Status Consistency Reason Active vlans
-- ------------ ------ ----------- ------ ---------------
20 Po20 up success success 10,12-13,15-19,21,
28,89,91,412-413
30 Po30 up success success 10,12-13,91,412

31 Po31 up success success 10,12

With the config above, when peer-switch command is entered the keep alive status says: 'peer is not reachable through peer-keepalive'

[balaji.bandi]

how is your spanning-tree config? is this identical

peer-switch :

This feature allows a pair of Cisco Nexus switches to appear as a single spanning-tree root in the Layer 2 topology.
It eliminates the need to pin the spanning-tree root to the vPC primary switch and improves vPC convergence if the vPC primary switch fails.

[Puts219]

That is correct, I made sure for that to match. I have the following command entered on both units:
spanning-tree vlan 1-3967 priority 8192

[balaji.bandi]

Do you mean role priority or spanning-tree priority 

Another observation you do not have role priority on other switches? is this missing or config typo?

can you post-show spa config both the switch ?

[Puts219]

spanning-tree vlan 1-3967 priority 8192

The command above is the only spanning-tree command on both nexus switches, the other switches are left with default spanning tree priority values making the nexus switches root bridges for all VLANs. At the moment, nexus-primary switch is showing at the root for all VLANs.

I do have role priority set up for nexus-primary switch to a value that makes it the primary for the VPC domain.

[balaji.bandi]

I was in agreement that its something to do with spanning tree here as the suggested routed interface is good, as i mentioned peer-switch is only optional when you required an identical spanning tree.

glad you were able to resolve the issue. appreciate your input.

[Christopher Hart · Accepted solution]

Hello!

I would want to analyze the logs (especially Spanning Tree Protocol finite state machine/event history logs) to be confident in this diagnosis, but if I had to guess, the spanning tree in VLAN 51 carrying your vPC Peer-Keepalive connectivity reconverged after peer-switch was configured under the vPC domain. As a result of this reconvergence, Ethernet1/37 on the vPC Secondary went through a Blocking, Learning, and Forwarding cycle. Since VLAN 51 is presumably only carried on Ethernet1/37, the SVI for VLAN 51 would then transition into an up/down status.

Typically, we see the vPC Peer-Keepalive link between two vPC peers configured as a routed interface (either a dedicated link between the two like you have, or using the mgmt0 interfaces of each switch). In other words, I would expect Ethernet1/37 to be a routed link (via no switchport) and to have the IP address currently assigned to the VLAN 51 SVI assigned to Ethernet1/37 instead. It's fairly uncommon to see customers use an SVI over a dedicated link for the vPC Peer-Keepalive link because that requires Spanning Tree Protocol to run over the VLAN, which introduces unnecessary control plane overhead (and can cause issues just like the one you observed!)

For that reason, I would recommend converting Ethernet1/37 to a routed interface and running your vPC Peer-Keepalive across it instead of utilizing an SVI for the same purpose. If you implement the vPC Peer-Keepalive this way, I do not think you would see this issue again while implementing peer-switch.

I hope this helps - thank you!

-Christopher

[MHM Cisco World · Accepted solution]

Yes, the Keep-alive must be L3, and for issue please check the below link
https://netcraftsmen.com/troubleshooting-svis-cisco-nexus-switches-vpc-environment/

[Puts219]

Thank you all for the helpful comments. The issue was spanning tree and both suggestions worked, not using an SVI and the fix from the article that @MHM Cisco World posted also worked. I will go with @Christopher Hart  suggestion and not use an SVI for this.