vlan suspend on nexux 3064

[malmsteen81]

Hi guys,

i have configured vpc with 2 nexus 3064 and worked fine for a couple of weeks. Yesterday i have powered off on siwtch to test network HA with VMware and after reboot the switch had  all interface vlan down.

i use a vlan with portchannel for keep alive

interface port-channel122
  description *** keepalive ***
  switchport mode trunk
  switchport trunk allowed vlan 23

n3k-ced-01# show int brief | i Vlan
Vlan1     --                                      down   VLAN/BD is down
Vlan23    --                                      down   suspended

the STP on first switch

LAN0023
  Spanning tree enabled protocol rstp
  Root ID    Priority    32791
             Address     0023.04ee.be01
             Cost        1
             Port        4217 (port-channel122)
             Hello Time  2  sec  Max Age 20 sec  Forward Delay 15 sec

  Bridge ID  Priority    32791  (priority 32768 sys-id-ext 23)
             Address     fc5b.39fb.9141
             Hello Time  2  sec  Max Age 20 sec  Forward Delay 15 sec

Interface        Role Sts Cost      Prio.Nbr Type
---------------- ---- --- --------- -------- --------------------------------
Po122            Root FWD 1         128.4217 P2p

and the vpc configuration is 

vpc domain 1
  peer-switch
  peer-keepalive destination 192.168.23.2 source 192.168.23.1 vrf keepalive
  delay restore 360
  peer-gateway
  auto-recovery
  ip arp synchronize

so, why the SVI are down? i have tried  to shutdown and then active manually but they dont' work. i also deleted vlan23 and interface and created again but not luck.

thank you

Andrea

[Muhammad Awais Khan]

Hi,

it sounds VPC configuration inconsistency to me. You can check the inconsistency by following:

show vpc consistency-parameters

it can be either mismatch on the vlans or VLAN SVI's. Make sure you have the same VLANs on both sides,

Some example output mentioned in below link might be useful

https://www.cisco.com/c/m/en_us/techdoc/dc/reference/cli/nxos/commands/vpc/show-vpc-consistency-parameters.html

[malmsteen81]

hi, i have checked the vlan trunk on all port-channel and i found a mismatch. i have correct the problem and restart the first switch.

Now the vlan1 is up, but vlan23 (use for keep-alive is down). 

vpc brief first nexus 

n3k-ced-01(config)# sh vpc brief
Legend:
(*) - local vPC is down, forwarding via vPC peer-link

vPC domain id : 1
Peer status : peer adjacency formed ok
vPC keep-alive status : peer is not reachable through peer-keepalive
Configuration consistency status : success
Per-vlan consistency status : success
Type-2 consistency status : success
vPC role : secondary
Number of vPCs configured : 2
Peer Gateway : Enabled
Dual-active excluded VLANs : -
Graceful Consistency Check : Enabled
Auto-recovery status : Enabled, timer is off.(timeout = 240s)
Delay-restore status : Timer is off.(timeout = 360s)
Delay-restore SVI status : Timer is off.(timeout = 10s)
Operational Layer3 Peer-router : Disabled

vPC Peer-link status
---------------------------------------------------------------------
id Port Status Active vlans
-- ---- ------ -------------------------------------------------
1 Po123 up 1,10,15-16,20-21,25-27,30,50-51,151-152,200

vPC status
----------------------------------------------------------------------------
Id Port Status Consistency Reason Active vlans
-- ------------ ------ ----------- ------ ---------------
14 Po14 up success success 1,10,15-16,20-
21,25-26,30,50
-51,151-152,20
0
23 Po23 down* Not Consistency Check Not -
Applicable Performed

second nexus 

n3k-ced-02# sh vpc brief
Legend:
(*) - local vPC is down, forwarding via vPC peer-link

vPC domain id : 1
Peer status : peer adjacency formed ok
vPC keep-alive status : peer is not reachable through peer-keepalive
Configuration consistency status : success
Per-vlan consistency status : success
Type-2 consistency status : success
vPC role : primary
Number of vPCs configured : 2
Peer Gateway : Enabled
Dual-active excluded VLANs : -
Graceful Consistency Check : Enabled
Auto-recovery status : Enabled, timer is off.(timeout = 240s)
Delay-restore status : Timer is off.(timeout = 360s)
Delay-restore SVI status : Timer is off.(timeout = 10s)
Operational Layer3 Peer-router : Disabled

vPC Peer-link status
---------------------------------------------------------------------
id Port Status Active vlans
-- ---- ------ -------------------------------------------------
1 Po123 up 1,10,15-16,20-21,25-27,30,50-51,151-152,200

vPC status
----------------------------------------------------------------------------
Id Port Status Consistency Reason Active vlans
-- ------------ ------ ----------- ------ ---------------
14 Po14 up success success 1,10,15-16,20-
21,25-26,30,50
-51,151-152,20
0
23 Po23 down* success success -

vpc consistency-parameters 

n3k-ced-01(config)# show vpc consistency-parameters global

Legend:
Type 1 : vPC will be suspended in case of mismatch

Name Type Local Value Peer Value
------------- ---- ---------------------- -----------------------
STP MST Simulate PVST 1 Enabled Enabled
STP Port Type, Edge 1 Normal, Disabled, Normal, Disabled,
BPDUFilter, Edge BPDUGuard Disabled Disabled
STP MST Region Name 1 "" ""
STP Disabled 1 None None
STP Mode 1 Rapid-PVST Rapid-PVST
STP Bridge Assurance 1 Enabled Enabled
STP Loopguard 1 Disabled Disabled
STP MST Region Instance to 1
VLAN Mapping
STP MST Region Revision 1 0 0
Interface-vlan admin up 2 1 1,23
Interface-vlan routing 2 1 1,23
capability
QoS (Cos) 2 ([0-7], [], [], [], ([0-7], [], [], [],
[], []) [], [])
Network QoS (MTU) 2 (9216, 9216, 9216, (9216, 9216, 9216,
9216, 9216, 9216) 9216, 9216, 9216)
Network Qos (Pause: 2 (F, F, F, F, F, F) (F, F, F, F, F, F)
T->Enabled, F->Disabled)
Input Queuing (Bandwidth) 2 (100, 0, 0, 0, 0, 0) (100, 0, 0, 0, 0, 0)
Input Queuing (Absolute 2 (F, F, F, F, F, F) (F, F, F, F, F, F)
Priority: T->Enabled,
F->Disabled)
Output Queuing (Bandwidth 2 (100, 0, 0, 0, 0, 0) (100, 0, 0, 0, 0, 0)
Remaining)
Output Queuing (Absolute 2 (F, F, F, F, F, F) (F, F, F, F, F, F)
Priority: T->Enabled,
F->Disabled)
Allowed VLANs - 1,10,15-16,20-21,25-27 1,10,15-16,20-21,25-27
,30,50-51,151-152,200 ,30,50-51,151-152,200
Local suspended VLANs - - -

Legend:
Type 1 : vPC will be suspended in case of mismatch

Name Type Local Value Peer Value
------------- ---- ---------------------- -----------------------
STP MST Simulate PVST 1 Enabled Enabled
STP Port Type, Edge 1 Normal, Disabled, Normal, Disabled,
BPDUFilter, Edge BPDUGuard Disabled Disabled
STP MST Region Name 1 "" ""
STP Disabled 1 None None
STP Mode 1 Rapid-PVST Rapid-PVST
STP Bridge Assurance 1 Enabled Enabled
STP Loopguard 1 Disabled Disabled
STP MST Region Instance to 1
VLAN Mapping
STP MST Region Revision 1 0 0
Interface-vlan admin up 2 1 1
Interface-vlan routing 2 1 1
capability
QoS (Cos) 2 ([0-7], [], [], [], ([0-7], [], [], [],
[], []) [], [])
Network QoS (MTU) 2 (9216, 9216, 9216, (9216, 9216, 9216,
9216, 9216, 9216) 9216, 9216, 9216)
Network Qos (Pause: 2 (F, F, F, F, F, F) (F, F, F, F, F, F)
T->Enabled, F->Disabled)
Input Queuing (Bandwidth) 2 (100, 0, 0, 0, 0, 0) (100, 0, 0, 0, 0, 0)
Input Queuing (Absolute 2 (F, F, F, F, F, F) (F, F, F, F, F, F)
Priority: T->Enabled,
F->Disabled)
Output Queuing (Bandwidth 2 (100, 0, 0, 0, 0, 0) (100, 0, 0, 0, 0, 0)
Remaining)
Output Queuing (Absolute 2 (F, F, F, F, F, F) (F, F, F, F, F, F)
Priority: T->Enabled,
F->Disabled)
Allowed VLANs - 1,10,15-16,20-21,25-27 1,10,15-16,20-21,25-27
,30,50-51,151-152,200 ,30,50-51,151-152,200
Local suspended VLANs - - -

thanks

Andrea

[Mark Malone]

hi
can you ping between the keep alive interfaces of vlan 23, once you can ping between them should come back up and be up int vpc brief

[malmsteen81]

already tried but

n3k-ced-02# ping 192.168.23.1 vrf keepalive
PING 192.168.23.1 (192.168.23.1): 56 data bytes
36 bytes from 192.168.23.2: Destination Host Unreachable
Request 0 timed out
36 bytes from 192.168.23.2: Destination Host Unreachable
Request 1 timed out
36 bytes from 192.168.23.2: Destination Host Unreachable
Request 2 timed out
36 bytes from 192.168.23.2: Destination Host Unreachable
Request 3 timed out
36 bytes from 192.168.23.2: Destination Host Unreachable
Request 4 timed out

[malmsteen81]

n3k-ced-01(config)# sh spanning-tree vlan 23

VLAN0023
  Spanning tree enabled protocol rstp
  Root ID    Priority    32791
             Address     0023.04ee.be01
             This bridge is the root
             Hello Time  2  sec  Max Age 20 sec  Forward Delay 15 sec

  Bridge ID  Priority    32791  (priority 32768 sys-id-ext 23)
             Address     0023.04ee.be01
             Hello Time  2  sec  Max Age 20 sec  Forward Delay 15 sec

Interface        Role Sts Cost      Prio.Nbr Type
---------------- ---- --- --------- -------- --------------------------------
Po122            Back BLK 1         128.4217 P2p

n3k-ced-02# sh spanning-tree vlan 23

VLAN0023
  Spanning tree enabled protocol rstp
  Root ID    Priority    32791
             Address     0023.04ee.be01
             This bridge is the root
             Hello Time  2  sec  Max Age 20 sec  Forward Delay 15 sec

  Bridge ID  Priority    32791  (priority 32768 sys-id-ext 23)
             Address     0023.04ee.be01
             Hello Time  2  sec  Max Age 20 sec  Forward Delay 15 sec

Interface        Role Sts Cost      Prio.Nbr Type
---------------- ---- --- --------- -------- --------------------------------
Po122            Desg FWD 1         128.4217 P2p

on firs switch STP has blocked vlan23, but why? the vlan is tagged only on port-channel keep alive

23   keepalive                        active    Po122, Eth1/47, Eth1/48

[Mark Malone]

Hi
The keepalive is a direct link yes ? just a vlan SVI with direct layer 2 port assigned to 23 connected between switches , interfaces are up yes


are you also sending that vlan over the peer link too ?
thats maybe why its blocked , looping somewhere between both switches , should only be seen on keepalive link

[malmsteen81]

hi, the two nexus are connected directly with two cables on port 1/47-48 by port channel (122).

interface port-channel122
  description *** keepalive ***
  switchport mode trunk
  switchport trunk allowed vlan 23

interface Vlan23
  no shutdown
  vrf member keepalive
  ip address 192.168.23.1/24

then peer-link has not access on vlan 23

interface port-channel123
  description *** VPC PEER LINKS ***
  switchport mode trunk
  switchport trunk allowed vlan 1,10,15-16,20-21,25-27,30,50-51,151-152,200
  spanning-tree port type network
  vpc peer-link

i can try to reboot the second nexus

[paul driver]

Hello

---

@malmsteen81 wrote:

Hi guys,

i have configured vpc with 2 nexus 3064 and worked fine for a couple of weeks. Yesterday i have powered off on siwtch to test network HA with VMware and after reboot the switch had  all interface vlan down.

i use a vlan with portchannel for keep alive

interface port-channel122
  description *** keepalive ***
  switchport mode trunk
  switchport trunk allowed vlan 23

n3k-ced-01# show int brief | i Vlan
Vlan1     --                                      down   VLAN/BD is down
Vlan23    --                                      down   suspended

the STP on first switch

LAN0023
  Spanning tree enabled protocol rstp
  Root ID    Priority    32791
             Address     0023.04ee.be01
             Cost        1
             Port        4217 (port-channel122)
             Hello Time  2  sec  Max Age 20 sec  Forward Delay 15 sec

  Bridge ID  Priority    32791  (priority 32768 sys-id-ext 23)
             Address     fc5b.39fb.9141
             Hello Time  2  sec  Max Age 20 sec  Forward Delay 15 sec

Interface        Role Sts Cost      Prio.Nbr Type
---------------- ---- --- --------- -------- --------------------------------
Po122            Root FWD 1         128.4217 P2p

and the vpc configuration is 

vpc domain 1
  peer-switch
  peer-keepalive destination 192.168.23.2 source 192.168.23.1 vrf keepalive
  delay restore 360
  peer-gateway
  auto-recovery
  ip arp synchronize

so, why the SVI are down? i have tried  to shutdown and then active manually but they dont' work. i also deleted vlan23 and interface and created again but not luck.

---

Have you checked that the L2 vlans for the L3 interfaces are actually in the vtp database for these switchs and for some reason it hasnt been over wrote after the reload losing the l2 vlans would icurr the loss of connectivity you are reporting?

sh vlans
sh vtp status

[malmsteen81]

Hi, i don't use vtp

[paul driver]

---

@malmsteen81 wrote:

Hi, i don't use vtp

---

Vtp is used to synchronize with the other switches if you dont use it then fine but do the switches have all the correct vlans in their vtp database, if not you need to re add them.

Another thing to try is to shutdown/restart the vpc domain to refesh the vpc.

[Mark Malone]

Ok peer looks fine ,before rebooting did you try bounce the interfaces , also Paul makes a good point confim your layer 2 database is ok for all vlans involved and something didnt happen on reboot of switch 1

[malmsteen81]

hi, i have tried to add vlan23 on port-channel 123 (peer link) and vlan is waked up. But is correct allow vpc peer to keepalive traffic??

[paul driver]

hello

---

@malmsteen81 wrote:

hi, i have tried to add vlan23 on port-channel 123 (peer link) and vlan is waked up. But is correct allow vpc peer to keepalive traffic??

---

not in the pc but in the L2 vlan database -

show vlan brief

[Mark Malone]

shouldn't be on the peer link no

keepalive doesent need to be a trunk either change it access as its only 1 vlan , on both sides

interface Vlan3003
description VPC Heartbeat
no shutdown
vrf member heartbeat
ip address x.x.x.x/30



interface Ethernet1/44
description VPC Heartbeat
switchport
switchport access vlan 3003
logging event port link-status
no shutdown