Re: VPC Suspended for from Nexus 7k to 6509

dan hale · ‎12-30-2018

Hi All, odd issue with setting up a VPC between two Nexus 7k and two 6509's.

Hardware
cisco Nexus7000 C7004
version 6.2(10)

I have two Nexus 7k's that I setup a VDC for our DMZ vlan. Goal is to span the DMZ vlan from the 6500's to the new DMZ VDC I created. The VPC peer link between the two nexus 7ks comes up good. I created a new Port channel on each nexus 7k for the DMZ link to the 6509's. On the 6509 side I connect to them via a trunk link.

However, when I turned on the interfaces I get a suspended link when I do a show ether channel summary. I've attached a picture of the topology and below are the interfaces configs from one side of the Nexus 7k and one side of the 6509. Both nexus 7k and 6509 interfaces are identical.

When I looked at the show log it did not say why they were suspended. Any debug I can run or issues with this setup? Why else would they be suspended?

Thanks,

Dan

Nexus 7k

interface port-channel1
description vPC Peerlink to VDC DMZ
switchport
switchport mode trunk
spanning-tree port type network
vpc peer-link

interface port-channel2
description vPC to DMZ to 6509
switchport
switchport mode trunk
switchport trunk allowed vlan 10
spanning-tree port type network
vpc 2

interface Ethernet1/25
description vPC Peerlink to VDC DMZ1 interface 1/25
switchport mode trunk
channel-group 1 mode active
no shutdown

interface Ethernet1/27
description vPC DMZ to 6509 to 1/11
switchport mode trunk
switchport trunk allowed vlan 10
channel-group 2 mode active
no shutdown

6509 Interfaces

interface GigabitEthernet1/11
description Uplink to DMZ Nexus 1/27
switchport
switchport trunk encapsulation dot1q
switchport trunk allowed vlan 10
switchport mode trunk

Francesco Molino · ‎12-30-2018

Hi

Your 2 Cat6k aren't connected to together. If there are not in VSS mode, You can't have a VPC from Nexus7k facing both 6k.
Your design would be good only if your 6k are in vss mode. Here on N7K, both 6k are sending different bridge ID and N7K knows then it's connected to 2 different switches, that's why you got a suspended state.

Thanks
Francesco
PS: Please don't forget to rate and select as validated answer if this answered your question

dan hale · ‎01-07-2019

Thanks Francesco, how about this setup see below:

I need to find out if there is a Trunk link from each 6500A to 6500B but, would this design work?

Thanks,

Dan

Hanfeng Cai · ‎01-07-2019

That's fine!

Micheline Murphy · ‎01-07-2019

Hello Dan--this second topology should work. You will need to amend your configuration, however. From the Cat 6500s point of view, they each have two ports that are in a single port channel. (Since you didn't label the ports on your diagram, I'm going to say ports e1/11 and e1/12.). Both of those ports are going to have to be configured in trunk as well as bound together in a single port-channel with channel-gro x mode active. Neither Cat will know that their e1/11 and e1/12 are connected to different switches.

On the vPC peer switches, you will have a port-channel with a single member. As with the Cat config, you will need to make sure that the port is configured for trunking.

For all VLANs that are trunked over vpc 3 and vpc4 must also be trunked over the peer-link. You can check this with the command show vpc consistency-parameters global. At the bottom, you will see "allowed VLANs" and "local suspended VLANs." For each vPC peer, the allowed VLANs should match, and there should be no suspended VLANs.

pod1-leaf1(config)# sh vpc consistency-parameters global

Legend:

Type 1 : vPC will be suspended in case of mismatch

Name Type Local Value Peer Value

------------- ---- ---------------------- -----------------------

QoS 2 ([], [], [], [], [], ([], [], [], [], [],

[]) [])

Network QoS (MTU) 2 (1538, 0, 0, 0, 0, 0) (1538, 0, 0, 0, 0, 0)

Network Qos (Pause) 2 (F, F, F, F, F, F) (F, F, F, F, F, F)

Input Queuing (Bandwidth) 2 (100, 0, 0, 0, 0, 0) (100, 0, 0, 0, 0, 0)

Input Queuing (Absolute 2 (F, F, F, F, F, F) (F, F, F, F, F, F)

Priority)

Output Queuing (Bandwidth) 2 (100, 0, 0, 0, 0, 0) (100, 0, 0, 0, 0, 0)

Output Queuing (Absolute 2 (F, F, F, F, F, F) (F, F, F, F, F, F)

Priority)

STP Mode 1 Rapid-PVST Rapid-PVST

STP Disabled 1 None None

STP MST Region Name 1 "" ""

STP MST Region Revision 1 0 0

STP MST Region Instance to 1

VLAN Mapping

STP Loopguard 1 Disabled Disabled

STP Bridge Assurance 1 Enabled Enabled

STP Port Type, Edge 1 Normal, Disabled, Normal, Disabled,

BPDUFilter, Edge BPDUGuard Disabled Disabled

STP MST Simulate PVST 1 Enabled Enabled

IGMP Snooping Group-Limit 2 4000 4000

Allowed VLANs - 101-102 101-102

Local suspended VLANs - - -

If for some reason, as VLAN is not trunked across the peer-link, the vPC will suspend that VLAN. When you are troubleshooting a vPC, keep in mind that vPC has a 4-minute timer until it brings up member ports after a peer-link has been bounced. So be patient.

I hope this helps. If it did, please consider marking this response "helpful." MM

Hanfeng Cai · ‎01-02-2019

Your design is incorrect, you should bundle two 65 as a vss. Since your current two 65 are individual, they can't bundle PC together.

Micheline Murphy · ‎01-07-2019

Hello Dan--on this topology, the reason you're getting suspended ports on the vPC peers is because you have named both member ports going to the Cat 6500s the same port-channel designation. Remember that the vPC peers will talk to each other back and forth about their ports, and if you name a port-channel the same thing on both peers, both switches will assume that the port goes to the same downstream device. The way you solve this problem is to (1) name the port-channel by the member device, or (2) specify vpc designation in the command vpc. (By default, NX-OS uses the port-channel designation, but you can configure a different designation. This just makes things overly complicated, however.).

Your config should look something like this:

pod1-leaf1(config)# sr vpc

!

version 7.3(1)N1(1)

feature vpc

vpc domain 100

peer-keepalive destination 10.0.0.1 source 10.0.0.2

delay restore 150

interface port-channel100

description **vpc 100 peer-link**

switchport trunk allowed vlan 100-199

spanning-tree port type network

speed 10000

vpc peer-link

interface port-channel101

description **to Device #1**

switchport mode trunk

switchport trunk allowed vlan 100-199

speed 10000

vpc 101

interface port-channel102

description **to Device #2**

switchport mode trunk

switchport trunk allowed vlan 100-199

speed 10000

vpc 102

Hope this helps. Please consider marking this response "helpful" if it was. MM

dan hale · ‎01-08-2019

Thank you!

Question my VDC is limited to 4 physical interfaces...I've read about using the Mgmt0 interface as the Keep_Alive. Since this is a VDC I was not sure since the MGMT0 is on the default VDC...can I still use that MGMT0?

This is what I was thinking?

vrf context management
vpc domain 2
peer-keepalive destination 1.1.1.2 source 1.1.1.1

interface mgmt0
vrf member management
ip address 1.1.1.1/30

Do I still need a physical interface or will it use the Default VDC MGMT0?

Thanks,

Dan

Micheline Murphy · ‎01-09-2019

Hello Dan--The vPC feature is agnostic about whether the switches are two physical switches or VDCs. You can even configure a VPC to run between two VDCs with the same parent switch.

By default, vPC will use the management vrf for the peer-keepalive. So as long as you can ping one management IP from the other management IP you're golden. Don't forget to use the keyword vrf management when you ping.

Does that answer your question? MM

mojafri · ‎01-10-2019

Hi @dan hale,

Below is the recommendation: https://www.cisco.com/c/dam/en/us/td/docs/switches/datacenter/sw/design/vpc_design/vpc_best_practices_design_guide.pdf

You can form peer-keep alive using mgmt for physical switch as well as VDCs considering all VDCs should have same subnet.

Eg:

! Out-of-band interface configuration in the default VDC
N7K# show running-config interface mgmt0
[output suppressed]
interface mgmt0
ip address 172.19.19.1/13

N7K# switchto vdc VDC2
[output suppressed]
! Out-of-band interface configuration in VDC2
N7K-VDC2# show running-config interface mgmt0

[output suppressed]
interface mgmt0
ip address 172.19.19.2/13

Hope this helps.

Regards,

MJ