NX-OS VPC topology

[networkinggeek]

I am new to nexus and trying to learn some practical applications of NX-OS. I have this lab built, I added two nexus switches so I can learn the concept of VPC. Now I need to connect my PC1 to PC2 so that they can ping each other going through vpc. If I create vpc on both sides it becomes layer 2 connection, how to complete this diagram (in blue circle) using vpc and get connectivity across between PCs?. Do I need to enable L3/igp protocols on nexus as well?

[Francesco Molino · Accepted solution]

Hi

You can create a vpc to your switch from Nexus side and a simple port-channel from access switch side.

Now, in terms of routing, all depends on your topology is setup.
Are your Nexus just acting as layer2 and your router in red box acting as layer3?
If so, there's no need to enable any routing protocol.
If your Nexus has svi and need to advertise them to your router, which i believe is running a routing protocol, then you will need to setup something like igp or bgp.

[balaji.bandi]

Adding to other post, you need to always add features before configuring IGP like example ospf.

you need to add feature ospf and then start configuring ospf.

[networkinggeek]

so I am able to bring my VPC up by removing peerlink port-channel1 and re-configuring it, for some reason my ports were in blocking state. right now my peer-link is up and so is keepalive

but now my VPC member link port-channel 10 does not come up, one link says sometimes not connected and not receiving lacp pdu.


NX-OSv9k-1(config-if)# sh int ethernet 1/1

Ethernet1/1 is down (suspended(no LACP PDUs))

admin state is up, Dedicated Interface

Belongs to Po10

Switch#show etherchannel summary
Flags: D - down P - bundled in port-channel
I - stand-alone s - suspended
H - Hot-standby (LACP only)
R - Layer3 S - Layer2
U - in use N - not in use, no aggregation
f - failed to allocate aggregator

M - not in use, minimum links not met
m - not in use, port not aggregated due to minimum links not met
u - unsuitable for bundling
w - waiting to be aggregated
d - default port

A - formed by Auto LAG

Number of channel-groups in use: 1
Number of aggregators: 1

Group Port-channel Protocol Ports
------+-------------+-----------+-----------------------------------------------
10 Po10(SN) LACP Gi1/0(w) Gi1/1(w)

[balaji.bandi]

since it vitual you see some time this issue, can you shutdown the port-channel and bring up, if you have still issue. turn off the node and turn on and test it.

please post the configuration to verify.

you can use below config i have tesed some time back using virtual nexus9K

http://www.balajibandi.com/?s=vpc

[Francesco Molino]

What is this port-channel 10 and where It's connected to?
Is it binded to a vpc? Never used N9k on gns3 but it should have same restrictions as Cisco virl where vpc aren't supported.

Can you details port-channel physical connections and share config output for both ends?

[networkinggeek]

so my Po10 is between L2Switch and Nexus (pair). It is not about support, I think it is going through some loop and blocks my ports.

 

My VPC peer link is up and so is keep-alive link.  But when I add my member link it puts one port in blocking and, I have rebooted nodes, reset/defaulted/bounced/ my port-channel/links each time it goes back to suspended. My iOS switch side says nexus is not configured for LACP (but it is configured) and nexus side complains it is not receiving LACP bpdu from iOS switch. It shows cdp neighbors but as soon as I add links in member link port-channel, cdp neigbors goes away, either spanning tree block my port. Here are my configs on three nodes.

Switch#show running-config interface po10
Building configuration...

Current configuration : 93 bytes
!
interface Port-channel10
switchport trunk encapsulation dot1q
switchport mode trunk
end

Switch#
Switch#
Switch#sh
Switch#show run
Switch#show running-config inter
Switch#show running-config interface gi
Switch#show running-config interface gigabitEthernet 1/0
Building configuration...

Current configuration : 173 bytes
!
interface GigabitEthernet1/0
switchport trunk encapsulation dot1q
switchport mode trunk
speed 1000
duplex full
no negotiation auto
channel-group 10 mode active
end

Switch#show running-config interface gigabitEthernet 1/1
Building configuration...

Current configuration : 148 bytes
!
interface GigabitEthernet1/1
switchport trunk encapsulation dot1q
switchport mode trunk
no negotiation auto
channel-group 10 mode active
end

Switch#show ete
Switch#show eth
Switch#show etherch
Switch#show etherchannel sum
Switch#show etherchannel summary
Flags: D - down P - bundled in port-channel
I - stand-alone s - suspended
H - Hot-standby (LACP only)
R - Layer3 S - Layer2
U - in use N - not in use, no aggregation
f - failed to allocate aggregator

M - not in use, minimum links not met
m - not in use, port not aggregated due to minimum links not met
u - unsuitable for bundling
w - waiting to be aggregated
d - default port

A - formed by Auto LAG

Number of channel-groups in use: 1
Number of aggregators: 1

Group Port-channel Protocol Ports
------+-------------+-----------+-----------------------------------------------
10 Po10(SN) LACP Gi1/0(w) Gi1/1(w)

Switch#
*Jan 23 04:50:23.084: %EC-5-L3DONTBNDL2: Gi1/1 suspended: LACP currently not enabled on the remote port.
*Jan 23 04:51:01.868: %EC-5-L3DONTBNDL2: Gi1/0 suspended: LACP currently not enabled on the remote port.
*Jan 23 04:51:30.909: %EC-5-L3DONTBNDL2: Gi1/0 suspended: LACP currently not enabled on the remote port.
*Jan 23 04:52:00.310: %EC-5-L3DONTBNDL2: Gi1/0 suspended: LACP currently not enabled on the remote port.
*Jan 23 04:52:30.503: %EC-5-L3DONTBNDL2: Gi1/0 suspended: LACP currently not enabled on the remote port.

===============================

NX-OSv9k-2# show running-config interface po1

!Command: show running-config interface port-channel1
!Time: Wed Jan 23 05:41:48 2019

version 7.0(3)I7(1)

interface port-channel1
description VPC Peer-Link
switchport mode trunk
spanning-tree port type network
vpc peer-link

NX-OSv9k-2# show running-config interface ethernet 1/8
show spanning-tree n

!Command: show running-config interface Ethernet1/8
!Time: Wed Jan 23 05:41:48 2019

version 7.0(3)I7(1)

interface Ethernet1/8
switchport mode trunk
channel-group 1 mode active

NX-OSv9k-2# show running-config interface ethernet 1/9

!Command: show running-config interface Ethernet1/9
!Time: Wed Jan 23 05:41:48 2019

version 7.0(3)I7(1)

interface Ethernet1/9
switchport mode trunk
channel-group 1 mode active

NX-OSv9k-2# sh vpc
Legend:
(*) - local vPC is down, forwarding via vPC peer-link

vPC domain id : 1
Peer status : peer adjacency formed ok
vPC keep-alive status : peer is alive
Configuration consistency status : success
Per-vlan consistency status : success
Type-2 consistency status : success
vPC role : secondary
Number of vPCs configured : 1
Peer Gateway : Disabled
Dual-active excluded VLANs : -
Graceful Consistency Check : Enabled
Auto-recovery status : Enabled, timer is off.(timeout = 240s)
Delay-restore status : Timer is off.(timeout = 30s)
Delay-restore SVI status : Timer is off.(timeout = 10s)
Operational Layer3 Peer-router : Disabled

vPC Peer-link status
---------------------------------------------------------------------
id Port Status Active vlans
-- ---- ------ -------------------------------------------------

...skipping one line

vPC status
----------------------------------------------------------------------------
Id Port Status Consistency Reason Active vlans
-- ------------ ------ ----------- ------ ---------------
10 Po10 down* Not Consistency Check Not -

Applicable Performed

Please check "show vpc consistency-parameters vpc <vpc-num>" for the
consistency reason of down vpc and for type-2 consistency reasons for
any vpc.

NX-OSv9k-2# how running-config interface po10
^
% Invalid command at '^' marker.
NX-OSv9k-2# show running-config interface ethernet 1/1

!Command: show running-config interface Ethernet1/1
!Time: Wed Jan 23 05:41:48 2019

version 7.0(3)I7(1)

interface Ethernet1/1
switchport mode trunk
channel-group 10 mode active

NX-OSv9k-2# show spanning-tree n
^
% Invalid parameter detected at '^' marker.
NX-OSv9k-2# show spanning-tree

VLAN0001
Spanning tree enabled protocol rstp
Root ID Priority 32769
Address 0cbf.d05c.c407
This bridge is the root
Hello Time 2 sec Max Age 20 sec Forward Delay 15 sec

Bridge ID Priority 32769 (priority 32768 sys-id-ext 1)
Address 0cbf.d05c.c407
Hello Time 2 sec Max Age 20 sec Forward Delay 15 sec

Interface Role Sts Cost Prio.Nbr Type
---------------- ---- --- --------- -------- --------------------------------
Po1 Desg FWD 3 128.4096 (vPC peer-link) Network P2p
Eth1/2 Desg FWD 4 128.2 P2p
Eth1/3 Desg FWD 4 128.3 P2p
Eth1/4 Desg FWD 4 128.4 P2p
Eth1/5 Desg FWD 4 128.5 P2p
Eth1/6 Desg FWD 4 128.6 P2p
Eth1/7 Desg FWD 4 128.7 P2p

==========================================

NX-OSv9k-1#

!Command: show running-config interface port-channel1
!Time: Wed Jan 23 05:39:20 2019

version 7.0(3)I7(1)

interface port-channel1
description VPC Peer-Link
switchport mode trunk
spanning-tree port type network
vpc peer-link

NX-OSv9k-1#
NX-OSv9k-1# show running-config interface ethernet 1/8

!Command: show running-config interface Ethernet1/8
!Time: Wed Jan 23 05:39:27 2019

version 7.0(3)I7(1)

interface Ethernet1/8
switchport mode trunk
channel-group 1 mode active

NX-OSv9k-1# show running-config interface ethernet 1/9

!Command: show running-config interface Ethernet1/9
!Time: Wed Jan 23 05:39:29 2019

version 7.0(3)I7(1)

interface Ethernet1/9
switchport mode trunk
channel-group 1 mode active

NX-OSv9k-1#
NX-OSv9k-1# sh vpc
Legend:
(*) - local vPC is down, forwarding via vPC peer-link

vPC domain id : 1
Peer status : peer adjacency formed ok
vPC keep-alive status : peer is alive
Configuration consistency status : success
Per-vlan consistency status : success
Type-2 consistency status : success
vPC role : primary
Number of vPCs configured : 1
Peer Gateway : Disabled
Dual-active excluded VLANs : -
Graceful Consistency Check : Enabled
Auto-recovery status : Enabled, timer is off.(timeout = 240s)
Delay-restore status : Timer is off.(timeout = 30s)
Delay-restore SVI status : Timer is off.(timeout = 10s)
Operational Layer3 Peer-router : Disabled

vPC Peer-link status
---------------------------------------------------------------------
id Port Status Active vlans
-- ---- ------ -------------------------------------------------
1 Po1 up 1

vPC status
----------------------------------------------------------------------------
Id Port Status Consistency Reason Active vlans
-- ------------ ------ ----------- ------ ---------------
10 Po10 down* Not Consistency Check Not -

Applicable Performed

Please check "show vpc consistency-parameters vpc <vpc-num>" for the
consistency reason of down vpc and for type-2 consistency reasons for
any vpc.

NX-OSv9k-1#
NX-OSv9k-1#
NX-OSv9k-1#
NX-OSv9k-1#
NX-OSv9k-1# show running-config interface po10

!Command: show running-config interface port-channel10
!Time: Wed Jan 23 05:39:38 2019

version 7.0(3)I7(1)

interface port-channel10
switchport mode trunk
vpc 10

NX-OSv9k-1# show running-config interface ethernet 1/1

!Command: show running-config interface Ethernet1/1
!Time: Wed Jan 23 05:39:43 2019

version 7.0(3)I7(1)

interface Ethernet1/1
switchport mode trunk
channel-group 10 mode active

NX-OSv9k-1#
NX-OSv9k-1#
NX-OSv9k-1# show sp
spanning-tree sprom
NX-OSv9k-1# show spanning-tree n
^
% Invalid parameter detected at '^' marker.
NX-OSv9k-1# show spanning-tree

VLAN0001
Spanning tree enabled protocol rstp
Root ID Priority 32769
Address 0cbf.d05c.c407
Cost 3
Port 4096 (port-channel1)
Hello Time 2 sec Max Age 20 sec Forward Delay 15 sec

Bridge ID Priority 32769 (priority 32768 sys-id-ext 1)
Address 0cbf.d0e4.7107
Hello Time 2 sec Max Age 20 sec Forward Delay 15 sec

Interface Role Sts Cost Prio.Nbr Type
---------------- ---- --- --------- -------- --------------------------------
Po1 Root FWD 3 128.4096 (vPC peer-link) Network P2p
Eth1/2 Desg FWD 4 128.2 P2p
Eth1/3 Desg FWD 4 128.3 P2p
Eth1/4 Desg FWD 4 128.4 P2p
Eth1/5 Desg FWD 4 128.5 P2p
Eth1/6 Desg FWD 4 128.6 P2p
Eth1/7 Desg FWD 4 128.7 P2p

NX-OSv9k-1# show port-channel summary
Flags: D - Down P - Up in port-channel (members)
I - Individual H - Hot-standby (LACP only)
s - Suspended r - Module-removed
b - BFD Session Wait
S - Switched R - Routed
U - Up (port-channel)
p - Up in delay-lacp mode (member)
M - Not in use. Min-links not met
--------------------------------------------------------------------------------
Group Port- Type Protocol Member Ports
Channel
--------------------------------------------------------------------------------
1 Po1(SU) Eth LACP Eth1/8(P) Eth1/9(P)
10 Po10(SD) Eth LACP Eth1/1(s)

NX-OSv9k-1# show int ethernet 1/1
Ethernet1/1 is down (suspended(no LACP PDUs))
admin state is up, Dedicated Interface
Belongs to Po10
Hardware: 100/1000/10000 Ethernet, address: 0cbf.d0e4.7108 (bia 0cbf.d0e4.7108
)
MTU 1500 bytes, BW 1000000 Kbit, DLY 10 usec
reliability 255/255, txload 1/255, rxload 1/255
Encapsulation ARPA, medium is broadcast
Port mode is trunk

[Francesco Molino]

You didn't share port-channel 10 configuration in Nexus2 and port-channel 1 on Nexus1.

Can you share output of show vpc please?

[networkinggeek]

Updated configs above

[networkinggeek]

I have attached my main config file to actual post/question.

NX-OSv9k-1# sh vpc
Legend:
(*) - local vPC is down, forwarding via vPC peer-link

vPC domain id : 1
Peer status : peer adjacency formed ok
vPC keep-alive status : peer is alive
Configuration consistency status : success
Per-vlan consistency status : success
Type-2 consistency status : success
vPC role : primary
Number of vPCs configured : 1
Peer Gateway : Disabled
Dual-active excluded VLANs : -
Graceful Consistency Check : Enabled
Auto-recovery status : Enabled, timer is off.(timeout = 240s)
Delay-restore status : Timer is off.(timeout = 30s)
Delay-restore SVI status : Timer is off.(timeout = 10s)
Operational Layer3 Peer-router : Disabled

vPC Peer-link status
---------------------------------------------------------------------
id Port Status Active vlans
-- ---- ------ -------------------------------------------------
1 Po1 up 1


vPC status
----------------------------------------------------------------------------
Id Port Status Consistency Reason Active vlans
-- ------------ ------ ----------- ------ ---------------
10 Po10 down* Not Consistency Check Not -

Applicable Performed


Please check "show vpc consistency-parameters vpc <vpc-num>" for the
consistency reason of down vpc and for type-2 consistency reasons for
any vpc.

=========

NX-OSv9k-1# show running-config int po1

!Command: show running-config interface port-channel1
!Time: Wed Jan 23 06:27:55 2019

version 7.0(3)I7(1)

interface port-channel1
description VPC Peer-Link
switchport mode trunk
spanning-tree port type network
vpc peer-link

==============

NX-OSv9k-2# show run interface po10

!Command: show running-config interface port-channel10
!Time: Wed Jan 23 06:28:02 2019

version 7.0(3)I7(1)

interface port-channel10
switchport mode trunk
vpc 10

[Francesco Molino]

On which platform are you running it? GNS3?

Please export your configs (N9k1, N9k2 and Switch) and attach them as text file. I’ll try to reproduce your design with your configs.
Just to let you know than doing this virtually could bring weird issues like you’re facing right now.

[networkinggeek]

@Francesco Molino I am using GNS3
I have a feeling as well that this is more of spanning tree issue or something, which is blocking my ports, or maybe port trunking. there is alot involved here, or maybe I am no bringing this up in right order, I have already tried with newer Nx9k images as well, and it throws exact same error and lacp does not come up

Thanks for trying and helping me out here, I am going to attach my current full configs here so it can be replicated if anyone wants to try 

@balaji.bandi

[balaji.bandi]

Long thread and since we read many post i think i lost here.. try to re-cap the issue here..we have provided detailed config and my self tried over eve-ng that config works.(not yours the one i have provided)

coming back to question, you have the problem between nexus switch forming the vpc or vpc not forming between nexus and switch ?  so we can concentrate on that rather review all the config.

on your nexus config both the device have same  why ?

peer-keepalive destination 10.1.1.2   <<-- 

I will send working config later day or over weekend when i get chance.

[networkinggeek]

(above is the new diagram)  I have made my topology more smaller,

My vpc is up and so is my vpc peer-keepalive (please look at the attached config files, @balaji.bandi  But I am unable to bring up my member port-channel i-e between IoSL2 switch and nexus pair. I have also attached my current configurations.

my spanning tree keeps blocking my et1/1 on both nexus, I don't know why. right now this is sh spanning tree and I removed et/1 from po10

NX-OSv9k-2(config-if)# sh spanning-tree

VLAN0001
Spanning tree enabled protocol rstp
Root ID Priority 32769
Address 0cbf.d05c.c407
This bridge is the root
Hello Time 2 sec Max Age 20 sec Forward Delay 15 sec

Bridge ID Priority 32769 (priority 32768 sys-id-ext 1)
Address 0cbf.d05c.c407
Hello Time 2 sec Max Age 20 sec Forward Delay 15 sec

Interface Role Sts Cost Prio.Nbr Type
---------------- ---- --- --------- -------- --------------------------------
Po1 Desg FWD 3 128.4096 (vPC peer-link) Network P2p 
Eth1/1 Desg BLK 4 128.1 P2p 
Eth1/2 Desg FWD 4 128.2 P2p 
Eth1/3 Desg FWD 4 128.3 P2p 
Eth1/4 Desg FWD 4 128.4 P2p 
Eth1/5 Desg FWD 4 128.5 P2p 
Eth1/6 Desg FWD 4 128.6 P2p 
Eth1/7 Desg FWD 4 128.7 P2p

[balaji.bandi]

On both the nexus please change as below :

interface Ethernet1/1
switchport mode trunk
channel-group 10 mode active  << - to  channel-group 10 mode on

Switch side try below config.

interface GigabitEthernet1/0
switchport trunk encapsulation dot1q
switchport mode trunk
media-type rj45
speed 1000
duplex full
no negotiation auto
channel-group 10 mode active   <-- channel-group 10 mode on
!
interface GigabitEthernet1/1
switchport trunk encapsulation dot1q
switchport mode trunk
media-type rj45
no negotiation auto
channel-group 10 mode active   <-- channel-group 10 mode on

Test and let us know...this more of virtual device issue, in real world your config works.