Nexus 5k - vpc to vpc

[ronald.spicka]

Hi all,

We are running 4x n5k and started with the vPC feature.

So my question is, if i can connect a vpc-pair to another vpc-pair?

In the cisco docs i can find examples for connecting a vpc-pair to a single switch, or server (with and without fex)

But there is nothing about how to connect 4 n5k via vPC feature.

thanks a lot

[gnijs]

Yes. Very interesting question. I would like to know the answer also.

Basically, i want to connect 2 pairs of N5000 , togethere with a portchannel.

Like:

N1 ---- VPC link --- N2

|                            |

|                            |

Po200                Po200

|                            |

|                            |

N3 ---- VPC link--- N4

According to me, it should be possible, but i haven't tried it.

[royalblues]

The attached topology is possible between the N5K and the N7K

Havent checked between 5K's but should support

Narayan

[mbadali]

Narayan, have you configured this in the field?  I've been attempting to configure this but am running into issues with LACP not meeting the min-links requirement as part of the bundle formation.  Between the log messages you'll see this error  port-channel100(0x16000063) does not meet min-links req num_intrans_active 0, pc_min_links_config 1 Suspend all member ports.   My configuration is identical to your vpc.jpg diagram.  If I remove the vpc config from the 5548s and essentially connect a single 5548 to the N7K pair (in vPC mode) the port channel comes up.

Configuration below errors...

Thanks

Mike

lab7010b %ETHPORT-5-IF_RX_FLOW_CONTROL: Interface port-channel100, operational Receive Flow Control state changed to on

2011 Jul  8 09:48:49 wklab7010b %ETHPORT-5-IF_TX_FLOW_CONTROL: Interface port-channel100, operational Transmit Flow Control state changed to on

2011 Jul  8 09:48:52.588711 lacp: lacp_proto_suspend_miscfg_members(1999): Ethernet4/9(0x1a188000): Port misconfigured is true 

2011 Jul  8 09:48:52.588741 lacp: lacp_proto_suspend_miscfg_members(2006): Ethernet4/9(0x1a188000): cmp_chk_pass=1 port_miscfg=0 winner-end=0 port_half_duplex=0 

2011 Jul  8 09:48:52.588771 lacp: lacp_proto_suspend_miscfg_members(1999): Ethernet7/9(0x1a308000): Port misconfigured is true 

2011 Jul  8 09:48:52.588786 lacp: lacp_proto_suspend_miscfg_members(2006): Ethernet7/9(0x1a308000): cmp_chk_pass=1 port_miscfg=0 winner-end=0 port_half_duplex=0 

2011 Jul  8 09:48:52.588886 lacp: lacp_proto_proc_sel(453):  Port-channel port-channel100(0x16000063) does not meet min-links req num_intrans_active 0, pc_min_links_config 1 Suspend all member ports

2011 Jul  8 09:48:52 wklab7010b %ETH_PORT_CHANNEL-5-PORT_SUSPENDED: Ethernet4/9: Ethernet4/9 is suspended

2011 Jul  8 09:48:52 wklab7010b %ETH_PORT_CHANNEL-5-PORT_SUSPENDED: Ethernet7/9: Ethernet7/9 is suspended

2011 Jul  8 09:48:52 wklab7010b %ETHPORT-5-SPEED: Interface port-channel100, operational speed changed to 10 Gbps

2011 Jul  8 09:48:52 wklab7010b %ETHPORT-5-IF_DUPLEX: Interface port-channel100, operational duplex mode changed to Full

2011 Jul  8 09:48:52 wklab7010b %ETHPORT-5-IF_RX_FLOW_CONTROL: Interface port-channel100, operational Receive Flow Control state changed to on

2011 Jul  8 09:48:52 wklab7010b %ETHPORT-5-IF_TX_FLOW_CONTROL: Interface port-channel100, operational Transmit Flow Control state changed to on

2011 Jul  8 09:48:53 wklab7010b %ETHPORT-5-SPEED: Interface port-channel100, operational speed changed to 10 Gbps

2011 Jul  8 09:48:53 wklab7010b %ETHPORT-5-IF_DUPLEX: Interface port-channel100, operational duplex mode changed to Full

2011 Jul  8 09:48:53 wklab7010b %ETHPORT-5-IF_RX_FLOW_CONTROL: Interface port-channel100, operational Receive Flow Control state changed to on

2011 Jul  8 09:48:53 wklab7010b %ETHPORT-5-IF_TX_FLOW_CONTROL: Interface port-channel100, operational Transmit Flow Control state changed to on

2011 Jul  8 09:48:56.489039 lacp: lacp_proto_suspend_miscfg_members(1999): Ethernet7/9(0x1a308000): Port misconfigured is true 

2011 Jul  8 09:48:56.489067 lacp: lacp_proto_suspend_miscfg_members(2006): Ethernet7/9(0x1a308000): cmp_chk_pass=1 port_miscfg=0 winner-end=0 port_half_duplex=0 

2011 Jul  8 09:48:56.489090 lacp: lacp_proto_suspend_miscfg_members(1999): Ethernet4/9(0x1a188000): Port misconfigured is true 

2011 Jul  8 09:48:56.489107 lacp: lacp_proto_suspend_miscfg_members(2006): Ethernet4/9(0x1a188000): cmp_chk_pass=1 port_miscfg=0 winner-end=0 port_half_duplex=0 

*****************************************************************************

*******both N7Ks have this same config ******

wklab7010a# sho run int po 100 membership

!Command: show running-config interface port-channel100 membership

!Time: Fri Jul  8 11:39:26 2011

version 5.1(3)

interface port-channel100

  description Portchannel to wklab5548_ab

  switchport

  switchport mode trunk

  switchport trunk allowed vlan 110,112-114,120-121,3450-3451

  spanning-tree port type network

  logging event port link-status

  logging event port trunk-status

  flowcontrol receive on

  flowcontrol send on

  mtu 9216

  load-interval counter 2 30

  vpc 100

interface Ethernet7/9

  description Portchannel to wklab5548_ab

  lacp rate fast

  switchport

  switchport mode trunk

  switchport trunk allowed vlan 110,112-114,120-121,3450-3451

  spanning-tree port-priority 128

  flowcontrol receive on

  flowcontrol send on

  link debounce time 0

  mtu 9216

  logging event port link-status

  logging event port trunk-status

  bandwidth 10000000

  load-interval counter 2 30

  udld enable

  channel-group 100 mode active

interface Ethernet4/9

  description Portchannel to wklab5548_ab

  lacp rate fast

  switchport

  switchport mode trunk

  switchport trunk allowed vlan 110,112-114,120-121,3450-3451

  spanning-tree port-priority 128

  flowcontrol receive on

  flowcontrol send on

  link debounce time 0

  mtu 9216

  logging event port link-status

  logging event port trunk-status

  bandwidth 10000000

  load-interval counter 2 30

  udld enable

  channel-group 100 mode active

  no shutdown

*******both N5Ks have this same config ******

wklab5548a# sho running-config int po 2 membership

!Command: show running-config interface port-channel2 membership

!Time: Fri Jul  8 11:43:24 2011

version 5.0(3)N1(1)

interface port-channel2

  description Portchannel to wklab7010ab

  switchport mode trunk

  vpc 2

  switchport trunk allowed vlan 110,112-114,120-121,3450-3451

  spanning-tree port type network

  flowcontrol receive on

  flowcontrol send on

  load-interval counter 2 30

interface Ethernet1/1

  description vPortchannel to wklan7010ab

  lacp rate fast

  switchport mode trunk

  switchport trunk allowed vlan 110,112-114,120-121,3450-3451

  spanning-tree port type network

  flowcontrol receive on

  flowcontrol send on

  logging event port link-status

  load-interval counter 2 30

  udld aggressive

  channel-group 2 mode active

interface Ethernet1/7

  description vPortchannel to wklan7010ab

  lacp rate fast

  switchport mode trunk

  switchport trunk allowed vlan 110,112-114,120-121,3450-3451

  spanning-tree port type network

  flowcontrol receive on

  flowcontrol send on

  logging event port link-status

  load-interval counter 2 30

  udld aggressive

  channel-group 2 mode active

[aaronanderson]

I can confirm this works.  I successfully completed a four node / two nexus 5k pair configuration today and I've noticed no issues thus far (attached two 5020's to an existing/production 5010 pair).  I'm over a year late responding to this, but I stumbled upon this looking for confrimation that this works, and hopefully this confirmation is of use to someone in the future.

Others have referenced n7k to n5k connection instructions.  These instructions are spot on and should be referenced if possible.  I've included sample / partial config below if it helps:

[N5k-1]------[N5k-2]

    |      \     /    |

    |        \ /      |

    |        /\       |

    |      /    \     |

[N5k-3]------ [N5k-4]

=========== n5k-1 ===========
vpc domain 10
role priority 2000
system-priority 4000

interface port-channel1
description vPC Peer Link
vpc peer-link

interface port-channel2
description vPC link to second N5k pair
vpc 2

interface eth 1/1
description N5k-3 Eth 1/1
channel-group 2 mode active

interface eth 1/2
description N5k-4 Eth 1/1
channel-group 2 mode active

=========== n5k-2 ===========
vpc domain 10
role priority 2100
system-priority 4000

interface port-channel1
description vPC Peer Link
vpc peer-link

interface port-channel2
description vPC link to second N5k pair
vpc 2

interface eth 1/1
description N5k-3 Eth 1/2
channel-group 2 mode active

interface eth 1/2
description N5k-4 Eth 1/2
channel-group 2 mode active

=========== n5k-3 ===========
vpc domain 20
role priority 3000
system-priority 4000

interface port-channel1
description vPC Peer Link
vpc peer-link

interface port-channel2
description vPC link to first N5k pair
vpc 2

interface eth 1/1
description N5k-1 Eth 1/1
channel-group 2 mode active

interface eth 1/2
description N5k-2 Eth 1/1
channel-group 2 mode active

=========== n5k-4 ===========
vpc domain 20
role priority 3100
system-priority 4000

interface port-channel1
description vPC Peer Link
vpc peer-link

interface port-channel2
description vPC link to first N5k pair
vpc 2

interface eth 1/1
description N5k-1 Eth 1/2
channel-group 2 mode active

interface eth 1/2
description N5k-2 Eth 1/2
channel-group 2 mode active

[e-chuah]

Hi, I am just wondering if the setup will work when you have only two links between the two vpc (i.e, remove the cross connection) Rgds Eng Wee

[Lucien Avramov]

yes it will work with vPC

[karthik.gopalakrishnan]

Hello,

I am setting up something similar too, but have a small concern with the design. Instead of having a L2 vPC trunk between N5K-1&2 and 3&4 pair, I am thinking of having a point-to-point L3 between N5K-1 & N5K-3, and N5K-2 & N5K-4, and do OSPF between them, as there will be different sets of vlans behind the two N5K pairs ( on the FEX ).

Will that work?

Thanks,

KG

[Jerry Ye]

I am assuming you want something like this:

[N5k-1]---vpc_pl---[N5k-2]

    |                            |

    |                            |

    |                            |

    |                            |

[N5k-3]---vpc_pl---[N5k-4]

if yes, you will need another L3 link between N5K-1<->N5K-2 and N5K-3<->N5K-4, it is because if N5K-1 is down, assuming you are running HSRP, traffic hits N5K-3 will be blackholed unless you have a L3 link between N5K-3 and N5K-4.

HTH,

jerry

[n.lenevez]

I need to setup something like this too. 2 VPC pairs between 2 datacentres (Prod and DR sites). All 4 switches need to participate in the same HSRP group:

[Jigar Dave]

Guys,

have a look at below link from Cisco N7K Product Manager:

https://supportforums.cisco.com/thread/2101574

hope you will get all your answers and some more knowledge as well.

[seungyoul-lee]

I was on the exactly same issue. The cause of this issue is the same vPC domain id between two N5K switch pairs. If you want extend layer 2 with vPC from one Nexus switch pair to another pair, the vPC domain id should be different from each other. Cisco suggests to change vPC domain id on one of N5K pairs in this case as a doc below.

http://www.cisco.com/c/en/us/support/docs/switches/nexus-5000-series-switches/116271-problem-vpc-00.html