Nexus 5k assistance with VPC and routing

[Milan Peyrac]

hello guys,

we are trying to implement a new solution for a client of ours who has purchased a pair of nexus 5596UP devices.

We have this topology attached in jpeg. They want to use the pair of 5k's for local lan and WAN connectivity.

Background

Customer wants a VPC setup between the pair of nexus 5k's beucase at some point they will want to purchase FEX modules and VPC servers directly, in which case the VPC will be required (VPC Vlans L3 will terminate on 5k's using HSRP).

Quesitons

1. Can I have the same vlan with SVI built on each nexus and pass the vlan across the peer link so I can build IBGP and peer EBGP as per the diagram. Will this work?

2. Is it possible to build a layer 3 link from each nexus to remote PE device and then setup another SVI on each nexus and allow that accross the peer link? Would this configuration work and would traffic pass across the peer link for IBGP connectivity?

3. Or can I have it as per question 1 above and use a seperate port-channel (non-vpc) between the Nexus 5k pair to trunk the vlan across?

What is the best design around this kind of solution?

The alternative is to have the Layer 2 switch connect to both Nexus 5k's without port-channel and let spanning tree manage the loop. In this case would I need to build another trunk between the 5k's or could I just allow the vlan across the VPC Peer link.

Thanks a lot in advance.

[Reza Sharifi · Accepted solution]

Hi,

Do the 5ks have layer-3 daughter cards installed? The 5K support BGP, but the max amount of BGP routes you can have is 8000.

HTH

[Milan Peyrac]

Yes they have correct cards and license...

[LA-Engineer]

You have to be very careful when configuring L3 services and interfaces while using VPC. 

Take a look at this document:

http://www.cisco.com/c/dam/en/us/td/docs/switches/datacenter/sw/design/vpc_design/vpc_best_practices_design_guide.pdf

Also, take a look at this post:

http://bradhedlund.com/2010/12/16/routing-over-nexus-7000-vpc-peer-link-yes-and-no/

You can create a vlan used exclusively for Nexus-to-Nexus iBGP peering.  Use a new 'access' link between the two switches and place them on the new vlan.  Make sure that this VLAN does not traverse the VPC peer link.  Then, create SVIs on each switch for that VLAN and peer over that link.  Then, you can create a L3 link on each nexus to peer with your eBGP neighbors.

The point you want to make sure you understand is the VPC loop prevention mechanism that says "If a packet is received on a VPC port, traverses the VPC peer link, it is not allowed to egress on a VPC port."

[MARK BAKER]

Milan,

With your alternative in the last paragraph, you wouldn't have to rely on spanning-tree if you configure the attached n5k ports as layer 3 interfaces instead of SVIs and VLANs. You can set the L2 switch interface with portfast on both the n5k and eBGP peer links.

You should replace the "portchannel" label on the L2 to n5k switch with "vPC" and replace the "vPC" on the link between the two n5k switches with "Peer Link". At least on the n7k, the routing protocol issue has to do with the vPC member links and not the peer link between the n7ks. It is a valid config to route on the peer link, but it is not the first choice. First choice would be to use a separate L3 link between them.

Thank you,

Mark