L3 over back to back vPC

[de1denta]

Hi All,

I need some advise on a particular design that I am looking at

I have a customer that has a 4 x Nexus 9k switches, 2 in each DC, with a back-to-back vPC as the DCI to carry a number of stretched VLANs.

The customer now needs to establish an OSPF adjacency between the Nexus 9Ks in each DC to provide a backup path for an MPLS network. I'm aware that certain L3 designs are not supported over vPC and was wondering what the best option is for this scenario? Can I establish L3 adjacencies over the vPC or should I be looking to install dedicated L3 point-to-point circuits for this? Another option is removing the back-to-back vPC and configuring as L3 with VXLAN?

Any advice is appreciated

[balaji.bandi]

Generally use case of vPC to avoid loops of Spanning tree, If you looking to be L3 p2p, why you need vPC  ?

Please look at the following guide regarding vPC on the Nexus.

 http://www.cisco.com/en/US/prod/collateral/switches/ps9441/ps9670/C07-572835-00_NX-OS_vPC_DG.pdf

[de1denta]

Hi,

Thanks for the reply

The customer has 2 x dark fibre circuits between their data centres which have been connected between the Nexus 9K switches in a vPC to provide 20Gbps aggregate bandwidth. This vPC is used to extended VLANs between sites for certain applications.

The need for layer3 between sites is to provide a backup path for an MPLS network. What I need to understand is if I can run L3 routing over the vPC or if dedicated links should be installed for this purpose.

[mcnairiain]

Hi Will

We have a similar setup, 2 9ks in 1 DC (VPC Domain 10) - DWDM - 2 9ks in other DC (VPC Domain 11) - B2B VPC between them for L2 vlan propagation. 

It used to be that if you have 2 switches in the same DC (same VPC domain) that have a peer link and KA link between them, and those switches were layer 3 aware (ie: they had SVI interfaces and or needed to provide a conduit between them for IGP adjacency/routes) - then you needed a dedicated L3 link between those 2 switches. This link needed to be either a L3 link or had to use a vlan that was not a VPC vlan. You could not use the VPC peer link for this process. 

In our Network we have a dedicated separate dot1Q trunk that carries non VPC vlans, each 7k (in our case) has an SVI on those non VPC vlans for OSPF.

Im not sure how it works between VPC domains over a B2B VPC, as technically the vlan you would want to use would be a VPC vlan - Just North/South as opposed to East West between VPC peers? I would imagine the limitation would still be there.

Have you actually tried to run IGP protocols over a vlan between sites on your setup?

[SIMMN]

Wonder if you have implemented this OSPF over B2B vpc? I am looking for the options for N5K pair and N9K pair scenario...Would like to use SVI for ospf peering but the SVI vlan would have to be vpc vlan...Also seems like my N5K running v7.1 doesn’t have “layer3 peer-router” feature...suggestion? 

[MHM Cisco World]

You can use vpc for L3 with layer3 peer-router.

[SIMMN]

Understood. But “layer3 peer-router” is not available on my N5K...assuming I can’t upgrade the firmware, looking for options to run OSPF on top of the B2B vPC.

[Sergiu.Daniluk]

Hi @SIMMN 

If you cannot upgrade to a supported version for Layer3 Peer-router, the only option left is to use non-vpc vlans or L3 dedicated interfaces. OSPF over back-2-back is not supported without layer3 peer router.  Check out the "Best Practices for Layer 3 and vPC" from this excellent whitepaper https://www.cisco.com/c/dam/en/us/td/docs/switches/datacenter/sw/design/vpc_design/vpc_best_practices_design_guide.pdf (written for 7k, valid for all Nexus families)

Stay safe,

Sergiu

[SIMMN]

Thanks! I do believe I went through that doc but didn’t find related or specific on the B2B with OSPF...

Since the setup is B2B vPC inter-switch pairs, in order to use either of the two other options you suggested, it would still require separate link besides the vPC, No matter for using routed ports or non-vPC vlan svi, right?

[Sergiu.Daniluk]

Well.. the document does not discuss about OSPF specifics, but it discuss about the general behavior. Based on that you can draw the conclusions and available options.

Regarding the separate links: Yes, you need dedicated interfaces (separate from b2b vpc), regardless of the option you go - non-vpc svis or l3 ports.

In my opinion, I would suggest doing the upgrade first, then configure l3 peer router and use routing over vpc. It's a more elegant solution.

Cheers,

Sergiu

[SIMMN]

Thanks!