Datacenter interconnect using VPC

[Arjen K]

A cliënt has two datacenters which are connected with two DCI (DataCenter Interconnect) links.

In the past they would use a 3750 stack on both sides with a port-channel for the DCI. Life was easy then.

In the new datacenter they have 4 nexus 5600's connected in a redundant way. 

- All nexus devices are connected to 1 local nexus (vpc peerlink) and 1 remote nexus (vpc).

- All vlans are stretched

- HSRP will be active on all Nexus devices, with ACL's for filtering HSRP, so Nexus L3 routing is local in both datacenters.

- The firewalls (2) are connected directly to only 1 nexus in each datacenter.

- Only one of the two datacenters has an active firewall. Firewall routing will travel over the DCI VPC.

The network has been running for 4 weeks without problems, however, after a network hickup we discovered that some L3 traffic is VPC blackholed.

We solved the problem (for now) by disabling one of the DCI links.

- I could remove the VPC between the datacenters, but then I would have spanning-tree.

Is there any way to configure the DCI in a smart way so we have a redundant active/active setup without spanning-tree? 

[Sergio Gonzalez]

Hi Arjen K,


You need to connect one of the wire of the DCI to each 5K by DC.


DC1-5K1 <-------------------------------->DC2-5K1

DC1-5K2 <-------------------------------->DC2-5K2


using VPC to has both link active-active.

Sergio

[ggmeza1983]

Sergio, buenas noches

Como sería la configración para tener active/active el Gateway en los dos Sites, asumiendo que las VLAN´s se extienden entre los dos Sites?

Otra consulta, es si se puede generar mas de un vPC para interconectar los dos Sites, o la tecnologia vPC no lo permite?

Gracias,

Gustavo.

[ggmeza1983]

Hi... Can I have more than one vPC between the two Sites?

Thanks,

Gustavo.

[Rick1776]

You could, but as I stated I would make those links non-VPC links and Routed Layer 3 and use EIGPR and or OSFP and let the ECMP do it's thing.

[Rick1776]

Never use a Layer3 link to vPC member ports. Have them seperated and have the IGP do their own ECMP. 

With vPC you'll run into an issue where a quarter of the traffic (packets) will go out one vPC switch and then the next flow will go from second vPC link and will get blocked because it will go over the vPC peer-link and will not be able to go out any member port.

This is a very common problem that I see all the time.

[robertbonyongwe]

Fabric Path between the two DCs with VPC+

[Rick1776]

So do you have Nexus 7K's/ASR's on the edge to do Fabricpath, Nexus 5K's don't support Fabricpath.

As I side before the only want to have a Active/Active DC would be to have higher layer protocols like OTV and LISP so as one host moves from DC1 to DC2 it's tracked and IGP updates are sent to the other locations. This is an important discussion and a lot of things have to be taking into consideration such as IPAM, DNS, SAN, and Applications.

Depending on where you are in the US or outside I work for a VAR that can help if required.

[robertbonyongwe]

How many data centres are you looking to interconnect, whats the
connection between the DCs (distance, transmission media )

*Fabric path gives you an active/active setup with ECMP
forwarding capabilities, most common use cases for fabric path is
interconnecting two DC connected via lets says 2* 10Gbps(layer 2) Fibre
DWDM links. It's for not big DC deployments with multiple geographically
spaced DCs. Fabric path eliminates STP on your core were the DC's
interconnect by using IS-IS control plane. Also fabric path gives you
Active/Active FHRP ( hsrp ) by using HSRP anycast ( hsrp version 2 ).
Note: Fabric path runs at layer two ( requires dedicated layer 2 link )


*But if you looking to interconnect multiple DC spaced via large
Geographical regions your best option will be VxLANs BGP EVPN or OTV as
these can route via a media ( mpls , fibre , eompls etc ) in short u can
tunnel via layer 3 domains.

But remember Cisco Nexus switches don't run all the above features on one
model depends on the switch model: Nexus 9000 , 3000 , 5000 , 7000 etc (
its all business for Cisco , lol )

[Rick1776]

I think he said they are already running Fabricpath with VPC+, but it's not in the diagram he listed in his first port.

[dineshperera]

Hi Rick,

Could you please able to grab the Nexus 5K configurations from both data centers if possible. We have on a DCI deployment with Nexus 5K Layer 2 extension and it will help me to build the Nexus configurations and I can get the output / leanings and share with you all for reference.