Design considerations Nexus 7k and Cisco ACI in data Center - OSPF or BGP - best choice

[Kumar99]

 Hi All ,

We have two data centers as shown in the picture. Data center 1 is primary and data center 2 is secondary.

 At each data center 

1) Nexus 7ks have 3 VDC's configured . WAN_VDC is connected to Cisco ACI using L3 Port channel .

note : At WAN_VDC , default route is pointed towards HSRP of ACI and static routes are configured on Cisco ACI towards HSRP ip of Nexus 7ks as we have so many L3 vlans on 3 VDC's.  

2) Nexus 7ks are connected using VPC+ and IBGP is running between them.

3) EBGP is running between data center 1 and data center 2 using DCI link .

4) We have legacy network (cisco 6500) connected to nexus 7ks using OSPF at data center 2 .

5) Redistribution is done between bgp to ospf and vice versa at data center 2 on WAN_VDC.

6) prefix lists are configured on nexus 7ks in data center 2 to advertise lan subnets of it over ebgp towards data center 1 over DCI link.

route-maps are mapped under bgp and ospf to restrict incoming and outgoing routes.

Now we would like to introduce dynamic routing between Cisco and Nexus 7ks at both data centers.

first , it will be done at Data Center 2.

What are the pros and cons of OSPF if we go with it.

what are pros and cons fo BGP if we go with it.

since ospf is already running on nexus 7ks for legacy network at data center 2, will it creates any problem if we go with ospf between cisco aci and nexus 7ks at data center 2. 

Design documents of ospfsays

1) we have some limitations on nexus 7ks for ospf

2) Only Type 3 LSAs can be filtred at ABR not type 5 and 7.

3) We must use table map to filter incoming and outgoing routes in OSPF.

4)Distribute-list is not supported by ospf.

5)Moreover ospf uses more CPU 

any advice on this would be appreciated.

Many Thanks,

Kumar

[Kasper Mortensen]

It's seems like a mess, honestly. Your topology looks like a prime scenario for a redistribution loop. How big is this environment? Amount of VM/server hosts and how many routers are participating in OSPF/BGP?

Questions:

1) Why do you have a default static route pointing TO your ACI fabric? I'd imagine there's nothing that needs to default into your server environment. 

2) Why are you running EBGP between the datacenters? Is there a scalability issue or a regional separation? And why run EBGP between the C6800 and NX7K and then IBGP between the NX7K? 

3) What line cards do you have in the NX7Ks?

4) How many processes/top policies are configured in your ACI? 

CPU usage of OSPF shouldn't be an issue unless something is wrong in the network or you have a serious amount of routers participating in area 0. 

You need to get rid of all those redistribution points. I'd suggest you decide to either have the 6880 or NX7 be the core of your network. Meaning your either remove the connection between ACI and 6880 or ACI and NX7K. It will greatly simplify your routing. 

Running BGP in the datacenter is usually done because of the features provided, such as better route visibility and path control. It also enables/prepares the infrastructure for technologies like EVPN or MPLS. If you don't have those needs and the datacenters are suitable to run one OSPF process in area 0, then you can simply do that as well. But it's impossible to make a real recommendation for your setup based on so little information. Depending on how much redistribution and how many devices you have in that setup, I'd suggest you document everything very well and understand the consequences of changes you make. I have cleaned up an extremely messy datacenter network before and it's usually a lot of small steps/reconfigurations to fix. I suggest you only get ideas from here and don't take any configuration change advice from someone who hasn't had hands on with your network there.