Hello All, I'm looking for some suggestions on a new network design. We have two data centers (directly connected) and many remote sites (DMVPN), all running OSPF. We are looking at adding both MPLS and SD-WAN solutions to replace DMVPN. These services normally prefer to run BGP, so I'm looking at how best to integrate BGP with our infrastructure.
The issue i'm running into is BGP/OSPF route preference. I would like to do mutual redistribution from OSPF to BGP, so that all WAN and MPLS sites follow shortest path to data center resources, and allow allow for primary path fail over, etc. The main sticking point right now is that if I advertise data center subnets to WAN/MPLS, then those same subnets are learned back through the second data center, the eBGP routes (AD 20) are preferred over the directly-connected OSPF routes (AD 110). I currently do not have separate sets of WAN routers, so my WAN/MPLS connections are terminating on my core routers, thus the eBGP vs OSPF route preference issue.
Basically I need the WAN subnets preferred over eBGP, and the LAN subnets preferred over OSPF, but each acting as a fail over for the other. I'm trying as much as possible to accomplish this without having to rely on prefix lists and ACLs, keeping it all dynamic. The reason for this is just because our data centers contain hundreds of prefixes, listing and maintaining that list would be a pain. Same with summarizing or aggregation.
Some things I've tried or looked into:
1. Raise AD of eBGP, or lower AD of OSPF. This creates a secondary issue, since it is basically 'all or nothing.' I can get the OSPF routes preferred over eBGP, but then it also prefers ALL routes learned through eBGP from the opposite data center over OSPF, and tries to route the long way around to get to WAN sites.
2. Run iBGP across the two data centers rather than two separate ASNs. I actually got a config working for this, because I can use weight/path to prefer iBGP routes to eBGP, then OSPF will beat out iBGP routes. I'd like to stay away from this just thinking long term, separate ASNs for the two data centers seems to be a better way to go, especially as we might look at converting our current point-to-point links over to MPLS.
3. Use a VRF for MPLS/WAN BGP. I was able to get this working in a lab, on newer code versions using the VRF import/export commands. I was hoping that the VRF BGP routes would show up differently in the global routing table, allowing me to control them more like OSPF routes with cost. However it looks like they are still importing into the GRT as BGP routes with AD 20. With BGP running in a VRF, and OSPF running in the GRT, is there any way to get those VRF routes into the GRT as OSPF E1 routes, or similar?
4. Use a VRF with GRE tunnels and OSPF neighbors between VRF and global. I haven't tested this one, but it seems like it would work, just a bit hacky and probably not great for production.
5. Use a separate set of WAN routers to terminate WAN/MPLS BGP, then connect those to core via OSPF. I'm sure this would resolve all my issues, but getting the extra hardware may not happen. Trying to do this via VRFs is my best chance as of right now.
6. Route tagging/communities/filters. I can't find a good way to get any of this working, since both data centers are in OSPF area 0, I can't find any unique values to do matching on (without having to list specific prefixes, or advertising routers, etc).
This 'seems' like a fairly straight forward setup, hopefully I am just missing something and there is a better way to do this. Any advise is appreciated.
Hi
Ok if you don't want to use any acls/prefix-list, I guess you don't want to use network statement as well (thinking about BGP backdor command).
You can acomplish by changing the AD.
Let me explain:
- By default eBGP has 20 and OSPF 110
- On OSPF, you can change the default distance per routes type.
- On BGP, you can change the AD per neighbor, let's say you set it to 115
- On OSPF, you can have define 110 for inter and intra area and 200 for external
Here and example: (sorry I did it using IOU as I'm not in the office and not able to do a real lab)
IOU3 advertises 192.168.10.0/24 subnet
IOU2 advertises 10.10.0.0/16 (few /24 subnets)
IOU1 advertises 10.1.0.0/16 (few /24 subnets)
I assume, you have network statements in OSPF to advertise your local subnets? Am i right? (that's why I asked for your routing config)
On DC1 and DC2:
router ospf 10
distance ospf external 200
Here, if you didn't do any filter in OSPF, routes learned from DC2 through BGP will be redistributed in OSPF to DC1 as external routes and local connected subnets of DC2 learned as intra-area routes (and vice versa).
Now, in terms of BGP:
On DC1, for example my BGP peer (IOU3) has IP 172.16.2.3. The below command says that every subnets I learn from this peer should have AD 115 instead of 20:
distance 115 172.16.2.3 0.0.0.0
in terms of routing (RIB) on IOU1: (you can see subnets from DC2 are in RIB using OSPF but all others external from DC2 are in BGP)
O 10.10.0.0/24 [110/20] via 172.16.1.2, 00:09:53, Ethernet0/2
O 10.10.1.0/24 [110/20] via 172.16.1.2, 00:09:53, Ethernet0/2
O 10.10.2.0/24 [110/20] via 172.16.1.2, 00:09:53, Ethernet0/2
B 192.168.10.0/24 [115/0] via 172.16.2.3, 00:09:22
and I shutdown BGP neighbor on DC1, the 192.168.10.0/24 will be learned as OSPF E2 routes:
O E2 192.168.10.0/24 [200/1] via 172.16.1.2, 00:00:05, Ethernet0/2
This is a quick example showing that routes learned from remote sites will have an AD of 115 instead of 20 but always prefered over BGP than OSPF and fallback to OSPF (AD 200) in case BGP is down on 1 router.
