Change AD Default Route in OSPF

[davparker]

I'm working on a project to enable Internet failover for any of our offices (see diagram). We have four paths to the Internet and three pairs of firewalls. So far my failover configuration is good for the remote offices. It was challenging but we went through a live test to verify all worked as expected. I use ip sla with route tracking to test for Internet connectivity. I'm on the last leg now to do the same for the Core-Office. I've run into an issue with OSPF. Turns out the Core switch and two ISRs each have static default routes configured. I tried enabling default-information originate on the Core OSPF router but the other ISRs don't pick up that route after removing the static default from the ISRs. Instead the are learning an eBGP default route from one of the other PE peerings. I'm guessing this is why the previous admin used static default routes for each layer 3 device. The learned eBGP route has an AD of 20 as opposed to OSPF's 110. I am unable to change the AD using a route-map with OSPF. Documentation indicates this should be doable by using a set distance cmd after matching the default route using a prefix-list. The problem is set distance is not a valid cmd in either my lab environment or the production equipment. (4331 routers and C9300 switch). My goal is to end up with one default route at the core to use with ip sla and route tracking. I'm open to ideas on how to solve this.

WBT-CORE(config-route-map)#match ip add prefix-list RM-SET-DEFAULT-AD
WBT-CORE(config-route-map)#set distance 10
^
% Invalid input detected at '^' marker.

 

[M02@rt37]

Hello @davparker 

set distance command is available on NX-OS.

You want ISR have 2 default-route? One from PE (via eBGP) and the other (prefer via OSPF) from CORE ?

[paul driver]

Hello
Looking at your topology, The site with the ASR-ISRs is running OSPF as the IGP and has two internet breakouts
Each hanging off each ASR rtr, so shouldn't the defaults be originating from these rtrs and not the OSPF core?
As such this will then allow to further advertise the defaults into the ospf domain by differing metrics / metric-type for preference?

[M02@rt37]

@davparker 

On your topology,

ASR should send default route to CORE via iBGP and CORE redistribute default to ISR via OSPF, no ?

[paul driver]

Hello M02@rt37 
TBH I wouldn't even have the cores running BGP , just OSPF, and perform the redistribution of OSPF into BGP on the ASRs, and send a default into OSPF towards the core

[davparker]

Weird, so I shut down my lab last night and powered up this morning. The routing is working as expected now. The CORE is advertising the default route to the ASRs on the Global routing vrf. Must be an anomaly on migrating OSPF config. Maybe I should have cleared the routing processes on the CORE after making the OSPF changes. I'll have to put my scripts through the paces.

[paul driver]

Hello
TBH that doesn’t seem correct - maybe its just down to a race condition between bgp and ospf 
I would imagine it needs to be more deterministic than what you are describing.

[davparker]

Yeah, running into issues. I'm wondering if I can accomplish my objective using OSPF at the Core using this topology. If I allow a default route to get redistributed into OSPF from BGP on the ISRs it gets inserted when the Core loses the default route due to ip sla with tracking, When the default route gets reinserted at the Core, the two ISRs won't reestablish the default route to the Core, instead they hang onto the default route learned from BGP pointing to either one of the ASRs. (ISR1 peers BGP with ASR1, ISR2 peers with ASR2). If I clear ip bgp on the ISRs, the routes get inserted properly. I haven't given up yet.

[paul driver]

Hello
so what your are describing is a cfg issue-
can you post the run cfg of the ASRs and the core  ( in text files ) and attach you this post ?

[davparker]

So the ASRs each advertise a default route. It is there to be picked up by the ISRs when they lose a default route through the CORE. I kind of have it working. In the OSPF process, I use default-information originate metric 1 for CORE, metric 2 for ISR1 and metric 3 for ISR2. Everything works fine. Then I simulate a path failure leading from the CORE to break Internet. Ip sla kicks in withdrawing the default route at the CORE, ISR1 & ISR2 each insert the default route via each ASR. ISR1 has the better metric, so it is preferred as expected. When I enable the Internet path at the CORE again, ip sla sees that and the default route through the CORE gets enabled but the two ISRs keep the default route learned from the ASRs. They don't use the default route learned via OSPF originate. If I clear ip bgp * on the ISRs, everything resets and the OSPF default route gets installed. I may have a hard time sharing configs due to the amount of sanitation involved.