prefix from 3 different sources not installed according to ad

[Alexander Gaertner]

on ISR4451-X/K9 with isr4400-universalk9.03.13.03.S.154-3.S3 the following scenario:

there's 3 sources for the prefix 10.68.0.0/16, EBGP with default ad of 20, OSPF with artificially reduced AD of 15 and a static route with artificially increased ad set to 250

problem is as soon as OSPF route disappears static route is installed and is never superseded by EBGP route afterwards which is what I would expect even if static route would take effect faster than EBGP for some internal reason, like daemon<->kernel communication lagging behind already present static route.

FYI if I tie the static route to a track object for a different prefix form the same sources with a delay up/down of 1 sec it works as I expect it should.

meaning if OSPF drops, EBGP prefix is installed, once OSPF returns it re-installs it, if both drop static is installed and undone once track object route is re-installed from a dynamic source.

is there something I'm missing that explains this behavior?

[Georg Pauwen]

Hello,

post your configuration...

Is the eBGP route locally originated ?

[Alexander Gaertner]

>Is the eBGP route locally originated ?

no received over MPLS Core.

>post your configuration...

I had to redact it so heavily I doubt it'll be useful. Besides the BGP<->OSPF interaction is working as desired on thousands of devices including this one, it's the static route that is the only meaningful deviation and that introduces the unexpected behavior.

router ospf $pid
router-id $id
auto-cost reference-bandwidth 10000
area 0 authentication
redistribute bgp $as metric 5000 subnets route-map BGP-TO-OSPF
passive-interface default
no passive-interface $lanif
network $ip 0.0.0.0 area 0
neighbor $ip cost 5000
neighbor $ip cost 5000
neighbor $ip cost 5000
default-information originate metric 5000
distance 15 $ip 0.0.0.0 80
distance 15 $ip 0.0.0.0 81

router bgp $as
bgp log-neighbor-changes
redistribute connected route-map REDISTRIBUTE-CONNECTED
redistribute static route-map $as-TAGGED-STATIC
redistribute ospf $id match internal external 1 external 2 route-map OSPF-TO-BGP
neighbor $ip remote-as $MPLS-as
neighbor $ip description - $desc
neighbor $ip fall-over bfd
neighbor $ip next-hop-self
neighbor $ip send-community
neighbor $ip allowas-in
neighbor $ip route-map $IN in
neighbor $ip route-map $OUT out

route-map BGP-TO-OSPF deny 10
match community 20
route-map BGP-TO-OSPF deny 20
description - $desc
match community 50
route-map BGP-TO-OSPF deny 25
match ip address prefix-list FILTER-TRACK-OBJECT
route-map BGP-TO-OSPF permit 30
set tag 150

route-map $OUT permit 10
description - $desc
description $desc
match ip address prefix-list $acl
set as-path prepend $as $as
route-map $OUT permit 20
description $desc
match ip address $acl
set as-path prepend $as

ip route $ip $nm $nh 250 tag $as name $name

workaround that works:

ip route $ip $nm $nh 250 tag $as name $name track 2

track 2 ip route $ip $nm reachability
delay down 1 up 1

alternative workaround that would work is implementing static route with a different tag and redistributing static in OSPF when matching tag. 

I'm more interested in why it doesn't work as it should than in a nice workaround though.

[Jon Marshall]

If you are redistributing the static route into BGP then I suspect this is a race condition and it is to do with BGP weight. 

A route received from an EBGP peer has a weight of 0 whereas a locally generated route (which would include your redistrbuted static) has a weight of 32768 and the highest weight wins. 

So what happens is that the OSPF route is withdrawn which means there is no route in the IP routing table. What looks to be happening is that the static route is then entered into the IP routing table and then redistributed into BGP and because of the higher weight it takes precedence over the EBGP learned route. 

If you introduce a delay to the static route then this allows the router to install the EBGP learned route into IP routing table and so the static route is then ignored because of the higher AD. 

The usual way around this is to apply a weight to EBGP learned routes (or just that specific route) and make that weight > 32768 so it should then install the EBGP route although obviously not suggesting to do this on your live network. 

All the above is a best guess without testing it myself. 

Jon

[Alexander Gaertner]

I'm an idiot. It was indeed a race condition as you had anticipated. Thank you very much for your help.