Floating Static route (Backup path) redistributed to OSPF>BGP not being able to get back to primary path once remote site goes up

[joseponceiii]

Hello guys,

Wondering if someone can help with the current scenario. To explain further, I'll try to summarize in each normal and backup condition. I'll just put an example of just (2) sites; but in reality there are other remote sites connecting to DC.

Under normal operation:

1) Remote Site 1 (single WAN router) is advertising 192.168.1.0/24 to the MPLS cloud using BGP. 

2) DC (single WAN router) is receiving the subnet 192.168.1.0/24 from the same MPLS cloud then redistributing this route to the OSPF. 

3) At DC LAN segment, the core switches (N5Ks) are using this route with no problem. 

When Remote Site 1's ISP goes down:

1) DC won't be able to receive this subnet anymore in the MPLS cloud;

2) At DC N5Ks, there is a floating static (ip route 192.168.1.0/24 <next hop IP> 200) which will be triggered once the original route/OSPF route goes down, and this static route needs to be redistributed back to OSPF (already configured under prefix list and redistribute static route-map) so the DC's WAN router will be able to advertise this subnet out of the MPLS, as there is no other path available for this subnet - basically other remote sites should take this DC's path once Remote Site 1 MPLS provider goes down.  This next hop in static route is pointing to our IPSec VPN router. 

3) At Remote Site 1, this subnet will also be at the path of IPSec tunnel, which is expected and working normally.

Now, here's the real problem. Once normal condition, then Site 1 goes down - it would work with no problem. However, when site 1 goes up, DC WAN router is still advertising this subnet since the static route in the N5K has been triggered thus the redistribution from static to OSPF is still there, making it advertise to the MPLS cloud (OSPF to BGP redistribution). 

The only way to resolve this is to remove the redistribution from static to OSPF in N5Ks, then put it back (so once the remote site 1 goes down again, other remote sites would be able to reach this subnet via DC path).

Is there any solution you could suggest? Apologies if the explanation is not clear. 

Thanks in advance.

[rais]

When backup path is redistributed into OSPF it should have a higher cost than the one redistributed from BGP into OSPF.

Similarly, route advertised from original remote site should be preferred by the ISP.

HTH

[paul driver]

Hello

---

@joseponceiii wrote:

However, when site 1 goes up, DC WAN router is still advertising this subnet since the static route in the N5K has been triggered thus the redistribution from static to OSPF is still there, making it advertise to the MPLS cloud (OSPF to BGP redistribution). 

---

I think what is occurring here is a race condition between an ebgp learned route taking preference due to its appended redistribution bgp weight path attribute.

So in this instance its possible before the remote site 1 fails an ebgp route is learned from remote site 1 for 192.168.1.0/24 with an admin distance of 20 (no weight value) However upon a remote site 1 failure, The redistributed static from ospf-bgp is learned and this ebgp route will now have a weight value appended to it in the bgp rib table so as/when the remote site 1 comes back up, the DC wan rtr will see two ebgp routes one with and one without the weight path attribute and by default the bgp selection process will always choose the weight first over admin distance which in this case is the redistributed ospf-bgp static route.

To negate this race condition, what you could do apply a default weight to all ebgp learned routes from the remote site 1 bgp neighbour with an higher weight value of 32768

[georgehewittuk1]

How high is the metric on the configuration for the redistribution of the static floating route into ospf? By default if nothing is set it is a cost of 20.

If the routes are both a metric of 20 as both have not been configured with a metric and assuming they are both E2 (default type) the next determining factor will be the metric to the advertising router (i.e. the router that is performing the redistribution) & that is choosen as best path and installed in RIB.

[MHM Cisco World]

I have a Idea Here but you must check it, 
there are two Route one from Site1 and other from DC 
so let config two route 
in DC have short prefix and in Site1 have long prefix, 
this way all site and DC have two route and it prefer forward the traffic toward the Long prefix not short prefix.
now if the route is missing then other Site and DC will automatic select the route from DC which is short prefix and this way there is no need redistribute and re-redistrubte again.

this My Idea for this issue.

[georgehewittuk1]

That will sort it