BGP Redundancy how to best handle broken paths between BGP Endpoints

[NPT_2]

We have 2 geographically diversified Internet connections with separate providers at 2 separate ends of our network.  The edge routers connecting to each internet providers are advertising all routes via BGP with preference given to the higher bandwidth ISP for BGP Advertisements for each network. 

There are approximately a 7 hops (and routers) between the 2 edge BGP routers on the shortest path over our network running OSPF on the routers other than the BGP Edge routers which run both BGP and OSPF. 

The problem I am trying to solve is that if the link between the OSPF router on the internal network and the Edge BGP router goes down both BGP routers still advertise all routes causing connectivity issues for many of the networks as both BGP routers are advertising all networks since their connection to the upstream ISP's is still up and operational. 

One edge router has redundant paths to the core, but due to location and costs the far end Edge BGP router has only a single path on the last couple hops. 

What is the best way to address this situation.  I have came up with a couple ideas, but would like some feedback on these and possible other options. 

1.  Run iBGP on all the internal routers that are now running OSPF and either fully mesh the routers (very large number of peers) or use route reflectors.  This is not something that I would like to do since it requires quite a bit of complexity that I am hesitant to jump into unless it is definitely the best route to go.

2.  Setup rules/scripts/etc. on the remote Edge BGP router to ping the network core router (where redundant links start), and if the pings fail shutdown the BGP process or pull the BGP routes from being advertised to the upstream ISP until connectivity is re-established. 

3.  Remove static BGP advertisement of networks on the Edge BGP routers and instead redistribute OSPF Aggregates dynamically from the core network into the Edge BGP routers that way if the network gets segmented the OSPF routes will not be available and automatically be pulled from redistribution into BGP.  Thinking this might work quite well, but not sure if it has any potential downsides.

What do you think of these options, or do you have any other thoughts or suggestions? 

The simplified connection (not including redundant links is as follows:

ISPA-------(EDGEBGP/OSPF RTR)-------(OSPF RTR)-------(OSPF RTR)-------(OSPF RTR)-------(OSPF RTR)-------(EDGEBGP/OSPF RTR)-----ISPB

Jim

[Joseph W. Doherty]

Disclaimer

The Author of this posting offers the information contained within this posting without consideration and with the reader's understanding that there's no implied or expressed suitability or fitness for any purpose. Information provided is for informational purposes only and should not be construed as rendering professional advice of any kind. Usage of this posting's information is solely at reader's own risk.

Liability Disclaimer

In no event shall Author be liable for any damages whatsoever (including, without limitation, damages for loss of use, data or profit) arising out of the use or inability to use the posting's information even if Author has been advised of the possibility of such damage.

Posting

For #1, you would have iBGP insert routes to avoid null statics?  But if so, you still have the issue you'll want to aggregate?

For #2, another approach might be to key on next downstream device (where redundant links start) routed loopback in a route map and redistribute statics into BGP if it's seen.  Perhaps a little simpler than using custom scripts.

For #3, one downside might be, depending on your topology, losing access to interior OSPF that's on same side as "break".  You would also have to be careful how you generate summaries, i.e. you don't want one /30 to incorrectly generate a summary for a larger subnet that's not reachable.

A possible 4th solution is to run a GRE tunnel between your two edge BGP/OSPF routers, on the ISP side, and run OSPF across the tunnel (with high cost).  You may have traffic "needlessly" transit the edge BGP router which has lost some or all interior OSPF connectivity, but if that's a concern, it could be mitigated with something like number 2 or 3, above.