Beginner BGP Preference question

[cco@leferguson.com]

I'm less than a novice at BGP, and helping a client.  They have two sites in separate counties that have both redundant equipment and paths between them.  Each site connects, separately, to an AT&T state run network that is semi-private (meaning it has both private and public addresses).  

The current vendor's firewall (some weird non-branded thing) connects to the AT&T network, and advertises BGP into AT&T from each end.  Each end advertises the opposite ends internal (yes, private) addresses.  I'll make up some addresses: 

Site 1: 
Local VLAN addresses 10.1.0.0/16
Advertised addresses: 10.1.0.0/16 and 10.2.0.0/16

Site 2: 
Local VLAN addresses: 10.2.0.0/16
Advertised addresses: 10.1.0.0/16 and 10.2.0.0/16

The result of this is that traffic from Site 1 to/from the VOIP connections AT&T is providing go like this: 

Site 1 originated path: 
10.1.x.y to AT&T goes out Site 1's firewall 
AT&T returns the connection to Site 1's firewall, to 10.1.x.y
Everything's happy and works.

Site 2 originated path:
10.2.x.y to AT&T goes out Site 2's firewall 
AT&T returns the connection to Site 1's firewall addressed to 10.2.x.y
Site 1's router forwards over internal connection to other end, and thence to 10.2.x.y

This means the return path for a site 2 originating session is asymmetric, returning through site 1.  This is working for many things but not some.  Note that NAT is not involved, but the firewalls are apparently not happy with a TCP session beginning on one and ending on another, as you would expect.  (What is not clear after literally days of discussion is why it was designed this way by the vendor). 

Here's the thing - they say they can't change their firewall to do this "right" (in any of the several possibilities), and are suggesting that the AT&T network connection move to our Cisco router, and we advertise BGP in a way that will work.

This implies I need to advertise (for example) 10.1.0.0 from Site 1, with a lower preference for 10.2.0.0 and the reverse from the other side.  Except my head is hearing reading about BGP and route preference inbound.

I THINK what I want to do is specific MED with a route map, and I see some indication how in this document. 

But I am really a bit gun shy as to whether this is a proper fix (notably because if it is, it would seem they could do it in their firewall now (to which we have no access, by the way)). 

I should also note this is a critical system that while highly redundant, shifting the BGP advertisements could take down all portions at once, so I cannot experiment practically (though I have asked to set up a separate pair of test connections for that purpose). 

To the extent it matters, eigrp is runing in the internal network between and among sites 1 and 2, and we have complete control over the transport portions of that network, just not the voip systems and that firewall at each end, which another vendor maintains (the vendor that has more or less thrown up their hands and given up). 

Bottom line -- is MED the correct path (pun intended) I should be pursuing?  If not, can you give me a pointer to continue my research?

Linwood

[cco@leferguson.com]

IGP is not, EIGRP is being used internally (there's a very large network also attached to the Cisco's, but not involved in this application, and eigrp routes all throughout it)

yes, the topology looks correct.  Does this require the AS for each site be different?  (That's one thing I do not know yet). 

Thank you!

[MHM Cisco World]

Both site use eigrp?

AT&T advertisers default route to both site? 

For AS no need to be same. 

MHM

[cco@leferguson.com]

Yes, both sites use the same EIGRP instance.

I actually do not know what AT&T is advertising into the firewalls.  The Cisco's on each side are not exchanging any routing protocols with the firewalls (or via them AT&T) at present.  For the Cisco's, the Firewall is the default gateway (static).  For the firewalls, the adjacent Cisco is the static route to 10.0.0.0.  This is the vendors design, we had no say in it.

This means that right now if the vendor's system on (for example) site 2 needs to communicate to AT&T and the Site 2 connection is down, it cannot send outbound.  It can receive via Site 1 and routing over the Site 1 -> Site 2 link, but there is no path from Site 2's firewall to AT&T if the Site 2 connection to AT&T is down.

There is no good reason for that, but it was the vendor's design.  As I understand it, the application at Site 2 notices the situation and changes how it communicates in some fashion, an application level not IP routing level failover.  This is another aspect of this setup that just makes no sense to me.  But it is not causing a functional problem, or so I am told. 

I am told this is a configuration widely used in this government application, but to me this looks like a design done by someone who didn't understand that routers should ... well, route.

But thank you @MHM Cisco World , this helps.  I'm hopeful we can bring AT&T into this.  And maybe the state that oversees this vendor and have a multi-way discussion to better understand.

But I think you have answered the core problem -- that if we took this over and into the Ciscos, we CAN make it work directly. 

[paul driver]

Hello

---

cco@leferguson.com  wrote:
**IF** we redo the connection to cut these firewalls out, clearly AT&T is going to have to work out how we peer with them, AS numbers, routes, etc.  But the core of the question before we want to start down this path is, if we do that, are we going to be able to properly give a preference so that Site 1's return traffic goes to Site 1, and same for Site 2, but each will fail over to the other (for return traffic -- outbound we can handle). 

Us taking over BGP was their suggestion, not ours.

---

Taking over bgp would be applicable as a result it would allow you to be able to provide preference to each site, but you obviously need to make some changes.

At a high level perspective, its a quite a simple approach tbh

- run ibgp between the fws and the ciscos
- Change the link between each site cisco rtrs to bgp ( assumption is each site runs their own bgp ASN)
- redistribute eigrp into bgp
- redistribute default routes from bgp into eigrp ( higher metric from the cisco rtrs)

The result will be:
Each site will run eigrp internally but peer to each other via ebgp.
By default, you would need NOT to perform any path preference and all s2s traffic will run between the ciscos as per bgp best path section process.
If either site experienced wan outage, they would advertise their own internal routes and receive any external routes and a default from the to/from the other site and still have internal/s2s and external reachabilty