BGP Conditional Advertisement Feature

Dennis Mink · ‎03-06-2018

This is actually a pretty cool feature, i didn't even know it existed until I was looking for a solution to advertise a subnet (prefix in BGP talk), only if a certain condition existed. This is exactly what conditional advertisements does

"The Border Gateway Protocol (BGP) conditional advertisement feature provides additional control of route advertisement, depending on the existence of other prefixes in the BGP table."

I am assuming, for those who want to read this post, that you have some understanding of BGP and its use of prefix-lists and route maps, otherwise this post might be hard to understand. Mind you, conditional advertisement is part of the CCIE R&S exam.

So let me go straight to the scenario:

So the routers under my admin domain are BEN and IBM. My primary router is BEN and my public IP range I am advertising is 203.11.11.0/24.

My two ISPs are Telstra and Next.
BEN has an eBGP neighbour with Telstra,
IBM has an eBGP peer with Next.
Then BEN and IBM from an iBGP neighbourship.

Nothing new so far. Now I have found that when advertising out the same public IP address (prefix) towards 2 different providers, even with AS path prepend, trying to make one ISP more preferable over the other, is highly unpredictable. This is because some providers prefer other providers no matter how often you AS prepend the crap out of your public prefix. This can cause asynchronous routing where your exit path is the primary ISP and entry through your secondary router. So I was looking for another solution; only route my public IP addresses out to the backup provider (Next in my case), in the event the primary fails. Or even better; fail over when the primary ISP stops advertising a default route into my organisation through the primary router.

In order to put all this in place, most, if not all configuration is done on the secondary router; IBM, so lets dive in.

As you can see below, the secondary internet router (IBM) has 2 default gateways

IBM#sh ip bgp topology *

For address family: IPv4 Unicast

BGP table version is 26, local router ID is 160.100.100.231

Status codes: s suppressed, d damped, h history, * valid, > best, i - internal,

r RIB-failure, S Stale, m multipath, b backup-path, f RT-Filter,

x best-external, a additional-path, c RIB-compressed,

Origin codes: i - IGP, e - EGP, ? - incomplete

RPKI validation codes: V valid, I invalid, N Not found

Network Next Hop Metric LocPrf Weight Path

*>i 0.0.0.0 203.11.11.5 0 200 0 3000 i

* 160.100.100.230 100 0 4000 i

The most preferred on comes from the BEN router, which in turn is being advertised by the Telstra Router (23.23.23.113). Initially I was going to use ip sla tracking on the IBM router to advertise 203.11.11.0/24 out if BEN lost the connection to Telstra, but this is not as fool proof as checking if the default gateway is still being advertised by BEN, because if my primary internet router no longer sends a default route 0.0.0.0 to my secondary internet router, the either my primary router is down, the link to Telstra is down, or Telstra is for some other reason no longer advertising a default route.

OK so on my IBM i set up a conditional advertisement to my Next BGP peer:

router bgp 5000

address-family ipv4

neighbor 160.100.100.230 advertise-map ADVERTISE non-exist-map NON-EXIST

what this means is that route map ADVERTISE is being invoked when the condition in route map NON-EXIST no longer exists.

route-map ADVERTISE permit 10

match ip address 60

route-map NON-EXIST permit 10

match ip address prefix-list TEST

match community 1

So the ADVERTISE route map is the easy part, it constitutes our public IP prefix 203.11.11.0/24

access-list 60 permit 203.11.11.0 0.0.0.255

the NON-EXIST route map is the condition that needs checking, and has in fact two conditions in it; it checks the prefix for a certain community and it checks if the actual prefix is available in the BGP table:

ip prefix-list TEST seq 5 permit 0.0.0.0/0

The reason there are two conditions, is that (refer to the sh ip bgp topology * output above), there are two 0.0.0.0 prefixes in the table; one from each provider. Now I am only interested in checking one of them; namely the one that comes from BEN 203.11.11.5. I though it would be easiest to add a check for a certain community in (although AS path would have worked as well).

ip community-list 1 permit 362000

So basically this second condition check to see if the route has 362000 as the community.

You can check the route to see if the community attribute is set and has the correct value. see below

IBM#sh ip bgp 0.0.0.0

BGP routing table entry for 0.0.0.0/0, version 25

Paths: (3 available, best #1, table default)

Not advertised to any peer

Refresh Epoch 1

3000, (received & used)

203.11.11.5 from 203.11.11.5 (203.11.11.5)

Origin IGP, metric 0, localpref 200, valid, internal, best

Community: 36200

rx pathid: 0, tx pathid: 0x0

Refresh Epoch 1

4000

So at this stage both conditions should be met; a) a default route in the BGP table and b)a route with community attribute 36200. So our public prefix 23.11.11.0/24 should NOT be advertised out IBM to Next. To verify:

IBM#sh ip bgp nei 160.100.100.230

BGP neighbor is 160.100.100.230, remote AS 4000, external link

---<output omitted>

Condition-map NON-EXIST, Advertise-map ADVERTISE, status: Withdraw

---<output omitted>

As you can see the conditional advertisement states "withdraw" which means the condition to start advertising is not met; ie.e we have a valid default route coming from BEN. So let me break something to trigger the condition to change. For this I will shut the connection between Telstra and BEN. (Remember BEN does not originate 0.0.0.0, its receives it from Telstra and as soon as that link breaks, it should no longer receive a default route either).

when debugging routing on IBM:

IBM#

*Mar 7 04:45:48.908: RT: updating bgp 0.0.0.0/0 (0x0) :

via 160.100.100.230 0 1048577

*Mar 7 04:45:48.915: RT: closer admin distance for 0.0.0.0, flushing 1 routes

*Mar 7 04:45:48.919: RT: add 0.0.0.0/0 via 160.100.100.230, bgp metric [20/0]

as you can see the 0.0.0.0 from BEN gets purged from the bgp table. and consequently the conditional advertisement kicks in:

IBM#sh ip bgp nei 160.100.100.230

Condition-map NON-EXIST, Advertise-map ADVERTISE, status: Advertise

To double check this, we check what routes the IBM router is sending to Next:

IBM#sh ip bgp nei 160.100.100.230 advertised-routes

BGP table version is 13, local router ID is 160.100.100.231

Status codes: s suppressed, d damped, h history, * valid, > best, i - internal,

r RIB-failure, S Stale, m multipath, b backup-path, f RT-Filter,

x best-external, a additional-path, c RIB-compressed,

Origin codes: i - IGP, e - EGP, ? - incomplete

RPKI validation codes: V valid, I invalid, N Not found

Network Next Hop Metric LocPrf Weight Path

*> 203.11.11.0/24 203.11.11.1 0 32768 i

Any questions, drop me a line.

Namaste

https://www.cisco.com/c/en/us/support/docs/ip/border-gateway-protocol-bgp/16137-cond-adv.html

Jeroen Huysmans · ‎03-12-2018

Hi Dennis, thanks, I was unaware of this option...

But, what happens if your primary eBGP peer is isolated in your ISP's network...?

Surely you can trust that they would no longer be advertising a default route towards your AS, but what if they would still be advertising a default route towards your network...?

You're public ip-space would be isolated, because your route-maps condition is still valid: there is a default route received from your primary ISP so you're not advertising your public ip-space towards the "backup" ISP...

regards, Jeroen

AARON WEINTRAUB · ‎03-13-2018

So the routers under my admin domain are BEN and IBM. My primary router is BEN and my public IP range I am advertising is 203.11.11.0/24.

My two ISPs are Telstra and Next.

BEN has an eBGP neighbour with Telstra,

IBM has an eBGP peer with Next.

Then BEN and IBM from an iBGP neighbourship.

Nothing new so far. Now I have found that when advertising out the same public IP address (prefix) towards 2 different providers, even with AS path prepend, trying to make one ISP more preferable over the other, is highly unpredictable. This is because some providers prefer other providers no matter how often you AS prepend the crap out of your public prefix. This can cause asynchronous routing where your exit path is the primary ISP and entry through your secondary router. So I was looking for another solution; only route my public IP addresses out to the backup provider (Next in my case), in the event the primary fails. Or even better; fail over when the primary ISP stops advertising a default route into my organisation through the primary router.

Going the long way around - what is the issue you see with asynchronous routing? It happens all the time in the public internet nowadays and shouldn't be cause for any concern at all in this example.

Conditional advertisement is a good tool, but I'm not completely sure that it's the right tool for this situation.

Dennis Mink · ‎03-20-2018

@Jeroen Huysmans you are absolutely right, if and when the default route is still being advertised to the primary router, whist there is a problem with the primary provide, indeed the conditional advertisement would not kick in. This is not so much a flaw but a limitation. the same applies when the two routers loose their iBGP neighbourship and the whole thing becomes splitbrain.

@AARON WEINTRAUB. Aaron. there is not per definition a problem with asynchronous routing in this scenario. asynchronous routing does rely on more paths than when your public range is advertised out of one router at a time. The illustration is provided is by no means a silver bullet, but should be considered as an alternative to AS path prepend.

matthewyauch · ‎03-23-2018

@AARON WEINTRAUB

If the primary circuit is a line-rate circuit (it's 1 Gb with no rate-limit), however your backup circuit is say a 100 Mb "burstable" to 1 Gbps, you will pay for any traffic on your backup connection beyond 100 Mbps. All connections being equal there's no issue with asynchronous routing, but sometimes the backup cannot handle the traffic whether for a technical or desired financial reason.

cheeseburger · ‎03-27-2018

Something I thought of, what sending community to the ISP eBGP peer and having them update the local pref for your routes to lower than what they're receiving from an eBGP peer at a connecting ISP. For example:

Send community xxx:50 to NEXT ISP and then all the routes matching will have the local pref set to 50 vs whatever their default is for the ISP connected. So, if traffic enters the NEXT ISP AS, the preferred route will be getting to TELSTRA somehow (I think ISPs generally use Local Pref of 70 between them??? Need an ISP person to confirm that but that's what I heard) and then if that BGP route somehow dies through TELSTRA, the best path selection process will recalculate and traffic will then come in the backup link in your AS.

Now, I've only been in networking for about a year and a half so please correct me if I'm missing something. I think this is how it would work though. I've never used the conditional advertisement feature so this was pretty cool to read about and I will definitely lab it up! I love BGP, it's so fun! :)

matthewyauch · ‎03-27-2018

@cheeseburger

That is actually the standard way of configuring a redundant circuit.

Providers usually have a 'customer default' for local preference to their customers. Say 100. This means that any traffic between their customers does not leave their AS if they have a direct link.

They also have a 'peer default' (example: 80) that they set all for all routes received from other ISPs.

Then they usually have communities to set your local preference, say from 70 to 90. This would allow you to set it to 70 and prefer another route, even if it arrives from another ISP (local pref 80).

The problem is that not all ISPs have communities for you to use to set local preference, nor do they tell you how their BGP is setup. There's also nothing stating how any ISP has to setup their BGP, so we have one provider where the default is 140 and you can set it from 90 to 150. This requires communicating with each provider to understand their setup and how you can tweak it to work for you. A lot of times calling them to get this information I get someone that knows less about BGP than I do.

For the providers that don't give any BGP tweaking control these conditional advertisements may be the only way outside of prepends to affect your routing at all.

sudkamt · ‎04-04-2018

In My case, I do not have BGP peering between my CE router, what will be a solution in that case?

Dennis Mink · ‎04-04-2018

sudkampt, do you mean you have 2 ISPs and two routers? but no BGP to your ISPs?

ar_djp · ‎11-04-2018

Is there a way for multiple route advertisement and conditions?

example:

advertise IP1 if IP100 exists

advertise IP2 if IP101 exists

advertise IP3 if IP102 exists

tellis002 · ‎11-20-2018

Thanks for the overview. I was aware of this tool, but you helped with clarification as well. Another tool in the network bag in corner case situations.