BGP - Can you split/prefer advertised block between multihomed routers

[bernardssupport]

We use BGP and have our own ASN and have a class C block.  Currently our block 172.X.X.0/24 is broadcast to both ISPs on separate edge routers for each ISP.  We are currently in the process of upgrading one circuit (ISPA) as it is getting saturated with traffic.  ISPA gets all the incoming traffic as it has a shorter AS_PATH compared to ISPB for the majority of incoming traffic. The majority of the traffic is on one IP (172.X.X.5). Is there a way to split the block using prefix-list or route-maps to have half the block be advertised on one ISP and the other block on the other ISP but still maintain failover redundancy? Here is a simplified and edited example of our config.

ROUTER_A
router bgp 72345
 bgp log-neighbor-changes
 network 172.X.X.0 mask 255.255.255.0
 network 12.X.X.0 mask 255.255.255.0
 neighbor MYCOMPANY peer-group
 neighbor MYCOMPANY remote-as 72345
 neighbor MYCOMPANY update-source GigabitEthernet0/1
 neighbor MYCOMPANY next-hop-self
 neighbor ISPA peer-group
 neighbor ISPA remote-as 7123
 neighbor ISPA description ISPA Link
 neighbor ISPA update-source GigabitEthernet0/0
 neighbor ISPA version 4
 neighbor ISPA send-community
 neighbor 12.X.X.8 peer-group ISPA
 neighbor 172.X.X.1 peer-group MYCOMPANY

ROUTER_B
router bgp 72345
 bgp log-neighbor-changes
 network 172.X.X.0 mask 255.255.255.0
 network 209.X.X.0 mask 255.255.255.0
 neighbor MYCOMPANY peer-group
 neighbor MYCOMPANY remote-as 72345
 neighbor MYCOMPANY update-source GigabitEthernet0/1
 neighbor MYCOMPANY next-hop-self
 neighbor MYCOMPANY prefix-list DEFAULT-ROUTE in
 neighbor ISPB peer-group
 neighbor ISPB remote-as 7124
 neighbor ISPB description ISPB Link
 neighbor ISPB update-source Vlan1
 neighbor ISPB version 4
 neighbor ISPB send-community
 neighbor 209.X.X.48 peer-group ISPB
 neighbor 172.X.X.253 peer-group MYCOMPANY

[Joseph W. Doherty]

Can you split you block?  Technically, yes (as Julio shows), but your ISPs may balk at accepting a prefix smaller than a /24.

You might want to discuss this issue with your ISPs.  They might suggest another way to deal with your link load issue.

[paul driver]

Hello

in addtion to Julio comments you can also manipulate on your incoming ebgp prefixes from either ISP to influence Ibgp outgoing traffic using local preference Path attribute -

As the weight pa in this case would not a viable option 

res

Paul

[Julio E. Moisa · Accepted solution]

Hi

Yes you can split the block and have redundancy, for example:

You could use AS-Prepend to advertise the prefixes to the IPS's, Please let me create an example:

ACL or Prefix list can be used, the divided routes should be known on both routers, displayed on the routing table and advertised on BGP with the subnet masks.

ROUTER 1

access-list 10 permit 192.168.10.0 0.0.0.127

access-list 11 permit 192.168.10.128 0.0.0.127

Route-map ISP-A permit 5
match ip address 10 
set as-path prepend 100

Route-map ISP-A permit 10
match ip address 11 
set as-path prepend 100 100 100

Route-map ISP-A permit 100

router bgp 100
neigh 1.1.1.1 remote 1
neigh 1.1.1.1 route-map ISP-A out

ROUTER 2

access-list 10 permit 192.168.10.0 0.0.0.127

access-list 11 permit 192.168.10.128 0.0.0.127

Route-map ISP-B permit 5
match ip address 10
set as-path prepend 100 100 100

Route-map ISP-B permit 10
match ip address 11 
set as-path prepend 100

Route-map ISP-B permit 100

router bgp 100
neigh 2.2.2.2 remote 2
neigh 2.2.2.2 route-map ISP-B out

This will help you split the prefixes and advertise that to the ISP's. Now if you want to receive routes from the ISP, Weigh, local preference can be used. 

[vcjones · Accepted solution]

A few quick comments...

1- Inbound load balancing and outbound load balancing are independent issues, and each requires its own solution. AS Path prepending only adjusts inbound traffic and it's effectiveness is dependent on how your upstreams are configured and how they connect to the rest of the Internet.

2- Basing your outbound routing on accepting a default path from your upstream provider is only useful for detecting a problem with your link to that provider. Most providers do not make that advertisement conditional on their BGP router having useful connectivity to the rest of the world, or even the rest of their network.

2.1- Be careful how your default routes are distributed internally...can you survive outbound packets arriving at your router with the failed uplink? If not, you need to configure it to not redistribute the default route it learns via IBGP from your other uplink's router.

3- Depending upon your firewalls, and where they are placed, and the rules they enforce, you may need to ensure that routing is symmetrical (i.e., all traffic returns via the same ISP it initially arrived or got sent out via.)

4- If your are using firewalls, are they a single point of failure? Even if not, can existing conversations survive the routing changes required if one of your ISPs fails?

FWIW: there are solutions to all these issues, and multiple others which simply didn't come to mind in the few minutes I devoted to this question, but they all have side effects which are usually neglected until they are painfully exposed by failure of the expensive redundant solution to maintain communications when real world failures occur.

Good luck and have fun! Just remember the bottom line: robustness vs. load sharing vs. cost/complexity is a tradeoff, and you only get to pick two out of three.

P.S.

I wrote a book about high availability network design over a decade ago (long since out of print), and other books have been written as well. The fundamental issues have not changes as technology has evolved, nor has the tendency to ignore the fundamentals and merely provide the illusion of redundancy.

[bernardssupport]

Thanks for the info. Yes redundancy works in a failover scenario. I am currently channeling outbound traffic thru ISPB (local_prefernce). My problem is how can I shape the inbound traffic (if at all possible – noting I have already tried AS-path and MED) to force whatever I can over ISPB when ISPA has the preferred routes to external faces customers.

[Masoud Pourshabanian]

Hello,

You can also use conditional advertisement.

Send 192.168.1.0/25 normally and 192.168.1.128/25 by a condition to ISPA 

Send 192.168.1.128/25 normally and 192.168.1.0/25 by a condition to ISPB

Condition :

On router connected to ISPB

Choose a chunk of IP address belonged to ISPA.

neighbor ISPB advertise-map 192.168.1.0/25(route-map) non-exist-map [chunk of IP from ISPA in a route-map]

It means you only advertise 192.168.1.0/25 to ISPB when you do not receive that chunk of IP from ISPA

How you make sure that chunk comes from ISPA? You can add a community when you receive that chunk from ISPA  (inbound)  and match it in your non-exist-map

You do the same on the router connected to ISPA.

Hope it helps,

Masoud

[vcjones]

While technically correct, all "solutions" based on splitting the allocated prefix will only work on the public internet if the pieces are all at least a /24. No ISP is going to propagate a /25 or longer to their upstreams or peers, and any decent ISP is going to filter your advertisements so that only your prearranged prefixes will be accepted (so you can't claim to be the best route to someone else's prefix).

If prepending is not working, and it is the ISP who provided your public prefix, they are probably aggregating your prefix into their block that it was allocated from, so all peers and upstreams will route via the longer prefix coming through the other ISP. Check on a looking glass site to see what your prefix advertisements look like outside your ISPs.

Solutions:

1- Talk to your provider and if they are aggregating, ask them not to aggregate your prefix. Then prepending should work.

2- If they won't/can't, you can do a conditional advertisement to your backup ISP so that your longer prefix only escapes through them when you need it to. Note that this will slow down failure recovery, but since typical BGP recovery is in minutes, a extra minute or two is not big deal. You will have to put in some safeguards to protect against a bouncing primary link dampening your backup path.

3- Talk to both ISPs to see what services they provide BGP users, they may be able to do the conditional advertising for you, speeding up recovery.

Good luck and have fun!

[bernardssupport]

Looking Glass site! Geeze sometimes the answers are so simple.  After pulling up our block on a looking glass site it was revealed that ISPB wasn't announcing any routes! What! Turns out they had a conditional statement on there side not to advertise our routes unless ISPA was unreachable. This was inserted 10 years ago when our circuits were T1's and DSL's. They removed the statement and we now have working BGP multihomed routers. Thanks for everyone's help and advice. 

[bernardssupport]

I marked this as the correct answer because it directly answers my question. Unfortunately the as-path prepend method doesn’t work. I have tried both AS-path prepend and MED (metric) setting on the entire block to no avail. All traffic seems to be preferred over ISPA regardless. The majority of my traffic problems stem from one internal IP on my 172.X.X.X block. Is there a way I can filter out that one address from being advertised to ISPA (I know this would kill redundancy)?

[walter baziuk]

NO_ADVERTISE is intended to prevent a given BGP speaker from propagating the route any further.

So in this case adding this attribute to given route advertised to a peer will actually prevent the route from being pushed to the remainder of the carrier's network. You might want to consider