Multihomed eBGP load balancing with 3 ISP's

[grobinson23]

We currently peer with 2 ISPs using BGP in an active/failover configuration.  My company wants to move to a 3 ISP model where Internet traffic is split across the 3 providers so that bandwidth is equally distributed on outgoing traffic across our 2 /22 ARIN IP ranges.  This is from our 2 edge switches that have VSS.  

Within my limited knowledge of BGP, I have determined that we could do load sharing pretty easily by adding multiple default routes and breaking up our /22's into /24 and advertising them that way.  However, I don't think this satisfies the request that downtime must be seamless, should one link drop.  

Currently, our ISP's advertise default routes.  From the research that I've done, we could get close to load balanced links if we receive full BGP routes and community settings and definitions.  I'm nervous about this because it looks really complicated, and I don't want our AS to turn into a transit AS.  I've been told the same can be accomplished with only partial BGP routes and community settings and definitions.  

Personally, I think we just need a WAN load balancer.  However, given the request, is there a thread out there that can explain this, or can someone discuss this requested scenario a little bit?  

Thanks!

[alexander.richard]

Hi there

First question would be what is the required reconvergence time for the applications using the Internet? Should an outage occur, when do they lose their state? Once you know that, you then have a target to aim for in terms of recovery

With regards load-balancing, with BGP we are always talking inbound and outbound.

The outbound solution is relatively simple - each ISP advertises a default route to your Internet edge router(s). Create an eBGP session from each edge router to the core, advertise the default route and redistribute into the IGP. Ensure the IGP cost to each BGP next hop is equal and you have ECMP for outbound routing.

Inbound influence is usually via MED (not likely in this case given 3 ISPs), adjusting local-pref in the ISP via BGP EXT communities configured your end, or via AS-PATH prepending for longer prefixes from your /22. Prepending would be simplest, but your unlikely to get an exact inbound traffic split, however a relatively even distribution should be sufficient. 

[grobinson23]

Ok, so with multiple providers, simply prepend them the same and let the providers pick the best path for themselves for inbound.

In this case, the iCore switches ARE the edge routers.  Would the outbound solution look the same?

It looks as if default routes from the ISP's would do the trick.  We wouldn't need full BGP routes to accomplish 3 ISP's?  What about 4 or 5?

Thanks!

[alexander.richard]

So for outbound, yes, 3 defaults will work as long as the IGP cost to reach each of the BGP next-hops is the same (and of course your IGP supports ECMP) If, the link speeds were different you could use the variance command via EIGRP and tune the delay metric on each link to distribute the load according to ISP link speeds...

Inbound requires that you split the /22 into say 4 /24s, and advertise the /22 out to each ISP as well as a number of the /24s (as I said because you have 3 links but only 4 /24s, the traffic isn't going to be perfect, but you'll have at least some distribution). This method does't actually require AS_PATH prepending as longest match always wins.

[manish arora]

Hi G Robinson,

You can have multiple type of policies configured with BGP and you can manipulate how traffic goes out of your network and how it comes back into your network ( coming back into your network is a little hard :-) ). Now let's say you have 3 ISP, primary with 10 GB and other two with 5 GB each in commit bandwidth, given that's the case I have, I would personally have all my ISPs send me all the routes that originate in their AS plus one more AS along with the a default route. I will then  manipulate the default routes that ISP1 with 10GB commit installs as primary then failover to ISP2 and than to ISP1.

This will require some expertise in configuring route-maps with as-path regex filtering and changing other attributes. I would say set a mock lab or something before you touch the production.

As far as not becoming a Transit AS, again you use route-map in the bound direction only advertise your /22 networks which should already be configured in your old setup.

Manish  

[grobinson23]

Thanks!