eBGP Load-Balancing

[john.pepper]

Hi,

We have site with a pair of 2 x 6800 VSS switches connected to a separate carrier using two routers - both running eBGP.  See attached.

This carrier is passing us all our global BGP routes (5,000+) via their backbone as they have links into our two European Data Centres.  So we see the full BGP AS paths of all our routes coming into our 6800 VSS switch.  This all works fine.

The problem is we want load-balancing across the two carrier routers.  And whilst our 6800 switch is seeing dual routes for all destinations it's only installing the route with the shortest AS path into the routing table - so not all prefixes show 2 hops out.  This is due to the differing AS path lengths being learned via our two Data Centres. And BGP only installing one route in the routing table where the AS Paths are different.

What we would like to do is influence all BGP routes being sent from the carrier routers to our 6800.  So that routes learned by our 6800 see equal BGP attributes. And therefore installs both learned routes into the routing table.

We have looked at manipulating the AS path from our Data Centres into the carrier network.  But with 5,000+ routes sent this isn't very scalable.

Has anyone come across this issue and has some suggestions.  Would we be better looking at setting the Local Pref from the carrier routers to us.  Or is there a way of resetting the BGP attributes so they're equal when we learn them via BGP?

Appreciate any help.

Thanks

John

[Georg Pauwen]

Hello,

from your drawing it is not clear if your are multihomed to one or two ASs, Post the configurations of both your core routers...

[john.pepper]

Hi Georg,

Thanks for the reply.  

- Our Core switch is in AS 11.  This is multi-homed to the carrier AS 10.  So 'we connect to one' carrier AS#.

Doing some more reading, this command would seem to fix our issue?

bgp bestpath as-path multipath-relax

[Jon Marshall]

John

I did some testing with this yesterday using both the command you quote and also "bgp bestpath as-path ignore".

Neither worked for me.

The problem with the "bgp bestpath as-path multipath-relax" command is, as far as I know, it expects the AS path length to be the same just the AS numbers within the path can be different and this does not match your scenario as I understand it.

I thought the "bgp bestpath as-path ignore" command should have worked ie. simply ignore the AS path as part of the BGP best path selection, which is really what you need,  but it didn't.

I am wondering whether the maximum paths command require that the AS paths be equal length even if you use that command.

Of course it could be that I was using an emulator and not real equipment or I am misunderstanding something (more likely !) so it may work for you.

Jon

[john.pepper]

Hi Jon,

Appreciate the feedback.  We're already using the "maximum-paths 2" and see all dual prefixes.  It's just BGP path selection that's stopping putting them into the routing table.

I believe the "bgp bestpath as-path ignore" command does ignore the AS path.  But BGP will then choose neighbour with the lowest router ID.   So maybe why you only see one hop out?

So my thoughts are we try this.

access-list 1 permit any

route-map Carrier_Local_Pref permit 10
match ip address 1
set local-preference 200

router bgp 10 

neighbor 10.208.92.10 route-map Carrier_Local_Pref in

neighbor 10.208.93.10 route-map Carrier_Local_Pref in

bgp bestpath as-path multipath-relax
 maximum-paths 2

[Jon Marshall]

I'm not following how that will work.

If you set the local preference on all routes to be the same then the BGP best path selection just moves on and you are back to the same problem you already have.

What am I misunderstanding ?

Jon

[john.pepper]

Jon,

Yes, you're right.  Did you try that in your in your simulation?

And did you use both maximum-paths as well as bgp bestpath as-path multipath-relax in your tests?

[Jon Marshall]

I did not use local preference but I did use both "bgp bestpath as-path multipath-relax" and maximum paths and it still only chose one route.

I had one router peering to two EBGP neighbors. The neighbors were in the same AS but the networks they were advertising had different AS path lengths which is the issue you are facing I believe.

Like I say I think with that command the AS path length still needs to be the same just the actual AS numbers in the path can differ.

I may try again this afternoon and see if I can get either command working.

Jon

[john.pepper]

Thanks Jon,

Appreciate your feedback and any test results you get.

We plan on doing some testing in the 'live' environment over the next few days.

Once we get a fix I'll share it here as I can see a number of posts on the same issue.

[cofee]

I don't think using this command "bgp bestpath as-path ignore" will accomplish the goal of load balancing, because it will just push down the path selection to MED and I believe end result will be same. I think prepending the shorter AS PATH to match longer AS PATH won't help either because since AS PATH is the same it will go down the list of path selection.

You should be able to use either local preference or weight for half the routes to use the standby bgp peer that's not being installed in RIB and this way you load balance across both links. I can't think of a solution for equal path load balancing if that's what you are looking for.

Please correct me if my understanding of bgp path selection is incorrect.

[Jon Marshall]

MED won't come into it unless it is being explicitly set and there is no mention of that so all routes will have equal metric.

According to the documents if you use maximum paths then as long as the routes are equal up to and including step 8 in the BGP best path selection then multiple routes to the same destination (defined by the maximum paths number) can be installed in the IP routing table.

However I tested with "bgp bestpath as-path ignore" and it only picked one route and that route was chosen as far as I could tell on lowest BGP router ID which is step 10.

So I'm not sure why it did not work other than maybe if you use that command maximum paths does not work as expected.

Hope that made sense :)

Jon

[cofee]

Correct. What I meant by MED was that it will keep going down the list and even if MED was explicitly defined it won't accomplish the goal since it will choose one path over the other.

It may not even getting to step 9 and probably choosing one path over the other based on IGP metric which is step 8.

[Jon Marshall]

Yes basically you need to get past step 8 and the routes still be equal for maximum paths to install multiple routes into the IP routing table.

I actually may be being a bit stupid here because maximum paths only works if the routes are received from the same neighbour AS (unless you use the "bgp bestpath as-path multipath-relax" command).

So if you tell it to ignore the AS path then perhaps that is why maximum paths does not work.

Just a guess.

Jon

[john.pepper]

Cheers for the feedback guys.  Reading and other threads on this issue it seems to work for some people - but not always :-o

We have this type of solution at numerous other sites and it works fine.  The difference being we're using EIGRP on the LAN side then redistributing into BGP on the WAN side.  It works fine as EIGRP load-balances great across equal paths.  But in this case we're having to work with this eBGP setup and all the challenges that brings.

We'll keep working this and test some of the above in our Live environment.

Thanks...John

[john.pepper]

Thanks to all who contributed here.  Appreciated and what makes this forum a great resource.

FYI - we managed to resolve our issue over the weekend taking a few actions.

1. We removed any redistribution and BGP Pre-pend from our one DC.  So that meant all BGP routes flowed into the remote site with the same AS path length.

2. We configured the 'bgp bestpath as-path multipath-relax' on the core 6800 switches.  We then saw dual paths for each destination.  See example below (addressing changed for example):

B 172.16.44.0/25     [20/0] via 10.208.93.10, 1d00h
                                 [20/0] via 10.208.92.10, 1d00h

B 172.16.44.144/32 [20/0] via 10.208.93.10, 1d00h
                                [20/0] via 10.208.92.10, 1d00h

B 172.16.56.0/21    [20/0] via 10.208.93.10, 1d00h
                               [20/0] via 10.208.92.10, 1d00h

So providing the AS path lengths are the same (but not same AS numbers), the 'bgp bestpath as-path multipath-relax' works.

I did look at another fix that could possibly achieved the same outcome (but not tested).  And that was to configure 'neighbor X.X.X default-originate' on both the carrier routers.

Thanks again to all for the input and support.