BGP Multihoming with two different ISPs (Load sharing)

[Mikey John]

Hi,

I need to configure Multihoming (Load sharing) with two different ISPs. I have 200M going into each of these ISPs, and Iam receiving Partial Internet routes from my ISPs (ISPs are passing all upstream connected routes to me)

I have 3 public IP segments 1.1.1.0/24, 1.1.2.0/24 and 1.1.3.0/24 (for example) which I would be advertising to my ISP. I cannot advertise anything less than /24 as the ISP will not take it.

I have read quite a few docs online about load sharing on a Multihomed environment. Each of the examples talk about taking a aggregate block (/19 for example) and breaking it into two /20 blocks and then doing load sharing.

But in my case, I cannot use any aggregate block. I only have 3 x /24 blocks with me. How can I load share in this case?

Appreciate your help.

Thanks
Mikey

[Mikey John]

Could someone please help reply on this?

Thanks

Mikey

[Richard Burts]

Mikey

If you have read quite a few docs about multi homing then you recognize that it gets tricky and that balancing of outbound traffic must be considered independent of balancing inbound traffic. So you need to create a solution which addresses both aspects.

Balancing outbound traffic is easier since that part is under your control. If you have negotiated with both ISP to receive partial routes then it suggests that you plan to balance outbound traffic so that traffic whose destination is in ISP 1 and its connected upstream routes would use ISP 1 and traffic whose destination is in ISP 2 and its connected upstream routes would use ISP 2. That does leave a question about how you will balance traffic whose destination is not in either ISP and their connected upstream routes. An alternative approach would be to use Policy Based Routing and send all traffic whose source was in 1.1.1.0 to ISP 1 and all traffic whose source was 1.1.2.0 to ISP 2. You would need to determine which ISP to use for 1.1.3.0.

Balancing inbound traffic is more difficult since that is not under your control. If you have chosen to balance based on the partial routing updates for the ISPs and their connected upstream routes then you can hope that traffic generated from them and their connected upstream addresses would come to you on the appropriate interface. But sometimes that may not work as expected. And it does not address how traffic from the Internet will come to you. If you chose to balance using PBR then you could advertise your address space to your ISPs using prepending so that for 1.1.1.0 the path through ISP 1 is more attractive and for 1.1.2.0 the path through ISP 2 is more attractive.

HTH

Rick

[Mikey John]

Hello Rick,

 Many thanks for your inputs. Wouldn't PBR be more complex ( and static in nature) to configure and maintain? 

Can we not advertise all the three /24 prefix on both the ISP links using AS path prepend to get some preferencing upstream?  I mean on a "Per prefix" basis, so if we mix the AS path prepends, some traffic will flow down ISP1 and some over ISP2.

Your thoughts?

Thanks

Mikey

[Richard Burts]

Mikey

PBR would be a bit more complex than just using the partial routes from each ISP. And PBR would give you a deterministic way to balance all outbound traffic. Using the partial routes will give you deterministic balancing for only some of your outbound traffic. What would you do with outbound traffic that did not match the partial routes from either ISP?

You certainly can advertise all three /24 prefixes to both ISP and use prepending to cause some traffic to prefer ISP 1 while other traffic prefers ISP 2. That is what I mention in my last paragraph of my previous response.

There are several alternatives that you can consider. Each has some pluses and some minuses. Choose the ones that seem the best fit for your environment.

HTH

Rick

[Mikey John]

Thanks Rick. I actually got confused looking at that statement of yours. There should have been a NOT in the sentence ;)

"If you chose to balance using PBR then you could advertise your address space to your ISPs using prepending so that for 1.1.1.0 the path through ISP 1 is more attractive and for 1.1.2.0 the path through ISP 2 is more attractive."

I agree, there are several alternatives. I'll try and go ahead with the prepend option and see the results before switching over to PBR. 

Appreciate your help and guidance.

Cheers

Mikey

[Mikey John]

Hi Rick,

I have pasted below a sample config that Iam planning to use for this setup.

For Inbound load sharing, I will be using AS-path prepend on a per prefix level.

R1 (ISP1)
==========
router bgp 1111
network 1.1.1.0 mask 255.255.255.0
network 1.1.2.0 mask 255.255.255.0
network 1.1.3.0 mask 255.255.255.0

neighbor x.x.x.x route-map ISP1_OUT

route-map ISP1_OUT permit 10
match ip address prefix-list LOCAL_SUBNET
set as-path prepend 1111 1111 1111

ip prefix-list LOCAL_SUBNET seq 10 permit 1.1.3.0/24

R2 (ISP2)
==========
router bgp 2222
network 1.1.1.0 mask 255.255.255.0
network 1.1.2.0 mask 255.255.255.0
network 1.1.3.0 mask 255.255.255.0

neighbor x.x.x.x route-map ISP2_OUT

route-map ISP2_OUT permit 10
match ip address prefix-list LOCAL_SUBNET
set as-path prepend 2222 2222 2222

ip prefix-list LOCAL_SUBNET seq 10 permit 1.1.1.0/24
ip prefix-list LOCAL_SUBNET seq 20 permit 1.1.2.0/24

And for outbound load sharing, since we are receiving partial routes (around 20k+ routes), Iam planning to divide the IPv4 space between these two ISPs. I have even sub divided Class A address space into two segments since majority of the big companies use this space. Are these ACLs covering all the IPv4 space, and a good enough strategy?

R1 (ISP1)
=========

access-list 10 permit 1.0.0.0 0.255.255.255 64.0.0.0 0.255.255.255
access-list 10 permit 128.0.0.0 0.0.255.255 191.255.0.0 0.0.255.255

router bgp 1111

neighbor x.x.x.x route-map ISP1_IN

route-map ISP1_IN permit 10
match ip address 10
set local-preference 120

R2 (ISP2)
==========

access-list 10 permit 65.0.0.0 0.255.255.255 126.0.0.0 0.255.255.255
access-list 10 permit 192.0.0.0 0.255.255.255 223.255.255.0 0.255.255.255

router bgp 2222

neighbor x.x.x.x route-map ISP2_IN

route-map ISP2_IN permit 10
match ip address 10
set local-preference 120

Please let me know your thoughts?

[Richard Burts]

Mikey

You are using access list 10 which would be a standard access list but are configuring as if it were an extended access list. Also you seem to think that with an extended access list you can define the beginning and end of a range. But that is not how an extended access list works. You would be better off using a prefix list to define your address ranges.

HTH

Rick

[Mikey John]

Sorry, my bad. Thanks for the correction :). So, I believe you understood my intention.

How can I divide the IPv4 address space equally between these two routers?

Roughly, I need to have

1.x.x.x - 126.x.x.x take ISP1 route, and 

128.x.x.x - 223.x.x.x take the ISP2 route.

I know the "ge" and "le" operators can be used for prefix lists, but not sure how I will achieve the above?

Could you please help guide.

Thanks

Mikey

[Mikey John]

Hi Rick,

 Could you please help have a look at the sample configs I have pasted below using prefix lists. Would this work?

Appreciate your help/guidance.

Thanks

Mikey

[Mikey John]

Hi Rick,

 I have come up with something like below using prefix lists as you suggested. Would this work?

R1
===

router bgp 1111
neighbor x.x.x.x route-map ABC IN

route-map ABC permit 10
match ip address prefix-list INTERNET_ROUTES
set local-preference 120
route-map ABC permit 20

ip prefix-list INTERNET_ROUTES seq 5 permit 0.0.0.0/1 ge 8 le 32 (Allows all Class A addresses)

R2
==

router bgp 2222
neighbor x.x.x.x route-map ABC IN

route-map ABC permit 10
match ip address prefix-list INTERNET_ROUTES
set local-preference 120
route-map ABC permit 20

ip prefix-list INTERNET_ROUTES seq 10 permit 128.0.0.0/8 ge 9 le 32 (For all Class B addresses)
ip prefix-list INTERNET_ROUTES seq 15 permit 192.0.0.0/16 ge 17 le 32 (For all Class C addresses)

Thanks

Mikey

[Richard Burts]

Mikey

This is an interesting approach. I have a few comments about it:

- Your route map on R1 permits classful class A advertisement but does not allow subnetted advertisements. Your route map on R2 does permit subnetted advertisements. Was this difference intentional?

- in both R1 and R2 you are referencing route-map ABC. Then you have route-map Bharti-IN permit 20

what is this route map?

I would have expected route-map ABC permit 20 if the intent is to allow advertisement of other routes from the neighbor.

- in the route map in R2 you really only want to check the first 2 bits for class B (must equal 10) and the first 3 bits for class C (must equal 110)

HTH

Rick

[Mikey John]

Hi Rick,

'Bharti' is the actual route-name I would be using ;). That was a typo error on my part (My apologies). I was actually intending to refer to route-map ABC in permit 20 statement. I have gone ahead and rectified that config (edited that thread).

"in the route map in R2 you really only want to check the first 2 bits for class B (must equal 10) and the first 3 bits for class C (must equal 110)"

I could not understand this last statement of yours. Could you please elaborate on this further?

Thanks Mikey

[Richard Burts · Accepted solution]

Mikey

First let us be sure about some aspects of IP addressing. For certain topics it is helpful to think of IP addresses in binary and the class A, B, C are one topic where thinking in binary is quite helpful. Many of us leaned about class of address in terms of class A is 1 through 127 (or some may say through 126), and class B is 128 through 191, and class C is 192 through 223. But the number ranges do not really define the class of address. What really distinguishes class A from B and from C is the binary pattern in the first octet.

Any class A network will have a binary 0 in the first bit and any network that has binary 0 in the first bit is a class A network. A class B network will have binary 10 in the first two bits and any network that has binary 10 in the first two bits will be class B. A class C network will have binary 110 in the first three bits and any network with binary 110 in the first 3 bits will be class C.

Your prefix list for class A correctly says that it only needs to check one bit

ip prefix-list INTERNET_ROUTES seq 5 permit 0.0.0.0/1 ge 8 le 32

and this statement will match any network with binary 0 in the first bit.

However your prefix list for class B says to check 8 bits

ip prefix-list INTERNET_ROUTES seq 10 permit 128.0.0.0/8 ge 9 le 32

so this prefix list would match 128.0.0.0 but what about 129.0.0.0 and any other class B network that do not match the 8 bit binary for 128? It would work if you change it to this to match only 2 bits

ip prefix-list INTERNET_ROUTES seq 10 permit 128.0.0.0/2 ge 9 le 32

And there is a similar issue with your prefix list statement for class C

ip prefix-list INTERNET_ROUTES seq 15 permit 192.0.0.0/16 ge 17 le 32

you have it matching on 16 bits where it really needs to match only on 3 bits. It would match 192.0.0.0 but would not match 192.1.0.0. It should match on only 3 bits

ip prefix-list INTERNET_ROUTES seq 15 permit 192.0.0.0/3 ge 17 le 32

HTH

Rick

[Mikey John]

Thanks Rick. That explanation was really helpful. 

Appreciate all your help :)

Cheers

Mikey