iBGP Route-Reflector MultiPath Load Sharing Mechanizm

mhiyoshi · ‎07-31-2017

Dear all

I would like to find more information below.

[Restrictions on iBGP Multipath Load Sharing]
http://www.cisco.com/c/en/us/td/docs/ios-xml/ios/iproute_bgp/configuration/xe-3s/irg-xe-3s-book/ibgp_multipath_load_sharing.html#GUID-6202DAFB-59AD-4562-BAB9-555D78A9D8E7

Route Reflector Limitation

With multiple iBGP paths installed in a routing table, a route reflector will advertise only one of the paths (one next hop).

##################################################

In my understanding when it implement BGP-RR multipath, the RR can advertise only one path however

It looks like two paths, one of the two paths is bestpath however there are two equal AD routes in the routing table.

If I use traffic generator then it looks load sharing, so I would like to know what the information is correct.

[topology]

R3---iBGP(AS-65000)---R1/R2----eBGP(AS-1)---R4-Lo0:11.11.11.11/32

10.0.10.0/24 192.168.1.0/16

[RR-Client-Config-R3]

router bgp 65000
neighbor 10.0.10.1 remote-as 65000
neighbor 10.0.10.2 remote-as 65000
maximum-paths ibgp 16

[RR-Config-R1/R2]

router bgp 65000
bgp log-neighbor-changes
network 192.168.0.0 mask 255.255.0.0
neighbor 10.0.10.3 remote-as 65000
neighbor 10.0.10.3 route-reflector-client
neighbor 10.0.10.3 next-hop-self
neighbor 192.168.1.3 remote-as 1

[Verification]

R3#sh ip bgp

BGP table version is 4, local router ID is 172.31.0.254
Status codes: s suppressed, d damped, h history, * valid, > best, i - internal,

r RIB-failure, S Stale
Origin codes: i - IGP, e - EGP, ? - incomplete

Network Next Hop Metric LocPrf Weight Path
*>i11.11.11.11/32 10.0.10.1 0 100 0 1 i
* i 10.0.10.2 0 100 0 1 i
*>i192.168.0.0/16 10.0.10.1 0 100 0 i
* i 10.0.10.2 0 100 0 i

R3#sh ip route bgp

11.0.0.0/32 is subnetted, 1 subnets
B 11.11.11.11 [200/0] via 10.0.10.2, 00:01:16
[200/0] via 10.0.10.1, 00:01:16
B 192.168.0.0/16 [200/0] via 10.0.10.2, 00:01:16
[200/0] via 10.0.10.1, 00:01:16

I am not sure however if I configre RR then within the RR it can not advertise more than two routes etc.

I appreciate if there is any related information. thanks

Best Regards

Masanobu Hiyoshi

Kevin Rivest · ‎07-31-2017

What it means by the route reflector limitation for multipath is not that the route reflector won't install both routes in it's own routing table, but that it will not reflect multiple routes to the same destination to downstream route reflector clients.

As an example, a slight variation of your topology:

---iBGP(AS-65000)---R1----eBGP(AS-1)---
R5---iBGP(AS-65000)---R3--- | |---R4-Lo0:11.11.11.11/32
---iBGP(AS-65000)---R2----eBGP(AS-1)---

R3 is the router reflector for R5. Having R1 and R2 as route reflectors for R3 is not needed as they will already advertise routes learned from eBGP to R3 via iBGP. Therefore the route reflector multipath limitation does not apply.

In this topology, R3 will receive both a route to 11.11.11.11 from R1 and R2, and because all BGP attributes match up to lowest IGP metric, it will install both paths in the routing table. It will still select one path as the best path via the final four tie breakers (oldest BGP path, lowest router ID, shortest cluster list, or lowest BGP neighbor ID). This path will be selected as the best path in the BGP RIB and will be the only path reflected to route reflector client R5.

In the current topology it would not matter as any traffic from R5 will still go through R3, as the next hop of either R1 or R2 are both through R3. But take the following modified topology as an example.

R6---iBGP(AS-65000)---
|
|--iBGP(AS-65000)---R1----eBGP(AS-1)---
R5---iBGP(AS-65000)---R3---| |---R4-Lo0:11.11.11.11/32
---iBGP(AS-65000)---R2----eBGP(AS-1)---

R3 has an iBGP connection to R1, R2, and R6 out the same interface via a switch. R6 only has an iBGP connection to R3 via the switch. Both R5 and R6 are route reflector clients.

In this example R3 still installs both routes in its own routing table. However again it only selects one path as the best path in the BGP table. It reflects this one best path to both R6 and R5. Again you will see load balancing on R5 for traffic to 11.11.11.11 as it must go through R3. Now the issue is you will not see load balancing for R6. The reason being, the next hop of the route will still be either R1 or R2's interface to the switch segment (with next-hop-self on R1/R2). R3 will not reflect a route for both a next hop of R1 and R2. When R6 sends traffic to 11.11.11.11 it will forward it directly to either R1 or R2 (whichever was sent from R3) as it has a direct connection with them, it will not forward the traffic to R3 or load balance to R1 and R2.

Hope this is clear and explains everything.

mhiyoshi · ‎07-31-2017

Dear Kevin

Thank you very much for a lot of information. According to your message, I have made the topology and there is one additional question for this matter. Please see the attached "double-NH-self-question.png"

For example one Dual RR(R1,R2) can send the BGP routes with next-hop-self, but when another single RR(R3) can receive the route and forward it to R5 with next-hop-self however R5 can receive it as BGP table but not change the next-hop so it can not install the routing table.

The example output is below, are there any reason or specification?

########################################################################################################

*Cat3560G

R3#sh ip bgp
BGP table version is 2, local router ID is 10.0.20.254
Status codes: s suppressed, d damped, h history, * valid, > best, i - internal,
r RIB-failure, S Stale, m multipath, b backup-path, x best-external, f RT-Filter
Origin codes: i - IGP, e - EGP, ? - incomplete

Network Next Hop Metric LocPrf Weight Path
*>i192.168.1.0 10.0.10.1 0 100 0 i
*mi 10.0.10.2 0 100 0 i
R3#

R3#sh ip route bgp
*omit
Gateway of last resort is not set

B 192.168.1.0/24 [200/0] via 10.0.10.2, 00:05:29
[200/0] via 10.0.10.1, 00:05:29
R3#

########################################################################################################

*C1812J

R5#sh ip bgp
BGP table version is 1, local router ID is 11.11.11.11
Status codes: s suppressed, d damped, h history, * valid, > best, i - internal,
r RIB-failure, S Stale
Origin codes: i - IGP, e - EGP, ? - incomplete

Network Next Hop Metric LocPrf Weight Path
* i192.168.1.0 10.0.10.1 0 100 0 i

R5#sh ip route bgp

########################################################################################################

Best Regards,

Masanobu Hiyoshi

Kevin Rivest · ‎07-31-2017

When a route reflector advertises prefixes to route reflector clients it will only change the next hop via the neighbor next-hop-self command for routes that are learned via eBGP. If the route was learned via iBGP and reflected to a route reflector client it will not update the next hop.

There are two solutions to this as follow.

The first would be to run an IGP (such as OSPF, EIGRP, IS-IS, or RIP) under the BGP topology for reachability to the next hop. If you do this the route will be installed in the routing table and selected as best in the BGP RIB as the next hop is reachable.

If you do not want to run an IGP underneath BGP, the next option would be to reset the next hop via an outbound route-map to the neighbor. An example configuration for R3 follows:

router bgp ASN
neighbor X.X.X.X route-map BGP-NEXT-HOP out
route-map BGP-NEXT-HOP permit 10
set ip next-hop R3-IF-IP-TO-R5

If you configure this it will show the advertised next hop in R3's advertised-routes table to R5 with the updated next hop. Also, it will show the same next hop (R3 interface IP to R5) as the next hop in R5's BGP table. Because the next hop is now directly connected and reachable it will be selected as best in the BGP RIB and placed in the routing table of R5.

Again this does not mean you will see both paths via R1 and R2 reflected to R5. R3 will still only reflect the best path in its BGP RIB to R5. But, it doesn't matter because all traffic will flow from R5 via R3 anyway which is multipathing.

Additionally, with my second example in my previous post if you would like to reflect both routes from R3 to R6 via R1 and R3 there is a solution. You can use the BGP add-path feature. Take the following example configuration:

R3

router bgp ASN
bgp additional-paths select all
neighbor X.X.X.X advertise additional-paths all
neighbor X.X.X.X additional paths send

R6

router bgp ASN
neighbor X.X.X.X additional paths receive
maximum-paths ibgp 2

R3 will now advertise both paths from R1 and R2 to R6. R6 will receive both paths and will install both paths in its routing table as the BGP attributes match up to lowest IGP metric with a different next hop and receive path ID. Because the next hop is unchanged to R1 and R2 and R6 has a direct connection via the switch, R6 will multipath between R1 and R2 to reach 11.11.11.11.

An example of R3's and R6's BGP table in this configuration follows:

R3#show ip bgp
BGP table version is 2, local router ID is 10.10.10.10
Status codes: s suppressed, d damped, h history, * valid, > best, i - internal,
r RIB-failure, S Stale, m multipath, b backup-path, f RT-Filter,
x best-external, a additional-path, c RIB-compressed,
Origin codes: i - IGP, e - EGP, ? - incomplete
RPKI validation codes: V valid, I invalid, N Not found

Network Next Hop Metric LocPrf Weight Path
*mia 11.11.11.11/32 10.0.0.2 0 100 0 20 i
*>i 10.0.0.3 0 100 0 20 i

R6#sh ip bgp
BGP table version is 11, local router ID is 10.0.0.4
Status codes: s suppressed, d damped, h history, * valid, > best, i - internal,
r RIB-failure, S Stale, m multipath, b backup-path, f RT-Filter,
x best-external, a additional-path, c RIB-compressed,
Origin codes: i - IGP, e - EGP, ? - incomplete
RPKI validation codes: V valid, I invalid, N Not found

Network Next Hop Metric LocPrf Weight Path
*>i 11.11.11.11/32 10.0.0.3 0 100 0 20 i
*mi 10.0.0.2 0 100 0 20 i

Hope that clears everything up and gives you the solutions you want..