Choosing routes from Multiple Route Reflectors

[oscardenizjensen]

Hej
Unfortunately I can not test this right now, so I would like some help from community

If I establish IBGP connection on a device to 3 different RRs, and if they all send the same route, will I always install routes from only 1 of the RRs into my RIB? Or will the routes do a "round-robin" style and get installed.

Is there a RR selection process like router-id when receiving same routes?

So if IBGP session to "main" RR goes down (and assuming I don't use Add-path or PIC-EDGE), then will my device need to re-populate the RIB with all the routes again from another RR or only part of the RIB?

Regards

[M02@rt37]

Hello @oscardenizjensen,

In BGP, when you establish iBGP connections to multiple Route Reflectors, the selection of which route to install in the RIB  can depend on the BGP attributes, specifically the BGP path selection algorithm, rather than a "round-robin" style distribution of routes from the RRs.

Each RR can receive the same route from different sources or clients. When it reflects these routes to its clients, it includes the BGP path attributes in the updates. This means that all the routes coming from the different RRs will have different path attributes, even though they are advertising the same prefix. The receiving router (the one with iBGP sessions to multiple RRs) will perform the BGP best path selection process based on the BGP attributes of the routes it receives. The best path is then installed in the RIB. In most cases, the BGP path selection algorithm will choose one "best" route based on factors like AS path length, next-hop address, local preference, etc. This best route is installed in the RIB. Other routes received from different RRs may still be kept in the BGP table (Loc-RIB) but are not necessarily installed in the RIB.

If you lose your iBGP session to the RR where the best path was learned, the router will not re-populate the entire RIB. It will rely on the next best path in its BGP table to become the best path, and that route will be installed in the RIB.

[David Ruess · Accepted solution]

Hello,

The BGP process is the same for picking its routes regardless how it learned it. To answer the question yes, by default it will only install one path. If nothing else is changes as far as metrics then it will choose the same one each time. I labbed this up to help me understand this and show you the process as well since you couldn't test it. To understand the selection process it helps to check the BGP Best Path Selection document linked below:

https://www.cisco.com/c/en/us/support/docs/ip/border-gateway-protocol-bgp/13753-25.html

Lets walk through the steps: (assuming nothing extra is configured)

1. Choose the route with the highest weight. (same weight of 0) 
2. If weight is not set, choose the route with the highest local preference. (Same LocalPRef = 100)
3. Choose routes that this router originated. (None)
4. Choose the path with the shortest Autonomous System path. (same path length)
5. Choose the path with the lowest origin code (i is lowest, e is next, ? is last). (all are 'i')
6. Choose the route with the lowest MED, if the same Autonomous System advertises the possible routes. (all have same MED)
7. Choose an EBGP route over an IBGP route. (all are iBGP routes)
8. Choose the route through the nearest IGP neighbor as determined by the lowest IGP metric. (all have same IGP metric) <this could possibly be your tie breaker if your IGP metric is different.
9. Determine if multiple paths require installation in the routing table for BGP Multipath. (multipath not configured)
10. When both paths are external, prefer the path that was received first (the oldest one). (not external paths)

11. Prefer the route that comes from the BGP router with the lowest router ID.

    Note: If a path contains route reflector (RR) attributes, the originator ID is substituted for the router ID in the path selection process. (both routes have the same Originator ID)

12. If the originator or router ID is the same for multiple paths, prefer the path with the minimum cluster list length. (same cluster length)

    This is only present in BGP RR environments. It allows clients to peer with RRs or clients in other clusters. In this scenario, the client must be aware of the RR-specific BGP attribute.

13. Prefer the path that comes from the lowest neighbor address. (here is where we land) 
    
    I have a neighbor of 22.22.22.22 (with RID of 2.2.2.2) and a neighbor at 5.5.5.5 (with RID of 5.5.5.5). It chooses the neighbor of 5.5.5.5 as its path since its the lowest neighbor IP. When I shut down the neighbor of 5.5.5.5 it chooses the 22.22.22.22. When I bring the 5.5.5.5 neighbor back online it reselects that one as the best path again.

Hope this helps clear anything up

Config from example above

R1#
R1#sh ip bgp 77.77.77.77
BGP routing table entry for 77.77.77.0/24, version 2
Paths: (2 available, best #2, table default)
Multipath: eBGP
Not advertised to any peer
Refresh Epoch 1
Local
7.7.7.7 from 22.22.22.22 (2.2.2.2) (RID of 2.2.2.2) - neighbor IP is 22.22.22.22
Origin IGP, metric 0, localpref 100, valid, internal
Originator: 77.77.77.77, Cluster list: 2.2.2.2
rx pathid: 0, tx pathid: 0
Refresh Epoch 2
Local
7.7.7.7 from 5.5.5.5 (5.5.5.5) (RID of 5.5.5.5) - neighbor IP is 5.5.5.5
Origin IGP, metric 0, localpref 100, valid, internal, best
Originator: 77.77.77.77, Cluster list: 5.5.5.5
rx pathid: 0, tx pathid: 0x0
R1#
R1#conf t
Enter configuration commands, one per line. End with CNTL/Z.
R1(config)#router bgp 100
R1(config-router)#neighbor 5.5.5.5 shutdown <-shutdown neighborship to the best path
R1(config-router)#
*Nov 8 10:47:26.214: %BGP-3-NOTIFICATION: sent to neighbor 5.5.5.5 6/2 (Administrative Shutdown) 0 bytes 
*Nov 8 10:47:26.215: %BGP-5-NBR_RESET: Neighbor 5.5.5.5 reset (Admin. shutdown)
*Nov 8 10:47:26.244: %BGP-5-ADJCHANGE: neighbor 5.5.5.5 Down Admin. shutdown
*Nov 8 10:47:26.245: %BGP_SESSION-5-ADJCHANGE: neighbor 5.5.5.5 IPv4 Unicast topology base removed from session Admin. shutdown
R1(config-router)#do sh ip bgp 77.77.77.77 
BGP routing table entry for 77.77.77.0/24, version 3
Paths: (1 available, best #1, table default)
Multipath: eBGP
Not advertised to any peer
Refresh Epoch 1
Local
7.7.7.7 from 22.22.22.22 (2.2.2.2)
Origin IGP, metric 0, localpref 100, valid, internal, best <-it now picks the 22.22.22.22 neighbor as best (only) path.
Originator: 77.77.77.77, Cluster list: 2.2.2.2
rx pathid: 0, tx pathid: 0x0
R1(config-router)# 
R1(config-router)#
R1(config-router)#no neighbor 5.5.5.5 shutdown <- bring neighbor back up
R1(config-router)#
*Nov 8 10:47:52.688: %BGP-5-ADJCHANGE: neighbor 5.5.5.5 Up 
R1(config-router)#
R1(config-router)#do sh ip bgp 77.77.77.77 
BGP routing table entry for 77.77.77.0/24, version 4
Paths: (2 available, best #1, table default)
Multipath: eBGP
Not advertised to any peer
Refresh Epoch 2
Local
7.7.7.7 from 5.5.5.5 (5.5.5.5)
Origin IGP, metric 0, localpref 100, valid, internal, best <-shows neighbor 5.5.5.5 as best again even though it has higher RID
Originator: 77.77.77.77, Cluster list: 5.5.5.5
rx pathid: 0, tx pathid: 0x0
Refresh Epoch 1
Local
7.7.7.7 from 22.22.22.22 (2.2.2.2)
Origin IGP, metric 0, localpref 100, valid, internal
Originator: 77.77.77.77, Cluster list: 2.2.2.2
rx pathid: 0, tx pathid: 0
R1(config-router)#

-David

[oscardenizjensen]

Thanks for labbing it out, so it was kind of how I was thinking. So if I have 2 RRs that advertise 10 routes that are same (metric etc), I will always install the routes advertised from one RR only.

[MHM Cisco World]

I was right it router-ID,
this lab I test 2.2.2.2 with neighbor 20.0.0.2 is more prefer (best path) than 3.3.3.3 with 10..0.0.3 

@David Ruess can you share the config of lab you use with topology I need to check. 

[Torbjørn]

This is what the documentation for BGP bestpath selection states regarding this:

Note: If a path contains route reflector (RR) attributes, the originator ID is substituted for the router ID in the path selection process.

https://www.cisco.com/c/en/us/support/docs/ip/border-gateway-protocol-bgp/13753-25.html

[MHM Cisco World]

I know and read this doc. 
and as I mention bofre the last attributes is complicated. 
I test and the router-ID is select 
@David Ruess  test and neighbor is select 

[MHM Cisco World]

Same route with identical attributes 

Router select only path via only one RR and Router-id will be tie breaker 

If you receive same route and you dont use add-path then if RR is down the router will process other path via other RR.

[David Ruess]

I am not sure that is quite correct. In my example above it references the best path selection from Cisco. Since the OP is using a Route-reflectors the RID selection is replaced by Originator ID which would likely be the same. In this case the next criteria is used and that's lowest neighbor address.
My example:

-David

[MHM Cisco World]

can you run lab and test this. 

Thanks A Lot
MHM

[David Ruess]

Yes, I already did. My output is in my original response.

[MHM Cisco World]

but the Best is lowest router-ID not originator.
2.2.2.2 not 5.5.5.5

[David Ruess]

Apologies. I had some of the wrong output in there from another test. I updated my response and the output with more notes to help make it clear. Thank you for catching it.

The router picks the lower neighbor IP of 5.5.5.5 even though the RID of the 22.22.22.22 neighbor is lower with 2.2.2.2.

[MHM Cisco World]

It OK freind it now clear.

Can you shut 5.5.5.5 wait for bgp to select 22.22.22.22 then no shut 5.5.5.5'

See if path shift to 5.5.5.5 again or not.

These last bgp criteria is so complicate.

Thanks A Lot
MHM

[David Ruess]

Yes, I also did that in my original response. You can see the bold text showing what commands I entered and what path it chooses each time.

It does indeed choose 5.5.5.5 again.