BGP MPLS VPN w/ route reflectors - convergence

[simone.c]

Hi all,

I have a question about BGP MPLS VPN convergence, with route reflectors.

Let's imagine the following topology: 2 route reflectors (RR1 & RR2, different cluster IDs) and 2 MPLS PEs route-reflector-clients (PE1 & PE2, each peering with both RRs, but not peering with each other).

PE1 has network

A (10.10.10.0/24)

and PE2 has network B

(10.10.11.0/24)

Network A and network B are part of the MPLS network and advertised to the RRs and reflected by the RRs.

If we check the

vpnv4

routes, we can see that PE1 receives network B twice - once from each RR - and installs only one of them (my assumption, based on empirical evidence, is that it installs the one from the RR with lowest router-ID). In our example it installs the one from RR1.

Question: assuming there's non-stop traffic flowing between network A and network B, what happens when RR1 (the RR which provides the preferred route) fails? Can someone explain in details how does BGP react, and what should happen to the traffic flow between network A and network B?

I simulated this in GNS3, running an extensive ping between network A and network B and shutting down RR1. I was expecting at least some packet lost, but there was none. After the holdtime for RR1 expired, I saw that the PEs had installed the

VPNV4

route from RR2, but the fact that there was no packet loss puzzles me, I don't get it.

The show output below shows that for both network A and network B, we receive a

vpnv4

route twice but one is chosen.

10.49.0.1 & 10.49.0.2

are RR1 and RR2.

PE1#show ip bgp vpnv4 all neighbors 10.49.0.1 routes 

Status codes: s suppressed, d damped, h history, * valid, > best, i - internal, 

              r RIB-failure, S Stale, m multipath, b backup-path, f RT-Filter, 

              x best-external, a additional-path, c RIB-compressed, 

              t secondary path, L long-lived-stale,

Origin codes: i - IGP, e - EGP, ? - incomplete

RPKI validation codes: V valid, I invalid, N Not found

     Network          Next Hop            Metric LocPrf Weight Path

Route Distinguisher: 64512:6900

 *>i  10.10.11.0/24   10.49.49.222              0    100      0 64512 i

          

PE1#show ip bgp vpnv4 all neighbors 10.49.0.2 routes 

Status codes: s suppressed, d damped, h history, * valid, > best, i - internal, 

              r RIB-failure, S Stale, m multipath, b backup-path, f RT-Filter, 

              x best-external, a additional-path, c RIB-compressed, 

              t secondary path, L long-lived-stale,

Origin codes: i - IGP, e - EGP, ? - incomplete

RPKI validation codes: V valid, I invalid, N Not found

     Network          Next Hop            Metric LocPrf Weight Path

Route Distinguisher: 64512:6900

 * i  10.10.11.0/24   10.49.49.222              0    100      0 64512 i




PE1#show ip bgp vpnv4 vrf vrf_A 10.10.11.0/24

BGP Bestpath: deterministic-med: cost-community-ignore

Paths: (1 available, best #1, table PE1)

  Not advertised to any peer

  Refresh Epoch 2

  64512, imported path from 64512:6900:10.10.11.0/24 (global)

    10.49.49.222 (metric 210) (via default) from 10.49.0.1 (10.49.0.1)

      Origin IGP, metric 0, localpref 100, valid, internal, best

      Extended Community: RT:11:9999

      Originator: 10.49.49.222, Cluster list: 10.49.0.1

      mpls labels in/out nolabel/24123

      rx pathid: 0, tx pathid: 0x0

      Updated on Aug 26 2023 00:24:21 CEST

--------------------------------------------------------  

PE2#show ip bgp vpnv4 all neighbors 10.49.0.1 routes 

Status codes: s suppressed, d damped, h history, * valid, > best, i - internal, 

              r RIB-failure, S Stale, m multipath, b backup-path, f RT-Filter, 

              x best-external, a additional-path, c RIB-compressed, 

              t secondary path, L long-lived-stale,

Origin codes: i - IGP, e - EGP, ? - incomplete

RPKI validation codes: V valid, I invalid, N Not found

     Network          Next Hop            Metric LocPrf Weight Path

Route Distinguisher: 64500:6900

 *>i  10.10.10.0/24   10.49.49.170              0    100      0 64500 i

          

PE2#show ip bgp vpnv4 all neighbors 10.49.0.2 routes 

Status codes: s suppressed, d damped, h history, * valid, > best, i - internal, 

              r RIB-failure, S Stale, m multipath, b backup-path, f RT-Filter, 

              x best-external, a additional-path, c RIB-compressed, 

              t secondary path, L long-lived-stale,

Origin codes: i - IGP, e - EGP, ? - incomplete

RPKI validation codes: V valid, I invalid, N Not found

     Network          Next Hop            Metric LocPrf Weight Path

Route Distinguisher: 64500:6900

 * i  10.10.10.0/24   10.49.49.170              0    100      0 64500 i




PE2#show ip bgp vpnv4 vrf vrf_B 10.10.10.0/24

BGP Bestpath: deterministic-med: cost-community-ignore

Paths: (1 available, best #1, table PE1)

  Not advertised to any peer

  Refresh Epoch 2

  64500, imported path from 64500:6900:10.10.10.0/24 (global)

    10.49.49.170 (metric 210) (via default) from 10.49.0.1 (10.49.0.1)

      Origin IGP, metric 0, localpref 100, valid, internal, best

      Extended Community: RT:10:9999

      Originator: 10.49.49.170, Cluster list: 10.49.0.1

      mpls labels in/out nolabel/24131

      rx pathid: 0, tx pathid: 0x0

      Updated on Aug 13 2023 05:11:25 CEST

[M02@rt37]

Hello @simone.c,

The behavior you're observing, where the PE router chooses one of the reflected routes for a given network (B in this case), is expected. The behavior of BGP and VPN route selection in this context is influenced by factors such as BGP attributes, tie-breaking rules, and internal metrics. The choice of one route over the other might be based on a variety of factors, including MED, AS path, origin type, and more.

About convergence and failover when RR1 fails, BGP has mechanisms to handle route withdrawals and re-advertisements. When a route reflector (RR1) fails, BGP detects that the routes it was reflecting are no longer available. The BGP routers (PE1 and PE2) that were using the reflected routes from RR1 will react to the withdrawal of routes and initiate a new route selection process.

Also, where you experienced no packet loss, it's possible that the network reconverged quickly after the route reflector failure due to the nature of BGP convergence and the use of route reflectors. BGP routers can quickly detect changes and adjust their routing tables accordingly.

[Giuseppe Larosa]

Hello @simone.c ,

MP BGP is in the control plane and it is needed to build the VRF routing table , to learn the

VPNv4 prefix

and the VPN label that is the inner label ( more internal) in the label stack used in the forwarding plane.

When you send traffic over an

MPLS L3 VPN

you are using the data plane that is an LSP with destination the remote PE loopback address with added the VPN label.

You have two

MP BGP RRS

and yes the one with the lowest BGP RID is selected as best this is expected, however the other

VPNv4 path via RR2

is present and avaiable on PE1.

The specific type of fault you emulate is a fault of RR1. You say you have shut RR1 on

GNS3

if RR1 is not in the path used by the

MPLS LSP 

between PE1 and PE2 it is possibile to have no losses when you disable RR1, as RR2 MP BGP is there and ready to be used and the

MPLS LSP

does not need to be re-routed.

you can check this with

show mpls forwarding 10.49.49.222 

show ip cef vrf vrf_A 10.10.11.0

on PE1

Edit:

we don't know your topology however even if there are two equal cost paths betweeen PE1 and PE2 going via RR1 and via  RR2 respectively. This does not mean that the path via RR1 is the one used. Actually for

MPLS L3 VPN

traffic can be load balanced based on source and destination IP.

If your test ping was going over the unaffected path again you have an explanation of what you see.

Edit 2:

if on RR1 you only disabled BGP or BGP neighbors, it is still a working P node

(LDP and OSPF or IS-IS or EIGRP)

and it can be used to send MPLS traffic between PE1 and PE2.

Hope to help

Giuseppe

[simone.c]

Hi Giuseppe,

thanks for the valuable input. Let me clarify immediately, the route reflectors are not in the forwarding path, so their outage does not directly affect the traffic flow.

Focusing on your sentence "it is possibile to have no losses when you disable RR1, as RR2 MP BGP is there and ready to be used and the MPLS LSP does not need to be re-routed", this point is at the core of my question: so while PE1 flushes the

vpnv4

route from RR1 and installs the

vpnv4

route from RR2 as best route, the forwarding plane is not affected at all, traffic will not be dropped. Is that so?

Thanks,

Simone

[Giuseppe Larosa · Accepted solution]

Hello @simone.c ,

>> this point is at the core of my question: so while PE1 flushes the

vpnv4

route from RR1 and installs the

vpnv4

route from RR2 as best route, the forwarding plane is not affected at all, traffic will not be dropped. Is that so?

Yes, because the two RRS RR1 and RR2 are not on path there is no impact in the forwarding plane because all the info used in the forwarding plane CEF FIB, adjacency table and MPLS LFIB do not change.

The change in the control plane does not change the BGP

next-hop

and it does not change the VPN label for remote prefix so the forwarding plane is not affected by this type of fault.

As soon as the

VPNv4

route via RR1 is removed the

VPNv4

route via RR2 is installed as best path but as noted above this does not cause changes in the forwarding plane.

Hope to help

Giuseppe