BGP route reflection problems

[j.restaino]

Hi,

I'm having trouble with BGP's route selection criteria. I have a network with NCS 5500; one of them is route reflector and receives route 1.46.0.0/19 via iBGP through three of my routers (R1, R2 and R3) with the following metric.

R1:
AS-PATH: 2914 10089 9587 24378
MED: 30263
IGP Metric: 12609

R2:
AS-PATH: 6939 10089 9587 24378
MED: 2179
IGP Metric: 13103

R3:
AS-PATH: 6939 10089 9587 24378
MED: 3664
IGP Metric: 952

My route reflector is choosing R2's route as best.

I understand the Route Reflector is comparing the route with next hop R2 and R3 through the MED, but it is not comparing the route of R1. I understand that this behavior should not be appropriate, since R1's route is learned by AS 21914 and not 6939, so the selected route should be that of R3 due the lower IGP Metric.

The problem that is arising is that when router R3 receives from the reflector that the best route is that of R2, MED assumes that the one it learns through iBGP is better and stops announcing the route to the reflector. This has the consequence that the reflector now receives the routes of R1 and R2 and between the two it takes the route of R1 as the best, thereby announcing the route to R3 with next hop R1. This causes R3 to announce the route again since the one learned by eBGP is preferred over the one learned by R1.

This situation is constantly repeating itself, so my routers have the route in the routing table for a few seconds and are constantly changing.

I would appreciate if someone can help me understand how to solve this problem.

Kind regards
José

[VladimirRoutin]

Sir, have you tried filtering the oscillating prefix in the eBGP session with AS 2914?

Can you share the bgp state in that case?

[Harold Ritter]

Hi @j.restaino ,

This behaviour is known as persistent route oscillation condition and is described in RFC3345.

https://www.rfc-editor.org/rfc/rfc3345

One way to workaround the issue is to use the always compare MED knob in on all the BGP routers.

RFC3345, section 2.3 also has some other ways to workaround this issue.

XR:

router bgp xxx

 bgp bestpath med always

IOS and IOS-XE:

router bgp xxx

 bgp always-compare-med

Regards, 

[j.restaino]

Thanks for your response @Harold Ritter, I was unaware of RFC 3345.

I would not always want to compare the MED based on the different criteria that the AS use to mark them. Furthermore, as an IGP metric we establish the physical distance in the links and I would not want to overwrite this information when the AS are different.

Looking for more information I found the following link https://www.cisco.com/c/en/us/support/docs/ip/border-gateway-protocol-bgp/16046-bgp-med.html
It describes that there is a temporary variable in the routes that receive the routes. This behavior is consistent with what happens to us since not all routes have problems and we have observed that routes that present this behavior later stop having it and others that did not have it begin to have it.

I understand that if we could activate "bgp deterministic-med" we could have a solution to the problem. In the case I described, the route announced by R1 would be chosen in all cases. The problem is that I couldn't find how to enable this in IOS XR (I'm using 7.3.2)

Another solution that I find viable is that the reflector, when it receives the route, adds the MED value of the IGP metric of the received route to the MED value, but I also did not find if this is feasible to do.

Thank you again for your response.

Regards
José

[Harold Ritter]

Hi @j.restaino ,

"bgp deterministic med" is enabled by default in IOS XR. Are all of your core devices running XR?

If not, you would probably want to deploy this feature manually on all non XR BGP devices, as recommended in the document you pointed out.

Regards, 

[j.restaino]

Thanks for the response @Harold Ritter.

In this case i don´t understand why the route reflector choose the route from R2, the reflector has IOS XR version 7.3.2.

If i am not wrong if the bgp deterministic med is enable, the correct route will be the route from R1 and this descicion is made by the reflector despite of his clients has or not bgp deterministic med.

Let me know if i am wrong.

Regards
José

[Harold Ritter]

You are very welcome @j.restaino ,

The best way to find out why the RR selects the route from R2 is to run the following command from the RR.

show bgp ipv4 unicast 1.46.0.0/19 bestpath-compare

Regards,

[j.restaino]

Thanks for your response @Harold Ritter ,

AS 6939 began making a new publication to another of our PEs. This happened on Friday itself, so I could not send the requested command. In any case, analyzing its output I still don't understand it.

The current image now is as follows.

R1:
AS-PATH: 2914 10089 9587 24378
MED: 30263
IGP Metric: 12609

R2:
AS-PATH: 6939 10089 9587 24378
MED: 2179
IGP Metric: 13103

R4: AS-PATH: 6939 10089 9587 24378
MED: 2354
IGP Metric: 12608

With bgp deterministic-med I understand that the selected route would be that of R1, however the selected one is that of R2. I understand that this should not be the best since the distance to the IGP is less and it is learned from another AS.

I attach the complete command output. I will appreciate if you can help me to understand why the best route is the number one.

Regards José

[Harold Ritter]

Hi @j.restaino ,

I see this is a VRF route. Could you please share the BGP configuration for the RR? Also, what is the XR version you are running on the RR?

Regards, 

[Harold Ritter]

Hi @j.restaino ,

It is also interesting to note that of the other RRs, two selects R1 (192.168.197.9, 192.168.197.12) and one selects R2 (192.168.197.2). What platform and what version do they run?

Path #4: Received by speaker 0
Flags: 0x2000000084008005, import: 0xa0
Not advertised to any peer
6939 10089 9587 24378, (received & used)
192.168.194.2 (metric 13103) from 192.168.197.2 (192.168.194.2), if-handle 0x00000000

Path #5: Received by speaker 0
Flags: 0x2000000084008005, import: 0x80
Not advertised to any peer
2914 10089 9587 24378, (received & used)
192.168.194.1 (metric 12609) from 192.168.197.9 (192.168.194.1), if-handle 0x00000000

Path #6: Received by speaker 0
Flags: 0x2000000084028005, import: 0x80
Not advertised to any peer
2914 10089 9587 24378, (received & used)
192.168.194.1 (metric 12609) from 192.168.197.12 (192.168.194.1), if-handle 0x00000000

Regards,

[j.restaino]

Hi @Harold Ritter 

The configuration i think is very standard i attach the template

The threes route reflectors (the one that i send the show bgp route and the other two) are trhee NCS 5500, two of them has Cisco IOS XR Software, Version 7.3.2 and the other has 7.3.1 (192.168.197.12)

Regards
José

[Harold Ritter]

Hi @j.restaino ,

Can you please provide the output for the following command:

show bgp vpnv4 unicast rd 7167:6057 1.46.0.0/19 bestpath-compare 

Regards,

[j.restaino]

Hi @Harold Ritter ,

I send it attached

Regards

[MHM Cisco World]

Cluster list: 192.168.197.9
      Longer cluster length than best path (path #2)

I dont see config of cluster' did yoh config cluster ID in RR client? That can be reason here.

As I mention there is other attributes effect selection Cluster id length is effect here select best path.

Try disable it.

[j.restaino]

Hi @MHM Cisco World,

The situation there is that we are receiving the same route from another reflector so it is ok that this path lose. But the doubt is between the routes that we are receiving from the others reflector Clients.

Regards