EBGP under VRF and BGP multipath on core routers

[IBEngTeam]

Hi all,

I have the following topology, two EBGP to two PE routers, each BGP is terminated in a different VRF. Each VRF receives same routes from the external ISP, and export it to the main VRF configured on a route reflector. I wat to implement BGP multipath in the main VRF, on the route received from external BGP.

I tried many configuration but with no success, this is the output for one of the routes:

sh bgp vrf internet 2.0.1.66 

BGP routing table entry for 2.0.1.0/24, Route Distinguisher: 10.255.255.60:1

Versions:

  Process           bRIB/RIB  SendTblVer

  Speaker              77042       77042

Last Modified: May 26 14:04:26.712 for 00:00:14

Paths: (3 available, best #1)

  Not advertised to any peer

  Path #1: Received by speaker 0

  Not advertised to any peer

  3257, (Received from a RR-client), (received & used)

    10.255.255.72 (metric 210) from 10.255.255.72 (10.255.255.72)

      Received Label 24125

      Origin incomplete, metric 0, localpref 1000, valid, internal, best, group-best, import-candidate, imported

      Received Path ID 0, Local Path ID 1, version 77042

      Community: 3257:1 25003:10 25003:99

      Extended community: RT:1234:1234 RT:3257:3257

      Source AFI: VPNv4 Unicast, Source VRF: default, Source Route Distinguisher: 10.255.255.72:3257

  Path #2: Received by speaker 0

  Not advertised to any peer

  3356, (Received from a RR-client), (received & used)

    10.255.255.73 (metric 210) from 10.255.255.73 (10.255.255.73)

      Received Label 24126

      Origin incomplete, metric 0, localpref 1000, valid, internal, group-best, imported

      Received Path ID 0, Local Path ID 0, version 0

      Community: 3356:1 25003:10 25003:99

      Extended community: RT:1234:1234 RT:3356:3356

      Source AFI: VPNv4 Unicast, Source VRF: default, Source Route Distinguisher: 10.255.255.73:3356

Attached is a diagram of the network. Can anyone help?

Thanks in advance.

Adi.

[Harold Ritter]

Hi @IBEngTeam ,

You need to configure the "bgp bestpath as-path multipath-relax" command for ECMP to work between paths coming from different ASNs. 

router bgp xxx
bgp bestpath as-path multipath-relax
!

address-family ipv4 vrf Internet
maximum-paths 2
exit-address-family

Regards,

[IBEngTeam]

Hi,

Thanks for the reply.

We have configured "bgp bestpath as-path multipath-relax" under router bgp, with no help.

When configuring a static route on the PE routers, all works well on the route reflector:

sh bgp vrf internet 123.123.123.123
Sun May 29 06:52:51.122 IDT
BGP routing table entry for 123.123.123.123/32, Route Distinguisher: 10.255.255.60:1
Versions:
Process bRIB/RIB SendTblVer
Speaker 83296 83296
Last Modified: May 28 06:06:12.063 for 1d00h
Paths: (2 available, best #1)
Not advertised to any peer
Path #1: Received by speaker 0
Not advertised to any peer
Local, (Received from a RR-client), (received & used)
10.255.255.72 (metric 210) from 10.255.255.72 (10.255.255.72)
Received Label 24134
Origin incomplete, metric 0, localpref 100, valid, internal, best, group-best, multipath, import-candidate, imported
Received Path ID 0, Local Path ID 1, version 83296
Extended community: RT:1234:1234 RT:3257:3257
Source AFI: VPNv4 Unicast, Source VRF: default, Source Route Distinguisher: 10.255.255.72:3257
Path #2: Received by speaker 0
Not advertised to any peer
Local, (Received from a RR-client), (received & used)
10.255.255.73 (metric 210) from 10.255.255.73 (10.255.255.73)
Received Label 24131
Origin incomplete, metric 0, localpref 100, valid, internal, multipath, import-candidate, imported
Received Path ID 0, Local Path ID 0, version 0
Extended community: RT:1234:1234 RT:3356:3356
Source AFI: VPNv4 Unicast, Source VRF: default, Source Route Distinguisher: 10.255.255.73:3356

But multipath for ebgp routes from PE routesrs (under vrf), is not working.

Any ideas?

Thanks.

[Harold Ritter]

Hi @IBEngTeam ,

I see C1 is an iBGP neighbor to PE1 and PE2. You would need to configure " maximum-paths ibgp 2" under the vrf Internet section of the BGP configuration as follow:

router bgp xxx

address-family ipv4 vrf Internet
maximum-paths ibgp 2
exit-address-family

Regards,

[IBEngTeam]

Hi,

just configured "maximum-paths ibgp 2", same result.

this is the bgp output:

sh bgp vrf internet 2.0.1.0
Wed Jun 1 07:22:49.822 IDT
BGP routing table entry for 2.0.1.0/24, Route Distinguisher: 10.255.255.60:1
Versions:
Process bRIB/RIB SendTblVer
Speaker 98987 98987
Last Modified: May 31 13:54:00.593 for 17:28:49
Paths: (3 available, best #1)
Not advertised to any peer
Path #1: Received by speaker 0
Not advertised to any peer
3257, (Received from a RR-client), (received & used)
10.255.255.72 (metric 210) from 10.255.255.72 (10.255.255.72)
Received Label 24008
Origin incomplete, metric 0, localpref 1000, valid, internal, best, group-best, import-candidate, imported
Received Path ID 0, Local Path ID 1, version 98987
Community: 6:25003 3257:1 25003:10 25003:99
Extended community: RT:1234:1234 RT:3257:3257
Source AFI: VPNv4 Unicast, Source VRF: default, Source Route Distinguisher: 10.255.255.72:3257
Path #2: Received by speaker 0
Not advertised to any peer
4455, (Received from a RR-client), (received & used)
10.255.255.72 (metric 210) from 10.255.255.72 (10.255.255.72)
Received Label 24006
Origin incomplete, metric 0, localpref 900, valid, internal, group-best, imported
Received Path ID 0, Local Path ID 0, version 0
Community: 6:25003 4455:1 25003:10 25003:99
Extended community: RT:1234:1234 RT:4455:4455
Source AFI: VPNv4 Unicast, Source VRF: default, Source Route Distinguisher: 10.255.255.72:4455
Path #3: Received by speaker 0
Not advertised to any peer
3356, (Received from a RR-client), (received & used)
10.255.255.73 (metric 210) from 10.255.255.73 (10.255.255.73)
Received Label 24006
Origin incomplete, metric 0, localpref 1000, valid, internal, group-best, imported
Received Path ID 0, Local Path ID 0, version 0
Community: 6:25003 3356:1 25003:10 25003:99
Extended community: RT:1234:1234 RT:3356:3356
Source AFI: VPNv4 Unicast, Source VRF: default, Source Route Distinguisher: 10.255.255.73:3356

Only one path is selected, what can it be?

Thanks,

Adi.

[Harold Ritter]

Hi @IBEngTeam ,

Did you try clearing the sessions after changing the configuration?

clear bgp vrf Internet ipv4 uni * soft

Also, could you please post the bgp configuration.

Regards,

[IBEngTeam]

Hi,

thanks for the reply.

yes, i tried "clear bgp vrf Internet ipv4 uni * soft" - did not help.

This is the configuration:

router bgp 1
vrf internet
rd 10.1.1.60:1
address-family ipv4 unicast
maximum-paths ibgp 6
maximum-paths ebgp 6
redistribute connected
!
address-family ipv6 unicast
maximum-paths ibgp 8
!
!

vrf internet
address-family ipv4 unicast
import from default-vrf route-policy rp_route_leak_from_global advertise-as-vpn
import route-target
1234:1234
!
export route-policy rp_internet_export
export to default-vrf route-policy rp_route_leak_to_global allow-imported-vpn
!
address-family ipv6 unicast
import from default-vrf route-policy rp_route_leak_from_global advertise-as-vpn
import route-target
1234:1234
!
export route-policy rp_internet_export
export to default-vrf route-policy rp_route_leak_to_global allow-imported-vpn
!
!

route-policy rp_internet_export
set extcommunity rt (5432:5432) additive
end-policy
!

prefix-set ps_all_ip_ext_adv
1.1.0.0/22
end-set
!
prefix-set ps_ib_public
2.2.0.0/22
end-set
!
community-set cs_allow_adv_ext
1:1
end-set
!
route-policy rp_route_leak_from_global
if community matches-any cs_allow_adv_ext and destination in ps_all_ip_ext_adv then
pass
elseif destination in ps_all_ip_ext_adv or destination in ps_ib_public then
pass
else
drop
endif
end-policy
!

prefix-set ps_leak_to_global
5.5.0.0/16 le 24
end-set
!
route-policy rp_route_leak_to_global
if destination in ps_leak_to_global then
pass
elseif destination in (9.8.7.6/32) then
pass
else
drop
endif
end-policy
!

this is very strange, because static routes being advertised form PE routers to vrf internet are working ok:

sh bgp vrf internet 123.123.123.123
Wed Jun 1 13:46:41.403 IDT
BGP routing table entry for 123.123.123.123/32, Route Distinguisher: 10.255.255.60:1
Versions:
Process bRIB/RIB SendTblVer
Speaker 101475 101475
Last Modified: Jun 1 07:26:45.725 for 06:19:55
Paths: (2 available, best #1)
Not advertised to any peer
Path #1: Received by speaker 0
Not advertised to any peer
Local, (Received from a RR-client), (received & used)
10.255.255.72 (metric 210) from 10.255.255.72 (10.255.255.72)
Received Label 24004
Origin incomplete, metric 0, localpref 100, valid, internal, best, group-best, multipath, import-candidate, imported
Received Path ID 0, Local Path ID 1, version 101475
Extended community: RT:1234:1234 RT:3257:3257
Source AFI: VPNv4 Unicast, Source VRF: default, Source Route Distinguisher: 10.255.255.72:3257
Path #2: Received by speaker 0
Not advertised to any peer
Local, (Received from a RR-client), (received & used)
10.255.255.73 (metric 210) from 10.255.255.73 (10.255.255.73)
Received Label 24005
Origin incomplete, metric 0, localpref 100, valid, internal, multipath, import-candidate, imported
Received Path ID 0, Local Path ID 0, version 0
Extended community: RT:1234:1234 RT:3356:3356
Source AFI: VPNv4 Unicast, Source VRF: default, Source Route Distinguisher: 10.255.255.73:3356

Any ideas?

Thanks,

Adi.

[Harold Ritter · Accepted solution]

Hi @IBEngTeam ,

The reason it works with the static routes redistributed from the PE is that these routes are considered local (empty AS path). The reason it does not work for the routes coming from outside the AS is that you do not have "bgp bestpath as-path multipath-relax" configured. 

router bgp 25003

vrf Internet
bgp bestpath as-path multipath-relax <++++ This command is needed to ignore the fact that the paths have different AS paths
address-family ipv4 unicast
maximum-paths ibgp 2
!

Regards,

[IBEngTeam]

Hi Harold,

Thanks, that solved the issue.

We thought the command "bgp bestpath as-path multipath-relax" is only configured under main BGP, it was not clear that it is needed under the vrf.

Learned a lot!

Thanks again,

Adi.

[Harold Ritter]

Hi Adi,

I am glad it worked for you.

Have a great day,