ASR9K BGP Load balance between diffrent vrf on same device

[IBEngTeam]

Hi all,

We have the following topology:

Router pe-1 have two vrf's, vrf A and vrf B, each connected to a different ISP via ebgp. routes from both vrf's are advertised to core-1 vrf internet, with same metrics. Is it possible to install multiple routes on core-1, received from same device - pe-1, but from different vrf's?

This is the configuration on vrf internet:

vrf internet
address-family ipv4 unicast
import route-target
1234:1234
!
!
!

router bgp 1
vrf internet
rd 1.1.1.60:1
bgp bestpath as-path multipath-relax
address-family ipv4 unicast
maximum-paths eibgp 6
redistribute connected
!
!

Here is a route for example, only one route is chosen, and the routes are not multipath:

sh bgp vrf internet 2.52.0.0/15
Sun Aug 14 11:51:17.897 IDT
BGP routing table entry for 2.52.0.0/15, Route Distinguisher: 1.1.1.60:1
Versions:
Process bRIB/RIB SendTblVer
Speaker 852258 852258
Last Modified: Aug 14 08:01:09.457 for 03:50:08
Paths: (2 available, best #1)
Not advertised to any peer
Path #1: Received by speaker 0
Not advertised to any peer
8888 12, (Received from a RR-client), (received & used)
1.1.1.71 (metric 210) from 1.1.1.71 (1.1.1.71)
Received Label 24143
Origin IGP, localpref 560, valid, internal, best, group-best, import-candidate, imported
Received Path ID 1, Local Path ID 1, version 852258
Community: 6:25003 5585:1 5585:60 25003:10 25003:99
Extended community: RT:1234:1234 RT:5585:5585
Source AFI: VPNv4 Unicast, Source VRF: default, Source Route Distinguisher: 1.1.1.71:5585
Path #2: Received by speaker 0
Not advertised to any peer
9999 12, (Received from a RR-client), (received & used)
1.1.1.71 (metric 210) from 1.1.1.71 (1.1.1.71)
Received Label 31819
Origin IGP, localpref 560, valid, internal, imported
Received Path ID 1, Local Path ID 0, version 0
Community: 6:25003 8551:1 8551:60 25003:10 25003:99
Extended community: RT:1234:1234 RT:8551:8551
Source AFI: VPNv4 Unicast, Source VRF: default, Source Route Distinguisher: 1.1.1.71:8551

Can anyone help?

Regards,

Adi.

[MHM Cisco World]

maximum-paths eibgp 6 <<- why eibgp ? it only two eBGP so only maximum-paths 6.

[IBEngTeam]

Hi,

Thanks for the reply.

Please note that this is a cisco 9k device, and the command maximum-paths eibgp 6 is for both ibgp and ebgp:

"To configure multipath load sharing for external BGP (eBGP) and internal (iBGP) routes, use the maximum-paths eibgp".

Anyway, we tried all options and same result.

Adi.

[MHM Cisco World]

maximum-paths eibgp <<- when one path is ebgp and other is ibgp

Maximum-paths <<- when both path ebgp

Maximum-paths ibgp  <<- when both path ibgp 

LET ME CHECK MY NOTE AGAIN.

[IBEngTeam]

Hi,

Thanks for the clarification.

So, We configure the following:

router bgp 25003
vrf internet
rd 1.1.1.61:1
bgp bestpath as-path multipath-relax
address-family ipv4 unicast
maximum-paths ebgp 6
maximum-paths ibgp 6
!
!

But same result, only one path is selected, and no multipath:

sh bgp vrf internet 2.52.0.0/15
Sun Aug 14 13:32:37.702 IDT
BGP routing table entry for 2.52.0.0/15, Route Distinguisher: 1.1.1.61:1
Versions:
Process bRIB/RIB SendTblVer
Speaker 875113 875113
Last Modified: Aug 14 08:01:09.588 for 05:31:28
Paths: (2 available, best #1)
Not advertised to any peer
Path #1: Received by speaker 0
Not advertised to any peer
8888 12, (Received from a RR-client), (received & used)
1.1.1.71 (metric 210) from 1.1.1.71 (1.1.1.71)
Received Label 24143
Origin IGP, localpref 560, valid, internal, best, group-best, import-candidate, imported
Received Path ID 1, Local Path ID 1, version 875113
Community: 6:25003 5585:1 5585:60 25003:10 25003:99
Extended community: RT:1234:1234 RT:5585:5585
Source AFI: VPNv4 Unicast, Source VRF: default, Source Route Distinguisher: 1.1.1.71:5585
Path #2: Received by speaker 0
Not advertised to any peer
9999 12, (Received from a RR-client), (received & used)
1.1.1.71 (metric 210) from 1.1.1.71 (1.1.1.71)
Received Label 31819
Origin IGP, localpref 560, valid, internal, imported
Received Path ID 1, Local Path ID 0, version 0
Community: 6:25003 8551:1 8551:60 25003:10 25003:99
Extended community: RT:1234:1234 RT:8551:8551
Source AFI: VPNv4 Unicast, Source VRF: default, Source Route Distinguisher: 1.1.1.71:8551

Could this be because both routes are comming from the same router - 1.1.1.71?

Thanks,

Adi

[IBEngTeam]

One more thing:

When checking a route that is received from two different routers all seem to work correctly:

sh bgp vrf internet 10.5.5.0
Sun Aug 14 13:36:47.490 IDT
BGP routing table entry for 10.5.5.0/24, Route Distinguisher: 1.1.1.71:5585
Versions:
Process bRIB/RIB SendTblVer
Speaker 223745 223745
Last Modified: Jul 31 17:13:15.203 for 1w6d
Paths: (4 available, best #1)
Advertised to CE peers (in unique update groups):
192.114.62.105
Path #1: Received by speaker 0
Advertised to CE peers (in unique update groups):
192.114.62.105
65133, (received & used)
1.1.1.60 (metric 210) from 1.1.1.60 (1.1.1.60)
Received Label 27112
Origin incomplete, metric 0, localpref 500, valid, internal, best, group-best, multipath, import-candidate, imported
Received Path ID 1, Local Path ID 1, version 223745
Extended community: RT:5432:5432
Source AFI: VPNv4 Unicast, Source VRF: default, Source Route Distinguisher: 1.1.1.60:1
Path #2: Received by speaker 0
Not advertised to any peer
65133, (received & used)
1.1.1.60 (metric 210) from 1.1.1.60 (1.1.1.61)
Received Label 24455
Origin incomplete, metric 0, localpref 500, valid, internal, imported
Received Path ID 1, Local Path ID 0, version 0
Extended community: RT:5432:5432
Originator: 1.1.1.61, Cluster list: 1.1.1.60
Source AFI: VPNv4 Unicast, Source VRF: default, Source Route Distinguisher: 1.1.1.61:1
Path #3: Received by speaker 0
Not advertised to any peer
65133, (received & used)
1.1.1.61 (metric 210) from 1.1.1.61 (1.1.1.60)
Received Label 24456
Origin incomplete, metric 0, localpref 500, valid, internal, multipath, import-candidate, imported
Received Path ID 1, Local Path ID 0, version 0
Extended community: RT:5432:5432
Originator: 1.1.1.60, Cluster list: 1.1.1.61
Source AFI: VPNv4 Unicast, Source VRF: default, Source Route Distinguisher: 1.1.1.60:1
Path #4: Received by speaker 0
Not advertised to any peer
65133, (received & used)
1.1.1.61 (metric 210) from 1.1.1.61 (1.1.1.61)
Received Label 24895
Origin incomplete, metric 0, localpref 500, valid, internal, import-candidate, imported
Received Path ID 1, Local Path ID 0, version 0
Extended community: RT:5432:5432
Source AFI: VPNv4 Unicast, Source VRF: default, Source Route Distinguisher: 1.1.1.61:1

[MHM Cisco World]

I do lab one week ago with RR and because I config next-hop-self in RR, the client refuse add additional path, 
and I remove the next-hop-self and the client still refuse add path because the next-hop now is the PE IP which is unknown for RR client, 
I add static route only for this next-hop and the RR client success add both path. 

[IBEngTeam]

Hi,

We are using next hop self on the RR client, so the issue is that both routes have the same next hop?

Any idea on how to bypass this?

[IBEngTeam]

Hi,

So I configured on the RR client the following:

route-policy rp_test
if destination in (2.52.0.0/15) and community matches-any (5585:1) then
set next-hop 2.2.2.2
elseif destination in (2.52.0.0/15) and community matches-any (8551:1) then
set next-hop 3.3.3.3
else
pass
endif
end-policy
!

And now all is well, and both routes are installed:

sh bgp vrf internet 2.52.0.0/15
Sun Aug 14 15:14:16.548 IDT
BGP routing table entry for 2.52.0.0/15, Route Distinguisher: 1.1.1.60:1
Versions:
Process bRIB/RIB SendTblVer
Speaker 923019 923019
Last Modified: Aug 14 15:07:40.526 for 00:06:36
Paths: (2 available, best #1)
Not advertised to any peer
Path #1: Received by speaker 0
Not advertised to any peer
8888 12, (Received from a RR-client), (received & used)
2.2.2.2 (metric 210) from 1.1.1.71 (1.1.1.71)
Received Label 24143
Origin IGP, localpref 560, valid, internal, best, group-best, multipath, import-candidate, imported
Received Path ID 1, Local Path ID 1, version 923019
Community: 6:25003 5585:1 5585:60 25003:10 25003:99
Extended community: RT:1234:1234 RT:5585:5585
Source AFI: VPNv4 Unicast, Source VRF: default, Source Route Distinguisher: 1.1.1.71:5585
Path #2: Received by speaker 0
Not advertised to any peer
9999 12, (Received from a RR-client), (received & used)
3.3.3.3 (metric 210) from 1.1.1.71 (1.1.1.71)
Received Label 31819
Origin IGP, localpref 560, valid, internal, multipath, import-candidate, imported
Received Path ID 1, Local Path ID 0, version 0
Community: 6:25003 8551:1 8551:60 25003:10 25003:99
Extended community: RT:1234:1234 RT:8551:8551
Source AFI: VPNv4 Unicast, Source VRF: default, Source Route Distinguisher: 1.1.1.71:8551

So the issue is with the next hop self on the RR client, any way around this?

[MHM Cisco World]

in MPLS Core the RR can config with Next-hop-unchanged command, BUT 
let me mention 
in MPLS Core the RR will only use to forward prefix from one PE to other PE, but the real data path not pass through the RR, and hence the are meaning of using next-hop-unchanged 
in your case since the RR/RR client is source of both Path and data path pass the RR/RR client  then logically the BGP add only one path. 
but as I mention we can use next-hop-unchanged command to workaround.

https://www.cisco.com/c/en/us/td/docs/ios-xml/ios/iproute_bgp/configuration/xe-16/irg-xe-16-book/bgp_next_hop_unchanged.pdf

[IBEngTeam · Accepted solution]

Hi,

The only solution i found for this issue, is setting the nexthop using route policy:

route-policy rp_test
if destination in (2.52.0.0/15) and community matches-any (5585:1) then
set next-hop 2.2.2.2
elseif destination in (2.52.0.0/15) and community matches-any (8551:1) then
set next-hop 3.3.3.3
else
pass
endif
end-policy
!

Thanks,

All