Solved: Triple ISP with Cisco IOS XR

Marc du · ‎06-13-2016

Hi Cisco Experts

Greeting

I lost my way with RPL config , calling all of you to help me how the RPL config looks like for the below scenario , 2 X ASR 9006

0utbound policy
===============

1. AS 52X accepts local routes from the 3 providors , along with default for the rest of the internet routes
2. Traffic destined to ISP-1 goes via ASR-1
3. Traffic destined to ISP-2 goes via ASR-1
4. Traffic destined to ISP-3 goes via ASR-2
5. all other traffic prefer ASR-1 ISP-1
6. if link to ISP-1 goes down traffic should re-route it via ASR-1 ISP-2
7. if link to ISP-2 goes down traffic should re-route it via ASR-2 ISP-3

Inbound policy
==============

1. Traffic is destinet to PI Subnet 91.X.X.X/24 from Internet shoud come Via ASR-1 ISP-1
2. Traffic is destinet to PI Subnet 92.X.X.X/24 from Internet shoud come Via ASR-1 ISP-2
3. Traffic is destinet to PI Subnet 93.X.X.X/24 from Internet shoud come Via ASR-2 ISP-3
4. if one ISP or Link Fails , other ISP should route traffic back to AS 52X from internet

Thanks all

Sayed hasan moubarak

xthuijs · ‎06-20-2016

that looks good now! multipath is enabled and both default routes will be used on a per flow bases. and if there is good distribution this will be near 50%.

if you want to take the bw into consideration and ratio it 2:1 you need to use dmz link bandwidth to help BGP understand the bandwidth of each link so instead of 50/50 we do 66/33. You can read here how to set up dmz link bw.

rule 5 of BGP best path selection is the as-path. the shorter path will win.

in many cases for loadbalancing over multipath, the different available paths have different lengths of the AS path showing a shorter AS hop count towards the destination. With the as-path ignore we can omit looking at the AS-PATH as a tie breaker for a preferred route.

the AS relax would override the stnadard rule for mpath that both AS-paths should match from all routes. this is also a silly rule, since like in your case you have different ISP's, and they will have different as-path sets and or lengts, each of those knobs will override either of these two rules for as-path consideration.

xander

View solution in original post

xthuijs · ‎06-14-2016

hi cosco,

ah thanks for that picture that helps a lot!

good that you have an iBGP peering so we can easily manage a few things.

you would need to enable the max-paths eiBGP to both process e and i bgp paths

since the as-path is not the same from different ISP's and possibly the length netiher, you

also need the override knobs for: multipath ignore (length) and relax (content of the as-path)

then when that is working, you will receive all routes from all ISP's either directly from their eBGP peer OR via the iBGP path.

Then you can set either on the neighbor statement that this neighbor has a higher weight , but that is local.

You can also set local preference on the routes received from that neighbor.

Noting that LP is advertised to other BGP peers in iBGP. weight is not advertised so keeps it local to this router under config.

For advertisement outbound, this is trickier, one way to manipulate inbound traffic by an outbound policy is by AS-path prepending, but not many ISP's like that and they may strip it.

there is very little transitive attributes that can be used to affect inbound routing while maintaining redundancy.

One option is to use the advertise statement via the network command on each designated ISP peering router, this so we ONLY advertise X on one, Y on the other.

but this means that if "ONE" dies, X route is not accessible at all, which then can be leveraged with some conditional routing on two to have an IPSLA probe to test for ONE existence, and based on that inject a static route in the routing table that than BGP can pick up. (hacky :)

cheers!

xander

Marc du · ‎06-15-2016

Hi xander

I am happy you xander answer my thread ,,,thanks for your reply ,

bro pls pls help me in RPL config , how the route policy looks like on both ASR 9006

Thanks bro

xthuijs · ‎06-16-2016

Sample configs here you go.

let me know when I need to be onsite where, to implement this and collect the check ;)

xander

router static
 address-family ipv4 unicast
  91.0.0.0/24 Null0 track NH

router bgp 64524
 address-family ipv4 unicast
  network 91.0.0.0/24

ipsla
 operation 1
  type icmp echo
   destination address 1.1.1.1
   timeout 1000
   frequency 5

 schedule operation 1
  start-time now
  life forever

track NH
 type rtr 1 reachability
end

RPL:

On R1/R2/R3 configure the weight parameter on inbound for their ISP neighbors.

route-policy weight-in

set weight 80000

end

Prefix set for R1/R2/R3. Change 91 to 92/93 for all devices.

prefix-set 9X
91.0.0.0/24
end

route-policy Rx-out 
if not destination in 9X then 
 prepend as-path 10 3
else
 pass
end

Marc du · ‎06-16-2016

Hi Bro Alexander

thanks for ur quick reply and Help

Don't imagine bro how much i was under pressure after we bought this boxes and found out the we need RPL instead of Route-map for BGP route manipulation , i am blind on RPL config , there is no such Scenario in Cisco Docs with regard to RPL on XR to deal with BGP Multihoming on IOS XR , only found one but for regular IOS

millions thanks bro for ur answer as this kind of setup require Cisco TAC not others contributor in this Forum ,also no one answer me

xthuijs · ‎06-16-2016

ouch sorry to hear, but glad it helped!

this reference on RPL may help in future maybe.

cheers!!

xander

Marc du · ‎06-17-2016

Hi Xander

thanks for you

Bro am a bit confused in the config , which XR/RPL syntax fit as per the below policy , line by line pls ,

as per the template provided i m confused which to which ,

I dont have R1/R2/R3 , only 2 boxes B-ASR-1 and B-ASR-2

RPL:

On R1/R2/R3 configure the weight parameter on inbound for their ISP neighbors.

route-policy weight-in

set weight 80000

end

Prefix set for R1/R2/R3. Change 91 to 92/93 for all devices.

0utbound policy
===============

1. AS 52X accepts local routes from the 3 providors , along with default for the rest of the internet routes
2. Traffic destined to ISP-1 goes via ASR-1
3. Traffic destined to ISP-2 goes via ASR-1
4. Traffic destined to ISP-3 goes via ASR-2
5. all other traffic prefer ASR-1 ISP-1
6. if link to ISP-1 goes down traffic should re-route it via ASR-1 ISP-2
7. if link to ISP-2 goes down traffic should re-route it via ASR-2 ISP-3

Inbound policy
==============

1. Traffic is destinet to PI Subnet 91.X.X.X/24 from Internet shoud come Via ASR-1 ISP-1
2. Traffic is destinet to PI Subnet 92.X.X.X/24 from Internet shoud come Via ASR-1 ISP-2
3. Traffic is destinet to PI Subnet 93.X.X.X/24 from Internet shoud come Via ASR-2 ISP-3
4. if one ISP or Link Fails , other ISP should route traffic back to AS 52X from internet

thanks and best regards

sayed hasan moubarak

xthuijs · ‎06-17-2016

"traffic destined for ISP-1" is a tricky thing... an ISP generally doesn't "host anything" but we can prefer prefixes from his ASN. However when we do that, since he is not hosting anything we wouldn't be gaining much. If the train of thought is to prefer routes that ISP-1 has the shortest AS-PATH to, that is general nature already.

At any rate, for RTR-1, ISP-1 neighbor the policy could look like:

in: (1234 is the ASN for ISP-1)

route-policy ISP1-in

if as-path originates-from '1234' then

set local-preference 200

endif

end-policy

this means that both RTR-1 and RTR-2 will see and prefer paths originating from 1234 all sent through the link ISP-1 of RTR-1.

For the outbound policies, the same configlet I provided will apply.

You'll advertise your 9y.x.x.x prefix prepended with your own AS (10 in my earlier example I used).

so you as-prepend 92 and 93 networks towards ISP-1, you prepend 91 and 93 for ISP2 etc.

xander

Marc du · ‎06-17-2016

Hi Bro Alexander

your kind input bro :

"traffic destined for ISP-1" is a tricky thing... an ISP generally doesn't "host anything"

my output:

1. AS 52X accepts local routes from the 3 providors , along with default for the rest of the internet routes

thanks for answers , i will simulate this setup on XRv , I will make template for the real production

let you know how it goes ,

Millions thanks bro

Marc du · ‎06-20-2016

Hi Alexander

Pls have a look to the output , why i cant get load balance , even though i have to default route from both ISP ( ISP-1 and ISP-2) , what else do i need

router bgp 52
bgp router-id 1.1.1.1
bgp log neighbor changes detail
bgp bestpath as-path multipath-relax
address-family ipv4 unicast
maximum-paths ebgp 8
maximum-paths ibgp 8

Origin codes: i - IGP, e - EGP, ? - incomplete
   Network            Next Hop            Metric LocPrf Weight Path
*> 0.0.0.0/0          192.168.31.3 (ISP-1)                  250      0 300 i
*                     192.168.51.2 (ISP-2)                200      0 500 i

RP/0/0/CPU0:XR-1#TRAceroute ipv4 60.60.60.60 source 92.2.3.254
Mon Jun 20 03:27:19.573 UTC
% Invalid source address
RP/0/0/CPU0:XR-1#TRAceroute ipv4 60.60.60.60 source 92.2.2.254
Mon Jun 20 03:27:25.813 UTC

Type escape sequence to abort.
Tracing the route to 60.60.60.60

1 192.168.31.3 9 msec 0 msec 0 msec
2 10.1.1.10 0 msec * 0 msec

RP/0/0/CPU0:XR-1#TRAceroute ipv4 60.60.60.60 source 91.1.1.254
Mon Jun 20 03:27:42.052 UTC

Type escape sequence to abort.
Tracing the route to 60.60.60.60

1 192.168.31.3 0 msec 0 msec 0 msec
2 10.1.1.10 0 msec * 0 msec

RP/0/0/CPU0:XR-1#sh route     60.60.60.60
Mon Jun 20 03:29:39.294 UTC

================================================================

Routing entry for 0.0.0.0/0
Known via "bgp 52", distance 20, metric 0, candidate default path
Tag 300, type external
Installed Jun 20 03:22:57.961 for 00:06:41
Routing Descriptor Blocks
    192.168.31.3, from 192.168.31.3, BGP external
      Route metric is 0
No advertising protos.
RP/0/0/CPU0:XR-1#sh route     60.60.60.60
Mon Jun 20 03:29:59.582 UTC

Routing entry for 0.0.0.0/0
Known via "bgp 52", distance 20, metric 0, candidate default path
Tag 300, type external
Installed Jun 20 03:22:57.961 for 00:07:01
Routing Descriptor Blocks
    192.168.31.3, from 192.168.31.3, BGP external
      Route metric is 0
No advertising protos.

thanks a lot

Sayed Hasan Moubarak

xthuijs · ‎06-20-2016

on that device,check the show bgp 0.0.0.0/0 it should say multipath when the right multipath is enabled.

the show bgp will always indicate one prefix as ">" best path as per normal selection.

in that case it chose a path because the local preference is 250 over the other 200.

in order for multipath to succeed, weight, lp, aspath (can be relaxed/ignored) need to be the same.

cheers

xander

Marc du · ‎06-20-2016

Hi Alexander

thanks for ur reply

here my global BGP config , what else do i need

router bgp 52
bgp router-id 1.1.1.1
bgp log neighbor changes detail
bgp bestpath as-path multipath-relax
address-family ipv4 unicast
maximum-paths ebgp 8
maximum-paths ibgp 8

RP/0/0/CPU0:XR-1#   sh bgp 0.0.0.0/0
Mon Jun 20 23:02:21.326 UTC
BGP routing table entry for 0.0.0.0/0
Versions:
Process           bRIB/RIB SendTblVer
Speaker                 11          11
Last Modified: Jun 20 23:01:12.243 for 00:01:09
Paths: (2 available, best #1)
Advertised to peers (in unique update groups):
    192.168.21.2
Path #1: Received by speaker 0
Advertised to peers (in unique update groups):
    192.168.21.2
300
    192.168.31.3 from 192.168.31.3 (30.30.30.30)
      Origin IGP, localpref 250, valid, external, best, group-best
      Received Path ID 0, Local Path ID 1, version 11
      Origin-AS validity: not-found
Path #2: Received by speaker 0
Not advertised to any peer
500
    192.168.51.2 from 192.168.51.2 (50.50.50.50)
      Origin IGP, localpref 200, valid, external
      Received Path ID 0, Local Path ID 0, version 0
      Origin-AS validity: not-found
RP/0/0/CPU0:XR-1#

thanks a lot bro

xthuijs · ‎06-20-2016

oh it is not the show run router bgp

but the show bgp <prefix>/<mask> that is off interest that shows the prefix and path attributes.

but it shows that the localpref is higher for one of the paths, so there will be no loadbalancing.

this local pref you likely set in your rpl config somewhere

so you would want to put an if statement in there something like:

if destination in 0.0.0.0/0 then

set local pref 100

done

elseif ...<all your other stuff here>

end

this so to fix the LP to 100 and needs to be done on both border routers.

xander

Marc du · ‎06-20-2016

Hi Alexander

here you go bro for " sh bgp 0.0.0.0/0 "

RP/0/0/CPU0:XR-1#   sh bgp 0.0.0.0/0
Mon Jun 20 23:02:21.326 UTC
BGP routing table entry for 0.0.0.0/0
Versions:
Process           bRIB/RIB SendTblVer
Speaker                 11          11
Last Modified: Jun 20 23:01:12.243 for 00:01:09
Paths: (2 available, best #1)
Advertised to peers (in unique update groups):
    192.168.21.2
Path #1: Received by speaker 0
Advertised to peers (in unique update groups):
    192.168.21.2
300
    192.168.31.3 from 192.168.31.3 (30.30.30.30)
      Origin IGP, localpref 250, valid, external, best, group-best
      Received Path ID 0, Local Path ID 1, version 11
      Origin-AS validity: not-found
Path #2: Received by speaker 0
Not advertised to any peer
500
    192.168.51.2 from 192.168.51.2 (50.50.50.50)
      Origin IGP, localpref 200, valid, external
      Received Path ID 0, Local Path ID 0, version 0
      Origin-AS validity: not-found
RP/0/0/CPU0:XR-1#

thanks

xthuijs · ‎06-20-2016

yeah the path 192.168.31.3 has a local pref of 250, the other path from 192.168.51.2 has a local pref of 200.

they need to be equal to whatever value you like (as per example from previous note).

cheers

xander