topic BGP Problem (Two ISPs) in Routing and SD-WAN

BGP Problem (Two ISPs)

CSCO12001340 — Tue, 05 Mar 2019 17:39:30 GMT

Hello !
I am having trouble with my iBGP routes. I am really confused how to set it up properly. Can anyone tell me about the wrong configurations related to my topology ? (ISP-1 CORE and ISP-2 CORE are Route Reflectors)

*** This topology should work without any interruption. For Example if Branch-x-isp-1 loses the iBGP neighborship with ISP-Core-RR , traffic should continue from the Banch-x-isp-2 router.

ISP-1 CORE RR CONFIG

router bgp 65000
bgp log-neighbor-changes
bgp redistribute-internal

neighbor 172.16.0.64 remote-as 65000
neighbor 172.16.0.64 route-reflector-client
neighbor 172.16.0.64 next-hop-self
neighbor 172.16.0.64 default-originate

neighbor 172.16.0.72 remote-as 65000
neighbor 172.16.0.72 route-reflector-client
neighbor 172.16.0.72 next-hop-self
neighbor 172.16.0.72 default-originate

neighbor 172.16.110.110 remote-as 65000
neighbor 172.16.110.110 route-reflector-client
neighbor 172.16.110.110 next-hop-self

ISP-2 CORE RR CONFIG

neighbor 172.16.110.101 remote-as 65000
neighbor 172.16.110.101 route-reflector-client
neighbor 172.16.110.101 next-hop-self

neighbor 172.29.51.209 remote-as 65000
neighbor 172.29.51.209 route-reflector-client
neighbor 172.29.51.209 next-hop-self
neighbor 172.29.51.209 default-originate

neighbor 172.29.60.27 remote-as 65000
neighbor 172.29.60.27 route-reflector-client
neighbor 172.29.60.27 next-hop-self
neighbor 172.29.60.27 default-originate

BRANCH-X-ISP-1

router bgp 65001
bgp log-neighbor-changes
bgp redistribute-internal
network 10.10.50.0 mask 255.255.255.0
neighbor 10.10.50.11 remote-as 65000
neighbor 10.10.50.11 route-reflector-client
neighbor 10.10.50.11 next-hop-self
neighbor 172.16.0.40 remote-as 65000
neighbor 172.16.0.40 next-hop-self

neighbor 172.16.0.40 route-reflector-client
neighbor 172.16.0.40 route-map LOCAL-PREF in (Sets LP to 600)

BRANCH-X-ISP-2

router bgp 65001
bgp log-neighbor-changes
bgp redistribute-internal
network 10.10.50.0 mask 255.255.255.0
neighbor 10.10.50.10 remote-as 65000
neighbor 10.10.50.10 route-reflector-client
neighbor 10.10.50.10 next-hop-self
neighbor 172.27.107.220 remote-as 65000
neighbor 172.27.107.220 next-hop-self

neighbor 172.27.107.220 route-reflector-client
neighbor 172.27.107.220 route-map LOCAL-PREF in (Sets LP to 500)

BRANCH-Y-ISP-1

router bgp 65000
bgp log-neighbor-changes
bgp redistribute-internal
network 10.10.60.0 mask 255.255.255.0
neighbor 10.10.60.11 remote-as 65000
neighbor 10.10.60.11 route-reflector-client
neighbor 10.10.60.11 next-hop-self
neighbor 172.16.0.40 remote-as 65000
neighbor 172.16.0.40 next-hop-self

neighbor 172.16.0.40 route-reflector-client
neighbor 172.16.0.40 route-map LOCAL-PREF in

BRANCH-Y-ISP-2

router bgp 65000
bgp log-neighbor-changes
bgp redistribute-internal
network 10.10.60.0 mask 255.255.255.0
neighbor 10.10.60.10 remote-as 65000
neighbor 10.10.60.10 route-reflector-client
neighbor 10.10.60.10 next-hop-self
neighbor 172.27.107.220 remote-as 65000
neighbor 172.27.107.220 next-hop-self

neighbor 172.27.107.220 route-reflector-client

Re: BGP Problem (Two ISPs)

Julio E. Moisa — Mon, 18 Dec 2017 12:42:58 GMT

You should create a cluster list on the Core router to avoid any trouble. Now if everything is iBGP (they could be all over 65000, RR is used for iBGP only) and the branches are receiving the Internet or external routes from the Core routers you could remove the next-hop-self from them. Route-reflectors should be configured on the Core routers only.

Hope it is useful

🙂

Re: BGP Problem (Two ISPs)

chrihussey — Mon, 18 Dec 2017 12:39:43 GMT

On the surface it looks like there is way too much route reflecting going on. But you have some AS discrepancies and I think before any suggestion can be provided there needs to be some clarification. For example the ISP1-CORE-RR peering with BRANCH-X-ISP1 says it is AS65000, yet BRANCH-X-ISP1 is actually AS65001.

Can you identify which routers belong to which AS?

Thanks

Re: BGP Problem (Two ISPs)

CSCO12001340 — Mon, 18 Dec 2017 13:53:39 GMT

Hi Chris,
I corrected the AS numbers, it was a typing error just.

Re: BGP Problem (Two ISPs)

CSCO12001340 — Mon, 18 Dec 2017 13:55:16 GMT

Hi Julio, what is cluster list ? 😕 Something like bgp peer group ?

Re: BGP Problem (Two ISPs)

chrihussey — Mon, 18 Dec 2017 13:56:43 GMT

Sorry, but where are the corrections?

Re: BGP Problem (Two ISPs)

CSCO12001340 — Mon, 18 Dec 2017 13:58:16 GMT

I edited my first post, all AS numbers will be 65000

Re: BGP Problem (Two ISPs)

Julio E. Moisa — Mon, 18 Dec 2017 17:39:53 GMT

The cluster id is a BGP command used to avoid loops using more than 1 route-reflector. Please check this link:

http://network-101.blogspot.com/2011/06/bgp-cluster-id-loop-prevention.html

Re: BGP Problem (Two ISPs)

chrihussey — Mon, 18 Dec 2017 20:21:31 GMT

To provide some additional information, the topology you have doesn't really allow for route reflectors the way you have things set up. Others may have different ideas but here are two suggestions:

1 - Have both cores peer with all the branch routers, create a cluster ID as Julio has suggested and define the branches as route reflector clients. Do not defined the cores as route reflector clients in the branch routers' peering to the cores. The branches only peer with the two cores. Then have the two cores peer with each other but not as route reflector clients.

2- Don't used route reflector clients at all and have all routers IBGP peer with each other. It is not a large topology and should work fine.

Finally, the assumption is that you are running an IGP (EIGRP, OSPF, ISIS, etc) supporting the BGP.

Hope this is of some help.

Re: BGP Problem (Two ISPs)

CSCO12001340 — Tue, 19 Dec 2017 05:42:16 GMT

Hi Julio,

Just to clarify, should i remove next-hop-self from the route reflectors or from the branch routers ?

Re: BGP Problem (Two ISPs)

CSCO12001340 — Tue, 19 Dec 2017 07:05:12 GMT

Hi Chris,

Thanks for your offers.

2- Don't used route reflector clients at all and have all routers IBGP peer with each other. It is not a large topology and should work fine. - I should use RR coz this is just a small part of the topology.

Re: BGP Problem (Two ISPs)

paul driver — Sat, 13 Jan 2018 12:06:32 GMT

Hello

Do you have reachability to all of the rtrs within this ibgp domain, what routing protocol are you running for connectivity for each rtrs links?

FYI -

The RR's don’t need to be RR clients to each other and this also applies between the branch rts.
The branch rtrs don’t need ibgp peering to each other as that’s what the RR are for.
Give both RR a cluster id so each rtr attaching to the their respective RR is known from cluster it is originating from

ISP-1 CORE RR CONFIG

router bgp 65000
cluster-id 10.10.10.10
no neighbor 172.16.110.110 route-reflector-client

ISP-2 CORE RR CONFIG

router bgp 65000
cluster-id 20.20.20.20
no neighbor 172.16.110.102 route-reflector-client

Branch xx

router bgp 65000
no neighbor 10.10.50.11

Branch xx

router bgp 65000
no neighbor 10.10.50.10

etc...

res
Paul

Re: BGP Problem (Two ISPs)

CSCO12001340 — Wed, 20 Dec 2017 11:24:13 GMT

Anyone has more comments about the topology ? 🙂 Thanks to everybody.

Re: BGP Problem (Two ISPs)

CSCO12001340 — Sat, 13 Jan 2018 08:31:37 GMT

Hello guys,

Thanks for your great replies. I fixed most of the topology by your comments.

1-I removed the iBGP peering between BRANCHX-ISP1 and BRANCHX-ISP2 (also for BRANCHYISP1-BRANCHY-ISP2)

2-I fixed the unnecessary next-hop-self commands

3-I removed the router-reflector-client commands between Core routers. (ISP1CORE and ISP2CORE)

But I just have one problem right now. I configured a same BGP CLUSTER ID value for both ISP1CORE and ISP2CORE. Now I can see just BRANCHX-ISP1 and BRANCHY-ISP1 advertised routes on ISP1CORE.

But what I need is , I want to see BRANCHX-ISP2 and BRANCHY-ISP2 routes on ISP1CORE as well. (For redundancy). To achieve this goal I want to establish a peering between

BRANCHX-ISP2 <->ISP1CORE
BRANCHY-ISP2 <->ISP1CORE

I am trying to configure ISP1CORE as a RR again for BRANCHX-ISP2 and BRANCHY-ISP2. But configuration fails. When i type show ip bgp summary, peering is not coming UP. It s stuck on IDLE.

How can I achieve this goal ?

Re: BGP Problem (Two ISPs)

paul driver — Sat, 13 Jan 2018 09:22:45 GMT

Hello

@CSCO12001340 wrote:

Hello guys,

But I just have one problem right now. I configured a same BGP CLUSTER ID value for both ISP1CORE and ISP2CORE. Now I can see just BRANCHX-ISP1 and BRANCHY-ISP1 advertised routes on ISP1CORE.

Try applying ISP2 a different cluster ID

res

Paul

Re: BGP Problem (Two ISPs)

CSCO12001340 — Sat, 13 Jan 2018 09:29:09 GMT

Hello,

When I do it, ISP1 and ISP2 Cores are sharing routes with each other and that is what i don't want.

Re: BGP Problem (Two ISPs)

paul driver — Sat, 13 Jan 2018 11:14:09 GMT

Hello

@CSCO12001340 wrote:

Hello,

When I do it, ISP1 and ISP2 Cores are sharing routes with each other and that is what i don't want.

They need to for residency ? - as if either if the branches lose connectivity to their one of the two ISP core it will be redundant

res

Paul

Re: BGP Problem (Two ISPs)

CSCO12001340 — Sat, 13 Jan 2018 11:25:09 GMT

Hello Paul,

To avoid routing loops, I am using BGP Cluster ID already right ?

I am trying to do the same solution. If branches lose connectivity to ISP-1, they should go ahead by using ISP-2. So they should receive routes from both RRs.

Now I can see only ISP-1 routes on CoreISP1 and only ISP-2 routes on COREISP2. What I want is see both of them on both Cores and I can manipulate the routes by local preference or sth like that to choose between them.

Re: BGP Problem (Two ISPs)

paul driver — Sat, 13 Jan 2018 21:26:22 GMT

Hello
Correct but dont forget local preference can be used only to prefer the exit path of the RR ASN not within it.

RR are basically is used to cut done on the amount of fully meshed peering required between IBGP peers but from your branch ISP you should be seeing routes from either ISP rtrs.

Just reviewed your OP and I am not so sure you even require RR, Your Core ISP rtrs are in a different ASN then the branch rtrs which means you wont even be able to have RR.

My initial suggestion focused on RR hence the example i posted shown all rtr ( core/branch) all in the same ASN, but your topology doesn't show this, So maybe its a totally different approach is required?

Can you please confirm this?

res

Paul

Re: BGP Problem (Two ISPs)

CSCO12001340 — Mon, 15 Jan 2018 06:24:43 GMT

Hello Paul,

Thanks for your help, you are great.

My Cores are in same AS, there is a mistype I think.