IBGP routing loop

[Nicholas Wysocki]

I have two routers located in different parts of the country. One currently has has bgp setup and is working. The second router was just installed and is configured but bgp is in the shutdown state waiting till this problem is fixes. I have the same AS number seetup on both. There is no other connection between the sites except the link to the isp. i setup a test lab with 3 routers to test the setup. i am tring to setup a gre tunnel between the two routers to establish bgp connection so the AS on continuges. without applying the tunnel to bgp, the tunnel works and i can ping across it. when i apply the nei for bgp of the tunnel. i am getting routing loop errors.

what are some options to solve my problem? We are wanting to bring up both routers for a failover.

[Mahesh Gohil]

Hello Nicholas,

Can you share more about the lab you prepared. Config from routers and what you want to achieve. Also the role of third router.

I mean you use it as ISP routers or for anything else.

Please share more data so that people can help you easily and quickly

Regards

Mahesh

[Adrian Lazar]

Hello Nicholas,

Where are you getting the tunnel destination route from? This is just a guess but a routing lookup loop could be happening because when you bring up BGP you receive the tunnel destination IP through it. If tunnel IP is A and tunnel destination is B they will point to each other (A->B , B->A) to reach the next hop.

If this is not the case please include more information like Mahesh suggested.

Best regards,

Adrian

[milan.kulik]

Hi,

it's probably because of BGP is advertising the same prefixes you are using already in your routing table.

But as Mahesh said, you need to show more details here to get a help.

BR,

Milan

[Nicholas Wysocki]

I attached the same info:

but i wanting to advertise the same address out at a different site using the same AS number. in about two months we have a test fail over down time window. so i would like to have the other site already advertizing the address but with a lower community so the isp will not use it unless we remove the primary router. i am thinking this will allow for a faster failover instead of turning off one router and turning on another and hopfull will advertise without problem. this will also allow me to test and verify the bgp connected to the isp is working and advertising correctly.

(router A) connected to (router B)(acting as isp router) connected to (router C)

--------router c---------
config> ip bgp-community new-format
config> router bgp 30
config-router> bgp always-compare-med
config-router> bgp bestpath med confred
config-router> bgp log-neighbor-changes
config-router> no auto-summary
config-router> no synchronization
config-router> network 4.4.4.0 mask 255.255.255.0
config-router> network 65.208.98.0 mask 255.255.255.0
config-router> neighbor 3.3.3.1 remote-as 10
config-router> neighbor 3.3.3.1 send-community
config-router> neighbor 3.3.3.1 route-map to_B out
config-router> neighbor 173.0.0.2 remote-as 30

config> route-map to_B permit 10
config-route-map> set community 174:10

interface Loopback0
ip address 65.208.98.1 255.255.255.0
!
interface Loopback1
ip address 4.4.4.1 255.255.255.255
!
interface Tunnel0
ip address 173.0.0.1 255.255.255.252
tunnel source Loopback1
tunnel destination 5.5.5.1

--------router A---------
config> ip bgp-community new-format
config> router bgp 30
config-router> bgp always-compare-med
config-router> bgp bestpath med confred
config-router> bgp log-neighbor-changes
config-router> no auto-summary
config-router> no synchronization
config-router> network 5.5.5.0 mask 255.255.255.0
config-router> network 65.208.98.0 mask 255.255.255.0
config-router> neighbor 2.2.2.1 remote-as 10
config-router> neighbor 2.2.2.1 send-community
config-router> neighbor 2.2.2.1 route-map to_B out
config-router> neighbor 173.0.0.1 remote-as 30

config> route-map to_B permit 10
config-route-map> set community 174:135

interface Loopback0
ip address 65.208.98.2 255.255.255.0
!
interface Loopback1
ip address 5.5.5.1 255.255.255.255
!
interface Tunnel0
ip address 173.0.0.2 255.255.255.252
tunnel source Loopback1
tunnel destination 4.4.4.1

--------router B---------
config> ip bgp-community new-format
config> router bgp 10
config-router> bgp always-compare-med
config-router> bgp bestpath med confred
config-router> bgp log-neighbor-changes
config-router> no auto-summary
config-router> no synchronization
config-router> neighbor to_customers peer-group
config-router> neighbor to_customers remote-as 30
config-router> neighbor to_customers send-community
config-router> neighbor to_customers default-originate
config-router> neighbor to_customers route-map from_customers in
config-router> neighbor to_customers version 4
config-router> neighbor to-customers soft-reconfiguration inbound
config-router> neighbor 2.2.2.2 peer-group to_customers
config-router> neighbor 3.3.3.2 peer-group to_customers

config> route-map from_customers permit 10
config-route-map> match community 174:10
config-route-map> set local-preference 10
config-route-map> route-map from_customers permit 15
config-route-map> match community 174:70
config-route-map> set local-preference 70
config-route-map> route-map from_customers permit 20
config-route-map> match community 174:120
config-route-map> set local-preference 120
config-route-map> route-map from_customers permit 25
config-route-map> match community 174:125
config-route-map> set local-preference 125
config-route-map> route-map from_customers permit 30
config-route-map> match community 174:135
config-route-map> set local-preference 135
config-route-map> route-map from_customers permit 35
config-route-map> match community 174:140
config-route-map> set local-preference 140
config-route-map> route-map from_customers permit 35
config-route-map> set local-preference 5

-----------------------------------------------------------------------------------------------------------------------------
cogent community strings:

community string local pref effect

174:10   10  Set customer route local preference to 10 (below everything-least perferd)--cogent san diego
174:70   70  set customer route local preference to 70 (below peers)
   80             --verizon atlanta
174:120   120  Set customer route local preference to 120 (below customer defualt)
174:125   125  Set customer route local preference to 125 (below customer defualt)
174:135   135  Set customer route local preference to 135 (above customer defualt)--cogent atlanta
174:140   140  Set customer route local preference to 140 (above customer defualt)

Atlanta:
verizon out 701:80
verizon in set local preference 80
cogent out 174:135
cogent in set local preference 110

san digo:
cogent out 174:10
cogent in set local preference 110

[JoeKeegan3]

It's odd the tunnel works, since it doesn't look to me that ether side should should learn the loopback prefix via BGP since BGP is configured to advertise 4.4.4.0/24, but does not have a route for that prefix. From what you have shown us it would only have a route for 4.4.4.1/32. Maybe there is something else going on to install a route for 4.4.4.0/24 in the RIB, or maybe reachability for the tunnel endpoints is handheld some other way?

Also you need to be careful about route filtering. In your configuration you are announcing the same prefix to both the ISP and over your tunnel. So assuming that 4.4.4.0/24 is being announced then your ISP would learn the route via eBGP and your tunnle partner would learn the route via iBGP.

So your tunnel peer will learn the route for 4.4.4.0/24 two ways, iBGP over the tunnel and eBGP from the ISP. The eBGP path will be selected which could be causing some of the issue or at the very least lead to an undesirable path. I'd recommend you do more route filtering to ensure only public prefixes are announced to the ISP and only private prefixes are announced to between the tunnel peers.

It might also be helpful to get a show ip bgp from each of the routers when BGP is running.

Joe

[Nicholas Wysocki]

if you want to see the debug messages just let me know and i can upload to paste them in.

[cadet alain]

Hi,

advertizing the address but with a lower community so the isp will not use it unless we remove the primary router. 

You must decide with your ISP what to do with communities otherwise they will not be used for path selection.

Regards.

Alain.

[Nicholas Wysocki]

question 1

i have a null route for 4.4.4.0 255.255.255.0 in one router and the other as a null router for 5.5.5.0 255.255.255.0

then i network the address into bgp to advertize out. that is why the isp router has routes to the loopback address.

question 2

that is exactly what i am trying to do. the community that is shown is what my isp uses to set the local preference. thats not the problem. the problem is on the routers, without the tunnel (not running ibgp between the routers) the router continue to search for the orgin of the local networks that are configrued. with the tunnel and while the tunnel is working the end routers complete the map and finish finding the orgin of the local configured network. but then bgp starts to look at the tunnel and find "recursive routing" as the debug message puts it. it temp disables the tunnel, although the inter show up/up. cuaseing the bgp connection to disconnect and then start to the seach for the orgin on the local networks.

[JoeKeegan3]

Can you post the output of show ip bgp from each of the routers?

The info form the debug would be good too.

[cadet alain]

Hi,

ok so much for me, sorry.

 then bgp starts to look at the tunnel and find "recursive routing" as 
the debug message puts it. it temp disables the tunnel, although the 
inter show up/up

it's still up/up because there is no keepalive on tunnel interface

recursive routing is because router thinks the best route to go to tunnel destination is through the tunnel itself.

Regards.

Alain.

[Nicholas Wysocki]

sh ip bgp

rtrA#
   Network          Next Hop            Metric LocPrf Weight Path
* i0.0.0.0          3.3.3.1                  0    100      0 10 i
*>                  2.2.2.1                                0 10 i
*>i4.4.4.0/24       173.0.0.1                0    100      0 i
*> 5.5.5.0/24       0.0.0.0                  0         32768 i
* i65.208.98.0/24   173.0.0.1                0    100      0 i
*>                  0.0.0.0                  0         32768 i
rtrA#

rtrB#

   Network          Next Hop            Metric LocPrf Weight Path
*  4.4.4.0/24       2.2.2.2                                0 30 i
*>                  3.3.3.2                  0             0 30 i
*> 5.5.5.0/24       2.2.2.2                  0             0 30 i
*  65.208.98.0/24   2.2.2.2                  0             0 30 i
*>                  3.3.3.2                  0             0 30 i
rtrB#

rtrC#

   Network          Next Hop            Metric LocPrf Weight Path
*> 0.0.0.0          3.3.3.1                                0 10 i
*> 4.4.4.0/24       0.0.0.0                  0         32768 i
*> 65.208.98.0/24   0.0.0.0                  0         32768 i
rtrC#

[JoeKeegan3]

Yeah there is some weirdness here, like for example router B is learning two paths to 4.4.4.0/24 and likewise we can see router A is learning two defaults. I would recommend doing some filtering to get this cleaned up, while it may not solve your issue it will make things easier to troubleshoot.

Ideally A & C should only be learning the route to the tunnel endpoints over B and not via the tunnel. Likewise A & C should only be advertising their local routes to each other and not routes learned from B.

I'm presuming these sites have internal prefixes that A & C are suppose to learn over the tunnel, so I would look at using something like the following config changes to do the filtering (this is a delta to your existing config and not complete). I've also setup filter to only recieve a default from the ISP router (router B), assuming these sites are single homed then this would be best practice, but it's optional.

Router A

ip prefix list ISP_MAP_IN seq 5 permit 0.0.0.0/0

ip prefix list ISP_MAP_OUT seq 5 permit 65.208.98.0/24
ip prefix list ISP_MAP_OUT seq 10 permit 5.5.5.0/24

ip prefix list iBGP_MAP seq 5 permit

route-map to_B permit 10
match ip address prefix-list ISP_MAP_OUT

route-map from_B permit 10
match ip address prefix-list ISP_MAP_IN

route-map to_C permit 10
match ip address prefix-list iBGP_MAP

router bgp 30
neighbor 173.0.0.1 route-map to_C out

Router C

ip prefix list ISP_MAP_IN seq 5 permit 0.0.0.0/0

ip prefix list ISP_MAP_OUT seq 5 permit 65.208.98.0/24
ip prefix list ISP_MAP_OUT seq 10 permit 4.4.4.0/24

ip prefix list iBGP_MAP seq 5 permit

route-map to_B permit 10
match ip address prefix-list ISP_MAP_OUT

route-map from_B permit 10
match ip address prefix-list ISP_MAP_IN

route-map to_A permit 10
match ip address prefix-list iBGP_MAP

router bgp 30
neighbor 173.0.0.2 route-map to_A out

[JoeKeegan3 · Accepted solution]

I forgot to do the inbound filtering from the ISP, but I think you'll get the idea.

[Nicholas Wysocki]

The filtering corrected my problem. thanks for the help.