BGP default route not being installed in the routing table.

[ergamusai]

I have a multihoming BGP setup, one 6509 connected to two ISPs. One is the primary, and one is the backup. 

I am receiving only default routes from both ISPs. The BGP neighborship is established, but both default routes received have the rib-failure flag.

router bgp 55555
bgp log-neighbor-changes
neighbor 1.1.1.1 remote-as 111
neighbor 1.1.1.1 version 4
neighbor 2.2.2.2 remote-as 222
neighbor 2.2.2.2  version 4
!
address-family ipv4
network X.X.X.X mask 255.255.254.0
neighbor 1.1.1.1 activate
neighbor 1.1.1.1 weight 50
neighbor 1.1.1.1 soft-reconfiguration inbound
neighbor 1.1.1.1 distribute-list 1_IN in
neighbor 1.1.1.1 distribute-list 1-OUT out
neighbor 2.2.2.2 activate
neighbor 2.2.2.2  send-community
neighbor 2.2.2.2  weight 40
neighbor 2.2.2.2  soft-reconfiguration inbound 

neighbor 2.2.2.2  route-map 2_OUT out
exit-address-family

interface Vlan10
description ISP Connections
ip address 2.2.2.3 255.255.255.248 secondary
ip address 1.1.1.2 255.255.255.248

r> 0.0.0.0 1.1.1.1 50 111 i
r 2.2.2.2 40 222 i

The only way I can make this scenario work is by adding the static routes

ip route 0.0.0.0 0.0.0.0 1.1.1.1

ip route 0.0.0.0 0.0.0.0 2.2.2.2

Even when I remove the static routes both default routes I receive have rib-failure flag

Thank you in advance,

[Philip D'Ath]

The stub networks must be in the BGP route table for this to work (which they wont be if you are only getting a default route).

Something like this will help:

router bgp 55555
  network 1.1.1.0 mask 255.255.255.248
  network 2.2.2.0 mask 255.255.255.248

[ergamusai]

The stub network is directly connected so it is in the routing table.

I have the same setup in 3 other locations and it is working as it should. The stub network is not in the network statement there.

[Philip D'Ath]

I don't know what else to say.  You have a RIB failure.  Tease me and add the network statements.  BGP needs to know about it - or you will get a RIB failure.

[Krishan PalSingh]

Hi, Could you share output of ?

show ip route 0.0.0.0

sh ip bgp rib-failure

show ip bgp 0.0.0.0

show ip route connected

[ergamusai]

I cannot remove both static routes as this a production network. I removed one of them, the one that leads to the secondary circuit. (ip route 0.0.0.0 0.0.0.0 2.2.2.2)

I added the statement network 2.2.2.0 in the network statements

Router#sh ip route 0.0.0.0
Routing entry for 0.0.0.0/0, supernet
Known via "static", distance 1, metric 0, candidate default path
Routing Descriptor Blocks:
* 1.1.1.1
Route metric is 0, traffic share count is 1

Router#sh ip bgp rib-failure
Network Next Hop RIB-failure RIB-NH Matches
0.0.0.0 1.1.1.1 Higher admin distance n/a

Router#sh ip bgp 0.0.0.0
BGP routing table entry for 0.0.0.0/0, version 2911
Paths: (4 available, best #1, table default, RIB-failure(17))
Advertised to update-groups:
23
Refresh Epoch 1
111
1.1.1.1 from 1.1.1.1 (X.X.X.X)
Origin IGP, localpref 100, weight 50, valid, external, best
rx pathid: 0, tx pathid: 0x0
Refresh Epoch 1
111, (received-only)
1.1.1.1 from 1.1.1.1 (X.X.X.X)
Origin IGP, localpref 100, valid, external
rx pathid: 0, tx pathid: 0
Refresh Epoch 1
222
2.2.2.2 from 2.2.2.2 (Y.Y.Y.Y)
Origin IGP, localpref 100, weight 40, valid, external
rx pathid: 0, tx pathid: 0
Refresh Epoch 1
222, (received-only)
2.2.2.2.from 2.2.2.2 (Y.Y.Y.Y)
Origin IGP, localpref 100, valid, external
rx pathid: 0, tx pathid: 0

Router#show ip route connected

Gateway of last resort is 1.1.1.1 to network 0.0.0.0

1.0.0.0/8 is variably subnetted, 2 subnets, 2 masks
C 1.1.1.0/29 is directly connected, Vlan10
L 1.1.1.2/32 is directly connected, Vlan10

C 2.2.2.2/0 is directly connected, Vlan10
L 2.2.2.1/32 is directly connected, Vlan10

[Philip D'Ath]

Here is what you can do to test this.  Change:

ip route 0.0.0.0 0.0.0.0 1.1.1.1
ip route 0.0.0.0 0.0.0.0 2.2.2.2

to:

ip route 0.0.0.0 0.0.0.0 1.1.1.1 220
ip route 0.0.0.0 0.0.0.0 2.2.2.2 220

This should allow BGP to take priority.

You can add the new routes before removing the old routes to greater safety.

[Rolf Fischer]

<deleted>

[Rolf Fischer]

<deleted> (sorry for the confusion ...)

You're absolutly correct in being very careful in a production environment. However, I agree to the suggestion in the other postings to increase the AD of your static default-routes (30 should be enough). The static default-routes then will stay in the routing-table unless the BGP-routes become valid ("floating static routes"). I really cannot see any risk in this and it will simplify the troubleshooting.

[Pawan Raut]

sh ip bgp 0.0.0.0  output show you have default route in BGP routing table receiving from ISP but that routes are not installed in router's routing table (its showing received-only in output) as you may have default routing with better AD (Admin distance) receiving from other routing protocol like static route or ospf,egrp.

Action Plan -

Please remove below static default routes and check the out put sh ip bgp 0.0.0.0  and sh ip route 0.0.0.0

ip route 0.0.0.0 0.0.0.0 1.1.1.1

ip route 0.0.0.0 0.0.0.0 2.2.2.2

[ergamusai]

This is a production network and I cannot remove the routes without a maintenance. But I need to have a solution by that time. In the last maintenance, I removed the static routes, and the default routes were still not being installed, they still had the RIB-failure flag, and the switch could not get out.

Can I test this with just removing the second static route and see if that received route will show up. I did it today, and it still have the r flag.

The only idea i have right now is to delete all bgp configs and build them again, and see what happens.

[Philip D'Ath]

Did you add the network statements for the stub network I suggested?

I did you change the administrative distance of the default static routes to 220 as I suggested - then they will still work, but if BGP starts working it will take precedence.

[ergamusai]

I just changed the AD for the static routes to 220 and the site went down. Had to put everything back the way it was immediately.

Will have to troubleshoot this during the maintenance window.

[Krishan PalSingh]

You should avoid dual static default route to multiple ISP's, unless you're really aware of potential impact of doing so. By default static routes will same AD will load balance traffic across links.

Anyways,

To answer your query,

changing AD for static default routes to 30 would help BGP to install route in table. This way you can keep both static and BGP routes. Static routes will take precedence when BGP connections are lost.

Hope this helps.

Please dont forget to mark this thread answered if helps. 

[ergamusai]

During my last maintenance the backup circuit was replaced, so that neighbor was built from scratch. The route was still not installed. 

I didn't clear the session on the primary neighbor, just removed the static route. The route is still not being installed. There are no ACLs blocking it, the ACLs for incoming traffic have only one permit any statement (the distribute lists in the neighbor statements).

I have tried to do everything that was mentioned in this thread in the secondary circuit, the route still has a rib-failure flag. I did add the network statement for the stub network.

My next maintenance be will scheduled around mid-end May. I will delete all the bgp configs at that time and add them again.