BGP with Multiple ISP not functioning

[mbbmani]

Hi,

I have 3 BGP session with 3 ISP using 3 routers.

The BGP session works but only 1 at a time.

Below are my configs

router bgp 22222
 bgp log-neighbor-changes
 neighbor YYY.YYY.YYY.YYY remote-as 123456
 !
 address-family ipv4
  bgp dampening
  network XXX.XXX.XXX.0 mask 255.255.255.252
  network XXX.XXX.XXX.16 mask 255.255.255.240
  network XXX.XXX.XXX.32 mask 255.255.255.224
  network XXX.XXX.XXX.64 mask 255.255.255.192
  network XXX.XXX.XXX.128 mask 255.255.255.128
  network XXX.XXX.XXY.0 mask 255.255.255.0
  network XXX.XXX.XXZ.0 mask 255.255.255.0
  aggregate-address XXX.XXX.XXX.0 255.255.252.0 summary-only
  neighbor YYY.YYY.YYY.YYY activate
  neighbor YYY.YYY.YYY.YYY remove-private-as
  neighbor YYY.YYY.YYY.YYY soft-reconfiguration inbound
  neighbor YYY.YYY.YYY.YYY prefix-list inbound-route-filter in
  neighbor YYY.YYY.YYY.YYY prefix-list route-advertise out
 exit-address-family
!

What could it be the problem.

Please advice.

[Muhammad Uzair · Accepted solution]

Can you please more elaborate what do mean by "only 1 BGP work at a time" in which state they are (IDLE/Active) when they stop working ? can you please share the output of "show ip bgp summary" from 3 routers ? above configuration is showing only 1 router, appreciate if you can also share the topology table of your network as well.

Thank you.

Kindest regards,

uzzi 

[mbbmani]

Above is the network diagram. Blue color box is link balancer.

All 3 routers has the same config.

This is the sh ip bgp summary

Router 1 (Internet access good)

BGP router identifier 111.111.111.129, local AS number 111111
BGP table version is 15, main routing table version 15
8 network entries using 1088 bytes of memory
8 path entries using 448 bytes of memory
3/3 BGP path/bestpath attribute entries using 384 bytes of memory
1 BGP AS-PATH entries using 24 bytes of memory
0 BGP route-map cache entries using 0 bytes of memory
0 BGP filter-list cache entries using 0 bytes of memory
BGP using 1944 total bytes of memory
Dampening enabled. 0 history paths, 0 dampened paths
BGP activity 8/0 prefixes, 8/0 paths, scan interval 60 secs

Neighbor        V           AS MsgRcvd MsgSent   TblVer  InQ OutQ Up/Down  State/PfxRcd
11x.11x.11x.77   4         111111     852     855       15    0    0 12:49:39        1

Router 2 (No Internet Access)

BGP router identifier 222.222.222.158, local AS number 222222
BGP table version is 24, main routing table version 24
8 network entries using 1088 bytes of memory
8 path entries using 448 bytes of memory
3/3 BGP path/bestpath attribute entries using 384 bytes of memory
1 BGP AS-PATH entries using 24 bytes of memory
0 BGP route-map cache entries using 0 bytes of memory
0 BGP filter-list cache entries using 0 bytes of memory
BGP using 1944 total bytes of memory
Dampening enabled. 0 history paths, 0 dampened paths
BGP activity 9/1 prefixes, 11/3 paths, scan interval 60 secs

Neighbor        V           AS MsgRcvd MsgSent   TblVer  InQ OutQ Up/Down  State/PfxRcd
222.222.222.145  4        222222    1429    1441       24    0    0 21:38:24        1

Router 3  ( No internet access)

BGP router identifier 333.333.333.134, local AS number 333333
BGP table version is 16, main routing table version 16
8 network entries using 960 bytes of memory
8 path entries using 416 bytes of memory
3/3 BGP path/bestpath attribute entries using 372 bytes of memory
1 BGP AS-PATH entries using 24 bytes of memory
0 BGP route-map cache entries using 0 bytes of memory
0 BGP filter-list cache entries using 0 bytes of memory
BGP using 1772 total bytes of memory
Dampening enabled. 0 history paths, 0 dampened paths
BGP activity 8/0 prefixes, 8/0 paths, scan interval 60 secs

Neighbor        V           AS MsgRcvd MsgSent   TblVer  InQ OutQ Up/Down  State/PfxRcd
333.333.333.133 4        333333     863     866       16    0    0 12:59:40        1

[mbbmani]

hi,

Only Router 1 has internet access.

Router 2 and 3 does not have internet access.

router-1r#ping 8.8.8.8 so gi0/1
Type escape sequence to abort.
Sending 5, 100-byte ICMP Echos to 8.8.8.8, timeout is 2 seconds:
Packet sent with a source address of NNN.NNN.NNN.1
!!!!!
Success rate is 100 percent (5/5), round-trip min/avg/max = 1/2/4 ms

router-2#ping 8.8.8.8 so fa0/1
Type escape sequence to abort.
Sending 5, 100-byte ICMP Echos to 8.8.8.8, timeout is 2 seconds:
Packet sent with a source address of NNN.NNN.NNN.9

router-3#ping 8.8.8.8 so gi0/1

Type escape sequence to abort.
Sending 5, 100-byte ICMP Echos to 8.8.8.8, timeout is 2 seconds:
Packet sent with a source address of NNN.NNN.NNN.5
.....
Success rate is 0 percent (0/5)

.....

[Muhammad Uzair]

With your shown output it seems you are advertising 1 prefix to all of 3 upstream, I don't know why you are doing this because in a usual scenario 1 edge router and then you configure 3 ebgp in one instance (bgp AS) or if you do multi homing yoi configure dual ISP setup but use MED if ISP agree or AS PREPAND which is known attribute and accepted by all ISP over the internet, are you advertising single /24 to all 3 upstream from your 3 routers? From where 8.8.8.8 is not reachable can you please share the output of "show ip bgp 8.8.8.8 " and also trace from same router from where internet is not working? 

Thank you.

Kindest regards

Uzzi

[mbbmani]

Hi,

I attached the sh ip bgp 8.8.8.8

Router 1

Router# sh ip bgp 8.8.8.8
BGP routing table entry for 0.0.0.0/0, version 2
Paths: (1 available, best #1, table default, RIB-failure(17))
  Not advertised to any peer
  9930, (received & used)
    11x.11x.11x.77 from 11x.11x.11x.77 (yyy.yyy.yyy.16)
      Origin IGP, localpref 100, valid, external, best

Router 2 - now it is able to access internet.

Router#sh ip bgp 8.8.8.8
BGP routing table entry for 0.0.0.0/0, version 9
Paths: (1 available, best #1, table default, RIB-failure(17))
  Not advertised to any peer
  10204, (received & used)
    222.222.222.133 from 222.222.222.133 (yyy.yyy.yyy.6)
      Origin IGP, metric 0, localpref 100, valid, external, best

Router 3

Router##sh ip bgp 8.8.8.8
BGP routing table entry for 0.0.0.0/0, version 2
Paths: (1 available, best #1, table default, RIB-failure(17))
  Not advertised to any peer
  38278, (received & used)
    333.333.333.145 from 333.333.333.145 (yyy.yyy.yyy.32)
      Origin IGP, localpref 100, valid, external, best

[Muhammad Uzair]

My bad sir, show ip bgp will always show it as routing entry 0.0.0.0/0 means all your ISPs advertising you a default route, but we need to check the IGP, can you please share the output of "show ip route 8.8.8.8 " and also trace result from 3 routers to 8.8.8.8

Thank you.

Kindest regards

Uzzi

[mbbmani]

Hi,

I attached the sh bgp ip 8.8.8.8 output.

Thank you for your advice.

[Muhammad Uzair]

Further to below also can you please share "show ip bgp neighbor (neighbor ip) advertise-routes " from 3 routers as well.

Thank you.

Kindest regards

Uzzi

[mbbmani]

Hi,

I attached the full config from all the 3 routers.

Issues :

1. Internet is only possible for ROUTER1 via ISP1.

2. Internet for ROUTER2 and ROUTER3 is via iBGP connection via ISP1.

Appreciate your help.

[Muhammad Uzair]

Why you configured static route when you are receiving default route from ISP? default route will be always preferred over BGP due to Admin distance. Usually the problem in reachability via 2 different sources (from 1 is reachable and from 2nd is not reachable is most probably due to NLRI of the reverse path to the source) appreciate if you can share the output of "show ip route 8.8.8.8 " and trace route 8.8.8.8 with source sub interfaces (.8 and .10).

Thank you.

Kindest regards,

Uzzi

[mbbmani]

Hi,

I attached the traceroute and ip route output for router 2.

I have the default route for the ISP supplied IP address. Int gi0/3.

[Muhammad Uzair]

My question is when you are learning default route from ISP why you defined is manually ? Also I cannot see any Gi0/3 in your provided router configuration moreover about the IP address of gi0/1.8 from where you are unable to reach over the internet, are you advertising this to your ISP ? Gi0/1.8 and .gI0/1.10 both belongs to public IP address ? if yes then are you advertising /28 of Gi0/1.10 to ISP ? you can check this by "show ip bgp neighbor (Neighbor IP) advertise-route.

203.115.224.189 is visible over the internet at route views however in your case it seems you are not advertising Gi0/1.10 prefix to your ISP and as part of BGP maximum prefix acceptance over the internet is /24 I have no idea why you are advertising in your network /27 /28 and /29 networks.

[mbbmani]

Hi,

I attached the router config after some amendments.

I use private IP for iBGP peering.

I also remove most of the static routes.

ISP1 is working but ISP2 is not.

If I shutdown Router ISP1, then ISP2 works.

Sorry for the trouble. Appreciate your help.