BGP 4 sites non adjacent routers (ISP not joining BGP)

hmc2500 · ‎05-13-2016

Hi, we have L3 switch/routers in 4 sites and were wondering if we could configure BGP between routers without the ISP participating. Static routes have been configured between the routers of each site. What is the best way of getting this configured? Should we use ibgp or ebgp (I'd prefer to use ibgp to keep it simple)? Is it ok to configure bgp on non adjacent routers? Network diagram attached.

Vinit Jain · ‎05-13-2016

Few things that you need to keep to make the decision:

1. Check with your ISP that they are not blocking TCP port 179

2. Will your ISP router be participating in BGP? If yes, then it is automatically EBGP as you will have your own AS number. If its not participating, you can have IBGP session between your sites (unless you have blocked multiple AS numbers for your company, in which case you can have EBGP).

3. If you are looking for IBGP sessions, the challenge i see is you will have to have a Full Mesh sessions or you will have to decide on which router will be acting as a Route Reflector.

i dont think there should be any problem if you configure BGP between non-adjacent routers as long as you have connectivity issues and there are not many packet drops between your sites.

Hope this information helps.

Regards

Vinit

Thanks
--Vinit

hmc2500 · ‎05-16-2016

Thanks but I failed to mention that the sites will have dual homed ISP connections. Primary is q in q vLAN (like ethernet basically) and backup is a hub and spoke topology with one of the sites as central site. From what I understand ethernet is like a full mesh, so (I hope I'm right) ibgp and ebgp could be used? I'm confused however for the hub and spoke topology. What will happen if you use iBGP and the backup line kicks in? Should the hub and spoke topology be designed with a route reflector?

My ISP will not be participating in BGP. We were thinking of using private AS numbers since this is to manage an internal network and the routers will not directly connect to internet routers with BGP.

From all the examples that I've seen I have not seen any BGP neighborships in remote subnets (except loopback interface with iBGP maybe). All bgp neighbors (from all the examples that I've seen) are ussually configured with locally connected subnets.

I hope someone can make this clear.

hmc2500 · ‎05-27-2016

I've tested eBGP with static failover in a simple scenario and got it to work using the following settings. So I think Vinit Jain was right. would like to hear your comments.

(diagram and full running config attached):

R1#sh run
Building configuration...

Current configuration : 1440 bytes
!
! Last configuration change at 21:12:55 UTC Thu May 26 2016
!
version 15.2
service timestamps debug datetime msec
service timestamps log datetime msec
!
hostname R1
!
boot-start-marker
boot-end-marker
!
!
!
no aaa new-model
no ip icmp rate-limit unreachable
ip cef
!
!
!
!
!
!
no ip domain lookup
no ipv6 cef
!
!
multilink bundle-name authenticated
!
!
!
!
!
!
!
!
!
ip tcp synwait-time 5
!
!
!
!
!
!
!
!
!
!
!
!
interface Ethernet0/0
no ip address
shutdown
duplex auto
!
interface GigabitEthernet0/0
ip address 10.0.0.1 255.255.255.0
media-type gbic
speed 1000
duplex full
negotiation auto
!
interface GigabitEthernet1/0
ip address 192.168.1.1 255.255.255.0
negotiation auto
!
interface Serial2/0
no ip address
shutdown
serial restart-delay 0
!
interface Serial2/1
no ip address
shutdown
serial restart-delay 0
!
interface Serial2/2
no ip address
shutdown
serial restart-delay 0
!
interface Serial2/3
no ip address
shutdown
serial restart-delay 0
!
ip forward-protocol nd
!
!
no ip http server
no ip http secure-server
ip route 12.0.0.0 255.255.255.0 192.168.1.4
ip route 13.0.0.0 255.255.255.0 10.0.0.3
ip route 192.168.4.0 255.255.252.0 10.0.0.3
ip route 192.168.8.0 255.255.252.0 192.168.1.4
!
!
!
!
control-plane
!
!
line con 0
exec-timeout 0 0
privilege level 15
logging synchronous
stopbits 1
line aux 0
exec-timeout 0 0
privilege level 15
logging synchronous
stopbits 1
line vty 0 4
login
!
!
end

R1#

R3#sh run
Building configuration...

Current configuration : 2200 bytes
!
! Last configuration change at 14:15:52 UTC Fri May 27 2016
!
version 15.2
service timestamps debug datetime msec
service timestamps log datetime msec
!
hostname R3
!
boot-start-marker
boot-end-marker
!
!
!
no aaa new-model
no ip icmp rate-limit unreachable
ip cef
!
!
!
!
!
!
no ip domain lookup
no ipv6 cef
!
!
multilink bundle-name authenticated
!
!
!
!
!
!
!
!
!
ip tcp synwait-time 5
!
track 1 ip sla 10 reachability
!
track 2 ip sla 20
!
!
!
!
!
!
!
!
!
!
!
!
interface Loopback0
ip address 13.0.0.3 255.255.255.0
!
interface Ethernet0/0
no ip address
shutdown
duplex auto
!
interface GigabitEthernet0/0
ip address 10.0.0.3 255.255.255.0
media-type gbic
speed 1000
duplex full
negotiation auto
!
interface GigabitEthernet1/0
ip address 11.0.0.3 255.255.255.0
negotiation auto
!
interface Serial2/0
no ip address
shutdown
serial restart-delay 0
!
interface Serial2/1
no ip address
shutdown
serial restart-delay 0
!
interface Serial2/2
no ip address
shutdown
serial restart-delay 0
!
interface Serial2/3
no ip address
shutdown
serial restart-delay 0
!
interface GigabitEthernet3/0
ip address 192.168.4.1 255.255.252.0
negotiation auto
!
router bgp 3
bgp log-neighbor-changes
network 13.0.0.0 mask 255.255.255.0
network 192.168.4.0 mask 255.255.252.0
redistribute connected
redistribute static
neighbor 12.0.0.4 remote-as 4
neighbor 12.0.0.4 ebgp-multihop 4
neighbor 12.0.0.4 update-source Loopback0
!
ip forward-protocol nd
!
!
no ip http server
no ip http secure-server
ip route 0.0.0.0 0.0.0.0 11.0.0.4 track 1
ip route 12.0.0.0 255.255.255.0 11.0.0.4 track 2
ip route 0.0.0.0 0.0.0.0 10.0.0.1 10
ip route 12.0.0.0 255.255.255.0 10.0.0.1 10
ip route 192.168.1.0 255.255.255.0 10.0.0.1
!
ip sla 10
icmp-echo 12.0.0.4 source-interface GigabitEthernet1/0
frequency 10
ip sla schedule 10 life forever start-time now
ip sla 20
icmp-echo 12.0.0.4 source-interface GigabitEthernet1/0
frequency 10
ip sla schedule 20 life forever start-time now
!
!
!
control-plane
!
!
line con 0
exec-timeout 0 0
privilege level 15
logging synchronous
stopbits 1
line aux 0
exec-timeout 0 0
privilege level 15
logging synchronous
stopbits 1
line vty 0 4
login
!
!
end

R3#

R3#show ip bgp sum
BGP router identifier 13.0.0.3, local AS number 3
BGP table version is 23, main routing table version 23
7 network entries using 1008 bytes of memory
12 path entries using 960 bytes of memory
4/3 BGP path/bestpath attribute entries using 544 bytes of memory
1 BGP AS-PATH entries using 24 bytes of memory
0 BGP route-map cache entries using 0 bytes of memory
0 BGP filter-list cache entries using 0 bytes of memory
BGP using 2536 total bytes of memory
BGP activity 8/1 prefixes, 28/16 paths, scan interval 60 secs

Neighbor V AS MsgRcvd MsgSent TblVer InQ OutQ Up/Down State/PfxRcd
12.0.0.4 4 4 1469 1482 23 0 0 22:10:50 6
R3#

R4#sh run
Building configuration...

Current configuration : 2257 bytes
!
! Last configuration change at 14:15:39 UTC Fri May 27 2016
!
version 15.2
service timestamps debug datetime msec
service timestamps log datetime msec
!
hostname R4
!
boot-start-marker
boot-end-marker
!
!
!
no aaa new-model
no ip icmp rate-limit unreachable
ip cef
!
!
!
!
!
!
no ip domain lookup
no ipv6 cef
!
!
multilink bundle-name authenticated
!
!
!
!
!
!
!
!
!
ip tcp synwait-time 5
!
track 1 ip sla 10 reachability
!
track 2 ip sla 20
!
!
!
!
!
!
!
!
!
!
!
!
interface Loopback0
ip address 12.0.0.4 255.255.255.0
!
interface Ethernet0/0
no ip address
shutdown
duplex auto
!
interface GigabitEthernet0/0
ip address 11.0.0.4 255.255.255.0
media-type gbic
speed 1000
duplex full
negotiation auto
!
interface GigabitEthernet1/0
ip address 192.168.1.4 255.255.255.0
negotiation auto
!
interface Serial2/0
no ip address
shutdown
serial restart-delay 0
!
interface Serial2/1
no ip address
shutdown
serial restart-delay 0
!
interface Serial2/2
no ip address
shutdown
serial restart-delay 0
!
interface Serial2/3
no ip address
shutdown
serial restart-delay 0
!
interface GigabitEthernet3/0
ip address 192.168.8.1 255.255.252.0
negotiation auto
!
router bgp 4
bgp log-neighbor-changes
network 12.0.0.0 mask 255.255.255.0
network 192.168.8.0 mask 255.255.252.0
redistribute connected
redistribute static
neighbor 13.0.0.3 remote-as 3
neighbor 13.0.0.3 ebgp-multihop 4
neighbor 13.0.0.3 update-source Loopback0
neighbor 13.0.0.3 soft-reconfiguration inbound
!
ip forward-protocol nd
!
!
no ip http server
no ip http secure-server
ip route 0.0.0.0 0.0.0.0 11.0.0.3 track 1
ip route 13.0.0.0 255.255.255.0 11.0.0.3 track 2
ip route 0.0.0.0 0.0.0.0 192.168.1.1 10
ip route 10.0.0.0 255.255.255.0 192.168.1.1
ip route 13.0.0.0 255.255.255.0 192.168.1.1 10
!
ip sla 10
icmp-echo 13.0.0.3 source-interface GigabitEthernet0/0
frequency 10
ip sla schedule 10 life forever start-time now
ip sla 20
icmp-echo 13.0.0.3 source-interface GigabitEthernet0/0
frequency 10
ip sla schedule 20 life forever start-time now
!
!
!
control-plane
!
!
line con 0
exec-timeout 0 0
privilege level 15
logging synchronous
stopbits 1
line aux 0
exec-timeout 0 0
privilege level 15
logging synchronous
stopbits 1
line vty 0 4
login
!
!
end

R4#

R4#sho ip bgp sum
BGP router identifier 12.0.0.4, local AS number 4
BGP table version is 34, main routing table version 34
7 network entries using 1008 bytes of memory
12 path entries using 960 bytes of memory
4/3 BGP path/bestpath attribute entries using 544 bytes of memory
1 BGP AS-PATH entries using 24 bytes of memory
0 BGP route-map cache entries using 0 bytes of memory
0 BGP filter-list cache entries using 0 bytes of memory
BGP using 2536 total bytes of memory
BGP activity 13/6 prefixes, 35/23 paths, scan interval 60 secs

Neighbor V AS MsgRcvd MsgSent TblVer InQ OutQ Up/Down State/PfxRcd
13.0.0.3 4 3 1484 1471 34 0 0 22:12:21 6
R4#sho ip bgp neigh
BGP neighbor is 13.0.0.3, remote AS 3, external link
BGP version 4, remote router ID 13.0.0.3
BGP state = Established, up for 22:12:28
Last read 00:00:14, last write 00:00:31, hold time is 180, keepalive interval is 60 seconds
Neighbor sessions:
1 active, is not multisession capable (disabled)
Session: 13.0.0.3
Topology IPv4 Unicast
Neighbor capabilities:
Route refresh: advertised and received(new)
Four-octets ASN Capability: advertised and received
Address family IPv4 Unicast: advertised and received
Enhanced Refresh Capability: advertised and received
Multisession Capability:
Stateful switchover support enabled: NO for session 1
Message statistics:
InQ depth is 0
OutQ depth is 0

Sent Rcvd
Opens: 1 1
Notifications: 0 0
Updates: 9 16
Keepalives: 1460 1465
Route Refresh: 1 0
Total: 1471 1484
Default minimum time between advertisement runs is 30 seconds

For address family: IPv4 Unicast
Session: 13.0.0.3
BGP table version 34, neighbor version 34/0
Output queue size : 0
Index 6, Advertise bit 0
6 update-group member
Inbound soft reconfiguration allowed
Slow-peer detection is disabled
Slow-peer split-update-group dynamic is disabled
Sent Rcvd
Prefix activity: ---- ----
Prefixes Current: 6 6 (Consumes 480 bytes)
Prefixes Total: 10 19
Implicit Withdraw: 4 12
Explicit Withdraw: 0 1
Used as bestpath: n/a 1
Used as multipath: n/a 0

Outbound Inbound
Local Policy Denied Prefixes: -------- -------
Bestpath from this peer: 3 n/a
Total: 3 0
Number of NLRIs in the update sent: max 3, min 0
Last detected as dynamic slow peer: never
Dynamic slow peer recovered: never
Refresh Epoch: 2
Last Sent Refresh Start-of-rib: never
Last Sent Refresh End-of-rib: never
Last Received Refresh Start-of-rib: 21:41:23
Last Received Refresh End-of-rib: 21:41:23
Refresh-In took 0 seconds
Sent Rcvd
Refresh activity: ---- ----
Refresh Start-of-RIB 0 1
Refresh End-of-RIB 0 1

Address tracking is enabled, the RIB does have a route to 13.0.0.3
Connections established 4; dropped 3
Last reset 22:12:37, due to BGP Notification received of session 1, hold time expired
External BGP neighbor may be up to 4 hops away.
Transport(tcp) path-mtu-discovery is enabled
Graceful-Restart is disabled
Connection state is ESTAB, I/O status: 1, unread input bytes: 0
Connection is ECN Disabled
Mininum incoming TTL 0, Outgoing TTL 4
Local host: 12.0.0.4, Local port: 179
Foreign host: 13.0.0.3, Foreign port: 30155
Connection tableid (VRF): 0

Enqueued packets for retransmit: 0, input: 0 mis-ordered: 0 (0 bytes)

Event Timers (current time is 0x5147388):
Timer Starts Wakeups Next
Retrans 1468 0 0x0
TimeWait 0 0 0x0
AckHold 1474 1427 0x0
SendWnd 0 0 0x0
KeepAlive 0 0 0x0
GiveUp 0 0 0x0
PmtuAger 0 0 0x0
DeadWait 0 0 0x0
Linger 0 0 0x0

iss: 3815589679 snduna: 3815617963 sndnxt: 3815617963 sndwnd: 15909
irs: 2482517850 rcvnxt: 2482546592 rcvwnd: 15472 delrcvwnd: 912

SRTT: 300 ms, RTTO: 303 ms, RTV: 3 ms, KRTT: 0 ms
minRTT: 12 ms, maxRTT: 316 ms, ACK hold: 200 ms
Status Flags: passive open, gen tcbs
Option Flags: nagle, path mtu capable

Datagrams (max data segment is 1460 bytes):
Rcvd: 2943 (out of order: 0), with data: 1477, total data bytes: 28741
Sent: 2918 (retransmit: 0 fastretransmit: 0),with data: 1469, total data bytes: 28283

R4#

Vasilii Mikhailovskii · ‎05-15-2016

Hello.

The major problem I see here is - if ISP does not participate in BGP, then how would it route your traffic?

For example, Router A announced "prefix A" to all routers (except ISP); Router B forwards a packet to ISP (toward Router A - per BGP), but ISP has no route to "prefix A" and will either forward per 0.0.0.0/0 or silently drop it.

So, encapsulation (like mGRE or IPSec) could help to overcome the problem.

hmc2500 · ‎05-19-2016

We use static routing currently, would that not be enough to route bgp traffic between sites?