cancel
Showing results for 
Search instead for 
Did you mean: 
cancel
1614
Views
5
Helpful
5
Replies

BGP Peer Flapping, Tunnel & Interface seem stable

rwills
Level 1
Level 1

I have a BGP Peering that will come up as Established, but no routes are exchanged.  It sits there for 180 seconds, and once the timer expires, the peering is lost.  It re-establishes itself after a couple seconds, and the cycle repeats.  There is a GRE Tunnel between the sites that the BGP peering runs over.  Though the Tunnel itself does not appear to be bouncing.  The Hub router is an ASR 1002.  The Branch router is a C2951.

 

Here is what I am seeing in the log file of the branch router:

 

 

Branch Router Log:

Jan 24 02:02:23.825 MST: %BGP-5-ADJCHANGE: neighbor 1.1.1.1 Down BGP Notification sent
Jan 24 02:02:23.825 MST: %BGP_SESSION-5-ADJCHANGE: neighbor 1.1.1.1 IPv4 Unicast topology base removed from session BGP Notification sent
Jan 24 02:02:32.125 MST: %BGP-5-ADJCHANGE: neighbor 1.1.1.1 Up
Jan 24 02:05:32.243 MST: %BGP-3-NOTIFICATION: sent to neighbor 1.1.1.1 4/0 (hold time expired) 0 bytes
Jan 24 02:05:32.243 MST: %BGP-5-NBR_RESET: Neighbor 1.1.1.1 reset (BGP Notification sent)

 

Hub router log:

Jan 23 14:00:32.699 MST: %BGP-3-NOTIFICATION: received from neighbor 2.2.2.2 4/0 (hold time expired) 0 bytes
Jan 23 14:00:32.699 MST: %BGP-5-NBR_RESET: Neighbor 2.2.2.2 reset (BGP Notification received)
Jan 23 14:00:32.699 MST: %BGP-5-ADJCHANGE: neighbor 2.2.2.2 Down BGP Notification received
Jan 23 14:00:32.699 MST: %BGP_SESSION-5-ADJCHANGE: neighbor 2.2.2.2 IPv4 Unicast topology base removed from session BGP Notification received
Jan 23 14:00:42.030 MST: %BGP-5-ADJCHANGE: neighbor 2.2.2.2 Up

 

 

 

 

Branch Router relevant config:

!
class-map match-any VoiceSignaling
match ip dscp cs5
class-map match-any Assured
match ip dscp af21
match protocol icmp
match protocol telnet
match protocol ssh
match protocol vnc
match access-group name af21_traffic
class-map match-any Video
match ip dscp af41
match ip precedence 4
class-map match-any Voice
match ip dscp ef
match protocol rtp audio
match protocol skinny
match ip dscp cs3
!
policy-map WANQoS
class Voice
priority percent 10
set ip dscp ef
class VoiceSignaling
bandwidth percent 10
set ip dscp af31
class Video
priority percent 25
set ip dscp af31
class Assured
bandwidth percent 50
set ip dscp af21
class class-default
fair-queue
random-detect
policy-map 150MB
class class-default
shape average 100000000
service-policy WANQoS
!
!
interface Loopback0
ip address 10.9.0.4 255.255.255.255
!
interface Tunnel920902
bandwidth 100000
ip address 2.2.2.2 255.255.255.0
ip flow ingress
ip flow egress
load-interval 30
keepalive 1 3
tunnel source 192.168.254.9
tunnel destination 192.168.254.92
tunnel key 920902
service-policy output 150MB
!
interface Null0
no ip unreachables
!
interface Embedded-Service-Engine0/0
no ip address
shutdown
!
interface GigabitEthernet0/0
description to px-lan-cs-23a
ip address 10.9.255.9 255.255.255.248
ip flow ingress
duplex auto
speed auto
!
interface GigabitEthernet0/1
description Comcast ENS
mtu 9216
ip address 192.168.254.9 255.255.255.0
load-interval 30
duplex auto
speed auto
!
!
router eigrp 10
network 10.0.0.0
redistribute bgp 65039 metric 3000 2000 255 1 1500 route-map no_redistribute
passive-interface default
no passive-interface GigabitEthernet0/0
!
router bgp 65039
bgp log-neighbor-changes
neighbor 1.1.1.1 remote-as 65192
!
address-family ipv4
network 10.9.0.0 mask 255.255.0.0
network 10.9.1.0 mask 255.255.255.0
network 10.9.5.0 mask 255.255.255.0
network 10.109.5.0 mask 255.255.255.0
network 10.109.205.0 mask 255.255.255.0
network 172.16.21.0 mask 255.255.255.0
aggregate-address 10.9.0.0 255.255.0.0 summary-only
neighbor 1.1.1.1 activate
exit-address-family

 

 

 

 

 

Hub Router relevant config:


flow exporter Export-FNF-LiveAction
description FNF v9
destination 10.91.2.198
source Loopback0
transport udp 2055
option interface-table
option application-table
!
!
flow monitor Monitor-FNF
description FNF Traffic Analysis
exporter Export-FNF-LiveAction
cache timeout inactive 10
cache timeout active 60
record Record-FNF
!

!
!
class-map match-any Lync
match protocol ms-lync
class-map match-any Browsing
match access-group name browsing_traffic
class-map match-any Assured
match ip dscp af21
match access-group name af21_traffic
match qos-group 18
match protocol pcoip
match protocol citrix
class-map match-any VoiceNBAR
match protocol rtp audio
class-map match-any Video
match ip dscp af41
match ip dscp af31
match ip dscp cs4
match application rtp
match application webex-meeting
match application telepresence-data
match application telepresence-media
match application telepresence-control
class-map match-any Voice
match ip dscp ef
match qos-group 46
match ip dscp cs3
class-map match-any Replication
match access-group name replication_traffic
match qos-group 10
class-map match-any VoiceTagged
match ip dscp ef
class-map match-any IsilonReplication
match access-group name isilon_replication
class-map match-any LyncVideo
match protocol ms-lync-video
class-map match-any LyncAudio
match protocol ms-lync-audio
class-map match-any PriorityReplication
match access-group name priority_replication_traffic
!
policy-map WANQoSHigh
class Voice
priority 10000
set ip dscp ef
class Video
priority 25000
set ip dscp af31
class Assured
set ip dscp af21
bandwidth remaining percent 10
fair-queue
queue-limit 2048 packets
class Replication
bandwidth remaining percent 55
fair-queue
fair-queue queue-limit 1024
queue-limit 1024 packets
class PriorityReplication
bandwidth remaining percent 25
fair-queue
fair-queue queue-limit 1024
queue-limit 1024 packets
class class-default
bandwidth remaining percent 10
fair-queue
random-detect
queue-limit 1024 packets
policy-map 1000MB
class class-default
shape average 1000000000
service-policy WANQoSHigh
policy-map WANQoSMedium
class Voice
priority 2000
set ip dscp ef
class Video
priority 10000
set ip dscp af31
class Assured
set ip dscp af21
bandwidth remaining percent 25
fair-queue
queue-limit 1024 packets
class Replication
bandwidth remaining percent 10
fair-queue
queue-limit 1024 packets
class Browsing
bandwidth remaining percent 10
fair-queue
queue-limit 1024 packets
class class-default
fair-queue
random-detect
bandwidth remaining percent 55
policy-map 100MB
class class-default
shape average 100000000
service-policy WANQoSMedium
policy-map L2TPV3
class Voice
set ip dscp tunnel ef
class class-default
set ip dscp tunnel af21
policy-map 20MB
class class-default
shape average 18000000
service-policy WANQoSMedium
!
!

!
!
interface Loopback0
ip address 10.92.0.5 255.255.255.255
!

!
interface Tunnel920902
description Comcast Phoenix
bandwidth 100000
ip address 2.92.9.1 255.255.255.0
ip flow monitor Monitor-FNF input
ip flow monitor Monitor-FNF output
load-interval 30
keepalive 1 3
tunnel source 192.168.254.92
tunnel destination 192.168.254.9
tunnel key 920902
!

!
interface GigabitEthernet0/0/0
description WEST-7702-A E1/3
ip address 10.92.255.33 255.255.255.248
ip hello-interval eigrp 10 1
ip hold-time eigrp 10 3
ip flow monitor Monitor-FNF input
ip flow monitor Monitor-FNF output
load-interval 30
negotiation auto
cdp enable
!
interface GigabitEthernet0/0/1
description WEST-7702-B E1/3
ip address 10.92.255.41 255.255.255.248
ip hello-interval eigrp 10 1
ip hold-time eigrp 10 3
ip flow monitor Monitor-FNF input
ip flow monitor Monitor-FNF output
load-interval 30
negotiation auto
cdp enable
!
interface GigabitEthernet0/0/2
description Comcast 1GB ENS
mtu 9216
ip address 192.168.254.92 255.255.255.0
negotiation auto
!

!
router eigrp 10
network 10.92.0.5 0.0.0.0
network 10.92.255.33 0.0.0.0
network 10.92.255.41 0.0.0.0
redistribute bgp 65192 metric 10000 2000 255 1 1500
passive-interface default
no passive-interface GigabitEthernet0/0/0
no passive-interface GigabitEthernet0/0/1
!
router bgp 65192
bgp log-neighbor-changes
neighbor 2.2.2.2 remote-as 65039

!
address-family ipv4
network 0.0.0.0
network 10.92.0.5
network 10.92.0.5 mask 255.255.255.255
network 10.92.1.0 mask 255.255.255.0
network 10.92.5.0 mask 255.255.255.0
network 10.192.5.0 mask 255.255.255.0
network 10.192.205.0 mask 255.255.255.0
network 172.17.17.0 mask 255.255.255.0
network 192.168.92.0
network 192.168.101.0
network 192.168.102.0
aggregate-address 10.192.0.0 255.255.0.0 summary-only
aggregate-address 10.92.0.0 255.255.0.0 suppress-map unfilter

neighbor 2.2.2.2 activate
neighbor 2.2.2.2 soft-reconfiguration inbound

exit-address-family
!

 

 

 

Branch Router - show ip bgp neighbor
BGP neighbor is 1.1.1.1, remote AS 65192, external link
BGP version 4, remote router ID 10.92.0.5
BGP state = Established, up for 00:01:56
Last read 00:01:56, last write 00:00:06, hold time is 180, keepalive interval is 60 seconds
Neighbor sessions:
1 active, is not multisession capable (disabled)
Neighbor capabilities:
Route refresh: advertised and received(new)
Four-octets ASN Capability: advertised and received
Address family IPv4 Unicast: advertised and received
Enhanced Refresh Capability: advertised and received
Multisession Capability:
Stateful switchover support enabled: NO for session 1
Message statistics:
InQ depth is 0
OutQ depth is 0

Sent Rcvd
Opens: 1 1
Notifications: 0 0
Updates: 2 0
Keepalives: 4 1
Route Refresh: 0 0
Total: 7 3
Default minimum time between advertisement runs is 30 seconds

 

 


Hub Router - show ip bgp neighbor
BGP neighbor is 2.2.2.2, remote AS 65039, external link
BGP version 4, remote router ID 10.9.0.4
BGP state = Established, up for 00:02:50
Last read 00:00:01, last write 00:00:47, hold time is 180, keepalive interval is 60 seconds
Neighbor sessions:
1 active, is not multisession capable (disabled)
Neighbor capabilities:
Route refresh: advertised and received(new)
Four-octets ASN Capability: advertised and received
Address family IPv4 Unicast: advertised and received
Enhanced Refresh Capability: advertised and received
Multisession Capability:
Stateful switchover support enabled: NO for session 1
Message statistics:
InQ depth is 0
OutQ depth is 0

Sent Rcvd
Opens: 1 1
Notifications: 0 0
Updates: 49 2
Keepalives: 4 5
Route Refresh: 0 0
Total: 56 8
Do log neighbor state changes (via global configuration)
Default minimum time between advertisement runs is 30 seconds

 

 

 

 

1 Accepted Solution

Accepted Solutions

Harold Ritter
Cisco Employee
Cisco Employee

Hi @rwills ,

 

This type of flapping generally denotes a path MTU issue between the BGP peers.

 

Please refer to the following document on how to troubleshoot this issue:

 

https://www.cisco.com/c/en/us/support/docs/ip/border-gateway-protocol-bgp/116377-troubleshoot-bgp-mtu.html

 

Regards,

Harold Ritter
Sr Technical Leader
CCIE 4168 (R&S, SP)
harold@cisco.com
México móvil: +52 1 55 8312 4915
Cisco México
Paseo de la Reforma 222
Piso 19
Cuauhtémoc, Juárez
Ciudad de México, 06600
México

View solution in original post

5 Replies 5

Reza Sharifi
Hall of Fame
Hall of Fame

Hi,

In order for the BGP to be established, the interface IPs need to be in the same IP subnet.

example:

site-A 1.1.1.1/30

site-B 1.1.1.2/30

 

router bgp 65039
bgp log-neighbor-changes
neighbor 1.1.1.1 remote-as 65192

 

router bgp 65192
bgp log-neighbor-changes
neighbor 2.2.2.2 remote-as 65039

 

 

Hi

  As neighborship is being formed, then you have no problem with connectivity.  Firstly, if possible, try to not use GRE tunnel just to feel out how BGP behaves. Then, in case it stop bouncing, you know that, despite GRE tunnel is not bouncing, it is somehow interfering on BGP peering.  One hypotese I have is related to MTU inside GRE tunnel.

  Despite MTU, I´d recommend to use the BGP command "disable-connected-check" but is a long shot. MTU would be my main concern.

 

Harold Ritter
Cisco Employee
Cisco Employee

Hi @rwills ,

 

This type of flapping generally denotes a path MTU issue between the BGP peers.

 

Please refer to the following document on how to troubleshoot this issue:

 

https://www.cisco.com/c/en/us/support/docs/ip/border-gateway-protocol-bgp/116377-troubleshoot-bgp-mtu.html

 

Regards,

Harold Ritter
Sr Technical Leader
CCIE 4168 (R&S, SP)
harold@cisco.com
México móvil: +52 1 55 8312 4915
Cisco México
Paseo de la Reforma 222
Piso 19
Cuauhtémoc, Juárez
Ciudad de México, 06600
México

Thanks.  The MTU had been set for Jumbo Frames.  Some of the tunnels seem to be working with that setting, but this one clearly wasn't.  I set the MTU to 1500 on both sides of the tunnel, and BGP started working. 

Hello

 


@rwills wrote:

interface Tunnel920902
ip address 2.92.9.1 255.255.255.0
tunnel source 192.168.254.92
tunnel destination 192.168.254.9


router bgp 65192
network 0.0.0.0


Make sure you are not incurring recursive routing through the tunnel, meaning the scr/dst of the tunnel is not being seen through the tunnel itself, has that would cause the bgp peering to flap.


Please rate and mark as an accepted solution if you have found any of the information provided useful.
This then could assist others on these forums to find a valuable answer and broadens the community’s global network.

Kind Regards
Paul
Review Cisco Networking for a $25 gift card