Hello,
I'm having an issue with BGP sessions expiring. This issues seems to resolve itself after hours/days then works solidly for weeks before the issue then re-occurs and i'm drawing up a blank. All the iBGP sessions work without issues just the E-BGP sessions causing me grief. Im using BGP multihop for the Peering with dual 10GB Links and a multiarea OSPF underlay.
This is what i'm seeing in the logs
22 May 24 12:35:51.876119 bgp: [23668] (default) ADJ: 122.122.122.122 keepalive timer fired
2022 May 24 12:35:51.876144 bgp: [23668] (default) ADJ: 122.122.122.122 keepalive timer fired for peer
2022 May 24 12:35:51.876155 bgp: [23668] (default) ADJ: 122.122.122.122 sending KEEPALIVE
2022 May 24 12:35:51.876639 bgp: [23668] (default) ADJ: 122.122.122.122 next keepalive expiry due in 00:00:59
2022 May 24 12:32:22.329490 bgp: [23668] (default) ADJ: 121.121.121.121 keepalive timer fired
2022 May 24 12:32:22.329520 bgp: [23668] (default) ADJ: 121.121.121.121 keepalive timer fired for peer
2022 May 24 12:32:22.329537 bgp: [23668] (default) ADJ: 121.121.121.121 sending KEEPALIVE
2022 May 24 12:32:22.330033 bgp: [23668] (default) ADJ: 121.121.121.121 next keepalive expiry due in 00:00:59
On the other side i see this.
2022 May 24 12:34:18.077881 bgp: [26672] (default) ADJ: 221.221.221.221 keepalive timer fired
2022 May 24 12:34:18.077907 bgp: [26672] (default) ADJ: 221.221.221.221 keepalive timer fired for peer
2022 May 24 12:34:18.077918 bgp: [26672] (default) ADJ: 221.221.221.221 sending KEEPALIVE
2022 May 24 12:34:18.078387 bgp: [26672] (default) ADJ: 221.221.221.221 next keepalive expiry due in 00:00:59
2022 May 24 12:34:18.086460 bgp: [26672] (default) ADJ: Peer 221.221.221.221 has pending data on socket during recv, extending expiry timer
2022 May 24 12:34:18.086940 bgp: [26672] (default) ADJ: 221.221.221.221 KEEPALIVE rcvd
2022 May 24 12:34:03.584558 bgp: [26672] (default) ADJ: 222.222.222.222 keepalive timer fired
2022 May 24 12:34:03.584588 bgp: [26672] (default) ADJ: 222.222.222.222 keepalive timer fired for peer
2022 May 24 12:34:03.584598 bgp: [26672] (default) ADJ: 222.222.222.222 sending KEEPALIVE
2022 May 24 12:34:03.585085 bgp: [26672] (default) ADJ: 222.222.222.222 next keepalive expiry due in 00:00:59
2022 May 24 12:34:03.587219 bgp: [26672] (default) ADJ: Peer 222.222.222.222 has pending data on socket during recv, extending expiry timer
2022 May 24 12:34:03.587696 bgp: [26672] (default) ADJ: 222.222.222.222 KEEPALIVE rcvd
So keep alives are being sent in both directions but only received on one side. Everytime the other side sends one it get the error, has pending data on socket during recv, extending expiry timer. This causing the timers to expire and it to start all over again.
Site 1 Relationships.
show bgp l2vpn evpn summary
BGP summary information for VRF default, address family L2VPN EVPN
BGP router identifier 121.121.121.121, local AS number 65001
BGP table version is 879001, L2VPN EVPN config peers 4, capable peers 4
356 network entries and 505 paths using 104744 bytes of memory
BGP attribute entries [77/13244], BGP AS path entries [1/6]
BGP community entries [0/0], BGP clusterlist entries [0/0]
Neighbor V AS MsgRcvd MsgSent TblVer InQ OutQ Up/Down State/PfxRcd
111.111.111.111 4 65001 80151 67103 879001 0 0 6w5d 118
112.112.112.112 4 65001 69159 64569 879001 0 0 6w5d 89
221.221.221.221 4 65002 130290 71161 879001 0 0 00:00:50 149
222.222.222.222 4 65002 130307 71155 879001 0 0 00:00:05 149
Site 2
show bgp l2vpn evpn summary
BGP summary information for VRF default, address family L2VPN EVPN
BGP router identifier 221.221.221.221, local AS number 65002
BGP table version is 52220, L2VPN EVPN config peers 4, capable peers 4
149 network entries and 149 paths using 36356 bytes of memory
BGP attribute entries [36/6192], BGP AS path entries [0/0]
BGP community entries [0/0], BGP clusterlist entries [0/0]
Neighbor V AS MsgRcvd MsgSent TblVer InQ OutQ Up/Down State/PfxRcd
121.121.121.121 4 65001 48978 46551 52220 0 0 00:01:34 0
122.122.122.122 4 65001 48986 46548 52220 0 0 00:01:16 0
211.211.211.211 4 65002 48500 40812 52220 0 0 4w1d 92
212.212.212.212 4 65002 42174 39843 52220 0 0 4w1d 57
show run bgp
router bgp 65002
router-id 221.221.221.221
log-neighbor-changes
address-family ipv4 unicast
address-family l2vpn evpn
retain route-target all
template peer INTER-BGP-PEER
remote-as 65001
update-source loopback0
ebgp-multihop 10
address-family ipv4 unicast
send-community
send-community extended
address-family l2vpn evpn
send-community
send-community extended
route-map NH-Unchanged out
template peer INTRA-BGP-PEER
remote-as 65002
update-source loopback0
address-family ipv4 unicast
send-community
send-community extended
route-reflector-client
address-family l2vpn evpn
send-community
send-community extended
route-reflector-client
neighbor 121.121.121.121
inherit peer INTER-BGP-PEER
remote-as 65001
neighbor 122.122.122.122
inherit peer INTER-BGP-PEER
remote-as 65001
neighbor 211.211.211.211
inherit peer INTRA-BGP-PEER
neighbor 212.212.212.212
inherit peer INTRA-BGP-PEER
show run bgp
router bgp 65001
router-id 121.121.121.121
log-neighbor-changes
address-family ipv4 unicast
address-family l2vpn evpn
retain route-target all
template peer INTER-BGP-PEER
remote-as 65002
update-source loopback0
ebgp-multihop 10
address-family ipv4 unicast
send-community
send-community extended
address-family l2vpn evpn
send-community
send-community extended
route-map NH-Unchanged out
template peer INTRA-BGP-PEER
remote-as 65001
update-source loopback0
address-family ipv4 unicast
send-community
send-community extended
route-reflector-client
address-family l2vpn evpn
send-community
send-community extended
route-reflector-client
neighbor 111.111.111.111
inherit peer INTRA-BGP-PEER
neighbor 112.112.112.112
inherit peer INTRA-BGP-PEER
neighbor 221.221.221.221
inherit peer INTER-BGP-PEER
remote-as 65002
neighbor 222.222.222.222
inherit peer INTER-BGP-PEER
remote-as 65002
Software Version
BIOS: version 07.69
NXOS: version 9.3(8)
Any help would be appreciated. Like i say it will just fix its self without any changes but once its down i cant seem to get the peers to form.
Thanks
