BGP - Multihop and Multipath -

[yhaas1]

Hi, 

hope you can help me. I want to do bgp with two C1111 Routers with two links between the routers.

First direct connected and secound via multihop. 

But I am not able to bring be secound link up. 

Setup like this: 

All Networks are /28. 

This is my BGP config: 

router bgp 100
bgp log-neighbor-changes
neighbor 10.31.4.18 remote-as 200
neighbor 10.31.4.18 fall-over bfd
neighbor 10.31.4.65 remote-as 200
neighbor 10.31.4.65 ebgp-multihop 4
neighbor 10.31.4.65 fall-over bfd
!
address-family ipv4
network 10.31.4.0 mask 255.255.255.240
network 10.31.4.16 mask 255.255.255.240
neighbor 10.31.4.18 activate
neighbor 10.31.4.65 activate
maximum-paths eibgp 5
exit-address-family

router bgp 200
bgp log-neighbor-changes
neighbor 10.31.4.17 remote-as 100
neighbor 10.31.4.17 fall-over bfd
neighbor 10.31.4.49 remote-as 100
neighbor 10.31.4.49 ebgp-multihop 4
neighbor 10.31.4.49 fall-over bfd
!
address-family ipv4
network 10.31.4.16 mask 255.255.255.240
network 10.31.4.32 mask 255.255.255.240
neighbor 10.31.4.17 activate
neighbor 10.31.4.17 soft-reconfiguration inbound
neighbor 10.31.4.49 activate
neighbor 10.31.4.49 soft-reconfiguration inbound
maximum-paths eibgp 5
exit-address-family

Maybe there is only a simple trick I don't see... 

Thanks! 

[balaji.bandi]

config looks ok -  do they have reachability between IP ?

try source interface :

neighbor x.x.x.x update-source interface  x/x

[yhaas1]

Thanks for your quick answer. 

But it didn't bring it "up". 

Here is the bgp output: 

BGP neighbor is 10.31.4.65, remote AS 200, external link
Fall over configured for session
BFD is configured.
BGP version 4, remote router ID 0.0.0.0
BGP state = Active, down for never
Neighbor sessions:
0 active, is not multisession capable (disabled)
Stateful switchover support enabled: NO for session 0
Message statistics:
InQ depth is 0
OutQ depth is 0

Sent Rcvd
Opens: 0 0
Notifications: 0 0
Updates: 0 0
Keepalives: 0 0
Route Refresh: 0 0
Total: 0 0
Do log neighbor state changes (via global configuration)
Default minimum time between advertisement runs is 30 seconds

For address family: IPv4 Unicast
BGP table version 18, neighbor version 1/18
Output queue size : 0
Index 0, Advertise bit 0
Slow-peer detection is disabled
Slow-peer split-update-group dynamic is disabled
Sent Rcvd
Prefix activity: ---- ----
Prefixes Current: 0 0
Prefixes Total: 0 0
Implicit Withdraw: 0 0
Explicit Withdraw: 0 0
Used as bestpath: n/a 0
Used as multipath: n/a 0
Used as secondary: n/a 0

Outbound Inbound
Local Policy Denied Prefixes: -------- -------
Total: 0 0
Number of NLRIs in the update sent: max 0, min 0
Last detected as dynamic slow peer: never
Dynamic slow peer recovered: never
Refresh Epoch: 1
Last Sent Refresh Start-of-rib: never
Last Sent Refresh End-of-rib: never
Last Received Refresh Start-of-rib: never
Last Received Refresh End-of-rib: never
Sent Rcvd
Refresh activity: ---- ----
Refresh Start-of-RIB 0 0
Refresh End-of-RIB 0 0

Address tracking is enabled, the RIB does have a route to 10.31.4.65
Route to peer address reachability Up: 1; Down: 0
Last notification 2d05h
Connections established 0; dropped 0
Last reset never
External BGP neighbor may be up to 4 hops away.
External BGP neighbor NOT configured for connected checks (multi-hop no-disable-connected-check)
Interface associated: (none) (peering address NOT in same link)
Transport(tcp) path-mtu-discovery is enabled
Graceful-Restart is disabled
SSO is disabled
No active TCP connection

[balaji.bandi]

Route to peer address reachability Up: 1; Down: 0
Last notification 2d05h
Connections established 0; dropped 0
Last reset never
External BGP neighbor may be up to 4 hops away.
External BGP neighbor NOT configured for connected checks (multi-hop no-disable-connected-check)
Interface associated: (none) (peering address NOT in same link)

Can you able to ping 10.31.4.49 source 10.31.4.65 ?

[yhaas1]

Right router: 

ping 10.31.4.65
Type escape sequence to abort.
Sending 5, 100-byte ICMP Echos to 10.31.4.65, timeout is 2 seconds:
!!!!!
Success rate is 100 percent (5/5), round-trip min/avg/max = 1/2/4 ms

traceroute 10.31.4.65
Type escape sequence to abort.
Tracing the route to 10.31.4.65
VRF info: (vrf in name/id, vrf out name/id)
1 10.31.4.50 0 msec 4 msec 0 msec
2 10.31.4.82 0 msec 0 msec 4 msec
3 10.31.4.65 0 msec 4 msec *

Left router:

ping 10.31.4.49
Type escape sequence to abort.
Sending 5, 100-byte ICMP Echos to 10.31.4.49, timeout is 2 seconds:
!!!!!
Success rate is 100 percent (5/5), round-trip min/avg/max = 1/1/4 ms

traceroute 10.31.4.49
Type escape sequence to abort.
Tracing the route to 10.31.4.49
VRF info: (vrf in name/id, vrf out name/id)
1 10.31.4.66 0 msec 0 msec 0 msec
2 10.31.4.81 0 msec 0 msec 4 msec
3 10.31.4.49 0 msec 0 msec *

[balaji.bandi]

I do not see any issue, until we missing any config, post full config both the routers:

just tested simple lab with your IP config, works straight away below example :

[yhaas1]

Here it is, hope you see my mistake: 

sh run
Building configuration...


Current configuration : 7264 bytes
!
! Last configuration change at 21:28:50 UTC Fri Jan 21 2022
!
version 16.10
service timestamps debug datetime msec
service timestamps log datetime msec
platform qfp utilization monitor load 80
no platform punt-keepalive disable-kernel-core
!
hostname XXXXXX
!
boot-start-marker
boot-end-marker
!
!
enable secret 9 XXXX
enable password XXXX
!
no aaa new-model
call-home
 ! If contact email address in call-home is configured as sch-smart-licensing@cisco.com
 ! the email address configured in Cisco Smart License Portal will be used as contact email address to send SCH notifications.
 contact-email-addr sch-smart-licensing@cisco.com
 profile "CiscoTAC-1"
  active
  destination transport-method http
  no destination transport-method email
!
!
!
!
login on-success log
!
!
!
!
!
!
!
subscriber templating
multilink bundle-name authenticated
!
!
!
crypto pki trustpoint TP-self-signed-1932404587
 enrollment selfsigned
 subject-name cn=IOS-Self-Signed-Certificate-1932404587
 revocation-check none
 rsakeypair TP-self-signed-1932404587
!
crypto pki trustpoint SLA-TrustPoint
 enrollment pkcs12
 revocation-check crl
!
!
crypto pki certificate chain TP-self-signed-1932404587
 certificate self-signed 01
  
        quit
crypto pki certificate chain SLA-TrustPoint
 certificate ca 01
  
        quit
!
license udi pid C1111-8P sn XXXXX
!
diagnostic bootup level minimal
!
spanning-tree extend system-id
!
!
username admin privilege 15 password 0 default
!
redundancy
 mode none
bfd map ipv4  10.31.4.64/28  10.31.4.49/32 mh-template1
bfd-template multi-hop mh-template1
 interval min-tx 120 min-rx 100 multiplier 3
!
!
!
vlan internal allocation policy ascending
!
!
!
!
!
!
interface Loopback0
 no ip address
!
interface Loopback1
 ip address 10.0.0.1 255.255.255.0
!
interface GigabitEthernet0/0/0
 ip address 10.31.4.17 255.255.255.240
 negotiation auto
 bfd interval 50 min_rx 50 multiplier 5
!
interface GigabitEthernet0/0/1
 ip address 10.31.4.49 255.255.255.240
 negotiation auto
 bfd interval 50 min_rx 50 multiplier 5
!
interface GigabitEthernet0/1/0
!
interface GigabitEthernet0/1/1
!
interface GigabitEthernet0/1/2
!
interface GigabitEthernet0/1/3
!
interface GigabitEthernet0/1/4
!
interface GigabitEthernet0/1/5
!
interface GigabitEthernet0/1/6
!
interface GigabitEthernet0/1/7
!
interface Vlan1
 ip address 10.31.4.2 255.255.255.240
!
!
router eigrp 100
 bfd all-interfaces
!
router bgp 100
 bgp log-neighbor-changes
 neighbor 10.31.4.18 remote-as 200
 neighbor 10.31.4.18 fall-over bfd
 neighbor 10.31.4.65 remote-as 200
 neighbor 10.31.4.65 ebgp-multihop 4
 neighbor 10.31.4.65 update-source GigabitEthernet0/0/1
 neighbor 10.31.4.65 fall-over bfd
 !
 address-family ipv4
  network 10.31.4.0 mask 255.255.255.240
  network 10.31.4.16 mask 255.255.255.240
  neighbor 10.31.4.18 activate
  neighbor 10.31.4.65 activate
  maximum-paths eibgp 5
 exit-address-family
!
ip forward-protocol nd
ip http server
ip http authentication local
ip http secure-server
ip route static bfd 10.31.4.65 10.31.4.49 unassociate
ip route 0.0.0.0 0.0.0.0 10.31.4.1
ip route 10.31.4.64 255.255.255.240 10.31.4.50
ip route 10.31.4.80 255.255.255.240 10.31.4.50
!
!
!
!
!
!
control-plane
!
!
line con 0
 transport input none
 stopbits 1
line vty 0 4
 password YYYYY
 login
!
!
!
!
!
!
end

sh run
Building configuration...


Current configuration : 7384 bytes
!
! Last configuration change at 21:05:26 UTC Fri Jan 21 2022 by admin
!
version 16.10
service timestamps debug datetime msec
service timestamps log datetime msec
service password-encryption
service call-home
platform qfp utilization monitor load 80
no platform punt-keepalive disable-kernel-core
!
hostname YYYYYY
!
boot-start-marker
boot-end-marker
!
!
enable secret 9 YYYYYY
!
no aaa new-model
call-home
 ! If contact email address in call-home is configured as sch-smart-licensing@cisco.com
 ! the email address configured in Cisco Smart License Portal will be used as contact email address to send SCH notifications.
 contact-email-addr sch-smart-licensing@cisco.com
 profile "CiscoTAC-1"
  active
  destination transport-method http
  no destination transport-method email
!
ip domain name cisco.com
!
!
!
login on-success log
!
!
!
!
!
!
!
subscriber templating
multilink bundle-name authenticated
!
!
!
crypto pki trustpoint TP-self-signed-1605323481
 enrollment selfsigned
 subject-name cn=IOS-Self-Signed-Certificate-1605323481
 revocation-check none
 rsakeypair TP-self-signed-1605323481
!
crypto pki trustpoint SLA-TrustPoint
 enrollment pkcs12
 revocation-check crl
!
!
crypto pki certificate chain TP-self-signed-1605323481
 certificate self-signed 01
  
        quit
crypto pki certificate chain SLA-TrustPoint
 certificate ca 01
          quit
!
license udi pid C1111-8P sn YYYYYY
!
diagnostic bootup level minimal
!
spanning-tree extend system-id
!
!
username admin password 7 YYYYYYY
!
redundancy
 mode none
bfd map ipv4  10.31.4.48/28  10.31.4.65/32 mh-template1
bfd-template multi-hop mh-template1
 interval min-tx 120 min-rx 100 multiplier 3
!
!
!
vlan internal allocation policy ascending
!
!
!
!
!
!
interface Loopback2
 ip address 10.0.1.1 255.255.255.0
!
interface GigabitEthernet0/0/0
 ip address 10.31.4.18 255.255.255.240
 negotiation auto
 bfd interval 50 min_rx 50 multiplier 5
!
interface GigabitEthernet0/0/1
 ip address 10.31.4.65 255.255.255.240
 negotiation auto
 bfd interval 50 min_rx 50 multiplier 5
!
interface GigabitEthernet0/1/0
!
interface GigabitEthernet0/1/1
!
interface GigabitEthernet0/1/2
!
interface GigabitEthernet0/1/3
!
interface GigabitEthernet0/1/4
!
interface GigabitEthernet0/1/5
!
interface GigabitEthernet0/1/6
!
interface GigabitEthernet0/1/7
!
interface Vlan1
 ip address 10.31.4.34 255.255.255.240
!
!
router eigrp 200
 bfd all-interfaces
!
router bgp 200
 bgp log-neighbor-changes
 neighbor 10.31.4.17 remote-as 100
 neighbor 10.31.4.17 fall-over bfd
 neighbor 10.31.4.49 remote-as 100
 neighbor 10.31.4.49 ebgp-multihop 4
 neighbor 10.31.4.49 update-source GigabitEthernet0/0/1
 neighbor 10.31.4.49 fall-over bfd
 !
 address-family ipv4
  network 10.31.4.16 mask 255.255.255.240
  network 10.31.4.32 mask 255.255.255.240
  neighbor 10.31.4.17 activate
  neighbor 10.31.4.17 soft-reconfiguration inbound
  neighbor 10.31.4.49 activate
  neighbor 10.31.4.49 soft-reconfiguration inbound
  maximum-paths eibgp 5
 exit-address-family
!
ip forward-protocol nd
ip http server
ip http authentication local
ip http secure-server
ip route static bfd 10.31.4.49 10.31.4.65 unassociate
ip route 0.0.0.0 0.0.0.0 10.31.4.33
ip route 10.31.4.0 255.255.255.240 10.31.4.17
ip route 10.31.4.48 255.255.255.240 10.31.4.66
ip route 10.31.4.80 255.255.255.240 10.31.4.66
!
!
!
!
!
!
control-plane
!
!
line con 0
 transport input none
 stopbits 1
line vty 0 4
 login local
 transport input ssh
!
!
!
!
!
!
end

[Harold Ritter]

@yhaas1 ,

One more thing, this configure will not work for BFD. It needs to be changed as follow:

bfd map ipv4  10.31.4.65/32  10.31.4.49/32 mh-template1

In place of 

bfd map ipv4  10.31.4.64/28  10.31.4.49/32 mh-template1

and

bfd map ipv4  10.31.4.49/32  10.31.4.65/32 mh-template1

in place of 

bfd map ipv4  10.31.4.48/28  10.31.4.65/32 mh-template1

For more information, please refer to this IOS-XE configuration document.

https://www.cisco.com/c/en/us/td/docs/ios-xml/ios/iproute_bgp/configuration/xe-16/irg-xe-16-book/bfd-bgp-multihop-client-support-and-cbit-ipv-and-ipv.html

Regards,

[Harold Ritter]

Hi @yhaas1 ,

I doubt BFD multihop is supported on the C1111. Can you try removing BFD from the session and see if it fixes the issue.

router bgp 100
no neighbor 10.31.4.65 fall-over bfd
!

router bgp 200
no neighbor 10.31.4.49 fall-over bfd
!

Regards,

[yhaas1]

Thanks for the answer. 

I removed BFD like you wrote but BGP Session is not coming up. 

KR

[Harold Ritter]

Hi @yhaas1 ,

It looks like BGP BFD multihop is indeed supported after all. As @MHM Cisco World  mentioned, you will need to configure the following for the BGP BFD multihop to work:

router bgp 100
neighbor 10.31.4.65 fall-over bfd multi-hop
!

router bgp 200
neighbor 10.31.4.49 fall-over bfd multi-hop
!

Regards,

[Harold Ritter]

Hi @yhaas1 ,

It is a good idea to make sure your configuration works without the "fall-over bfd" first. Once you get it to work without BFD, you can go ahead and apply the "fall-over bfd".

To troubleshoot why the BGP session does not come up even without BFD, start by addressing the following status from the BGP output you provided:

Address tracking is enabled, the RIB does have a route to 10.31.4.65
Route to peer address reachability Up: 1; Down: 0
Last notification 2d05h
Connections established 0; dropped 0
Last reset never

Regards,

[yhaas1]

I removed all BFD config. 

But the output is like this: 

BGP neighbor is 10.31.4.65, remote AS 200, external link
BGP version 4, remote router ID 0.0.0.0
BGP state = Idle, down for never
Neighbor sessions:
0 active, is not multisession capable (disabled)
Stateful switchover support enabled: NO
Do log neighbor state changes (via global configuration)
Default minimum time between advertisement runs is 30 seconds

For address family: IPv4 Unicast
BGP table version 21, neighbor version 1/21
Output queue size : 0
Index 0, Advertise bit 0
Slow-peer detection is disabled
Slow-peer split-update-group dynamic is disabled
Sent Rcvd
Prefix activity: ---- ----
Prefixes Current: 0 0
Prefixes Total: 0 0
Implicit Withdraw: 0 0
Explicit Withdraw: 0 0
Used as bestpath: n/a 0
Used as multipath: n/a 0
Used as secondary: n/a 0

Outbound Inbound
Local Policy Denied Prefixes: -------- -------
Total: 0 0
Number of NLRIs in the update sent: max 0, min 0
Last detected as dynamic slow peer: never
Dynamic slow peer recovered: never
Refresh Epoch: 1
Last Sent Refresh Start-of-rib: never
Last Sent Refresh End-of-rib: never
Last Received Refresh Start-of-rib: never
Last Received Refresh End-of-rib: never
Sent Rcvd
Refresh activity: ---- ----
Refresh Start-of-RIB 0 0
Refresh End-of-RIB 0 0

Address tracking is enabled, the RIB does have a route to 10.31.4.65
Route to peer address reachability Up: 1; Down: 0
Last notification 3d01h
Connections established 0; dropped 0
Last reset never
External BGP neighbor may be up to 4 hops away.
External BGP neighbor NOT configured for connected checks (multi-hop no-disable-connected-check)
Interface associated: (none) (peering address NOT in same link)
Transport(tcp) path-mtu-discovery is enabled
Graceful-Restart is disabled
SSO is disabled
No active TCP connection

[Harold Ritter]

Hi @yhaas1 ,

Did you just remove the BFD config under BGP?

Can you please provide the "sh runn | sec bfd" output.

Also what IOS-XE version are you running on the C1111?

BTW, I did a quick recreate and I have this same setup working with BFD in my virtual lab using a CSR1000v in place of the C1111. Here is the relevant configuration.

hostname Router-Right

!

bfd map ipv4 10.31.4.65/32 10.31.4.49/32 mh-template1

bfd-template multi-hop mh-template1

 interval min-tx 120 min-rx 100 multiplier 3

!

interface Loopback1

 ip address 10.0.0.1 255.255.255.0

!

interface GigabitEthernet0/0/0

 ip address 10.31.4.17 255.255.255.240

 bfd interval 50 min_rx 50 multiplier 5

!

interface GigabitEthernet0/0/1

 ip address 10.31.4.49 255.255.255.240

 no bfd interval 50 min_rx 50 multiplier 5

!

router bgp 100

 bgp log-neighbor-changes

 neighbor 10.31.4.18 remote-as 200

 neighbor 10.31.4.18 fall-over bfd

 neighbor 10.31.4.65 remote-as 200

 neighbor 10.31.4.65 ebgp-multihop 4

 neighbor 10.31.4.65 fall-over bfd multi-hop

 !

 address-family ipv4

  network 10.31.4.0 mask 255.255.255.240

  network 10.31.4.16 mask 255.255.255.240

  neighbor 10.31.4.18 activate

  neighbor 10.31.4.65 activate

  maximum-paths eibgp 5

 exit-address-family

!

ip route 10.31.4.64 255.255.255.240 10.31.4.50

hostname Router-Left

bfd map ipv4  10.31.4.49/32  10.31.4.65/32 mh-template1

bfd-template multi-hop mh-template1

 interval min-tx 120 min-rx 100 multiplier 3

!

!

interface Loopback2

 ip address 10.0.1.1 255.255.255.0

!

interface GigabitEthernet0/0/0

 ip address 10.31.4.18 255.255.255.240

 bfd interval 50 min_rx 50 multiplier 5

!

interface GigabitEthernet0/0/1

 ip address 10.31.4.65 255.255.255.240

 no bfd interval 50 min_rx 50 multiplier 5

!

router bgp 200

 bgp log-neighbor-changes

 neighbor 10.31.4.17 remote-as 100

 neighbor 10.31.4.17 fall-over bfd

 neighbor 10.31.4.49 remote-as 100

 neighbor 10.31.4.49 ebgp-multihop 4

 neighbor 10.31.4.49 fall-over bfd multi-hop

 !

 address-family ipv4

  network 10.31.4.16 mask 255.255.255.240

  network 10.31.4.32 mask 255.255.255.240

  neighbor 10.31.4.17 activate

  neighbor 10.31.4.17 soft-reconfiguration inbound

  neighbor 10.31.4.49 activate

  neighbor 10.31.4.49 soft-reconfiguration inbound

  maximum-paths eibgp 5

 exit-address-family

!

Regards,

[yhaas1]

Hi, 

I tried your config on the C111 and it is not working. 

Version is: Cisco IOS XE Software, Version 16.10.01b

BGP neighbor is 10.31.4.65,  remote AS 200, external link
 Fall over configured for session
 BFD is configured.
  BGP version 4, remote router ID 0.0.0.0
  BGP state = Idle, down for never
  Neighbor sessions:
    0 active, is not multisession capable (disabled)
    Stateful switchover support enabled: NO
  Do log neighbor state changes (via global configuration)
  Default minimum time between advertisement runs is 30 seconds

 For address family: IPv4 Unicast
  BGP table version 29, neighbor version 1/29
  Output queue size : 0
  Index 0, Advertise bit 0
  Slow-peer detection is disabled
  Slow-peer split-update-group dynamic is disabled
                                 Sent       Rcvd
  Prefix activity:               ----       ----
    Prefixes Current:               0          0
    Prefixes Total:                 0          0
    Implicit Withdraw:              0          0
    Explicit Withdraw:              0          0
    Used as bestpath:             n/a          0
    Used as multipath:            n/a          0
    Used as secondary:            n/a          0

                                   Outbound    Inbound
  Local Policy Denied Prefixes:    --------    -------
    Total:                                0          0
  Number of NLRIs in the update sent: max 0, min 0
  Last detected as dynamic slow peer: never
  Dynamic slow peer recovered: never
  Refresh Epoch: 1
  Last Sent Refresh Start-of-rib: never
  Last Sent Refresh End-of-rib: never
  Last Received Refresh Start-of-rib: never
  Last Received Refresh End-of-rib: never
                                       Sent       Rcvd
        Refresh activity:              ----       ----
          Refresh Start-of-RIB          0          0
          Refresh End-of-RIB            0          0

  Address tracking is enabled, the RIB does have a route to 10.31.4.65
  Route to peer address reachability Up: 4; Down: 0
    Last notification 05:33:40
  Connections established 0; dropped 0
  Last reset never
  External BGP neighbor may be up to 4 hops away.
  External BGP neighbor NOT configured for connected checks (multi-hop no-disable-connected-check)
  Interface associated: (none) (peering address NOT in same link)
  Transport(tcp) path-mtu-discovery is enabled
  Graceful-Restart is disabled
  SSO is disabled
  No active TCP connection