Clarify next-hop-self in RR

[Ethan55]

I have a simple topo as below: 

R9 and R13 establish iBGP with RR. R9 is RR client of RR (10.10.10.10). R13 is a normal iBGP neighbor of RR.

I use OSPF and LDP.

R9 and R13 have VRF VPNA and exchange Lo1 as the VPNv4 route. Do Next-hop self in RR.

R13 and R9 learn the VPNv4 route of each other but can not ping each other.

R9#show ip route vrf VPN_A

Routing Table: VPN_A
Codes: L - local, C - connected, S - static, R - RIP, M - mobile, B - BGP
       D - EIGRP, EX - EIGRP external, O - OSPF, IA - OSPF inter area 
       N1 - OSPF NSSA external type 1, N2 - OSPF NSSA external type 2
       E1 - OSPF external type 1, E2 - OSPF external type 2
       i - IS-IS, su - IS-IS summary, L1 - IS-IS level-1, L2 - IS-IS level-2
       ia - IS-IS inter area, * - candidate default, U - per-user static route
       o - ODR, P - periodic downloaded static route, H - NHRP, l - LISP
       a - application route
       + - replicated route, % - next hop override

Gateway of last resort is not set

      172.16.0.0/16 is variably subnetted, 3 subnets, 2 masks
C        172.16.167.0/24 is directly connected, Loopback1
L        172.16.167.1/32 is directly connected, Loopback1
B        172.16.168.0/24 [200/0] via 10.10.10.10, 00:30:41

R13#show ip route vrf VPN_A

Routing Table: VPN_A
Codes: L - local, C - connected, S - static, R - RIP, M - mobile, B - BGP
       D - EIGRP, EX - EIGRP external, O - OSPF, IA - OSPF inter area 
       N1 - OSPF NSSA external type 1, N2 - OSPF NSSA external type 2
       E1 - OSPF external type 1, E2 - OSPF external type 2
       i - IS-IS, su - IS-IS summary, L1 - IS-IS level-1, L2 - IS-IS level-2
       ia - IS-IS inter area, * - candidate default, U - per-user static route
       o - ODR, P - periodic downloaded static route, H - NHRP, l - LISP
       a - application route
       + - replicated route, % - next hop override

Gateway of last resort is not set

      172.16.0.0/16 is variably subnetted, 3 subnets, 2 masks
B        172.16.167.0/24 [200/0] via 10.10.10.10, 00:31:55
C        172.16.168.0/24 is directly connected, Loopback1
L        172.16.168.1/32 is directly connected, Loopback1

ICMP request out but was dropped at RR. The current configuration:

R13
vrf definition VPN_A
 rd 13.13.13.13:1
 route-target export 999:1
 route-target import 999:1
 !
 address-family ipv4
 exit-address-family
!         

interface Loopback0
 ip address 13.13.13.13 255.255.255.255
!
interface Loopback1
 vrf forwarding VPN_A
 ip address 172.16.168.1 255.255.255.0
!

router ospf 1
 network 13.13.13.13 0.0.0.0 area 0
 network 192.168.23.0 0.0.0.255 area 0
!
router bgp 7552
 bgp log-neighbor-changes
 neighbor 10.10.10.10 remote-as 7552
 neighbor 10.10.10.10 update-source Loopback0
 !
 address-family ipv4
  neighbor 10.10.10.10 activate
 exit-address-family
 !
 address-family vpnv4
  neighbor 10.10.10.10 activate
  neighbor 10.10.10.10 send-community extended
 exit-address-family
 !
 address-family ipv4 vrf VPN_A
  redistribute connected
 exit-address-family
 

R9
vrf definition VPN_A
 rd 9.9.9.9:1
 route-target export 999:1
 route-target import 999:1
 !
 address-family ipv4
 exit-address-family
!
interface Loopback0
 ip address 9.9.9.9 255.255.255.255
!
interface Loopback1
 vrf forwarding VPN_A
 ip address 172.16.167.1 255.255.255.0
!

router ospf 1
 network 9.9.9.9 0.0.0.0 area 0
 network 192.168.29.0 0.0.0.255 area 0
!
router bgp 7552
 bgp log-neighbor-changes
 neighbor 10.10.10.10 remote-as 7552
 neighbor 10.10.10.10 update-source Loopback0
 !
 address-family ipv4
  neighbor 10.10.10.10 activate
 exit-address-family
 !
 address-family vpnv4
  neighbor 10.10.10.10 activate
  neighbor 10.10.10.10 send-community extended
 exit-address-family
 !
 address-family ipv4 vrf VPN_A
  redistribute connected
 exit-address-family
!

RR
router bgp 7552
 bgp router-id 10.10.10.10
 address-family ipv4 unicast
 !
 address-family vpnv4 unicast
 !
 neighbor 9.9.9.9
  remote-as 7552
  update-source Loopback0
  address-family ipv4 unicast
   route-reflector-client
   next-hop-self
   soft-reconfiguration inbound always
  !
  address-family vpnv4 unicast
   route-reflector-client
   next-hop-self
   soft-reconfiguration inbound always
  !
 !
 neighbor 13.13.13.13
  remote-as 7552
  update-source Loopback0
  address-family ipv4 unicast
   next-hop-self
   soft-reconfiguration inbound always
  !
  address-family vpnv4 unicast
  next-hop-self
  soft-reconfiguration inbound always
  !
 !
! 

If I don't Next-hop self in RR, I can ping between VPNA of R13 and R9 successfully. Please help me clarify this case.

Thanks.

[Ethan55]

Hi @MHM Cisco World 

"The next hop self is by defualt add for vpnv4 for PE to PE"

--> I don't think so, NHS is not by default. If I don't set NHS in RR, R9 will learn R13's VPNv4 route with Next-hop 13.13.13.13 and R13 will learn R9's VPNv4 route with Next-hop 9.9.9.9. And in this case, R9 and R13 can ping the VPNv4 route of each other successfully.

[MHM Cisco World]

To clarify That why I mention PE-to-PE 

In ipv4 bgp if fwo routers connect via iBGP by default the next-hop-self is disable 

In vpnv4 bgp (without RR) if two routers ibgp the next-hop-self is enabled by defualt.

[Ethan55]

The issue you have you active ipv4 and vpnv4 for the same neighbor' this I think issue you need to activate only vpnv4.

--> Also do your suggestion but no luck.

[MHM Cisco World]

the IOU2 is RR and I config next-hop-self all and the ping is success NO ISSUE at all 
sorry I have IOS and IOS XE only but the principle is same 
do show ip bgp vpnv4 all <<- in RR and check the route is add to correct VRF or not
THANKS 
MHM

[Harold Ritter]

Hi @MHM Cisco World ,

> sorry I have IOS and IOS XE only but the principle is same 

The principle is not the same. In IOS and IOS-XE, you need to configure the "all" keyword on the next-hop-self command for it to apply to both iBGP and eBGP learnt prefixes. By default, it only changes the next to self for the eBGP learnt prefixes.

Regards,

[MHM Cisco World]

I know we need all

I am talking about principle of RR with next-hop-self in mpls

[Harold Ritter · Accepted solution]

Hi ,

I see the next hop has been changed by the RR. This is an uncommon design and I had never tested it before, but it appears that the RR stitches the two LSP (R13 to RR and RR to R9). So yes, it should work after all. Thanks for testing it @MHM Cisco World .

I ran a quick test with XR (7.1.1) and it worked as well. 

Regards,

[MHM Cisco World]

Thanks alot for all your support' I every day learn new from your experiences.

Thanks again 

Have a nice summer 

MHM

[Ethan55]

Hi @MHM Cisco World 

Still not yet to get your idea to fix my case.

[Ethan55]

Hi @Harold Ritter ,

I ran a quick test with XR (7.1.1) and it worked as well.

--> How it can work, can you point me what is wrong in my set up.

Thanks.

[MHM Cisco World]

Share show ip bgp vpnv4 in RR

[Harold Ritter]

Hi @Ethan55 ,

Let's put it that way, it is not a common scenario. You normally would want the connectivity to be end to end between the two PEs without being forced through the RR. I ran a quick test though and see that the RR will perform the VPNv4 label swapping and forward traffic to the egress PEs (R9 and R13). The above mentioned behavior is caused by the RR forcing the next hop self on VPNv4 prefixes learnt from the PEs. 

It would help if you could post the full config for the RR so we can understand why it is not working in your case.

Regards,

[Harold Ritter]

Hi @Ethan55 ,

Please provide the full configuration for the RR so we can help determine what is wrong with your setup.

Regards,

[Ethan55]

Hi @Harold Ritter ,

Full config:

R9 (e0/0) ---- (gi0/0/0/7) RR (gi0/0/0/6) ---- (e0/0) R13

R9
R9#show running-config
Building configuration...

Current configuration : 2576 bytes
!
! Last configuration change at 16:46:45 +07 Sun Jul 16 2023 by juniper
!
version 15.5
service timestamps debug datetime msec
service timestamps log datetime msec
no service password-encryption
!
hostname R9
!
boot-start-marker
boot-end-marker
!
!
vrf definition VPN_A
rd 9.9.9.9:1
route-target export 999:1
route-target import 999:1
!
address-family ipv4
exit-address-family
!
!
no aaa new-model
!
!
!
bsd-client server url https://cloudsso.cisco.com/as/token.oauth2
clock timezone +07 7 0
mmi polling-interval 60
no mmi auto-configure
no mmi pvc
mmi snmp-timeout 180
!
!
!
!
!
no ip icmp rate-limit unreachable
!
!
!
!
!
!
!
!

!
!
!
!
no ip domain lookup
ip domain name VIETTEL
ip cef
no ipv6 cef
!
multilink bundle-name authenticated
mpls label protocol ldp
!
!
!
!
!
!
!
cts logging verbose
!
!
username juniper privilege 15 secret 5 $1$thTq$utqTIVaDCRFo8m.pPw82a/
username namnh privilege 15 secret 5 $1$NYtf$Sh1Cnof/jNQ1B9Yd1DSoz/
!
redundancy
!
no cdp log mismatch duplex
!
ip tcp synwait-time 5
!
!
!
!
!
!
!
!
!
!
!
!
!
interface Loopback0
ip address 9.9.9.9 255.255.255.255
!
interface Loopback1
vrf forwarding VPN_A
ip address 172.16.167.1 255.255.255.0
!
interface Ethernet0/0
ip address 192.168.29.9 255.255.255.0
ip ospf network point-to-point
ip ospf cost 1000
mpls ip
!
interface Ethernet0/1
no ip address
!
interface Ethernet0/1.2000
encapsulation dot1Q 2000
vrf forwarding VPN_A
ip address 172.16.124.1 255.255.255.252
!
interface Ethernet0/2
no ip address
!
interface Ethernet0/3
description TO_MGMT
ip address 10.99.89.12 255.252.0.0
!
router ospf 2000 vrf VPN_A
area 2000 nssa default-information-originate no-summary
network 0.0.0.0 255.255.255.255 area 2000
!
router ospf 1
network 9.9.9.9 0.0.0.0 area 0
network 192.168.29.0 0.0.0.255 area 0
!
router bgp 7552
bgp log-neighbor-changes
neighbor 10.10.10.10 remote-as 7552
neighbor 10.10.10.10 update-source Loopback0
!
address-family ipv4
neighbor 10.10.10.10 activate
exit-address-family
!
address-family vpnv4
neighbor 10.10.10.10 activate
neighbor 10.10.10.10 send-community extended
exit-address-family
!
address-family ipv4 vrf VPN_A
redistribute connected
exit-address-family
!
ip forward-protocol nd
!
!
no ip http server
no ip http secure-server
ip route 10.0.0.0 255.0.0.0 10.96.0.1
!
!
!
mpls ldp router-id Loopback0
!
control-plane
!
!
!
!
!
!
!
!
line con 0
exec-timeout 0 0
privilege level 15
logging synchronous
line aux 0
exec-timeout 0 0
privilege level 15
logging synchronous
line vty 0 4
login local
transport input ssh
!
!
end

R13
R13#show running-config
Building configuration...

Current configuration : 1892 bytes
!
! Last configuration change at 16:56:08 +07 Sun Jul 16 2023
!
version 15.5
service timestamps debug datetime msec
service timestamps log datetime msec
no service password-encryption
!
hostname R13
!
boot-start-marker
boot-end-marker
!
!
vrf definition VPN_A
rd 13.13.13.13:1
route-target export 999:1
route-target import 999:1
!
address-family ipv4
exit-address-family
!
!
no aaa new-model
!
!
!
bsd-client server url https://cloudsso.cisco.com/as/token.oauth2
clock timezone +07 7 0
mmi polling-interval 60
no mmi auto-configure
no mmi pvc
mmi snmp-timeout 180
!
!
!
!
!
!
!
!
!
!
!
!
!

!
!
!
!
ip cef
no ipv6 cef
!
multilink bundle-name authenticated
!
!
!
!
!
!
!
cts logging verbose
!
!
!
redundancy
!
!
!
!
!
!
!
!
!
!
!
!
!
!
!
interface Loopback0
ip address 13.13.13.13 255.255.255.255
!
interface Loopback1
vrf forwarding VPN_A
ip address 172.16.168.1 255.255.255.0
!
interface Ethernet0/0
ip address 192.168.23.13 255.255.255.0
ip ospf network point-to-point
ip ospf cost 1000
mpls ip
!
interface Ethernet0/1
no ip address
shutdown
!
interface Ethernet0/2
no ip address
shutdown
!
interface Ethernet0/3
no ip address
shutdown
!
router ospf 1
network 13.13.13.13 0.0.0.0 area 0
network 192.168.23.0 0.0.0.255 area 0
!
router bgp 7552
bgp log-neighbor-changes
neighbor 10.10.10.10 remote-as 7552
neighbor 10.10.10.10 update-source Loopback0
!
address-family ipv4
neighbor 10.10.10.10 activate
exit-address-family
!
address-family vpnv4
neighbor 10.10.10.10 activate
neighbor 10.10.10.10 send-community extended
exit-address-family
!
address-family ipv4 vrf VPN_A
redistribute connected
exit-address-family
!
ip forward-protocol nd
!
!
no ip http server
no ip http secure-server
!
!
!
mpls ldp router-id Loopback0
!
control-plane
!
!
!
!
!
!
!
!
line con 0
logging synchronous
line aux 0
line vty 0 4
login
transport input none
!
!
end

RR
Building configuration...
!! IOS XR Configuration 6.1.3
!! Last configuration change at Wed Jul 19 22:59:09 2023 by namnh
!
hostname CKV01
domain name VIETTEL
vrf TEST
address-family ipv4 unicast
import route-target
100:438
999:1
!
export route-target
100:438
999:1
!
!
!
line default
transport input ssh
!
control-plane
management-plane
out-of-band
interface MgmtEth0/0/CPU0/0
allow SSH peer
address ipv4 10.0.0.0/8
!
!
!
!
!
interface Loopback0
ipv4 address 10.10.10.10 255.255.255.255
!
interface MgmtEth0/0/CPU0/0
ipv4 address 10.99.89.13 255.252.0.0
!
interface GigabitEthernet0/0/0/0
ipv4 address 192.168.16.10 255.255.255.0
!
interface GigabitEthernet0/0/0/1
ipv4 address 192.168.19.10 255.255.255.0
!
interface GigabitEthernet0/0/0/2
ipv4 address 192.168.28.10 255.255.255.0
!
interface GigabitEthernet0/0/0/3
ipv4 address 192.168.27.10 255.255.255.0
!
interface GigabitEthernet0/0/0/4.1
ipv4 address 192.168.21.13 255.255.255.0
encapsulation dot1q 1
!
interface GigabitEthernet0/0/0/5
vrf TEST
ipv4 address 10.4.30.1 255.255.255.0
!
interface GigabitEthernet0/0/0/6
ipv4 address 192.168.23.10 255.255.255.0
!
interface GigabitEthernet0/0/0/7
ipv4 address 192.168.29.10 255.255.255.0
!
interface GigabitEthernet0/0/0/8
shutdown
!
interface GigabitEthernet0/0/0/9
shutdown
!
interface GigabitEthernet0/0/0/10
shutdown
!
interface GigabitEthernet0/0/0/11
shutdown
!
interface GigabitEthernet0/0/0/12
shutdown
!
interface GigabitEthernet0/0/0/13
shutdown
!
interface GigabitEthernet0/0/0/14
shutdown
!
route-policy PASS
pass
end-policy
!
router static
address-family ipv4 unicast
10.0.0.0/8 10.96.0.1
!
!
router ospf 1
area 0
interface Loopback0
passive enable
!
interface GigabitEthernet0/0/0/0
cost 2000
network point-to-point
!
interface GigabitEthernet0/0/0/1
cost 1000
network point-to-point
!
interface GigabitEthernet0/0/0/2
network point-to-point
!
interface GigabitEthernet0/0/0/3
cost 50
network point-to-point
!
interface GigabitEthernet0/0/0/4.1
cost 2000
network point-to-point
!
interface GigabitEthernet0/0/0/6
cost 1000
network point-to-point
!
interface GigabitEthernet0/0/0/7
cost 1000
network point-to-point
!
!
!
router bgp 7552
bgp router-id 10.10.10.10
address-family ipv4 unicast
!
address-family vpnv4 unicast
!
neighbor 9.9.9.9
remote-as 7552
update-source Loopback0
address-family vpnv4 unicast
route-reflector-client
next-hop-self
soft-reconfiguration inbound always
!
!
neighbor 13.13.13.13
remote-as 7552
update-source Loopback0
address-family vpnv4 unicast
next-hop-self
soft-reconfiguration inbound always
!
!
vrf TEST
rd 100:438
address-family ipv4 unicast
label mode per-prefix
redistribute connected
!
!
!
mpls oam
!
mpls ldp
router-id 10.10.10.10
interface GigabitEthernet0/0/0/0
!
interface GigabitEthernet0/0/0/1
!
interface GigabitEthernet0/0/0/2
!
interface GigabitEthernet0/0/0/3
!
interface GigabitEthernet0/0/0/4.1
!
interface GigabitEthernet0/0/0/6
!
interface GigabitEthernet0/0/0/7
!
!
ssh server v2
end

[Harold Ritter]

Hi @Ethan55 ,

I do not see anything in the RR configuration that would prevent R13 from pinging R9 a vice versa, except  for the version you are currently running (6.1.3). Could you load something more recent, at least 6.3.1 and try again.

After upgrading, you will have to add the following command for the RR to change the next hop on iBGP learnt routes, as it doesn't happen by default:

RR:

router bgp 7552

 ibgp policy out enforce-modifications

Regards,

Regards,