NNI Opiton B IOS XRv Data Plane Issue

[jaideepchoudhary91]

Hello,

I have tried to simulate a NNI option B setup , with My PE routers being IOSv and ASBRs being IOS XRv.

SP1-PE1 <-------> SP1-ASBR1<-------->SP2-ASBR<--------->SP2-PE2

I have made Loopbacks on SP1-PE1 and SP2-PE2 and tagged them under respective VRFs. Control plane seems to be fine. But I am not able to ge thorugh the Data Plane.

Also I got the below error message for BGP on the ASBRs:

RP/0/0/CPU0:Jun 13 10:28:30.484 : bgp[1052]: %ROUTING-BGP-4-LABEL_COLLISION : Label 24001 collision: prev: [T: 11 RD:10:10 PFX/NHID:10.10.10.10/32] curr: [T: 11 RD:10:10 PFX/NHID:10.10.10.10/32]
RP/0/0/CPU0:Jun 13 10:28:30.484 : bgp[1052]: %ROUTING-BGP-4-LABEL_COLLISION : Label 24002 collision: prev: [T: 11 RD:20:20 PFX/NHID:20.20.20.20/32] curr: [T: 11 RD:20:20 PFX/NHID:20.20.20.20/32]
RP/0/0/CPU0:PE1-ASBR1#

Below are the Routers Config.

SP1-PE1#sh running-config
Building configuration...

Current configuration : 3717 bytes
!
! Last configuration change at 09:36:40 UTC Thu Jun 13 2019
!
version 15.5
service timestamps debug datetime msec
service timestamps log datetime msec
no service password-encryption
!
hostname SP1-PE1
!
boot-start-marker
boot-end-marker
!
!
!
no aaa new-model
ethernet lmi ce
!
!
!
bsd-client server url https://cloudsso.cisco.com/as/token.oauth2
mmi polling-interval 60
no mmi auto-configure
no mmi pvc
mmi snmp-timeout 180
!
!
!
!
!
!
!
!
ip vrf Blue
rd 100:100
route-target export 1000:1000
route-target import 1000:1000
!
ip vrf red
rd 200:200
route-target export 2000:2000
route-target import 2000:2000
!
!
!
!
ip cef
no ipv6 cef
!
multilink bundle-name authenticated
!
!
cts logging verbose
!
!
!
redundancy
!
!
!
!
!
!
!
!
!
!
!
!
!
!
!
interface Loopback0
ip vrf forwarding Blue
ip address 100.100.100.100 255.255.255.255
!
interface Loopback1
ip vrf forwarding red
ip address 200.200.200.200 255.255.255.255
!
interface Loopback2
ip address 1.1.1.1 255.255.255.255
!
interface GigabitEthernet0/0
no ip address
shutdown
duplex auto
speed auto
!
interface GigabitEthernet0/1
ip address 192.168.10.1 255.255.255.252
duplex auto
speed auto
media-type rj45
mpls ip
!
router ospf 1
router-id 1.1.1.1
network 1.1.1.1 0.0.0.0 area 0
network 192.168.10.0 0.0.0.3 area 0
!
router bgp 100
bgp log-neighbor-changes
neighbor 192.168.10.2 remote-as 100
!
address-family vpnv4
neighbor 192.168.10.2 activate
neighbor 192.168.10.2 send-community both
exit-address-family
!
address-family ipv4 vrf Blue
redistribute connected
exit-address-family
!
address-family ipv4 vrf red
redistribute connected
exit-address-family
!
ip forward-protocol nd
!
!
no ip http server
no ip http secure-server
!
!
!
!
control-plane

------------------------------------------------------------------------------------

RP/0/0/CPU0:PE1-ASBR1#sh running-config
Thu Jun 13 09:35:47.691 UTC
Building configuration...
!! IOS XR Configuration 6.1.3
!! Last configuration change at Thu Jun 13 08:38:01 2019 by cisco
!
hostname PE1-ASBR1
line console
exec-timeout 0 0
!
interface Loopback0
ipv4 address 111.111.111.111 255.255.255.255
!
interface MgmtEth0/0/CPU0/0
shutdown
!
interface GigabitEthernet0/0/0/0
ipv4 address 192.168.10.2 255.255.255.252
!
interface GigabitEthernet0/0/0/1
ipv4 address 192.168.10.5 255.255.255.252
!
route-policy PASS
pass
end-policy
!
router static
address-family ipv4 unicast
192.168.10.6/32 GigabitEthernet0/0/0/1 192.168.10.6
!
!
router ospf 1
router-id 111.111.111.111
area 0
interface Loopback0
!
interface GigabitEthernet0/0/0/0
!
!
!
router bgp 100
address-family vpnv4 unicast
retain route-target all
!
neighbor 192.168.10.1
remote-as 100
address-family vpnv4 unicast
route-policy PASS in
route-policy PASS out
next-hop-self
soft-reconfiguration inbound always
!
!
neighbor 192.168.10.6
remote-as 200
address-family vpnv4 unicast
route-policy PASS in
route-policy PASS out
soft-reconfiguration inbound always
!
!
!
mpls ldp
router-id 111.111.111.111
address-family ipv4
!
interface GigabitEthernet0/0/0/0
!

RP/0/0/CPU0:PE1-ASBR1#

------------------------------------------------------------------------------------

SP2-PE2#sh running-config
Building configuration...

Current configuration : 3706 bytes
!
! Last configuration change at 09:37:24 UTC Thu Jun 13 2019
!
version 15.5
service timestamps debug datetime msec
service timestamps log datetime msec
no service password-encryption
!
hostname SP2-PE2
!
boot-start-marker
boot-end-marker
!
!
!
no aaa new-model
ethernet lmi ce
!
!
!
bsd-client server url https://cloudsso.cisco.com/as/token.oauth2
mmi polling-interval 60
no mmi auto-configure
no mmi pvc
mmi snmp-timeout 180
!
!
!
!
!
!
!
!
ip vrf Blue
rd 10:10
route-target export 1000:1000
route-target import 1000:1000
!
ip vrf red
rd 20:20
route-target export 2000:2000
route-target import 2000:2000
!
!
!
!
ip cef
no ipv6 cef
!
multilink bundle-name authenticated
!
!
cts logging verbose
!
!
!
redundancy
!
!
!
!
!
!
!
!
!
!
!
!
!
!
!
interface Loopback0
ip vrf forwarding Blue
ip address 10.10.10.10 255.255.255.255
!
interface Loopback1
ip vrf forwarding red
ip address 20.20.20.20 255.255.255.255
!
interface Loopback2
ip address 2.2.2.2 255.255.255.255
!
interface GigabitEthernet0/0
no ip address
shutdown
duplex auto
speed auto
!
interface GigabitEthernet0/1
ip address 192.168.10.10 255.255.255.252
duplex auto
speed auto
media-type rj45
mpls ip
!
router ospf 1
router-id 2.2.2.2
network 2.2.2.2 0.0.0.0 area 0
network 192.168.10.8 0.0.0.3 area 0
!
router bgp 200
bgp log-neighbor-changes
neighbor 192.168.10.9 remote-as 200
!
address-family vpnv4
neighbor 192.168.10.9 activate
neighbor 192.168.10.9 send-community both
exit-address-family
!
address-family ipv4 vrf Blue
redistribute connected
exit-address-family
!
address-family ipv4 vrf red
redistribute connected
exit-address-family
!
ip forward-protocol nd
!
!
no ip http server
no ip http secure-server
!
!
!
!
control-plane

-------------------------------------------------------------------------

RP/0/0/CPU0:SP2-ASBR#sh running-config
Thu Jun 13 09:36:27.358 UTC
Building configuration...
!! IOS XR Configuration 6.1.3
!! Last configuration change at Thu Jun 13 08:36:09 2019 by cisco
!
hostname SP2-ASBR
line console
exec-timeout 0 0
!
interface Loopback0
ipv4 address 222.222.222.222 255.255.255.255
!
interface MgmtEth0/0/CPU0/0
shutdown
!
interface GigabitEthernet0/0/0/0
ipv4 address 192.168.10.6 255.255.255.252
!
interface GigabitEthernet0/0/0/1
ipv4 address 192.168.10.9 255.255.255.252
!
route-policy PASS
pass
end-policy
!
router static
address-family ipv4 unicast
192.168.10.5/32 GigabitEthernet0/0/0/0 192.168.10.5
!
!
router ospf 1
router-id 222.222.222.222
area 0
interface Loopback0
!
interface GigabitEthernet0/0/0/1
!
!
!
router bgp 200
address-family vpnv4 unicast
retain route-target all
!
neighbor 192.168.10.5
remote-as 100
address-family vpnv4 unicast
route-policy PASS in
route-policy PASS out
soft-reconfiguration inbound always
!
!
neighbor 192.168.10.10
remote-as 200
address-family vpnv4 unicast
route-policy PASS in
route-policy PASS out
next-hop-self
soft-reconfiguration inbound always
!
!
!
mpls ldp
router-id 222.222.222.222
address-family ipv4
interface GigabitEthernet0/0/0/1
!
!
end

RP/0/0/CPU0:SP2-ASBR#

[jaideepchoudhary91]

Adding Below the VPN labels output from all the 4 routers.

SP1-PE1#sh ip bgp vpnv4 all labels
Network Next Hop In label/Out label
Route Distinguisher: 10:10
10.10.10.10/32 192.168.10.2 nolabel/24001

Route Distinguisher: 20:20
20.20.20.20/32 192.168.10.2 nolabel/24002

Route Distinguisher: 100:100 (Blue)
10.10.10.10/32 192.168.10.2 nolabel/24001
100.100.100.100/32 0.0.0.0 17/nolabel(Blue)

Route Distinguisher: 200:200 (red)
20.20.20.20/32 192.168.10.2 nolabel/24002
200.200.200.200/32 0.0.0.0 18/nolabel(red)

SP1-PE1#

--------------------------------------------------------------------------

SP1-ASBR#

Network Next Hop Rcvd Label Local Label
Route Distinguisher: 10:10
*> 10.10.10.10/32 192.168.10.6 24002 24001
Route Distinguisher: 20:20
*> 20.20.20.20/32 192.168.10.6 24003 24002
Route Distinguisher: 100:100
*>i100.100.100.100/32 192.168.10.1 17 24003
Route Distinguisher: 200:200
*>i200.200.200.200/32 192.168.10.1 18 24004

Processed 4 prefixes, 4 paths
RP/0/0/CPU0:PE1-ASBR1#

-------------------------------------------------------------------

SP2-PE2#sh bgp vpnv4 unicast all labels
Network Next Hop In label/Out label

Route Distinguisher: 10:10 (Blue)
10.10.10.10/32 0.0.0.0 17/nolabel(Blue)
100.100.100.100/32 192.168.10.9 nolabel/24004

Route Distinguisher: 20:20 (red)
20.20.20.20/32 0.0.0.0 18/nolabel(red)
200.200.200.200/32  192.168.10.9 nolabel/24005

Route Distinguisher: 100:100
100.100.100.100/32 192.168.10.9 nolabel/24004

Route Distinguisher: 200:200
200.200.200.200/32  192.168.10.9 nolabel/24005

-------------------------------------------------------------------

RP/0/0/CPU0:SP2-ASBR#sh bgp vpnv4 unicast labels
BGP router identifier 222.222.222.222, local AS number 200
BGP generic scan interval 60 secs
Table ID: 0x0 RD version: 0
BGP main routing table version 11
BGP NSR Initial initsync version 3 (Reached)
BGP NSR/ISSU Sync-Group versions 0/0
BGP scan interval 60 secs

Route Distinguisher: 10:10
*>i10.10.10.10/32 192.168.10.10 17 24002
Route Distinguisher: 20:20
*>i20.20.20.20/32 192.168.10.10 18 24003
Route Distinguisher: 100:100
*> 100.100.100.100/32 192.168.10.5 24003 24004
Route Distinguisher: 200:200
*> 200.200.200.200/32 192.168.10.5 24004 24005

Processed 4 prefixes, 4 paths
RP/0/0/CPU0:SP2-ASBR#

[Hector Gustavo Serrano Gutierrez · Accepted solution]

Can you modify the iBGP configuration to make SP1-PE1 peer with SP1-ASBR1 and SP2-ASBR peer with SP2-PE2 using the loopback IP instead of using the IP on the physical interfaces?

The loopback is a /32 which is needed by CEF to properly program the labels in IOS as well.

Cheers.

[jaideepchoudhary91]

Thanks a lot Hector. It worked !! :)

[Giuseppe Larosa · Accepted solution]

Hello Jaideep,

Edit:

I see it is an MPLS L3VPN context.

Hector is right you need to use loopback addresses /32 as BGP endpoints.

The reason is that MP BGP provides the BGP next hop for VPNv4 prefix as a Loopback address

With recursion the LSP to the destination PE loopback is used as the external label in the data plane for MPLS L3 VPN packets.

Also you may need to use next-hop self on each ASBR towards the internal PE node.

Edit: I see that you have already used next-hop self just change the iBGP sessions to use the loopbacks in global routing table as BGP endpoints.

This trick allows to join an LSP in ISPA to an LSP in ISPB at the ASBR node level.

Hope to help

Giuseppe

[jaideepchoudhary91]

Thank You so much Giuseppe !!

It worked after changing the mp-ibgp peering with loopbacks.

[Georg Pauwen]

Hello,

the problem might simply be that you have specified the next hop in the static routes (label forwarding doesn't work when you specify the next hop). Try and change the static routes on both ASBRs so that the next hop is not configured, e.g.:

router static
address-family ipv4 unicast
192.168.10.5/32 GigabitEthernet0/0/0/0

[jaideepchoudhary91]

Hello George,

I also read the same comment in some article. But surprisingly it worked for me by just putting next hop IP address as the next hop.

Note: I also made the VPNv4 eBGP peering with the loopback IPs instead of P2P IP addresses.

RP/0/0/CPU0:PE1-ASBR1#sh run router static
Thu Jun 13 17:57:20.345 UTC
router static
address-family ipv4 unicast
222.222.222.222/32 192.168.10.6

RP/0/0/CPU0:SP2-ASBR#sh run router static
Thu Jun 13 17:58:15.751 UTC
router static
address-family ipv4 unicast
111.111.111.111/32 192.168.10.5
!

SP1-PE1#traceroute vrf Blue 10.10.10.10 source 100.100.100.100 numeri
1 192.168.10.2 [MPLS: Label 24004 Exp 0] 8 msec 7 msec 6 msec
2 192.168.10.6 [MPLS: Label 24002 Exp 0] 5 msec 4 msec 4 msec
3 10.10.10.10 7 msec * 8 msec
SP1-PE1#

SP1-PE1#ping vrf Blue 10.10.10.10 source 100.100.100.100
!!!!!
Success rate is 100 percent (5/5), round-trip min/avg/max = 5/5/7 ms

SP2-PE2#ping vrf red 200.200.200.200 source 20.20.20.20
!!!!!
Success rate is 100 percent (5/5), round-trip min/avg/max = 4/6/8 ms

SP2-PE2#trace vrf red 200.200.200.200 source 20.20.20.20 numeric
1 192.168.10.9 [MPLS: Label 24005 Exp 0] 9 msec 5 msec 6 msec
2 192.168.10.5 [MPLS: Label 24003 Exp 0] 5 msec 6 msec 6 msec
3 200.200.200.200 5 msec * 6 msec

[Georg Pauwen]

Hello,

thanks for the feedback. I guess in theory, this should NOT work:

router static
address-family ipv4 unicast
192.168.10.5/32 GigabitEthernet0/0/0/0 192.168.10.5

And this should work:

router static
address-family ipv4 unicast
192.168.10.5/32 GigabitEthernet0/0/0/0

Either way, the loopback BGP peering was the solution. Good stuff...