Re: MPLS option C without RR

mnavada · ‎03-03-2020

ebgp-multihop 255 this wont work in the IOS XR but no issue in ios router.

the moment i execute this command in xr i see the pop action appear for loopback adress which is pointing to other AS.

2.2.2.2/32-65000-R2-----9.9.9.9/32-65001-XR9- MP-EBGP peering

The moment i changed the command from ebgp-multihop 255 to ebgp-multihop mpls then the transport labels magically appears.

ebgp-multihop mpls clearly states that disable bgp forwarding.

anyone knows why this is applicable for XR or is this the default behavior of XR ???????

Elvin Arias · ‎03-09-2020

Option C, my second favorite. - B is actually more exciting. ; )

1. The 'ebgp-multihop 255' command will work for inter-AS Option C application, can you post the results if otherwise? - Try using this command in conjunction with the 'mpls activate' and adding the corresponding boundary/edge egress interface on the Option C ASBR and let know the results. This is useful when you have a multihop peering at the ASBR layer like it is suggested in this case.

2. The 'ebgp-multihop mpls' command is a different animal, what it does is to prevent BGP from enabling MPLS on the peering interface and also prevents allocation of Implicit-NULL rewrite labels for nexthop addresses learned from the peer. This is useful in some scenarios in which MPLS forwarding labels to the nexthops have already been learned via BGP labeled-unicast or LDP.

Let know if you have any questions.

Elvin

mnavada · ‎03-09-2020

I

Let me explain

Scenario 1 : R2 and XR9 are MP-ebgp vpnv4 peering configured.

Scenario 2 : I have shut the RR link towards R4 and R8 this is purely option C without RR.

Scenario 3 : R2 to R9 one LSP will be formed (19-vpn label) when you do the trace route from R1 to XR10.

Result for scenario 3 below ( one LSP throughout the domain)

RP/0/0/CPU0:ios#traceroute 1.1.1.1 source 20.20.20.20
Mon Mar 9 19:58:05.344 UTC

Type escape sequence to abort.
Tracing the route to 1.1.1.1

1 10.1.91.9 9 msec 0 msec 0 msec
2 10.1.79.7 [MPLS: Labels 24002/19 Exp 0] 59 msec 49 msec 69 msec
3 10.1.67.6 [MPLS: Labels 24004/19 Exp 0] 89 msec 79 msec 69 msec
4 10.1.56.5 [MPLS: Labels 18/19 Exp 0] 89 msec 79 msec 79 msec
5 10.1.35.3 [MPLS: Labels 17/19 Exp 0] 79 msec 69 msec 59 msec
6 10.1.1.2 [MPLS: Label 19 Exp 0] 49 msec 39 msec 59 msec
7 10.1.1.1 69 msec * 59 msec
RP/0/0/CPU0:ios#

FYI

R2 PE config:

Router#sh run | s r bgp
router bgp 65000
bgp router-id 2.2.2.2
bgp log-neighbor-changes
no bgp default ipv4-unicast
neighbor 4.4.4.4 remote-as 65000
neighbor 4.4.4.4 update-source Loopback0
neighbor 5.5.5.5 remote-as 65000
neighbor 5.5.5.5 update-source Loopback0
neighbor 9.9.9.9 remote-as 65001
neighbor 9.9.9.9 ebgp-multihop 255
neighbor 9.9.9.9 update-source Loopback0
!
address-family ipv4
exit-address-family
!
address-family vpnv4
neighbor 4.4.4.4 activate
neighbor 4.4.4.4 send-community extended
neighbor 5.5.5.5 activate
neighbor 5.5.5.5 send-community extended
neighbor 9.9.9.9 activate
neighbor 9.9.9.9 send-community extended
exit-address-family
!
address-family ipv4 vrf MPLS-A
network 10.1.1.0 mask 255.255.255.0
neighbor 10.1.1.1 remote-as 100
neighbor 10.1.1.1 activate
exit-address-family

R5 ASBR config:

Router#sh run | s r bgp
router bgp 65000
bgp log-neighbor-changes
neighbor 2.2.2.2 remote-as 65000
neighbor 2.2.2.2 update-source Loopback0
neighbor 10.1.56.6 remote-as 65001
!
address-family ipv4
network 2.2.2.2 mask 255.255.255.255
network 4.4.4.4 mask 255.255.255.255
neighbor 2.2.2.2 activate
neighbor 10.1.56.6 activate
neighbor 10.1.56.6 send-label
exit-address-family
!
address-family vpnv4
neighbor 2.2.2.2 activate
neighbor 2.2.2.2 send-community extended
exit-address-family
Router#

XR6 ASBR config:

RP/0/0/CPU0:ios#sh run router bgp
Mon Mar 9 20:01:20.541 UTC
router bgp 65001
address-family ipv4 unicast
network 8.8.8.8/32
network 9.9.9.9/32
allocate-label all
!
address-family vpnv4 unicast
!
neighbor 9.9.9.9
remote-as 65001
update-source Loopback0
address-family vpnv4 unicast
!
!
neighbor 10.1.56.5
remote-as 65000
address-family ipv4 labeled-unicast
route-policy PASS in
route-policy PASS out
!
!
!

RP/0/0/CPU0:ios#
RP/0/0/CPU0:ios#
RP/0/0/CPU0:ios#

XR9 PE config:

Mon Mar 9 20:01:45.009 UTC
router bgp 65001
bgp router-id 9.9.9.9
address-family ipv4 unicast
!
address-family vpnv4 unicast
!
neighbor 2.2.2.2
remote-as 65000
ebgp-multihop 255 mpls
update-source Loopback0
address-family vpnv4 unicast
route-policy PASS1 in
route-policy PASS1 out
!
!
neighbor 6.6.6.6
remote-as 65001
update-source Loopback0
address-family vpnv4 unicast
!

!
!
vrf MPLS-A
rd 65001:1
address-family ipv4 unicast
redistribute eigrp 100

RP/0/0/CPU0:ios#
RP/0/0/CPU0:ios#

Question :

neighbor 2.2.2.2
remote-as 65000
ebgp-multihop 255 mpls why this will be configured like this?

ebgp-multihop 255---- This makes pop action for 2.2.2.2/32 (implicit null).

But this has no issue with the IOS router like R2. it just works.

let me know the thought on this

mnavada · ‎03-09-2020

XR9 configured with ebgp-multihop 255 see the result below

RP/0/0/CPU0:ios#config t
Mon Mar 9 20:10:31.543 UTC
RP/0/0/CPU0:ios(config)#router bgp 65001
RP/0/0/CPU0:ios(config-bgp)#neighbor 2.2.2.2
RP/0/0/CPU0:ios(config-bgp-nbr)#address-family vpnv4 unicast
RP/0/0/CPU0:ios(config-bgp-nbr-af)#exit
RP/0/0/CPU0:ios(config-bgp-nbr)#ebgp-multihop 255----------------->see Here
RP/0/0/CPU0:ios(config-bgp-nbr)#commit
Mon Mar 9 20:11:08.980 UTC
RP/0/0/CPU0:ios(config-bgp-nbr)#
RP/0/0/CPU0:ios(config-bgp-nbr)#
RP/0/0/CPU0:ios(config-bgp-nbr)#do sh mpls forwarding
Mon Mar 9 20:11:14.980 UTC
Local Outgoing Prefix Outgoing Next Hop Bytes
Label Label or ID Interface Switched
------ ----------- ------------------ ------------ --------------- ------------
24000 Pop 7.7.7.7/32 Gi0/0/0/0 10.1.79.7 3012
24001 Pop 10.1.67.0/24 Gi0/0/0/0 10.1.79.7 0
24002 24001 6.6.6.6/32 Gi0/0/0/0 10.1.79.7 3253
24003 Pop 2.2.2.2/32 Gi0/0/0/0 10.1.79.7 0-----------> Pop action
24004 Aggregate MPLS-A: Per-VRF Aggr[V] \
MPLS-A 2748
24005 Unlabelled 20.20.20.20/32[V] Gi0/0/0/1 10.1.91.10 8892
RP/0/0/CPU0:ios(config-bgp-nbr)#

XR9 configured with ebgp-multihop mpls 255

RP/0/0/CPU0:ios(config-bgp-nbr)#ebgp-multihop ?
<1-255> maximum hop count
mpls Disable BGP MPLS forwarding
<cr>
RP/0/0/CPU0:ios(config-bgp-nbr)#ebgp-multihop mpls-------> see Here
RP/0/0/CPU0:ios(config-bgp-nbr)#commit
Mon Mar 9 20:13:01.872 UTC
RP/0/0/CPU0:ios(config-bgp-nbr)#

RP/0/0/CPU0:ios(config-bgp-nbr)#do sh mpls forwarding
Mon Mar 9 20:13:08.652 UTC
Local Outgoing Prefix Outgoing Next Hop Bytes
Label Label or ID Interface Switched
------ ----------- ------------------ ------------ --------------- ------------
24000 Pop 7.7.7.7/32 Gi0/0/0/0 10.1.79.7 3330
24001 Pop 10.1.67.0/24 Gi0/0/0/0 10.1.79.7 0
24002 24001 6.6.6.6/32 Gi0/0/0/0 10.1.79.7 3467
24003 24002 2.2.2.2/32 Gi0/0/0/0 10.1.79.7 0---------> Label appears exception for XR ????
24004 Aggregate MPLS-A: Per-VRF Aggr[V] \
MPLS-A 2748
24005 Unlabelled 20.20.20.20/32[V] Gi0/0/0/1 10.1.91.10 8892
RP/0/0/CPU0:ios(config-bgp-nbr)#

Harold Ritter · ‎03-10-2020

In InterAS option C, you only need ebgp-multihop for the sessions between the two route reflectors. You also need to use the command "neighbor next-hop-unchanged" on the session between the route reflectors.

https://www.cisco.com/c/en/us/support/docs/multiprotocol-label-switching-mpls/mpls/200523-Configuration-and-Verification-of-Layer.html

Regards

Harold Ritter
Sr Technical Leader
CCIE 4168 (R&S, SP)
harold@cisco.com
México móvil: +52 1 55 8312 4915
Cisco México
Paseo de la Reforma 222
Piso 19
Cuauhtémoc, Juárez
Ciudad de México, 06600
México

mnavada · ‎03-10-2020

Hi Harold,

I agree with that solution but that is using RR as a control plane.

The Question i put up before is Option C without RR.

R2--->R5----MPIBGP VPNV4 Peering

R5--->XR6----EBGP Labeled unicast

XR6--->XR9----MPIBGP VPNV4 Peering

R2-->XR9----MPEBGP VPNV4 Peering

Elvin Arias · ‎03-11-2020

I see, in this case, it is quiet different, but the same thing... ; )

IOS-XR and IOS-XE are different platforms, and assumptions as to how they work/interoperate in these environments do not apply, since forwarding engines/logic of these two is very different.

Note: To follow the outputs, in this case, 2.2.2.2 is the IOS prefix, and 20.20.20.20 is the XR prefix. XR is suppose to impose label 18 with a local label of 16008.

The logic of this is in point 2. of my previous reply. In this case what is happening is that IOS-XR by default assumes that an eBGP connection is always directly connected, due to this, it will make a label rewrite to the peer address in the Label Switching Database (LSD). To fix this, you have to resolve the label rewrite, how can you see this RW operation? With the 'show mpls lsd trace verbose' command. Also, if you check the MPLS Table, you will see that there will be two clients for the label of the remote loopback [2.2.2.2/32] on the XR device.

RP/0/0/CPU0:XR2#show mpls label table
Wed Mar 11 15:06:14.044 UTC
Table Label Owner State Rewrite
----- ------- ------------------------------- ------ -------
0 16008 LDP(A): InUse Yes <<< LDP
  BGP-VPNv4(A):bgp-default InUse Yes <<< BGP

Note that LDP forwarding table [a separate table that shows LDP forwarding entries only] has the correct information indicating that the outgoing label is not IMP-NULL, but 18:

RP/0/0/CPU0:XR2#show mpls ldp forwarding 2.2.2.2/32 detail
Wed Mar 11 16:03:27.288 UTC

Codes:
- = GR label recovering, (!) = LFA FRR pure backup path
{} = Label stack with multi-line output for a routing path
G = GR, S = Stale, R = Remote LFA FRR backup

Prefix Label Label(s) Outgoing Next Hop Flags
In Out Interface G S R
--------------- ------- -------------- ------------ ------------------- -----
2.2.2.2/32 16008 18 Gi0/0/0/0.420 20.4.20.4 <<< Right label [18]
[ table-id 0xe0000000, RIB flags 0x0000, load-metric 0,
path-id 1;
label from: peer 4.4.4.4:0 ]
RIB route: table-id 0xe0000000, version 2, priority 7, metric 1
source 3, type 0x10, flags 0x0
Updates:
Routing : Total 4, Last at Mar 11 15:17:21.118 (00:46:06 ago)
Forwarding: Total 6, Last at Mar 11 15:17:21.118 (00:46:06 ago)

/

Both entries are active, and BGP will be preferred, which means that a label RW operation (as mentioned before) will be done, see below the results of the 'show mpls lsd trace verbose' command:

Mar 11 15:19:22.000 mpls/lsd/str 0/0/CPU0 t1 IM:647: remove-caps from ifh=0xf00, app=BGP-VPNv4:bgp-default:active, app-idx=7
Mar 11 15:17:21.138 mpls/lsd/rw 0/0/CPU0 t1 RW:2272: Outgoing Label:Pop
Mar 11 15:17:21.138 mpls/lsd/rw 0/0/CPU0 t1 RW:1965: IPv4 MOI - NH:20.4.20.4, IF:GigabitEthernet0/0/0/0.420 (ifh 0xf00), NHID:0x0, Lbl:3, path_flags:0x0
Mar 11 15:17:21.138 mpls/lsd/rw 0/0/CPU0 t1 RW:1218: RW Update [2/2] Label FPI (BCDL Pri:4) - Incoming Label:16008
Mar 11 15:17:21.138 mpls/lsd/rw 0/0/CPU0 t1 RW:1282: RW Update [1/2] IPv4 FPI (BCDL Pri:4) - Prefix:2.2.2.2/32, Route flags:0x0
Mar 11 15:17:21.138 mpls/lsd/rw 0/0/CPU0 t1 RW:2272: Outgoing Label:Pop
Mar 11 15:17:21.138 mpls/lsd/rw 0/0/CPU0 t1 RW:1965: IPv4 MOI - NH:20.4.20.4, IF:GigabitEthernet0/0/0/0.420 (ifh 0xf00), NHID:0x5, Lbl:3, path_flags:0x0

/

At the very moment you tell the routers that the eBGP session is not directly connected [by using the 'ebgp-multihop mpls' command], but that it is over loopbacks as multihop sessions, it will create another RW operation telling BGP not to RW the label since it is coming from LDP/BGP-LU:

ar 11 15:20:19.776 mpls/lsd/rw 0/0/CPU0 t1 RW:1218: RW Update [2/2] Label FPI (BCDL Pri:4) - Incoming Label:16008
Mar 11 15:20:19.776 mpls/lsd/rw 0/0/CPU0 t1 RW:1282: RW Update [1/2] IPv4 FPI (BCDL Pri:4) - Prefix:2.2.2.2/32, Route flags:0x0
Mar 11 15:20:19.776 mpls/lsd/rare 0/0/CPU0 t1 BCDL:805: LSDv4 - EOD sent to FIB for UG 0, all apps converged
Mar 11 15:20:19.776 mpls/lsd/common 0/0/CPU0 t1 BCDL:852: LSDv4 - Send RW successful for UG 0, updates=1, size=48, total_downloads=75, EOD=1
Mar 11 15:20:19.776 mpls/lsd/common 0/0/CPU0 t1 BCDL:729: LSDv4 - Send RW successful for UG 0, updates=1, size=372, total_downloads=74, EOD=1
Mar 11 15:20:19.776 mpls/lsd/common 0/0/CPU0 t1 BCDL:663: LSDv4 - For UG 0 Downloaded 1 RWs with BCDL priority 4
Mar 11 15:20:19.756 mpls/lsd/str 0/0/CPU0 t1 IM:501: Enabling intf (0xf00), app=BGP-VPNv4:bgp-default:active, ftypes:0x1
Mar 11 15:19:22.020 mpls/lsd/rw 0/0/CPU0 t1 RW:2272: Outgoing Label:18
Mar 11 15:19:22.020 mpls/lsd/rw 0/0/CPU0 t1 RW:1965: IPv4 MOI - NH:20.4.20.4, IF:GigabitEthernet0/0/0/0.420 (ifh 0xf00), NHID:0x5, Lbl:18, path_flags:0x0

/

This could be fixed in a few ways, the easiest happens to be we have that command. Note that there is a different variation of this scenario where you have a one-hop loopback peering at the ASBR layer, in that case, you will need 'mpls activate', since label forwarding will be deactivated - so many tricks ;).

Check this bug: 'CSCud27151 BGP rewrites LDP label when BGP confederation is used.' which talks about this, since you are a Cisco employee, you can 'topic' the defect internally and find more details, hopefully. ; )

/

HTH,

Elvin

mnavada · ‎03-11-2020

Hi Elvin,

Looks like one more solution we have for the same.

Remove ebggp-multihop mpls command replace with next hop unchanged along with ebgp-multihop 255.

See the output below

RP/0/0/CPU0:ios#sh run router bgp
Thu Mar 12 09:01:26.721 UTC
router bgp 65001
bgp router-id 9.9.9.9
address-family ipv4 unicast
!
address-family vpnv4 unicast
!
neighbor 2.2.2.2
remote-as 65000
ebgp-multihop 255----------
address-family vpnv4 unicast
next-hop-unchanged----------

But there is no way we can paste next hop unchanged in this case it wont affect anything this is purely Option c without RR (that command only valid for option c with RR to avoid extra LSP and symmetric path Through RR).

I Just want to tell this next hop unchanged command also tells the XR rewrite the label and use as LDP for forwarding.

RP/0/0/CPU0:ios#sh mpls forwarding
Thu Mar 12 09:06:08.981 UTC
Local Outgoing Prefix Outgoing Next Hop Bytes
Label Label or ID Interface Switched
------ ----------- ------------------ ------------ --------------- ------------
24000 Pop 7.7.7.7/32 Gi0/0/0/0 10.1.79.7 2730
24001 Pop 10.1.67.0/24 Gi0/0/0/0 10.1.79.7 0
24002 24001 6.6.6.6/32 Gi0/0/0/0 10.1.79.7 2655
24003 24002 2.2.2.2/32 Gi0/0/0/0 10.1.79.7 2176--------->>>LDP
24004 24003 4.4.4.4/32 Gi0/0/0/0 10.1.79.7 0
24005 Aggregate MPLS-A: Per-VRF Aggr[V] \
MPLS-A 520
24006 Unlabelled 20.20.20.20/32[V] Gi0/0/0/1 10.1.91.10 0
RP/0/0/CPU0:ios#sh mpls label table label 24003 detail
Thu Mar 12 09:06:26.770 UTC
Table Label Owner State Rewrite
----- ------- ------------------------------- ------ -------
0 24003 LDP(A) InUse Yes---------> LDP
(IPv4, vers:0, 'default':4U, 2.2.2.2/32)
RP/0/0/CPU0:ios#

See this without next hop unchanged command output below

RP/0/0/CPU0:ios#config t
Thu Mar 12 09:07:27.256 UTC
RP/0/0/CPU0:ios(config)#router bgp 65001
RP/0/0/CPU0:ios(config-bgp)#neighbor 2.2.2.2
RP/0/0/CPU0:ios(config-bgp-nbr)#ad
address-family advertisement-interval
RP/0/0/CPU0:ios(config-bgp-nbr)#adDress-family vpnv4 unicast
RP/0/0/CPU0:ios(config-bgp-nbr-af)#no next-hop-unchanged
RP/0/0/CPU0:ios(config-bgp-nbr-af)#commit
Thu Mar 12 09:07:45.145 UTC
RP/0/0/CPU0:ios(config-bgp-nbr-af)#do sh mpls label table label 24003
Thu Mar 12 09:08:01.114 UTC
Table Label Owner State Rewrite
----- ------- ------------------------------- ------ -------
0 24003 LDP(A) InUse Yes
BGP-VPNv4(A):bgp-default InUse Yes
RP/0/0/CPU0:ios(config-bgp-nbr-af)#
RP/0/0/CPU0:ios(config-bgp-nbr-af)#do sh mpls label table label 24003 detail
Thu Mar 12 09:08:04.853 UTC
Table Label Owner State Rewrite
----- ------- ------------------------------- ------ -------
0 24003 LDP(A) InUse Yes--------> LDP
BGP-VPNv4(A):bgp-default InUse Yes---------> BGP in use
(IPv4, vers:0, 'default':4U, 2.2.2.2/32)
RP/0/0/CPU0:ios(config-bgp-nbr-af)#

: RW Update [0/2] IPv4 FPI (BCDL Pri:4) - Prefix:2.2.2.2/32, Route flags:0x0
RW:1416: RW Update [0/2] Label FPI (BCDL Pri:4) - Incoming Label:24003, ELC:0, rw_flags:0x80000
RW:2084: IPv4 MOI - NH:10.1.79.7, IF:GigabitEthernet0/0/0/0 (ifh 0x100), NHID:0x2, Lbl:24002, path_flag

As you said BGP preferred it works top of MPLS but we came to know there r many ways we can rewrite the label and i tell you this is not the convincing solution.

May be soon it will be resolved.

Elvin Arias · ‎03-12-2020

XE and XR are different, and these kind of behaviors and dependencies you will find it in a few features. This is a non-issue and is the expected behavior as explained in the root cause analysis above.

Standard solutions:

1. With RR - use the NH unchanged.
2. Without RR - use ebgp-multihop mpls

Workaround:

1. Create a TE tunnel directly between the PEs to construct the LSP end-to-end.
2. Other workarounds exist, too. ; )

It was a pleasure to help.

Elvin

I'mDos · ‎06-18-2020

Hi Elvin,

How do you fix the issue on label rewrite? Im testing Inter AS VPN Option C without RR. It's working fine so far but whenever there are some bgp flapping on ASBR1, the label on my PE1 get refreshed and change to "unlabelled". It stays there even after the BGP reconverge and I can see the next-hop label on my ASBR1. See below for details

My Topology is

CE1 -- PE1 (XR) -- ASBR1 (XE) -- ASBR2 (XR) -- PE2 (XE) - CE2

Here are my configs

PE1 (XR)

router bgp 20.20

neighbor 195.195.0.1. --> ASBR1 (XE)
remote-as 20.20
use neighbor-group AS2020-iBGP-NG
address-family ipv4 unicast
use af-group AS2020-iBGP-AF
address-family vpnv4 unicast
multipath
soft-reconfiguration inbound always

neighbor 130.130.0.1 --> PE2 (XE). This is redistributed from ASBR1 (XE) and learned via IGP
remote-as 60.60
use neighbor-group AS2020-EXTERNAL-NG
ebgp-multihop 255 mpls
update-source Loopback1
address-family vpnv4 unicast
route-policy ACCEPT-ALL in
route-policy AS2020-MPLS-OPTION_C out
soft-reconfiguration inbound always

route-policy AS2020-MPLS-OPTION_C
set next-hop 195.195.0.3. --> PE2 (XE) LDP Loopback IP

ASBR1 (XE)

router bgp 20.20

neighbor 12.12.12.2 remote-as 60.60 --> ASBR2 (XR)

neighbor 195.195.0.3 remote-as 20.20 --> PE1 (XR)

address-family ipv4

network 193.193.0.1 mask 255.255.255.255

network 195.195.0.3 mask 255.255.255.255

neighbor 195.195.0.3 activate
neighbor 195.195.0.3 send-community both

neighbor 12.12.12.2 activate

neighbor 12.12.12.2 send-label

address-family vpnv4

neighbor 195.195.0.3 activate
neighbor 195.195.0.3 send-community both
neighbor 195.195.0.3 next-hop-self

router ospf 10

redistribute bgp 20.20 subnets route-map BGP-TO-OSPF-INJECT

ip prefix-list BGP-TO-OSPF seq 10 permit 192.0.0.3/32
ip prefix-list BGP-TO-OSPF seq 11 permit 130.130.0.1/32

route-map BGP-TO-OSPF-INJECT permit 10
match ip address prefix-list BGP-TO-OSPF

ASBR2 (XR)

router bgp 60.62

bgp confederation peers
60.61
60.63

bgp confederation identifier 60.60

address-family ipv4 unicast

network 192.0.0.3/32 route-policy AS6060-POLICY-AIGP

network 130.130.0.1/32 route-policy AS6060-POLICY-AIGP

allocate-label all

neighbor 192.0.0.3 --> ASBR2 (XR)
remote-as 60.63
use neighbor-group AS6060-CONFEDERATION-NG
address-family ipv4 unicast
use af-group AS6060-CONFEDERATION-AF
next-hop-self
address-family vpnv4 unicast
use af-group AS6060-CONFEDERATION-VPNV4-AF

neighbor 12.12.12.1 --> ASBR1 (XE)
remote-as 20.20
use neighbor-group EXTERNAL-PEERS-NG
address-family ipv4 unicast
use af-group EXTERNAL-PEERS-AF
route-policy ACCEPT-ALL in
route-policy AS2020-POLICY-OUT out

address-family ipv4 labeled-unicast
send-community-ebgp
route-policy ACCEPT-ALL in
route-policy ACCEPT-ALL out
send-extended-community-ebgp
soft-reconfiguration inbound always

router ospf 10

redistribute bgp 60.62 route-policy BGP-TO-OSPF-INJECT

route-policy BGP-TO-OSPF-INJECT
if destination in (195.195.0.3/32, 193.193.0.1/32) then
pass
else
drop
endif

PE2 (XE)

router bgp 60.63

bgp confederation identifier 60.60
bgp confederation peers 60.61 60.62

neighbor 192.0.0.1 remote-as 60.61

neighbor 193.193.0.1 remote-as 20.20

neighbor 193.193.0.1 ebgp-multihop 5

address-family ipv4

neighbor 192.0.0.1 activate

address-family vpnv4

neighbor 192.0.0.1 activate
neighbor 192.0.0.1 send-community extended
neighbor 192.0.0.1 inherit peer-policy AS6060-CONFEDERATION-VPNV4-PP

neighbor 193.193.0.1 activate
neighbor 193.193.0.1 send-community extended
neighbor 193.193.0.1 next-hop-unchanged
neighbor 193.193.0.1 route-map AS2020-MPLS-OPTION_C out

route-map AS2020-MPLS-OPTION_C permit 10
set ip next-hop 192.0.0.3

CE1

CE1>sho ip route bgp
Codes: L - local, C - connected, S - static, R - RIP, M - mobile, B - BGP
D - EIGRP, EX - EIGRP external, O - OSPF, IA - OSPF inter area
N1 - OSPF NSSA external type 1, N2 - OSPF NSSA external type 2
E1 - OSPF external type 1, E2 - OSPF external type 2
i - IS-IS, su - IS-IS summary, L1 - IS-IS level-1, L2 - IS-IS level-2
ia - IS-IS inter area, * - candidate default, U - per-user static route
o - ODR, P - periodic downloaded static route, H - NHRP, l - LISP
a - application route
+ - replicated route, % - next hop override, p - overrides from PfR

Gateway of last resort is not set

60.0.0.0/30 is subnetted, 1 subnets
B 60.60.60.4 [20/0] via 15.15.15.13, 11:00:57 --> CE2
140.140.0.0/32 is subnetted, 1 subnets
B 140.140.0.3 [20/0] via 15.15.15.13, 11:00:57 --> CE2
192.192.0.0/32 is subnetted, 2 subnets
B 192.192.0.5 [20/0] via 15.15.15.13, 11:08:49

CE1>traceroute 140.140.0.3
Type escape sequence to abort.
Tracing the route to 140.140.0.3
VRF info: (vrf in name/id, vrf out name/id)
1 15.15.15.13 4 msec 5 msec 3 msec
2 12.12.12.17 [MPLS: Labels 50014/350002 Exp 0] 13 msec 6 msec 5 msec
3 12.12.12.2 [MPLS: Labels 320001/350002 Exp 0] 6 msec 7 msec 5 msec
4 60.60.60.5 [AS 20.20] [MPLS: Labels 0/350002 Exp 0] 8 msec 7 msec 7 msec
5 60.60.60.6 [AS 20.20] 6 msec 5 msec *

CE2

CE2#sho ip bgp
BGP table version is 199, local router ID is 140.140.0.3
Status codes: s suppressed, d damped, h history, * valid, > best, i - internal,
r RIB-failure, S Stale, m multipath, b backup-path, f RT-Filter,
x best-external, a additional-path, c RIB-compressed,
t secondary path,
Origin codes: i - IGP, e - EGP, ? - incomplete
RPKI validation codes: V valid, I invalid, N Not found

Network Next Hop Metric LocPrf Weight Path
*> 0.0.0.0 60.60.60.5 0 60.60 i
*> 15.15.15.12/30 60.60.60.5 0 60.60 20.20 60.60 ? --> CE1
*> 60.60.60.4/30 0.0.0.0 0 32768 ?
*> 140.140.0.3/32 0.0.0.0 0 32768 ?
*> 192.192.0.4/32 60.60.60.5 0 60.60 20.20 60.60 ? --> CE1
*> 192.192.0.5/32 60.60.60.5 0 60.60 20.20 ?

CE2#traceroute 192.192.0.4
Type escape sequence to abort.
Tracing the route to 192.192.0.4
VRF info: (vrf in name/id, vrf out name/id)
1 60.60.60.5 6 msec 3 msec 3 msec
2 30.30.30.13 [AS 60.60] [MPLS: Labels 330031/20003 Exp 0] 5 msec 7 msec 5 msec
3 12.12.12.1 [AS 60.60] [MPLS: Labels 40001/20003 Exp 0] 5 msec 6 msec 4 msec
4 12.12.12.18 [AS 60.60] 4 msec 4 msec 11 msec
5 15.15.15.14 [AS 60.60] 7 msec 5 msec *

-----------------------

PE1 (XR)

PE1#sho mpls forwarding
Thu Jun 18 07:59:41.202 UTC
Local Outgoing Prefix Outgoing Next Hop Bytes
Label Label or ID Interface Switched
------ ----------- ------------------ ------------ --------------- ------------
20000 50014 192.0.0.3/32 Gi0/0/0/2 12.12.12.17 2066252. --> This is the label when everything is well
20003 Aggregate A2-INTER-CORP5: Per-VRF Aggr[V] \
A2-INTER-CORP5 8528584
20005 350014 6060:20:60.60.60.4/30 \
192.0.0.3 0
20006 350002 6060:20:140.140.0.3/32 \
192.0.0.3 0

ASBR1 (XE)

ASBR1#clear ip bgp *
*Jun 18 08:03:31.489: %BGP-3-NOTIFICATION_MANY: sent to 3 sessions 6/4 (Administrative Reset) for all peers
*Jun 18 08:03:31.540: %BGP-5-ADJCHANGE: neighbor 12.12.12.2 Down User reset
*Jun 18 08:03:31.540: %BGP_SESSION-5-ADJCHANGE: neighbor 12.12.12.2 IPv4 Unicast topology base removed from session User reset
*Jun 18 08:03:31.541: %BGP-5-ADJCHANGE: neighbor 195.195.0.3 Down User reset
*Jun 18 08:03:31.541: %BGP_SESSION-5-ADJCHANGE: neighbor 195.195.0.3 VPNv4 Unicast topology base removed from session User reset
*Jun 18 08:03:31.541: %BGP_SESSION-5-ADJCHANGE: neighbor 195.195.0.3 IPv4 Unicast topology base removed from session User reset

ASBR1#sho ip bgp summary
BGP router identifier 12.12.12.1, local AS number 20.20
BGP table version is 30, main routing table version 30
26 network entries using 6448 bytes of memory
44 path entries using 5984 bytes of memory
23/12 BGP path/bestpath attribute entries using 6440 bytes of memory
6 BGP AS-PATH entries using 192 bytes of memory
3 BGP community entries using 72 bytes of memory
6 BGP extended community entries using 370 bytes of memory
20 BGP route-map cache entries using 1280 bytes of memory
14 BGP filter-list cache entries using 448 bytes of memory
BGP using 21234 total bytes of memory
Dampening enabled. 0 history paths, 0 dampened paths
3 received paths for inbound soft reconfiguration
BGP activity 420/373 prefixes, 767/702 paths, scan interval 60 secs

Neighbor V AS MsgRcvd MsgSent TblVer InQ OutQ Up/Down State/PfxRcd
12.12.12.2 4 60.60 18 11 30 0 0 00:02:12 16
195.195.0.3 4 20.20 15 16 30 0 0 00:02:16 2

ASBR1#sho mpls forwarding-table
Local Outgoing Prefix Bytes Label Outgoing Next Hop
Label Label or Tunnel Id Switched interface
40001 explicit-n 195.195.0.3/32 4537584 Gi2 12.12.12.18
50001 Pop Label IPv4 VRF[V] 0 aggregate/CORP5-INTER-AS5050
50009 No Label IPv4 VRF 424926 aggregate
50014 320001 192.0.0.3/32 0 Gi7 12.12.12.2

PE1 (XR)

PE1#sho mpls forwarding

Thu Jun 18 08:06:49.826 UTC
Local Outgoing Prefix Outgoing Next Hop Bytes
Label Label or ID Interface Switched
------ ----------- ------------------ ------------ --------------- ------------
20000 Unlabelled 192.0.0.3/32 Gi0/0/0/2 12.12.12.17 0 --> Remains Unlabelled even after the ASBR1 BGP reconverge

20003 Aggregate A2-INTER-CORP5: Per-VRF Aggr[V] \
A2-INTER-CORP5 8528584

20005 350014 6060:20:60.60.60.4/30 \
192.0.0.3 0
20006 350002 6060:20:140.140.0.3/32 \
192.0.0.3 0

To have it forcibly refresh, I need to do this on ASBR1

ASBR1(config)# ip route 192.0.0.3 255.255.255.255 null 0
ASBR1(config)#no ip route 192.0.0.3 255.255.255.255 null 0

Then the label will be reflected again on PE1

RP/0/RP0/CPU0:ISP-B-PE2#sho mpls forwarding
Thu Jun 18 08:15:18.345 UTC
Local Outgoing Prefix Outgoing Next Hop Bytes
Label Label or ID Interface Switched
------ ----------- ------------------ ------------ --------------- ------------
20000 50007 192.0.0.3/32 Gi0/0/0/2 12.12.12.17 0

Thanks in advance.

Regards,

Ed

Elvin Arias · ‎06-18-2020

I've encountered this issue many times when doing labs, each time I had to do the same as you (or even restart the instance). Since it is lightweight XRv, do not expected reliable results - You might find that XRv9000 would be a better choice (I think it is what's being used on the new iteration of the CCIESP lab).

In regular ASR9K, this _should_ not happen at all, BCDL process should be able to download the label information to the line cards just fine.

Regards,

Elvin

I'mDos · ‎06-18-2020

Thanks! I'll give it a try on a pysical lab soon. Im running XRv9K anyway.

Regards,

Ed