02-14-2025 12:03 PM
Hi guys
Thanks for a great resource!
I've set up an SP lab where i am right now working on an SR setup. This is all virtual running the following images:
- Cisco IOS XR Software, Version 7.4.1
- CSR1000V - Cisco IOS XE Software, Version 17.03.02
- ISRV - Cisco IOS XE Software, Version 17.03.03
The diagram below shows a bit of the parts of the lab that i am currently working on, for context.
So, ISPs 2 and 4 are up and running as described in their respective white text boxes. Traffic is exchanged across the Option C link between them. I have connectivity in the simulated vrf Internet between devices 4-ISP-7 (Lo47) and Internet (1.1.1.1). As seen in the following trace:
4-ISP-7#traceroute vrf Internet 1.1.1.1 source lo47
Type escape sequence to abort.
Tracing the route to 1.1.1.1
VRF info: (vrf in name/id, vrf out name/id)
1 10.4.37.3 [MPLS: Labels 40005/19/2302 Exp 0] 21 msec
10.4.47.4 [MPLS: Labels 40005/19/2302 Exp 0] 18 msec
10.4.37.3 [MPLS: Labels 40005/19/2302 Exp 0] 4 msec
2 10.4.46.6 [MPLS: Labels 40005/19/2302 Exp 0] 5 msec
10.4.23.2 [MPLS: Labels 40005/19/2302 Exp 0] 17 msec
10.4.46.6 [MPLS: Labels 40005/19/2302 Exp 0] 3 msec
3 10.4.25.5 [MPLS: Labels 19/2302 Exp 0] 41 msec
10.4.56.5 [MPLS: Labels 19/2302 Exp 0] 5 msec
10.4.25.5 [MPLS: Labels 19/2302 Exp 0] 3 msec
4 172.1.2.1 [MPLS: Label 2302 Exp 0] 3 msec 2 msec 3 msec
5 172.1.2.0 4 msec * 4 msec
What troubles me is this. There is no connectivity from 4-ISP-3 to the Internet router. I have routing entries both in the vrf and in the underlay and yet i get this when i look in the fib:
RP/0/RP0/CPU0:4-ISP-3#sh ip cef vrf Internet 1.1.1.1
Wed Dec 13 05:32:57.267 UTC
1.1.1.1/32, version 19, drop adjacency, internal 0x5000001 0x30 (ptr 0xebae450) [1], 0x600 (0xed4c0b0), 0xa08 (0xeea47d8)
Updated Dec 13 03:49:50.467
Prefix Len 32, traffic index 0, precedence n/a, priority 3
gateway array (0xebb4318) reference count 5, flags 0x3a, source rib (7), 0 backups
[6 type 1 flags 0x88401 (0xeee78e8) ext 0x0 (0x0)]
LW-LDI[type=1, refc=1, ptr=0xed4c0b0, sh-ldi=0xeee78e8]
gateway array update type-time 3 Dec 13 05:32:43.819
LDI Update time Dec 13 03:49:50.467
LW-LDI-TS Dec 13 03:49:50.467
via 10.2.255.3/32, 0 dependencies, recursive [flags 0x6000]
path-idx 0 NHID 0x0 [0xd6643f8 0x0]
recursion-via-/32
next hop VRF - 'default', table - 0xe0000000
unresolved
labels imposed {2302}
Load distribution: 0 (refcount 6)
Hash OK Interface Address
0 Y recursive drop
The RIB looks okay imho:
RP/0/RP0/CPU0:4-ISP-3#sh ip route vrf Internet 1.1.1.1
Wed Dec 13 06:22:35.759 UTC
Routing entry for 1.1.1.1/32
Known via "bgp 4", distance 200, metric 0
Tag 2, type internal
Installed Dec 13 03:49:50.460 for 02:32:45
Routing Descriptor Blocks
10.2.255.3, from 10.4.255.6
Nexthop in Vrf: "default", Table: "default", IPv4 Unicast, Table Id: 0xe0000000
Route metric is 0
No advertising protos.
The underlay looks good across the Option C:
RP/0/RP0/CPU0:4-ISP-3#ping 10.2.255.3 sour lo 0
Wed Dec 13 06:24:14.285 UTC
Type escape sequence to abort.
Sending 5, 100-byte ICMP Echos to 10.2.255.3, timeout is 2 seconds:
!!!!!
Success rate is 100 percent (5/5), round-trip min/avg/max = 3/4/8 ms
RP/0/RP0/CPU0:4-ISP-3#sh ip route 10.2.255.3
Wed Dec 13 06:24:23.325 UTC
Routing entry for 10.2.255.3/32
Known via "bgp 4", distance 200, metric 0
Tag 2, type internal
Installed Dec 11 00:01:58.824 for 2d06h
Routing Descriptor Blocks
10.4.255.5, from 10.4.255.6
Route metric is 0
No advertising protos.
Again, the XE box 4-ISP-7, has connectivity in the Internet vrf. I need help to figure out why the FIB entry is a drop adj...
Any ideas appreciated!
Thanks and have a great weekend.
Cheers
02-14-2025 12:51 PM
labels imposed is there, i am assuming its the vrf label 2302
can we check "show mpls forwarding prefix" need to confirm that outgoing label is present for 10.2.255.3/32
02-14-2025 01:43 PM - edited 02-14-2025 02:15 PM
Hey
I was wondering about that my self, this output shows a pr. vrf label association:
RP/0/RP0/CPU0:4-ISP-3#sh mpls forwarding
Wed Dec 13 07:54:34.177 UTC
Local Outgoing Prefix Outgoing Next Hop Bytes
Label Label or ID Interface Switched
------ ----------- ------------------ ------------ --------------- ------------
24000 Pop SR Adj (idx 1) Gi0/0/0/0 10.4.23.2 0
24001 Pop SR Adj (idx 3) Gi0/0/0/0 10.4.23.2 0
24002 Pop SR Adj (idx 1) Gi0/0/0/3 10.4.37.7 0
24003 Pop SR Adj (idx 3) Gi0/0/0/3 10.4.37.7 0
24004 Aggregate Internet: Per-VRF Aggr[V] \
Internet 0
40001 40001 SR Pfx (idx 1) Gi0/0/0/0 10.4.23.2 0
40002 Pop SR Pfx (idx 2) Gi0/0/0/0 10.4.23.2 0
40004 40004 SR Pfx (idx 4) Gi0/0/0/3 10.4.37.7 0
40005 40005 SR Pfx (idx 5) Gi0/0/0/0 10.4.23.2 4344
40006 40006 SR Pfx (idx 6) Gi0/0/0/0 10.4.23.2 346566
40007 Pop SR Pfx (idx 7) Gi0/0/0/3 10.4.37.7 992
Isnt it default on xr with per prefix for mpls? Is this then specific to SR and do we change it to see what changes?
Also, it must be the VPN label, according to this output:
RP/0/RP0/CPU0:4-ISP-3#sh bgp vpnv4 uni vrf Internet 1.1.1.1
Wed Dec 13 08:02:25.157 UTC
BGP routing table entry for 1.1.1.1/32, Route Distinguisher: 10.4.255.3:1
Versions:
Process bRIB/RIB SendTblVer
Speaker 47 47
Last Modified: Dec 13 03:49:50.917 for 04:12:34
Paths: (1 available, best #1)
Not advertised to any peer
Path #1: Received by speaker 0
Not advertised to any peer
2 1
10.2.255.3 (metric 20) from 10.4.255.6 (10.4.255.6)
Received Label 2302
Origin IGP, metric 0, localpref 100, valid, internal, best, group-best, import-candidate, imported
Received Path ID 0, Local Path ID 1, version 47
Extended community: RT:1:1
Source AFI: VPNv4 Unicast, Source VRF: default, Source Route Distinguisher: 10.2.255.3:1
According to the BGP-LU (I have BGP-LU across the Option C, think i forgot to mention) table it should be 19:
RP/0/RP0/CPU0:4-ISP-3#sh bgp ipv4 uni 10.2.255.3
Wed Dec 13 08:22:01.142 UTC
BGP routing table entry for 10.2.255.3/32
Versions:
Process bRIB/RIB SendTblVer
Speaker 4 4
Last Modified: Dec 11 00:01:58.917 for 2d08h
Paths: (1 available, best #1)
Not advertised to any peer
Path #1: Received by speaker 0
Not advertised to any peer
2
10.4.255.5 (metric 20) from 10.4.255.6 (10.4.255.5)
Received Label 19
Origin IGP, metric 0, localpref 100, valid, internal, best, group-best, labeled-unicast
Received Path ID 0, Local Path ID 1, version 4
Originator: 10.4.255.5, Cluster list: 10.4.255.6
And by looking at this output, the RIB looks to recurse to the nexthop out of ISP4:
RP/0/RP0/CPU0:4-ISP-3#sh ip route 10.2.255.3 det
Wed Dec 13 08:06:30.188 UTC
Routing entry for 10.2.255.3/32
Known via "bgp 4", distance 200, metric 0
Tag 2, type internal
Installed Dec 11 00:01:58.824 for 2d08h
Routing Descriptor Blocks
10.4.255.5, from 10.4.255.6
Route metric is 0
Label: 0x13 (19)
Tunnel ID: None
Binding Label: None
Extended communities count: 0
NHID:0x0(Ref:0)
Route version is 0x4 (4)
No local label
IP Precedence: Not Set
QoS Group ID: Not Set
Flow-tag: Not Set
Fwd-class: Not Set
Route Priority: RIB_PRIORITY_RECURSIVE (10) SVD Type RIB_SVD_TYPE_LOCAL
Download Priority 4, Download Version 69
No advertising protos.
It doesnt push the SR label for some reason...
On 4-ISP-7, where there is connectivity the RIB and BGP-LU tables match on the label:
4-ISP-7#sh ip route 10.2.255.3
Routing entry for 10.2.255.3/32
Known via "bgp 4", distance 200, metric 0
Tag 2, type internal
Last update from 10.4.255.5 04:32:17 ago
Routing Descriptor Blocks:
* 10.4.255.5, from 10.4.255.6, 04:32:17 ago
opaque_ptr 0x7F4D2FB1F0E8
Route metric is 0, traffic share count is 1
AS Hops 1
Route tag 2
MPLS label: 19
4-ISP-7#sh bgp ipv4 uni
4-ISP-7#
4-ISP-7#
4-ISP-7#
4-ISP-7#sh bgp ipv4 uni 10.2.255.3
BGP routing table entry for 10.2.255.3/32, version 2
Paths: (1 available, best #1, table default)
Flag: 0x100
Not advertised to any peer
Refresh Epoch 1
2
10.4.255.5 (metric 30) from 10.4.255.6 (10.4.255.6)
Origin IGP, metric 0, localpref 100, valid, internal, best
Originator: 10.4.255.5, Cluster list: 10.4.255.6
mpls labels in/out nolabel/19
rx pathid: 0, tx pathid: 0x0
Updated on Dec 13 2023 03:41:35 UTC
Cheers and thanks a bunch!
02-14-2025 02:40 PM
Found the error...
RP/0/RP0/CPU0:4-ISP-3(config)#router bgp 4
RP/0/RP0/CPU0:4-ISP-3(config-bgp)# address-family ipv4 unicast
RP/0/RP0/CPU0:4-ISP-3(config-bgp-af)#allocate-label all
RP/0/RP0/CPU0:4-ISP-3(config-bgp-af)#commit
Wed Dec 13 09:02:48.956 UTC
RP/0/RP0/CPU0:Dec 13 09:02:49.582 UTC: config[67433]: %MGBL-CONFIG-6-DB_COMMIT : Configuration committed by user 'xr'. Use 'show configuration commit changes 1000000031' to view the changes.
RP/0/RP0/CPU0:4-ISP-3(config-bgp-af)#RP/0/RP0/CPU0:Dec 13 09:02:51.121 UTC: bgp[1078]: [default-upd] (vpn4u): Started updgrp timer for updgrp 0.2:: delay=0.010, delaytype=0
RP/0/RP0/CPU0:Dec 13 09:02:51.124 UTC: ipv4_rib[1241]: %ROUTING-RIB-3-LABEL_ERR_ADD : Add local-label 24008 (2) for table 0xe0000000, prefix 10.4.255.5/32, by proto bgp client 27 bgp node0_RP0_CPU0 - existing label 40005 added by proto-id 5 client 28
RP/0/RP0/CPU0:Dec 13 09:02:51.129 UTC: fib_mgr[248]: %ROUTING-FIB-6-RETRYDB_OK : All objects in FIB IPv4 retry queue are resolved now.
RP/0/RP0/CPU0:Dec 13 09:02:51.133 UTC: bgp[1078]: [default-upd] (vpn4u): Starting updgen walk for updgrp 0.2:: targetver=57: tblver=57, labelver=57, minfwdver=57, ackdfwdver=57, standbyver=0
LC/0/0/CPU0:Dec 13 09:02:51.134 UTC: fib_mgr[187]: %ROUTING-FIB-6-RETRYDB_OK : All objects in FIB IPv4 retry queue are resolved now.
RP/0/RP0/CPU0:Dec 13 09:02:51.351 UTC: bgp[1078]: [default-upd] (vpn4u): Started updgrp timer for updgrp 0.2:: delay=0.010, delaytype=0
RP/0/RP0/CPU0:Dec 13 09:02:51.362 UTC: bgp[1078]: [default-upd] (vpn4u): Starting updgen walk for updgrp 0.2:: targetver=57: tblver=57, labelver=57, minfwdver=57, ackdfwdver=57, standbyver=0
end
RP/0/RP0/CPU0:Dec 13 09:02:53.500 UTC: config[67433]: %MGBL-SYS-5-CONFIG_I : Configured from console by xr
RP/0/RP0/CPU0:4-ISP-3#ping vrf Internet 1.1.1.1 sour Lo43
Wed Dec 13 09:03:01.801 UTC
Type escape sequence to abort.
Sending 5, 100-byte ICMP Echos to 1.1.1.1, timeout is 2 seconds:
!!!!!
Success rate is 100 percent (5/5), round-trip min/avg/max = 3/5/10 ms
I forgot to turn on allocate-label...
Thanks for the boost!
Cheers
02-14-2025 05:35 PM
Hi @kim ,
Glad you found the issue and thanks for the feedback.
Regards,
02-14-2025 05:03 PM - edited 02-14-2025 05:06 PM
Hi @kim ,
The "unresolved" may indicate that the egress interface towards the destination (gi0/0/0/1) is not MPLS enabled.
You can verify that using the "show mpls interface" command on 4-ISP-3.
You can add the following command on 4-ISP-3 to activate mpls on gi0/0/0/1:
router bgp 4
mpls activate
interface gi0/0/0/1
This is required when you use ibgp-lu to propagate the labels between the ASBR and the rest of the PEs in the AS.
Regards,
02-15-2025 12:52 AM
Hi Harold
Thanks! My first efforts here to connect ISP2 and ISP4 was across a border between two IOS-XE devices. Next up is connectivity between ISP 4 and an other ISPs that you cant see in the screenshot i uploaded here. Both another option C and B where there are XR-XR and XR-XE borders. I will definitely see the mpls activate command in action there
Cheers and have a great weekend!
Discover and save your favorite ideas. Come back to expert answers, step-by-step guides, recent topics, and more.
New here? Get started with these tips. How to use Community New member guide