L3 MPLS VPN, Custers Learning Remote sites but can't ping

[ThomasCampbell64]

Cust A - HQ  and remote sites have learned all their routes via OSPF through MP-BGP from however they can't ping through the MPLS network. 

[ThomasCampbell64 · Accepted solution]

Think its a bug in my

CSR1000v

getting a VRF doesn't exist message :

%VRF

specified does not match this router

When checking the config it clearly does

[Harold Ritter]

Hi @ThomasCampbell64 ,

This is not a bug. This message is usually caused if you are trying to configure ospf for the VRF, but use the process number already used by ospf for the global VRF (GRT).

R1(config)#do sh runn vrf
Building configuration...

Current configuration : 74 bytes
vrf definition test
!
address-family ipv4
exit-address-family
!
!
end

R1(config)#do sh runn | sec router ospf
router ospf 1
router-id 192.168.100.1
R1(config)#router ospf 1 vrf test 
%VRF specified does not match existing router

But it works like a charm if you use a different process number on the same router.

R1(config)#router ospf 10 vrf test
R1(config-router)#

Regards,

[ThomasCampbell64]

router ospf 1
prefix-suppression
area 0
interface Loopback0
!
interface GigabitEthernet0/0/0/1
!
interface GigabitEthernet0/0/0/2
!
!
!
router ospf 100
vrf CustB
redistribute bgp 100
address-family ipv4 unicast
area 0
interface GigabitEthernet0/0/0/0
network point-to-point
!
!
!
!
router bgp 100
bgp router-id 10.2.2.2
address-family vpnv4 unicast
!
address-family vpnv6 unicast
!
neighbor 10.12.12.12 <<< Route Reflector
remote-as 100
update-source Loopback0
address-family vpnv4 unicast
!
address-family vpnv6 unicast
!
!
vrf CustB
address-family ipv4 unicast
redistribute ospf 100 match internal external
!
!
!
mpls ldp
router-id 10.2.2.2
interface GigabitEthernet0/0/0/1
!
interface GigabitEthernet0/0/0/2
!
!
mpls label range table 0 24200 24299
end

RP/0/RP0/CPU0:XR2#show run vrf
Thu Aug 31 03:55:51.791 UTC
vrf CustB
rd 100:20
address-family ipv4 unicast
import route-target
100:2000
!
export route-target
100:2000
!
!
address-family ipv6 unicast
import route-target
100:2000
!
export route-target
100:2000
!
!
!

[ThomasCampbell64]

From each PE side I can ping the network addresses in the VRF from the PE's and routing information is being exchanged between the remote sites but they can't pass traffic.

Can see the Loopback address of the remote site from the Route-reflector and can ping the next hop of the CE of the Remote site but can't ping the remote site addresses:

RP/0/RP0/CPU0:XR2#show bgp vpnv4 unicast
Thu Aug 31 04:03:37.888 UTC
BGP router identifier 10.2.2.2, local AS number 100
BGP generic scan interval 60 secs
Non-stop routing is enabled
BGP table state: Active
Table ID: 0x0 RD version: 0
BGP main routing table version 11
BGP NSR Initial initsync version 1 (Reached)
BGP NSR/ISSU Sync-Group versions 0/0
BGP scan interval 60 secs

Status codes: s suppressed, d damped, h history, * valid, > best
i - internal, r RIB-failure, S stale, N Nexthop-discard
Origin codes: i - IGP, e - EGP, ? - incomplete
Network Next Hop Metric LocPrf Weight Path
Route Distinguisher: 100:20 (default for vrf CustB)
*> 10.7.7.7/32 192.168.27.7 2 32768 ?
*>i10.10.10.10/32 10.5.5.5 2 100 0 ?
*> 10.100.1.0/24 192.168.27.7 2 32768 ?
*>i10.100.10.0/24 10.5.5.5 2 100 0 ?
*> 192.168.27.0/24 0.0.0.0 0 32768 ?
*>i192.168.51.0/24 10.5.5.5 0 100 0 ?

Processed 6 prefixes, 6 paths
RP/0/RP0/CPU0:XR2#ping 10.5.5.5 source loopback 0
Thu Aug 31 04:04:02.210 UTC
Type escape sequence to abort.
Sending 5, 100-byte ICMP Echos to 10.5.5.5, timeout is 2 seconds:
!!!!!
Success rate is 100 percent (5/5), round-trip min/avg/max = 2/3/8 ms
RP/0/RP0/CPU0:XR2#ping vrf CustB 10.10.10.10
Thu Aug 31 04:04:36.523 UTC
Type escape sequence to abort.
Sending 5, 100-byte ICMP Echos to 10.10.10.10, timeout is 2 seconds:
.....
Success rate is 0 percent (0/5)

[Harold Ritter · Accepted solution]

Hi @ThomasCampbell64 ,

Can you provide the output for the following commands from XR2:

show mpls ldp neighbor

show cef vrf CustB 10.10.10.10/32 det

ping mpls ipv4 10.5.5.5/32 source loopback 0 (please configure "mpls oam" first)

Regards,

[ThomasCampbell64 · Accepted solution]

Wasn't an option to ping from loop back with the mpls oam so it put the loopback address in manually:

RP/0/RP0/CPU0:XR2#show mpls ldp neighbor
Thu Aug 31 04:57:49.547 UTC

Peer LDP Identifier: 10.3.3.3:0
TCP connection: 10.3.3.3:19948 - 10.2.2.2:646
Graceful Restart: No
Session Holdtime: 180 sec
State: Oper; Msgs sent/rcvd: 32/38; Downstream-Unsolicited
Up time: 00:17:51
LDP Discovery Sources:
IPv4: (1)
GigabitEthernet0/0/0/2
IPv6: (0)
Addresses bound to this peer:
IPv4: (6)
10.3.3.3 172.16.13.3 172.16.23.3 172.16.34.3
172.16.35.3 172.16.113.3
IPv6: (0)

Peer LDP Identifier: 10.4.4.4:0
TCP connection: 10.4.4.4:45347 - 10.2.2.2:646
Graceful Restart: No
Session Holdtime: 180 sec
State: Oper; Msgs sent/rcvd: 33/38; Downstream-Unsolicited
Up time: 00:17:51
LDP Discovery Sources:
IPv4: (1)
GigabitEthernet0/0/0/1
IPv6: (0)
Addresses bound to this peer:
IPv4: (6)
10.4.4.4 172.16.14.4 172.16.24.4 172.16.34.4
172.16.45.4 172.16.114.4
IPv6: (0)

RP/0/RP0/CPU0:XR2#show cef vrf CustB 10.10.10.10/32 detail
Thu Aug 31 04:58:42.382 UTC
10.10.10.10/32, version 9, internal 0x5000001 0x30 (ptr 0xdbd6c28) [1], 0x0 (0xe4dbfd8), 0xa08 (0xe868378)
Updated Aug 31 04:56:50.252
Prefix Len 32, traffic index 0, precedence n/a, priority 3
gateway array (0xe345af0) reference count 3, flags 0x38, source rib (7), 0 backups
[4 type 1 flags 0x8441 (0xe8ad428) ext 0x0 (0x0)]
LW-LDI[type=1, refc=1, ptr=0xe4dbfd8, sh-ldi=0xe8ad428]
gateway array update type-time 3 Aug 31 04:41:10.070
LDI Update time Aug 31 04:41:10.070
LW-LDI-TS Aug 31 04:56:50.253
via 10.5.5.5/32, 3 dependencies, recursive [flags 0x6000]
path-idx 0 NHID 0x0 [0xdd20060 0x0]
recursion-via-/32
next hop VRF - 'default', table - 0xe0000000
next hop 10.5.5.5/32 via 24203/0/21
next hop 172.16.23.3/32 Gi0/0/0/2 labels imposed {24300 24505}
next hop 172.16.24.4/32 Gi0/0/0/1 labels imposed {24401 24505}

Load distribution: 0 (refcount 4)

Hash OK Interface Address
0 Y recursive 24203/0

RP/0/RP0/CPU0:XR2#ping mpls ipv4 10.5.5.5/32 source 10.2.2.2
Thu Aug 31 05:01:15.762 UTC

Sending 5, 100-byte MPLS Echos to 10.5.5.5/32,
timeout is 2 seconds, send interval is 0 msec:

Codes: '!' - success, 'Q' - request not sent, '.' - timeout,
'L' - labeled output interface, 'B' - unlabeled output interface,
'D' - DS Map mismatch, 'F' - no FEC mapping, 'f' - FEC mismatch,
'M' - malformed request, 'm' - unsupported tlvs, 'N' - no rx label,
'P' - no rx intf label prot, 'p' - premature termination of LSP,
'R' - transit router, 'I' - unknown upstream index,
'X' - unknown return code, 'x' - return code 0

Type escape sequence to abort.

!!!!!
Success rate is 100 percent (5/5), round-trip min/avg/max = 5/10/25 ms

[ThomasCampbell64]

I am not sure but i think the mpls oam made things work, pings working fine now. Might of been bug in the EVE-NG or IOS-XRv side of things...

R7#sh ip route ospf
Codes: L - local, C - connected, S - static, R - RIP, M - mobile, B - BGP
D - EIGRP, EX - EIGRP external, O - OSPF, IA - OSPF inter area
N1 - OSPF NSSA external type 1, N2 - OSPF NSSA external type 2
E1 - OSPF external type 1, E2 - OSPF external type 2
i - IS-IS, su - IS-IS summary, L1 - IS-IS level-1, L2 - IS-IS level-2
ia - IS-IS inter area, * - candidate default, U - per-user static route
o - ODR, P - periodic downloaded static route, H - NHRP, l - LISP
a - application route
+ - replicated route, % - next hop override, p - overrides from PfR

Gateway of last resort is not set

10.0.0.0/8 is variably subnetted, 5 subnets, 2 masks
O E2 10.10.10.10/32 [110/2] via 192.168.27.2, 00:21:09, GigabitEthernet0/1
O E2 10.100.10.0/24 [110/2] via 192.168.27.2, 00:21:09, GigabitEthernet0/1
O E2 192.168.51.0/24 [110/1] via 192.168.27.2, 00:21:09, GigabitEthernet0/1
R7#ping 10.10.10.10
Type escape sequence to abort.
Sending 5, 100-byte ICMP Echos to 10.10.10.10, timeout is 2 seconds:
!!!!!
Success rate is 100 percent (5/5), round-trip min/avg/max = 4/5/6 ms

[ThomasCampbell64]

Thank you so much!!! @Harold Ritter 

R7#traceroute 10.10.10.10
Type escape sequence to abort.
Tracing the route to 10.10.10.10
VRF info: (vrf in name/id, vrf out name/id)
1 192.168.27.2 19 msec 3 msec 3 msec
2 172.16.23.3 [MPLS: Labels 24300/24505 Exp 0] 10 msec
172.16.24.4 [MPLS: Labels 24401/24505 Exp 0] 8 msec
172.16.23.3 [MPLS: Labels 24300/24505 Exp 0] 5 msec
3 192.168.51.5 [MPLS: Label 24505 Exp 0] 6 msec 6 msec 5 msec
4 192.168.51.10 5 msec * 6 msec
R7#

[Harold Ritter]

You are welcome @ThomasCampbell64 and thanks for the feedback