L3VPN lab help

[Neil Stephens1]

Hello experts, i am working on a lab for CCNP SP core - L3VPN between two PEs. However i am struggling to get connectivity between CE1 and CE2. both CE1s are getting routes redistributed into them so they have routes to either site. However pinging is failing. This a CML lab - so happy to share the yaml if anyone would like.

One thing that may add to the complexity of this setup - is that R1 is a route reflector and its in-line.

Configs and outputs:

hostname PE2

!
vrf definition RED
rd 100:110
route-target export 100:1000
route-target import 100:1000
!
address-family ipv4
exit-address-family
!

!
interface Loopback0
ip address 22.22.22.22 255.255.255.0
!
interface GigabitEthernet0/0
vrf forwarding RED
ip address 192.168.1.253 255.255.255.0
!
interface GigabitEthernet0/1
ip address 10.0.7.2 255.255.255.0
mpls ip
!
interface GigabitEthernet0/2
ip address 10.0.8.2 255.255.255.0
mpls ip
!
!
!
router eigrp 1
!
address-family ipv4 vrf RED autonomous-system 1
redistribute bgp 65000
redistribute connected
network 192.168.1.0
exit-address-family
!
router ospf 1
router-id 22.22.22.22
network 10.0.0.0 0.255.255.255 area 0
network 11.11.11.0 0.0.0.255 area 0
network 22.22.22.0 0.0.0.255 area 0
!
router bgp 65000
bgp log-neighbor-changes
neighbor 1.1.1.1 remote-as 65000
neighbor 1.1.1.1 update-source Loopback0
!
address-family ipv4
neighbor 1.1.1.1 activate
neighbor 1.1.1.1 send-community both
exit-address-family
!
address-family vpnv4
neighbor 1.1.1.1 activate
neighbor 1.1.1.1 send-community both
exit-address-family
!
address-family ipv4 vrf RED
redistribute connected
redistribute eigrp 1
exit-address-family
!
mpls ldp router-id Loopback0 force

hostname PE1
!
vrf definition RED
rd 100:110
route-target export 100:1000
route-target import 100:1000
!
address-family ipv4
exit-address-family
!
!

!
!
!
no ip domain lookup
ip cef
no ipv6 cef
!
multilink bundle-name authenticated
no mpls ip propagate-ttl forwarded

!
!
interface Loopback0
ip address 11.11.11.11 255.255.255.0
ip ospf 1 area 0
!
interface GigabitEthernet0/0
vrf forwarding RED
ip address 192.168.2.253 255.255.255.0
!
interface GigabitEthernet0/1
ip address 10.0.1.1 255.255.255.0
mpls ip
!
interface GigabitEthernet0/2
ip address 10.0.2.1 255.255.255.0
mpls ip
!
!
!
router eigrp 1
!
address-family ipv4 vrf RED autonomous-system 1
redistribute bgp 65000
redistribute connected
network 192.168.2.0
exit-address-family
!
router ospf 1
router-id 11.11.11.11
network 10.0.0.0 0.0.255.255 area 0
network 11.11.11.0 0.0.0.255 area 0
!
router bgp 65000
bgp log-neighbor-changes
neighbor 1.1.1.1 remote-as 65000
neighbor 1.1.1.1 update-source Loopback0
!
address-family ipv4
neighbor 1.1.1.1 activate
neighbor 1.1.1.1 send-community both
exit-address-family
!
address-family vpnv4
neighbor 1.1.1.1 activate
neighbor 1.1.1.1 send-community both
exit-address-family
!
address-family ipv4 vrf RED
redistribute connected
redistribute eigrp 1
exit-address-family
!
mpls ldp router-id Loopback0 force

hostname R1 (route reflector)
!
!
interface Loopback0
ip address 1.1.1.1 255.255.255.252
ip ospf 1 area 0
!
!
interface GigabitEthernet0/1
ip address 10.0.1.2 255.255.255.0
mpls ip
!
!
interface GigabitEthernet0/3
ip address 10.0.3.1 255.255.255.0
mpls ip
!
router ospf 1
router-id 1.1.1.1
network 1.1.1.0 0.0.0.255 area 0
network 10.0.0.0 0.255.255.255 area 0
!
router bgp 65000
bgp log-neighbor-changes
neighbor 2.2.2.2 remote-as 65000
neighbor 2.2.2.2 update-source Loopback0
neighbor 3.3.3.3 remote-as 65000
neighbor 3.3.3.3 update-source Loopback0
neighbor 4.4.4.4 remote-as 65000
neighbor 4.4.4.4 update-source Loopback0
neighbor 5.5.5.5 remote-as 65000
neighbor 5.5.5.5 update-source Loopback0
neighbor 6.6.6.6 remote-as 65000
neighbor 6.6.6.6 update-source Loopback0
neighbor 11.11.11.11 remote-as 65000
neighbor 11.11.11.11 update-source Loopback0
neighbor 22.22.22.22 remote-as 65000
neighbor 22.22.22.22 update-source Loopback0
!
address-family ipv4
neighbor 2.2.2.2 activate
neighbor 2.2.2.2 send-community both
neighbor 2.2.2.2 route-reflector-client
neighbor 3.3.3.3 activate
neighbor 3.3.3.3 send-community both
neighbor 3.3.3.3 route-reflector-client
neighbor 4.4.4.4 activate
neighbor 4.4.4.4 send-community both
neighbor 4.4.4.4 route-reflector-client
neighbor 5.5.5.5 activate
neighbor 5.5.5.5 send-community both
neighbor 5.5.5.5 route-reflector-client
neighbor 6.6.6.6 activate
neighbor 6.6.6.6 send-community both
neighbor 6.6.6.6 route-reflector-client
neighbor 11.11.11.11 activate
neighbor 11.11.11.11 send-community both
neighbor 11.11.11.11 route-reflector-client
neighbor 22.22.22.22 activate
neighbor 22.22.22.22 send-community both
neighbor 22.22.22.22 route-reflector-client
exit-address-family
!
address-family vpnv4
neighbor 2.2.2.2 activate
neighbor 2.2.2.2 send-community both
neighbor 2.2.2.2 route-reflector-client
neighbor 3.3.3.3 activate
neighbor 3.3.3.3 send-community both
neighbor 3.3.3.3 route-reflector-client
neighbor 4.4.4.4 activate
neighbor 4.4.4.4 send-community both
neighbor 4.4.4.4 route-reflector-client
neighbor 4.4.4.4 next-hop-self
neighbor 5.5.5.5 activate
neighbor 5.5.5.5 send-community both
neighbor 5.5.5.5 route-reflector-client
neighbor 6.6.6.6 activate
neighbor 6.6.6.6 send-community both
neighbor 6.6.6.6 route-reflector-client
neighbor 11.11.11.11 activate
neighbor 11.11.11.11 send-community both
neighbor 11.11.11.11 route-reflector-client
neighbor 22.22.22.22 activate
neighbor 22.22.22.22 send-community both
neighbor 22.22.22.22 route-reflector-client
exit-address-family
!
!
mpls ldp router-id Loopback0 force

PE1#show ip bgp all
For address family: IPv4 Unicast

For address family: VPNv4 Unicast

BGP table version is 19, local router ID is 11.11.11.11
Status codes: s suppressed, d damped, h history, * valid, > best, i - internal,
r RIB-failure, S Stale, m multipath, b backup-path, f RT-Filter,
x best-external, a additional-path, c RIB-compressed,
t secondary path,
Origin codes: i - IGP, e - EGP, ? - incomplete
RPKI validation codes: V valid, I invalid, N Not found

Network Next Hop Metric LocPrf Weight Path
Route Distinguisher: 100:110 (default for vrf RED)
*> 172.17.1.0/24 192.168.2.1 130816 32768 ?
*>i 172.18.1.0/24 22.22.22.22 130816 100 0 ?
*>i 192.168.1.0 22.22.22.22 0 100 0 ?
*> 192.168.2.0 0.0.0.0 0 32768 ?
*> 192.168.4.0 192.168.2.1 3072 32768 ?
*>i 192.168.6.0 22.22.22.22 3072 100 0 ?

PE2#show ip bgp all
For address family: IPv4 Unicast

For address family: VPNv4 Unicast

BGP table version is 17, local router ID is 22.22.22.22
Status codes: s suppressed, d damped, h history, * valid, > best, i - internal,
r RIB-failure, S Stale, m multipath, b backup-path, f RT-Filter,
x best-external, a additional-path, c RIB-compressed,
t secondary path,
Origin codes: i - IGP, e - EGP, ? - incomplete
RPKI validation codes: V valid, I invalid, N Not found

Network Next Hop Metric LocPrf Weight Path
Route Distinguisher: 100:110 (default for vrf RED)
*>i 172.17.1.0/24 11.11.11.11 130816 100 0 ?
*> 172.18.1.0/24 192.168.1.1 130816 32768 ?
*> 192.168.1.0 0.0.0.0 0 32768 ?
*>i 192.168.2.0 11.11.11.11 0 100 0 ?
*>i 192.168.4.0 11.11.11.11 3072 100 0 ?
*> 192.168.6.0 192.168.1.1 3072 32768 ?

CE1#show ip route
Codes: L - local, C - connected, S - static, R - RIP, M - mobile, B - BGP
D - EIGRP, EX - EIGRP external, O - OSPF, IA - OSPF inter area
N1 - OSPF NSSA external type 1, N2 - OSPF NSSA external type 2
E1 - OSPF external type 1, E2 - OSPF external type 2
i - IS-IS, su - IS-IS summary, L1 - IS-IS level-1, L2 - IS-IS level-2
ia - IS-IS inter area, * - candidate default, U - per-user static route
o - ODR, P - periodic downloaded static route, H - NHRP, l - LISP
a - application route
+ - replicated route, % - next hop override, p - overrides from PfR

Gateway of last resort is not set

172.17.0.0/16 is variably subnetted, 2 subnets, 2 masks
C 172.17.1.0/24 is directly connected, Loopback0
L 172.17.1.1/32 is directly connected, Loopback0
172.18.0.0/24 is subnetted, 1 subnets
D 172.18.1.0
[90/131072] via 192.168.2.253, 03:15:31, GigabitEthernet0/0
D 192.168.1.0/24 [90/3072] via 192.168.2.253, 03:15:31, GigabitEthernet0/0
192.168.2.0/24 is variably subnetted, 2 subnets, 2 masks
C 192.168.2.0/24 is directly connected, GigabitEthernet0/0
L 192.168.2.1/32 is directly connected, GigabitEthernet0/0
192.168.4.0/24 is variably subnetted, 2 subnets, 2 masks
C 192.168.4.0/24 is directly connected, GigabitEthernet0/1
L 192.168.4.1/32 is directly connected, GigabitEthernet0/1
D 192.168.6.0/24 [90/3328] via 192.168.2.253, 03:15:31, GigabitEthernet0/0

CE2#show ip route
Codes: L - local, C - connected, S - static, R - RIP, M - mobile, B - BGP
D - EIGRP, EX - EIGRP external, O - OSPF, IA - OSPF inter area
N1 - OSPF NSSA external type 1, N2 - OSPF NSSA external type 2
E1 - OSPF external type 1, E2 - OSPF external type 2
i - IS-IS, su - IS-IS summary, L1 - IS-IS level-1, L2 - IS-IS level-2
ia - IS-IS inter area, * - candidate default, U - per-user static route
o - ODR, P - periodic downloaded static route, H - NHRP, l - LISP
a - application route
+ - replicated route, % - next hop override, p - overrides from PfR

Gateway of last resort is not set

172.17.0.0/24 is subnetted, 1 subnets
D 172.17.1.0
[90/131072] via 192.168.1.253, 03:15:54, GigabitEthernet0/0
172.18.0.0/16 is variably subnetted, 2 subnets, 2 masks
C 172.18.1.0/24 is directly connected, Loopback0
L 172.18.1.1/32 is directly connected, Loopback0
192.168.1.0/24 is variably subnetted, 2 subnets, 2 masks
C 192.168.1.0/24 is directly connected, GigabitEthernet0/0
L 192.168.1.1/32 is directly connected, GigabitEthernet0/0
D 192.168.2.0/24 [90/3072] via 192.168.1.253, 03:15:54, GigabitEthernet0/0
D 192.168.4.0/24 [90/3328] via 192.168.1.253, 03:15:54, GigabitEthernet0/0
192.168.6.0/24 is variably subnetted, 2 subnets, 2 masks
C 192.168.6.0/24 is directly connected, GigabitEthernet0/1
L 192.168.6.1/32 is directly connected, GigabitEthernet0/1

PE1#show mpls forwarding-table
Local Outgoing Prefix Bytes Label Outgoing Next Hop
Label Label or Tunnel Id Switched interface
16 No Label 192.168.2.0/24[V] \
0 aggregate/RED
17 No Label 192.168.4.0/24[V] \
0 Gi0/0 192.168.2.1
18 30 22.22.22.22/32 0 Gi0/1 10.0.1.2
18 22.22.22.22/32 0 Gi0/2 10.0.2.2
19 20 6.6.6.6/32 44268 Gi0/2 10.0.2.2
20 28 5.5.5.5/32 0 Gi0/1 10.0.1.2
21 26 3.3.3.3/32 0 Gi0/1 10.0.1.2
22 No Label 2.2.2.2/32 40748 Gi0/2 10.0.2.2
23 No Label 1.1.1.1/32 123683 Gi0/1 10.0.1.2
24 24 10.0.6.0/24 0 Gi0/2 10.0.2.2
25 25 10.0.8.0/24 0 Gi0/2 10.0.2.2
26 29 10.0.7.0/24 0 Gi0/1 10.0.1.2
27 27 10.0.5.0/24 0 Gi0/1 10.0.1.2
28 Pop Label 10.0.4.0/24 0 Gi0/2 10.0.2.2
29 Pop Label 10.0.3.0/24 0 Gi0/1 10.0.1.2
30 30 4.4.4.4/32 44702 Gi0/2 10.0.2.2
31 No Label 172.17.1.0/24[V] 0 Gi0/0 192.168.2.1

[Harold Ritter · Accepted solution]

Hi @Neil Stephens1 ,

The issue is that you configured the loopback interface address as a /24 on both PE1 and PE2. OSPF will advertise the loopback interface address as a /32 by default, but LDP will consider the prefix advertisement as a /24. This breaks the end to end LSP between PE1 and PE2 and vice versa.

To fix the issue, make sure the loopback interface IP address is configured as a /32 on both PEs.

PE1:

interface Loopback0
ip address 11.11.11.11 255.255.255.255
!

PE2:

interface Loopback0
ip address 22.22.22.22 255.255.255.255
!

Regards,

[Neil Stephens1]

Thanks so much @Harold Ritter, i was left scratching my head with this for ages - thats fixed the problem!

I'm curious as to how best you would tshoot such an issue to verify the label path is broken?

For anyone who does come across this problem, the below should have been my clue to check my routing / addressing.

PE1
*Mar 7 08:04:11.835: %BGP-4-VPNV4NH_MASK: Nexthop 11.11.11.11 may not be reachable from neigbor 1.1.1.1 - not /32 mask

[Harold Ritter]

You are very welcome @Neil Stephens1 . 

>I'm curious as to how best you would tshoot such an issue to verify the label path is broken?

If you check on the respective penultimate hop router (PHR) you should see that the label entry for the PE loopback interface addresses (11.11.11.11 or 22.22.22.22) in the "show mpls for" output shows as a "No label", rather than a "POP label". This is due to the fact that OSPF advertises the loopback interface address as a /32 and LDP advertise the label binding as a /24.

One tool that can be used to troubleshoot such and issue it to perform a "ping mpls ipv4" or "traceroute mpls ipv4" and check whether it is successful.

> *Mar 7 08:04:11.835: %BGP-4-VPNV4NH_MASK: Nexthop 11.11.11.11 may not be reachable from neigbor 1.1.1.1 -   > not /32 mask

This message should definitely not be ignored. Configuring the loopback interface ip address as a /32 is a best practice.

Regards,

Harold 

[filopeter]

Just one remark to the accepted solution. If you need to advertise the Loopback address with the configured/real subnet mask over OSPF, please change the ospf network type to point-to-point. This will override the default behaviour of advertising Loopback interfaces with mask /32.
!
interface Loopback0
ip address 22.22.22.22 255.255.255.0
ip ospf network point-to-point
!

[Harold Ritter]

Hi @filopeter ,

This is also an option, but I personally don't see any reason to advertise the loopback address as a /24 in the context of MPLS VPN, where normally all core devices use a /32 out of the same block.

Regards,