Re: MPLS L3VPN: Hosts not reachable to each other - Page 2

tonitones10 · ‎05-15-2023

Hello everyone,

This will be my first discussion here. Just like the title says, host1 of CE1 and CE2 cannot reach each other. I'm not well versed with MPLS, nor an expert with BGP, OSPF, and the likes. This will be my first time configuring MPLS L3VPN.

Further details are provided below.

This is the current topology:

P1(AS65001) ---- P2(AS65001)
    |                |
PE1(AS65001)       PE2(AS65001)
    |                |
CE1(AS65004)       CE2(AS65004)
    |                |
HostA(PC)           HostB(PC)

A little bit background of the topology: P1, P2, PE1, and PE2 are existing routers in production. BGP, OSPF, P2P links are also existing in production (just replicated in simulator) except for the MPLS configs and CE1 & CE2 routers. P1 is the RR of P2. P2 is the RR of PE2. PE1 is linked to P1 through iBGP. CE1 and CE2 are linked to their respective PE's through eBGP. What I'm trying to accomplish is to make HostA and HostB reachable to each other with the existing configs + the MPLS configs.

Please see the configs below for the routers included in the topology. Not all of the configs are pasted, only the configs that are relevant (I think) to the current problem.

CE1 and CE2 configs:

CE1

interface Loopback0
 description CE1-Lo
 ip address 1.1.1.1 255.255.255.255
!
interface Port-channel1.303
 description PE1 LINK
 encapsulation dot1Q 303
 ip address 100.65.0.10 255.255.255.248
!
router bgp 65004
 bgp router-id 1.1.1.1
 bgp log-neighbor-changes
 neighbor 100.65.0.9 remote-as 65001
 !
 address-family ipv4
  network 192.168.101.0
  neighbor 100.65.0.9 activate
 exit-address-family
!
ip route 192.168.101.0 255.255.255.0 Null0 200

CE2

interface Loopback0
 description CE2-Lo
 ip address 2.2.2.2 255.255.255.255
!
interface Port-channel1.301
 description PE2 LINK
 encapsulation dot1Q 301
 ip address 100.65.0.2 255.255.255.252
!
router bgp 65004
 bgp router-id 2.2.2.2
 bgp log-neighbor-changes
 neighbor 100.65.0.1 remote-as 65001
 neighbor 100.65.0.1 description PE2
 !
 address-family ipv4
  network 192.168.99.0
  neighbor 100.65.0.1 activate
 exit-address-family
!
ip route 192.168.99.0 255.255.255.0 Null0 200

PE1 and PE2 Configs:

PE1

vrf definition CustA_CE1
 rd 65004:10
 route-target export 65004:10
 route-target import 65004:20
 !
 address-family ipv4
 exit-address-family
!
!
mpls label range 500 599
no mpls ip propagate-ttl forwarded
!
interface Loopback0
 description PE1-Lo0
 ip address 172.16.35.0 255.255.255.255
!
interface Port-channel1.100
 description P1 iBGP LINK
 encapsulation dot1Q 100
 ip address 100.75.0.2 255.255.255.252
 mpls ip
 mpls label protocol ldp
!
router bgp 65001
 bgp router-id 172.16.35.0
 bgp log-neighbor-changes
 neighbor 100.75.0.1 remote-as 65001
 neighbor 100.75.0.1 description P1
 !
 address-family ipv4
  neighbor 100.75.0.1 activate
 exit-address-family
 !
 address-family vpnv4
  neighbor 100.75.0.1 activate
  neighbor 100.75.0.1 send-community both
 exit-address-family
 !
 address-family ipv4 vrf CustA_CE1
  neighbor 100.65.0.10 remote-as 65004
  neighbor 100.65.0.10 description CustA_CE1
  neighbor 100.65.0.10 activate
  neighbor 100.65.0.10 as-override
 exit-address-family
!
ip route 0.0.0.0 0.0.0.0 100.75.0.1
!
mpls ldp router-id Loopback0 force

PE2

vrf definition CustA_CE1
 rd 65004:10
 route-target export 65004:10
 route-target import 65004:20
 !
 address-family ipv4
 exit-address-family
!
mpls label range 100 199
no mpls ip propagate-ttl forwarded
!
interface Loopback0
 description PE2-Lo
 ip address 172.16.16.0 255.255.255.255
!
interface Port-channel1.110
 description P1 OSPF LINK
 encapsulation dot1Q 110
 ip address 100.75.0.10 255.255.255.252
 ip ospf network point-to-point
 ip ospf hello-interval 2
 ip ospf cost 1
 mpls ip
 mpls label protocol ldp
!
interface Port-channel1.301
 description CE2 LINK
 encapsulation dot1Q 301
 vrf forwarding CustA_CE2
 ip address 100.65.0.1 255.255.255.252
!
router ospf 100
 router-id 172.16.16.0
 passive-interface default
 no passive-interface Port-channel1.110
 network 100.75.0.8 0.0.0.3 area 0
 network 172.16.16.0 0.0.0.0 area 16
!
router bgp 65001
 bgp router-id 172.16.16.0
 bgp log-neighbor-changes
 no bgp default ipv4-unicast
 neighbor 172.16.0.0 remote-as 65001
 neighbor 172.16.0.0 description P2
 neighbor 172.16.0.0 update-source Loopback0
 !
 address-family ipv4
  neighbor 172.16.0.0 activate
 exit-address-family
 !
 address-family vpnv4
  neighbor 172.16.0.0 activate
  neighbor 172.16.0.0 send-community both
 exit-address-family
 !
 address-family ipv4 vrf CustA_CE2
  neighbor 100.65.0.2 remote-as 65004
  neighbor 100.65.0.2 activate
  neighbor 100.65.0.2 as-override
 exit-address-family
 !

P1 and P2 Configs:

P1

mpls label range 400 499
!
interface Loopback0
 description P1-Lo0
 ip address 172.16.0.249 255.255.255.255
!
interface Port-channel1.100
 description PE1 iBGP LINK
 encapsulation dot1Q 100
 ip address 100.75.0.1 255.255.255.252
 mpls ip
 mpls label protocol ldp
!
interface Port-channel1.200
 description P2 OSPF LINK
 encapsulation dot1Q 200
 ip address 100.75.0.5 255.255.255.252
 ip ospf network point-to-point
 ip ospf hello-interval 2
 ip ospf cost 1
 mpls ip
 mpls label protocol ldp
!
router ospf 100
 router-id 172.16.0.249
 passive-interface default
 no passive-interface Port-channel1.200
 network 100.75.0.4 0.0.0.3 area 0
 network 172.16.0.249 0.0.0.0 area 0
!
router bgp 65001
 bgp router-id 172.16.0.249
 bgp log-neighbor-changes
 neighbor 100.75.0.2 remote-as 65001
 neighbor 100.75.0.2 description PE1
 neighbor 172.16.0.0 remote-as 65001
 neighbor 172.16.0.0 description P2
 neighbor 172.16.0.0 update-source Loopback0
 !
 address-family ipv4
  network 100.75.0.0 mask 255.255.255.252
  neighbor 100.75.0.2 activate
  neighbor 100.75.0.2 next-hop-self
  neighbor 172.16.0.0 activate
  neighbor 172.16.0.0 route-reflector-client
  neighbor 172.16.0.0 next-hop-self
 exit-address-family
 !
 address-family vpnv4
  neighbor 100.75.0.2 activate
  neighbor 100.75.0.2 send-community both
  neighbor 172.16.0.0 activate
  neighbor 172.16.0.0 send-community both
  neighbor 172.16.0.0 route-reflector-client
 exit-address-family
!

P2

mpls label range 100 199
no mpls ip propagate-ttl forwarded
!
interface Loopback0
 description P2-Lo0
 ip address 172.16.0.0 255.255.255.255
!
interface Port-channel1.110
 description PE2 LINK OSPF
 encapsulation dot1Q 110
 ip address 100.75.0.9 255.255.255.252
 ip ospf network point-to-point
 ip ospf hello-interval 2
 ip ospf cost 1
 mpls ip
 mpls label protocol ldp
!
interface Port-channel1.200
 description P1 LINK OSPF
 encapsulation dot1Q 200
 ip address 100.75.0.6 255.255.255.252
 ip ospf network point-to-point
 ip ospf hello-interval 2
 ip ospf cost 1
 mpls ip
 mpls label protocol ldp
!
router ospf 100
 router-id 172.16.0.0
 passive-interface default
 no passive-interface Port-channel1.110
 no passive-interface Port-channel1.200
 network 100.75.0.4 0.0.0.3 area 0
 network 100.75.0.8 0.0.0.3 area 0
 network 172.16.0.0 0.0.0.0 area 0
!
router bgp 65001
 bgp router-id 172.16.0.0
 bgp log-neighbor-changes
 neighbor 172.16.0.249 remote-as 65001
 neighbor 172.16.0.249 description P1
 neighbor 172.16.0.249 update-source Loopback0
 neighbor 172.16.16.0 remote-as 65001
 neighbor 172.16.16.0 description PE2
 neighbor 172.16.16.0 update-source Loopback0
 !
 address-family ipv4
  neighbor 172.16.0.249 activate
  neighbor 172.16.16.0 activate
  neighbor 172.16.16.0 route-reflector-client
 exit-address-family
 !
 address-family vpnv4
  neighbor 172.16.0.249 activate
  neighbor 172.16.0.249 send-community both
  neighbor 172.16.16.0 activate
  neighbor 172.16.16.0 send-community both
  neighbor 172.16.16.0 route-reflector-client
 exit-address-family
!

Additional details that might help:

PE1

PE1# show bgp vpnv4 unicast all
BGP table version is 14, local router ID is 172.16.35.0
Status codes: s suppressed, d damped, h history, * valid, > best, i - internal,
              r RIB-failure, S Stale, m multipath, b backup-path, f RT-Filter,
              x best-external, a additional-path, c RIB-compressed,
              t secondary path,
Origin codes: i - IGP, e - EGP, ? - incomplete
RPKI validation codes: V valid, I invalid, N Not found

     Network          Next Hop            Metric LocPrf Weight Path
Route Distinguisher: 65004:10
 *>i  192.168.99.0     172.16.16.0              0    100      0 65004 i
Route Distinguisher: 65004:30 (default for vrf CustA_CE1)
 *>i  192.168.99.0     172.16.16.0              0    100      0 65004 i
 *>   192.168.101.0    100.65.0.10              0             0 65004 i

PE2

PE2# show bgp vpnv4 unicast all
BGP table version is 23, local router ID is 172.16.16.0
Status codes: s suppressed, d damped, h history, * valid, > best, i - internal,
              r RIB-failure, S Stale, m multipath, b backup-path, f RT-Filter,
              x best-external, a additional-path, c RIB-compressed,
              t secondary path,
Origin codes: i - IGP, e - EGP, ? - incomplete
RPKI validation codes: V valid, I invalid, N Not found

     Network          Next Hop            Metric LocPrf Weight Path
Route Distinguisher: 65004:10 (default for vrf CustA_CE2)
 *>   192.168.99.0     100.65.0.2               0             0 65004 i
 *>i  192.168.101.0    100.75.0.2               0    100      0 65004 i
Route Distinguisher: 65004:20
 *>i  192.168.101.0    100.75.0.2               0    100      0 65004 i

I can see the subnets for each other but the hosts cannot reach each other. I may be missing something here, hoping that you guys can help.

-tonitones

David Ruess · ‎05-15-2023

After looking at this output I believe @MHM Cisco World had a good point. PE1 should have labels for networks all the way through to PE2 as PE2 does. Can you provide the output of show mpls ldp neighbor? I suspect PE1 does not have an LDP neighbor. If it does then its not learning labels from its neighbor. Can you check this please.

tonitones10 · ‎05-15-2023

As you guys suspected, no output on PE1 when doing show mpls ldp neighbor. For PE2, there is one neighbor, only P2.

PE1 : 
PE1#sh mpls ldp neighbor
PE1#


PE2 :
PE2#sh mpls ldp neighbor
    Peer LDP Ident: 172.16.0.0:0; Local LDP Ident 172.16.16.0:0
        TCP connection: 172.16.0.0.646 - 172.16.16.0.25791
        State: Oper; Msgs sent/rcvd: 1799/1806; Downstream
        Up time: 1d02h
        LDP discovery sources:
          Port-channel1.110, Src IP addr: 100.75.0.9
        Addresses bound to peer LDP Ident:
          172.16.0.0      100.75.0.9      100.75.0.6

I forgot to include that, routers PE1, P1, P2, and PE2 have this configured:

mpls ldp label
 allocate global prefix-list ALL-LOOPBACK
!
ip prefix-list ALL-LOOPBACK seq 15 permit 172.16.0.0/16 le 32
!
mpls ldp router-id Loopback0 force

I'm not sure, but I think this is what's causing the issue.

-tonitones

David Ruess · ‎05-15-2023

Correct. The MPLS label domain must include the entire path from PE to PE. MPLS L3VPN needs both labels (MPLS and VPNv4) in the stack to work. When one of those label paths is broken then MPLS L3VPN will not pass traffic. Routes will be delivered as you saw but connectivity fails.

Try getting rid of those extra commands to see if connectivity is established and you get an LDP neighbor on PE1.

-David

tonitones10 · ‎05-15-2023

It seems that still no connectivity is established, even after removing the extra commands. P1 cannot still see PE1 on ldp neighbor.

-tonitones

Harold Ritter · ‎05-15-2023

Hi @tonitones10 ,

I checked the PE1 and P1 configuration and see a few strange things.

PE1:

no ospf configured. PE1 needs to be part of the IGP domain. This is a requirement for LDP to work end to end between PE1 and PE2.

P1:

I see the following ospf configuration:

router ospf 100
 router-id 172.16.0.249
 passive-interface default
 no passive-interface Port-channel1.200
 network 100.75.0.4 0.0.0.3 area 0
 network 172.16.0.249 0.0.0.0 area 0
!

There is no configuration for Port-channel1.100 (interface to PE1).

This is why you do not have an LDP session between PE1 and P1 and that you can't reach CE2 from CE1 and vice versa.

Bottom line, you need to configure ospf between PE1 and P1, just like it is configured between PE2 and P2. This should fix it.

Regards,

Regards,
Harold Ritter, CCIE #4168 (EI, SP)

tonitones10 · ‎05-15-2023

Hi @Harold Ritter ,

Thanks for pointing out the OSPF config on PE1 and P1.

PE1 router never had OSPF config on production. I also suspected that CE1 wont be able to reach CE2 because of that. So I tried putting it on simulation without making any revisions on the current configs and adding mpls onto the current setup. Any changes to the present configs are subject for approval if I want to put it into production. I hope that explains why no ospf is configured on PE1.

The question is, will CE1 ever reach CE2 without having OSPF between PE1 and P1? Even if I had other PE's under P2, I know it wont have issues because PE's linked to P2 are running OSPF. In the case of PE1 and P1 is different, they're not running OSPF.

-tonitones

Harold Ritter · ‎05-16-2023

Hi @tonitones10 ,

> PE1 router never had OSPF config on production

That might have been fine before, but it looks like you are now deploying the first VRF and it will not work.

> The question is, will CE1 ever reach CE2 without having OSPF between PE1 and P1?

You could probably get it to work using bgp ipv4 +label, but it would be more complex than just adding ospf on PE1.

My question for you would be, why would this be an issue to configure ospf on PE1?

Regards,

Regards,
Harold Ritter, CCIE #4168 (EI, SP)

MHM Cisco World · ‎05-16-2023

as I mention before there are no label in PE1

check
1- CEF is enable
2- the LDP router-ID is reachable

ALSO
please share the RIB of PE1 (global)

Harold Ritter · ‎05-16-2023

Hi @MHM Cisco World ,

The reason there is no IGP label on PE1 is that the LDP session is down and the LDP session is down because there is no IGP between PE1 and P1. Please check the provided configurations at the beginning go this post.

Regards,

Regards,
Harold Ritter, CCIE #4168 (EI, SP)

MHM Cisco World · ‎05-16-2023

Thanks

That why I want to be sure and ask him show me the RIB of PE1.

David Ruess · ‎05-15-2023

Another thing is your import/export. They should be opposite of eachother. So if you export a number from one VRF on one PE you need to IMPORT that number on the other VRF on the other PE.

PE1

vrf definition CustA_CE1
 rd 65004:10
 route-target export 65004:10
 route-target import 65004:20

PE2

vrf definition CustA_CE1
 rd 65004:10
 route-target export 65004:20
 route-target import 65004:10

Hope that helps

-David

tonitones10 · ‎05-15-2023

Hello @David Ruess ,

Apologies for the confusion, what you pointed out is actually the VRF configs of PE1 and PE2. The one on top that I originally posted is a copy-paste error on my end.

This time, I followed what M02@rt37 suggested to have the same VRF names and import/export id for an effective troubleshooting, but to no avail, still the same outputs.

Thank you.

-tonitones

David Ruess · ‎05-15-2023

For some reason my original post was deleted.

It looks like you're using the same AS for both sites. The fundamental loop prevention mechanism in BGP is not to accept routes from its own AS. So on your neighbor statements you would either need to use the ALLOWAS-IN command on CE router or AS-OVERRIDE on PE router towards CE. That along with my other statement of import/export may fix your issue.

-David

tonitones10 · ‎05-15-2023

Hello @David Ruess ,

AS-OVERRIDE is already included on PE routers facing CE's. As for ALLOWAS-IN, I have yet to add on neighbor statements on CE's. I think it shouldn't have any issues without it since I have as-override on PE's.

-tonitones

David Ruess · ‎05-15-2023

Apologies. I looked through and swear I didn't see it. You only need one or the other, not both. Have you checked the CE routers routing table to see if the networks make it from the PE routers? sh ip route. If its only making it to the PE routers and not CE then can you show theire is a BGP relatiponship between the CE/PE routers?