Re: Ask the Expert- MPLS Best Practices- Configuration & Troubleshooting - Page 2

Cisco Moderador · ‎10-12-2018

This topic is a chance to discuss more about the Multi Protocol Label Switching (MPLS) technology and its typical applications. This session will cover all questions related to MPLS fundamentals including its supporting protocols (LDP, BGP), and its most common applications, such as BGP-less core networks, 6PE/6VPE migration mechanisms, L2 and L3 VPNs, both in intra-AS and inter-AS scope.

To participate in this event, please use thebutton below to ask your questions

Ask questions from Monday October 15th to Friday November 2nd, 2018

Featured expert

David Samuel Peñaloza Seijas works as a Senior Network Consulting Engineer at Verizon Enterprise Solutions in the Czech Republic. Previously, he worked as a Network Support Specialist in the IBM Client Innovation Center in the Czech Republic. David is an expert interested in all topics related to networks. However, he focuses mainly on data centers, enterprise networks, and network design, including software-defined networking (SDN). David has a long relationship with Cisco. He has been a Cisco Instructor for the Cisco Academy and was recognized as a Cisco Champion and a Cisco Designated VIP for 2017 and 2018. David holds a CCNP R&S, CCDP, CCNA Security, CCNA CyberOps and a CCNA SP certification. Currently, he is preparing for a CCDE.

David might not be able to answer each question due to the volume expected during this event. Remember that you can continue the conversation on the Service Providers community.

Find other events https://community.cisco.com/t5/custom/page/page-id/Events?categoryId=technology-support

**Helpful votes Encourage Participation! **
Please be sure to rate the Answers to Questions

tuda09081990 · ‎10-23-2018

Hi Sir,

Our network is running unified mpls as cisco standard and we want to deploy mpsl-te in access rings (Separate ospf process with core and agg domain). Can you advise best practice to reach switching < 50ms for ring topology. I'm trying with mpls-te fast-reroute use protect link but not really clear what we should do for full protect our service as L2VPN, L3VPN

Thanks so much

David Samuel Penaloza Seijas · ‎10-28-2018

Hello

For TE, you have to be careful as the tunnels are point to point and unidirectional. If tunnels are from PE to PE, it should be ok, but, if they are:

- P router to P router

- P router to PE

- PE router to P router

You need to be careful with the labels (transport) being popped up one hop before their actual destination. That would blackhole the traffic as the router receiving the packet will only see the VPN label, not the transport label, and it would not be able to forward the traffic (lacking transport label) as the only router understanding the VPN label is the destination PE. The other midpoints would only switch traffic based on topmost label (being transport label or TE label assigned by RSVP).

In an Unified MPLS deployment, the ASBRs would be the next hop for any LSP going through the IGP processes, consider them among your options for LPR (Local Point of Repair) in the case of TE tunnels.

In addition to that, in the case of a ring, i would be thinking on node protection (MPLS FRR) and make sure there is a backup LSP going in the other direction (clockwise/counter clockwise). On top of that, enhance fault detection with BFD so the LSP can be changed quickly.

Unfortunately, this session doesn't cover TE mechanisms for MPLS. I would recommend you to open a thread in the following community: https://community.cisco.com/t5/service-providers/ct-p/4441-service-providers

Thanks

Rami Ibrahim · ‎10-24-2018

Hi David,

First of all, Thank you for this session.

Actually I would like to know if there is a scalable way to encrypt L3VPN customers traffic inside the MPLS network, in other words to offload this to the provider side as opposed to the customer side.

Thank you.

Romio

David Samuel Penaloza Seijas · ‎10-24-2018

Hello Romio

I would add some questions to clarify your requirements:

- Is this only for MPLS?

- Are there any sites with Internet circuits?

- What are your traffic patterns? (I.e. Spoke to spoke, hub and Spoke, etc.).

- What is the scale of the network?

IPSEC and GETVPN could be among your options, but it depends on which are your specific requirements.

tuda09081990 · ‎10-25-2018

Hi Sir,

Our network is running unified mpls as cisco standard and we want to deploy mpsl-te in access rings (Separate ospf process with core and agg domain). Can you advise best practice to reach switching < 50ms for ring topology. I'm trying with mpls-te fast-reroute use protect link but not really clear what we should do for full protect our service as L2VPN, L3VPN

Thanks so much

Adam Vitkovsky · ‎11-02-2018

Hi

There’s no such a thing as full protect L2/L3VPN,

You can protect for core link/node failures –I recommend you configure node-link protection on all your links using many to one protection (Bypass LSPs).

You can protect for edge link/node failures. –for PCE-CE link failure protection I recommend you enable BGP PIC edge (prerequisites are: advertise best external + per-next-hop VRF labels) - For edge node (PE) failure protection I recommend enabling BGP PIC Core (cisco term).

Now with regards to the 50ms convergence,

Once the above backup options are enabled the backup path is pre-programmed in the HW so the switchover will be instant –however the bulk of the failover time is in the failure detection.

In best case scenario you can rely on L1 loss of light to detect the failure, but it’s a good practice to back that up with BFD on all links you’d want to achieve a sub 50ms convergence (you can set the BFD times to 15ms/45ms)

adam

RoKu · ‎10-27-2018

Hi, I've been trying to solve this problem for a quite long time and I gave up. I need some help ;)

I have two devices on my desk: C6500, sup720, pfc3c, msfc3 and C7600, rsp720, pfc3cxl, msfc4. Both are connected with a wire. Both ports are switchports with allowed vlan 2326. MPLS enabled. ping works

C6500 configuration:

mpls label protocol ldp
mpls traffic-eng tunnels


interface GigabitEthernet6/2
 description to_C7600
 switchport
 switchport trunk encapsulation dot1q
 switchport trunk native vlan 3425
 switchport trunk allowed vlan 2326,3425
 switchport mode trunk
 mtu 9216
 ip arp inspection trust
 load-interval 30
 no cdp enable
 spanning-tree vlan 3425 cost 10000
 ip dhcp snooping trust

interface Vlan2326
 description to_C7600_vlan
 ip address 10.4.253.15 255.255.255.254
 ip ospf mtu-ignore
 ip ospf 65405 area 0
 mpls traffic-eng tunnels
 mpls bgp forwarding
 mpls ip
 ip rsvp bandwidth

router ospf 65405
 router-id 10.4.100.12
 network 10.4.100.12 0.0.0.0 area 0
 mpls ldp sync
 mpls traffic-eng router-id Loopback69
 mpls traffic-eng area 0
 mpls traffic-eng interface Vlan2327 area 0
 mpls traffic-eng interface Vlan2326 area 0

interface Loopback69
 ip address 10.4.100.12 255.255.255.255


#ping 10.4.8.8 source 10.4.100.12
Type escape sequence to abort.
Sending 5, 100-byte ICMP Echos to 10.4.8.8, timeout is 2 seconds:
Packet sent with a source address of 10.4.100.12
!!!!!
Success rate is 100 percent (5/5), round-trip min/avg/max = 1/1/4 ms


#sh mpls interfaces
Interface              IP            Tunnel   BGP Static Operational
Vlan2326               Yes (ldp)     Yes      Yes No     Yes
Tunnel102              No            No       No  No     No


interface Tunnel102
 description tunnel_to_C7600_10_4_8_8
 ip unnumbered Loopback69
 tunnel source Loopback69
 tunnel mode mpls traffic-eng
 tunnel destination 10.4.8.8
 tunnel mpls traffic-eng autoroute announce
 tunnel mpls traffic-eng path-option 10 dynamic
 no routing dynamic
 ip rsvp bandwidth 8000


#sh mpls traffic-eng tunnels tunnel 102

Name: tunnel_to_C7600_10_4_8_8            (Tunnel102) Destination: 10.4.8.8
  Status:
    Admin: up         Oper: down   Path: not valid   Signalling: Down
    path option 10, type dynamic

  Config Parameters:
    Bandwidth: 0        kbps (Global)  Priority: 7  7   Affinity: 0x0/0xFFFF
    Metric Type: TE (default)
    AutoRoute announce: enabled  LockDown: disabled Loadshare: 0 [0] bw-based
    auto-bw: disabled

  Shortest Unconstrained Path Info:
    Path Weight: 1 (TE)
    Explicit Route: 10.4.253.15 10.4.253.14 10.4.8.8
  History:
    Tunnel:
      Time since created: 1 hours, 12 minutes
      Time since path change: 7 seconds
      Number of LSP IDs (Tun_Instances) used: 441
    Prior LSP: [ID: 441]
      ID: path option 10 [441]
      Removal Trigger: path error
      Last Error: RSVP:: Path Error from 10.4.253.14: Routing Problem (code 24, value 4, flags 0)

C7600 configuration:

mpls label protocol ldp
mpls traffic-eng tunnels


#sh ip vrf
  Name                             Default RD            Interfaces
  MPLS_back                        65203:1               Lo70
                                                         Vl2326
                                                         Tu102



interface GigabitEthernet1/1
 description to_C6500
 switchport
 switchport trunk encapsulation dot1q
 switchport trunk allowed vlan 2326
 switchport mode trunk
 mtu 9216
 no cdp enable

interface Vlan2326
 description to_C6500
 ip vrf forwarding MPLS_back
 ip address 10.4.253.14 255.255.255.254
 ip ospf mtu-ignore
 ip ospf 65203 area 0
 mpls ip
 mpls traffic-eng tunnels
 mpls traffic-eng administrative-weight 100
 mpls bgp forwarding
 ip rsvp bandwidth

router ospf 65203 vrf MPLS_back
 router-id 10.4.8.8
 network 10.4.8.8 0.0.0.0 area 0
 network 10.4.100.222 0.0.0.0 area 0
 mpls traffic-eng router-id Loopback70
 mpls traffic-eng area 0
 mpls traffic-eng interface Vlan2326 area 0
!

interface Loopback70
 ip vrf forwarding MPLS_back
 ip address 10.4.8.8 255.255.255.255


#ping vrf MPLS_back 10.4.100.12 source 10.4.8.8
Type escape sequence to abort.
Sending 5, 100-byte ICMP Echos to 10.4.100.12, timeout is 2 seconds:
Packet sent with a source address of 10.4.8.8
!!!!!
Success rate is 100 percent (5/5), round-trip min/avg/max = 1/1/4 ms


#sh mpls interfaces vrf MPLS_back
Interface              IP            Tunnel   BGP Static Operational

VRF MPLS_back:
Tunnel100              No            No       No  No     No
Vlan2326               Yes (ldp)     Yes      Yes No     Yes
Tunnel102              No            No       No  No     No


interface Tunnel102
 description tunnel_to_C6500_10_4_100_12
 ip vrf forwarding MPLS_back
 ip unnumbered Loopback70
 tunnel mode mpls traffic-eng
 tunnel destination 10.4.100.12
 tunnel mpls traffic-eng priority 7 7
 tunnel mpls traffic-eng bandwidth 8000
 tunnel mpls traffic-eng path-option 100 dynamic


sh mpls traffic-eng tunnels tunnel 102

Name: tunnel_to_C6500_10_4_100_12         (Tunnel102) Destination: 10.4.100.12
  Status:
    Admin: up         Oper: down   Path: not valid   Signalling: Down
    path option 100, type dynamic

  Config Parameters:
    Bandwidth: 8000     kbps (Global)  Priority: 7  7   Affinity: 0x0/0xFFFF
    Metric Type: TE (default)
    AutoRoute: disabled LockDown: disabled Loadshare: 8000 [0] bw-based
    auto-bw: disabled

  Shortest Unconstrained Path Info:
    Path Weight: 100 (TE)
    Explicit Route: 10.4.253.14 10.4.253.15 10.4.100.12
  History:
    Tunnel:
      Time since created: 1 hours, 29 minutes
      Number of LSP IDs (Tun_Instances) used: 544

Could you please tell me what I did wrong ;) ?

David Samuel Penaloza Seijas · ‎10-28-2018

Hello

If the tunnel is down due to signaling not happening, this usually means that the requested options could not be satisfied and a path couldn't be found to meet those requirements.

Last Error: RSVP:: Path Error from 10.4.253.14: Routing Problem (code 24, value 4, flags 0)

According to RFC 3209 ( RSVP-TE: Extensions to RSVP for LSP Tunnels):

7.3. Error Codes and Globally-Defined Error Value Sub-Codes

   The following list extends the basic list of Error Codes and Values
   that are defined in [RFC2205].

   Error Code    Meaning

     24          Routing Problem

                 This Error Code has the following globally-defined
                 Error Value sub-codes:

                  1       Bad EXPLICIT_ROUTE object
                  2       Bad strict node
                  3       Bad loose node
                  4       Bad initial subobject
                  5       No route available toward
                           destination
                  6       Unacceptable label value
                  7       RRO indicated routing loops
                  8       MPLS being negotiated, but a
                          non-RSVP-capable router stands
                            in the path
                  9       MPLS label allocation failure
                 10       Unsupported L3PID

There is a problem in your configuration, the router cannot even signal to find a path.

- There is a explicit path in the option but you config states dynamic.

interface Tunnel102
 description tunnel_to_C7600_10_4_8_8
 ip unnumbered Loopback69
 tunnel source Loopback69
 tunnel mode mpls traffic-eng
 tunnel destination 10.4.8.8
 tunnel mpls traffic-eng autoroute announce
 tunnel mpls traffic-eng path-option 10 dynamic
 no routing dynamic
 ip rsvp bandwidth 8000

 Shortest Unconstrained Path Info:
    Path Weight: 1 (TE)
    Explicit Route: 10.4.253.15 10.4.253.14 10.4.8.8

I could be mistaken in this statement. Therefore, please take this with a grain of salt ;)

Unfortunately, this session doesn't cover TE mechanisms for MPLS. I would recommend you to open a thread in the following community: https://community.cisco.com/t5/service-providers/ct-p/4441-service-providers

Thanks

Ugur Ersoy · ‎10-29-2018

EDIT: Nevermind, I didn't notice that your ospf process is also in vrf.

interface Loopback70
 ip vrf forwarding MPLS_back
 ip address 10.4.8.8 255.255.255.255

Is loopback address in vrf just for testing?

Ugur Ersoy · ‎10-29-2018

Hello David,

I was wondering on which conditions we can use same bgp router id between two peers. Normally, as far as I know, we are not allowed to use same router id neither in ebgp nor ibgp. As long as we do, we should receive a notification as follows: BGP identifier wrong

However, we've got some hairpins using in our network to make smooth transit between different VRFs. So actually, the box peers itself. Within these peerings the router ids are not configured explicitly under VRFs, so peers sharing same router id without any problem.

David Samuel Penaloza Seijas · ‎11-01-2018

Hello Ugur

I all honesty, i would like to think about a reason to peer with yourself, but i cannot come up with something. Why not leaking routes between VRFs instead?

This would even break Best Path Selection algorithm in BGP , the next-to-last step will be stabbed in the back, and any field where the router puts its own RID will be at risk of a conflict - E.g. AGGREGATOR, ORIGINATOR_ID, CLUSTER_LIST (initialized from the RID of the RR).

Not judging the setup, but instead pondering about the reasons behind it.

Hope to hear back from you!

Thanks

Ugur Ersoy · ‎11-01-2018

hello David,

Yes I know, it's kinda weird to implement such design and not safe also :) We are also trying to simplify this design by using vrf leaks.

Mostly this solution is implemented where devices working inline like dpi and nat exist. So, the router is connected to dpi/nat device with 2 interfaces, in this way, it is possible to get the traffic travel through nat/dpi devices and change vrf the traffic goes via the BGP peered over this link.

g-khaustov · ‎11-02-2018

Hello Team.

I have a design question.

We have networks from 19 hops ( 2 routers in each hop) connected as daisy chain along huge distance 1500 км. (Device are Cisco 7600/ASR9k)

Currently use BGP full mesh - without RR. /MPLS L3 VPN. EoMPLS.VPLS - is working services /

We need installed seperate RR and make migration from BGP full mesh configuration to BGP - RR configuration without interrrated services.

are there any recomendations or best method to approach this migration without interrupted services?

Thanks advance

Gennady

Adam Vitkovsky · ‎11-02-2018

Actually migration from full mesh to RRs is straightforward and seamless.

The best approach is:

Stage1) build a new RR infrastructure on top of your existing full-mesh and let it soak in (make sure it’s stable)

(routes received via RRs should not be preferred)

Stage2) once all PEs are connected to both full-mesh and new RRs make sure PEs are receiving all routes via full-mesh as well as via your RRs, also make sure each route has same paths as it used to over full-mesh, and if not try to understand what it will mean in terms of traffic routing and data flows in your network.

Stage3) once you are satisfied with the state of routing information each PE receives over the RR sessions then you can start disabling the full-mesh sessions on every PE at your convenience.

Note1:

What you need to verify upfront is whether all your PEs have enough memory to hold the extra routes (that will be received via RRs)

Note2:

When migrating from full-mesh to RRs –depending on how you’ll design your RR infrastructure (single plane, address-based-route-reflection, add-path, diverse-path, Type-1 RDs) when switching to RRs you may receive just a subset of the paths you used to receive for each prefix over the full mesh –this is because each RR as BGP speakers will advertise only what it considers as the best path (from its perspective) to its clients by default. So after you disable full-mesh BGP sessions and leave PE with just RRs sessions during final stage you may perceive a change in traffic flows –i.e. is the number or preference of the paths changed by switching to the RR infrastructure.

adam

David Samuel Penaloza Seijas · ‎11-02-2018

Thats excellent!

Thanks for passing by and sharing your thoughts! This was unexpected :D