02-18-2015 03:52 PM - edited 03-05-2019 12:50 AM
Hi,
I came up with a good article from Cisco for troubleshooting MPLS, but I couldn't access due to permissions. I need step by step instructions to troubleshoot, where to look, what to look for, and how to connect, this includes VPN with MPLS.
Please provide documentation to follow.
Regards,
02-18-2015 04:09 PM
http://www.cisco.com/c/en/us/support/docs/multiprotocol-label-switching-mpls/mpls/12492-mpls-tsh.html
http://www.cisco.com/c/en/us/support/docs/multiprotocol-label-switching-mpls/mpls/13734-mpls-vpn-tsh.html
This document describes how to troubleshoot Multiprotocol Label Switching (MPLS).
Readers of this document should have knowledge of this topic:
MPLS basics
This document is based on the Configuring Basic MPLS Using OSPF sample configuration and presumes that you have configured these elements:
IP address and a routing protocol such as Open Shortest Path First Protocol (OSPF Protocol) or Intermediate System-to-Intermediate System Protocol (IS-IS Protocol)
Cisco Express Forwarding (CEF) or distributed CEF switching on all routers
General MPLS or tag switching on all routers
MPLS or tag switching on all required interfaces
If you have doubts about which hardware or Cisco IOS® Software releases support MPLS, refer to the Software Advisor.
The information in this document was created from the devices in a specific lab environment. All of the devices used in this document started with a cleared (default) configuration. If your network is live, make sure that you understand the potential impact of any command.
Refer to Cisco Technical Tips Conventions for more information on document conventions.
This section contains several MPLS troubleshoot procedures.
Issue the show ip protocols command in order to display the parameters and current state of the active routing protocol process:
Pomerol# show ip protocols Routing Protocol is "ospf 1" Outgoing update filter list for all interfaces is not set Incoming update filter list for all interfaces is not set Router ID 10.10.10.3 Number of areas in this router is 1. 1 normal 0 stub 0 nssa Maximum path: 4 Routing for Networks: 10.1.1.0 0.0.0.255 area 9 10.10.10.0 0.0.0.255 area 9 Routing Information Sources: Gateway Distance Last Update 10.10.10.2 110 10:41:55 10.10.10.3 110 10:41:55 10.10.10.1 110 10:41:55 10.10.10.6 110 10:41:55 10.10.10.4 110 10:41:55 10.10.10.5 110 10:41:55 Distance: (default is 110)
Ensure that the protocol routes for the MPLS network and all neighbors are present. You can also issue the show ip route command in order to verify the routing table:
Pomerol# show ip route Codes: C - connected, S - static, I - IGRP, R - RIP, M - mobile, B - BGP D - EIGRP, EX - EIGRP external, O - OSPF, IA - OSPF inter area N1 - OSPF NSSA external type 1, N2 - OSPF NSSA external type 2 E1 - OSPF external type 1, E2 - OSPF external type 2, E - EGP i - ISIS, L1 - ISIS level-1, L2 - ISIS level-2, ia - ISIS inter area * - candidate default, U - per-user static route, o - ODR Gateway of last resort is 10.200.28.1 to network 0.0.0.0 10.0.0.0/8 is variably subnetted, 13 subnets, 3 masks C 10.1.1.8/30 is directly connected, Serial0/1.2 O 10.1.1.12/30 [110/390] via 10.1.1.5, 15:26:38, Serial0/1.1 O 10.10.10.2/32 [110/196] via 10.1.1.10, 15:26:38, Serial0/1.2 C 10.10.10.3/32 is directly connected, Loopback0 O 10.1.1.0/30 [110/390] via 10.1.1.5, 15:26:38, Serial0/1.1 [110/390] via 10.1.1.10, 15:26:38, Serial0/1.2 O 10.10.10.1/32 [110/196] via 10.1.1.5, 15:26:38, Serial0/1.1 O 10.10.10.6/32 [110/98] via 10.1.1.22, 15:26:38, Serial0/1.3 O 10.10.10.4/32 [110/391] via 10.1.1.5, 15:26:38, Serial0/1.1 C 10.1.1.4/30 is directly connected, Serial0/1.1 C 10.1.1.20/30 is directly connected, Serial0/1.3
If the routers or routes are not present, investigate the routing protocol process. Refer to the OSPF Support Page in order to investigate the routing protocol process.
Issue the show ip cef summary command in order to display specific entries in the Forwarding Information Base (FIB) with IP address information as a basis. This output shows Normal status:
Pomerol# show ip cef summary IP CEF with switching (Table Version 131), flags=0x0, bits=8 32 routes, 0 reresolve, 0 unresolved (0 old, 0 new) 32 leaves, 18 nodes, 23004 bytes, 125 inserts, 93 invalidations 1 load sharing elements, 336 bytes, 1 references universal per-destination load sharing algorithm, id B642EBCF 1 CEF resets, 6 revisions of existing leaves 6 in-place modifications refcounts: 4909 leaf, 4864 node
Issue the show ip cef and show ip cef interface commands in order to verify CEF status. If CEF has not been enabled, nothing appears:
Pomerol# show ip cef %CEF not running Prefix Next Hop Interface
Refer to the Cisco Express Forwarding Overview if you continue to have problems with the enablement of CEF.
Issue the show mpls interfaces command in order to ensure that MPLS is globally enabled. This command also verifies that a Label Distribution Protocol (LDP) runs on the requested interfaces:
Pomerol# show mpls interfaces Interface IP Tunnel Operational (...) Serial0/1.1 Yes (tdp) Yes Yes Serial0/1.2 Yes Yes No Serial0/1.3 Yes (tdp) Yes Yes (...)
show mpls interfaces Command Output Field Descriptions | |
---|---|
Field | Description |
IP | This field shows that MPLS IP is configured for an interface. The LDP appears in parentheses to the right of the IP status. The LDP is either:
|
Tunnel | This field indicates the capacity of traffic engineering on the interface. |
Operational | This field shows the status of the LDP. Note: In the example output, the Operational field is down on Serial0/1.2 because the interface is down. |
An unlabeled connection must be up between each pair of router neighbors. The routing protocol and the LDP use the unlabeled connection to build the routing table and the label forwarding information base (LFIB).
Pomerol# ping 10.10.10.6 Type escape sequence to abort. Sending 5, 100-byte ICMP Echos to 10.10.10.6, timeout is 2 seconds: !!!!! Success rate is 100 percent (5/5), round-trip min/avg/max = 56/56/60 ms
Issue the show tag-switching tdp discovery command in order to display the discovered neighbors:
Pomerol# show tag-switching tdp discovery Local TDP Identifier: 10.10.10.3:0 Discovery Sources: Interfaces: Serial0/1.1 (tdp): xmit/recv TDP Id: 10.10.10.1:0 Serial0/1.2 (tdp): xmit/recv TDP Id: 10.10.10.2:0 Serial0/1.3 (tdp): xmit/recv TDP Id: 10.10.10.6:0
In the show tag-switching tdp discovery command output, the use of TDP binds labels with routes. If any of the presumed neighbors is not present and you cannot ping the presumed neighbor, a connectivity problem exists and the LDP cannot run. If LDP runs correctly, it assigns one label per forwarding equivalent class.
Note: If the router ID for the LDP cannot be reached from the global routing table, the neighbor relationship fails to establish.
Issue the show tag-switching tdp bindings command in order to ensure the assignment of labels to each destination. You can use commands such as the show tag-switching forwarding-table {ip address | prefix} detail command in order to verify the different routes and the labels associated with the routes.
The output that this section shows contains label bindings for 10.10.10.x/32 networks, which are the interfaces of each label switch router (LSR):
Note: There are multiple labels for each LSR. Each label corresponds to a different path.
Pomerol# show tag-switching tdp bindings (...) tib entry: 10.10.10.1/32, rev 31 local binding: tag: 18 remote binding: tsr: 10.10.10.1:0, tag: imp-null remote binding: tsr: 10.10.10.2:0, tag: 18 remote binding: tsr: 10.10.10.6:0, tag: 21 tib entry: 10.10.10.2/32, rev 22 local binding: tag: 17 remote binding: tsr: 10.10.10.2:0, tag: imp-null remote binding: tsr: 10.10.10.1:0, tag: 19 remote binding: tsr: 10.10.10.6:0, tag: 22 tib entry: 10.10.10.3/32, rev 2 local binding: tag: imp-null remote binding: tsr: 10.10.10.2:0, tag: 17 remote binding: tsr: 10.10.10.1:0, tag: 20 remote binding: tsr: 10.10.10.6:0, tag: 23 tib entry: 10.10.10.4/32, rev 40 local binding: tag: 20 remote binding: tsr: 10.10.10.1:0, tag: 16 remote binding: tsr: 10.10.10.2:0, tag: 20 remote binding: tsr: 10.10.10.6:0, tag: 24 tib entry: 10.10.10.5/32, rev 44 local binding: tag: 22 remote binding: tsr: 10.10.10.1:0, tag: 17 remote binding: tsr: 10.10.10.2:0, tag: 22 remote binding: tsr: 10.10.10.6:0, tag: 25 tib entry: 10.10.10.6/32, rev 48 local binding: tag: 23 remote binding: tsr: 10.10.10.6:0, tag: imp-null remote binding: tsr: 10.10.10.1:0, tag: 22 remote binding: tsr: 10.10.10.2:0, tag: 24 (...) Pomerol# show tag-switching forwarding-table 10.10.10.4 detail Local Outgoing Prefix Bytes tag Outgoing Next Hoptag tag or VC or Tunnel Id switched interface 20 16 10.10.10.4/32 0 Se0/1.1 point2point MAC/Encaps=4/8, MTU=1500, Tag Stack{16} 48D18847 00010000 No output feature configured Per-packet load-sharing
Use the debug mpls packet command or the MPLS-aware traceroute command functionality in order to make sure that the labels are set.
Pesaro# traceroute 10.10.10.4 Type escape sequence to abort. Tracing the route to 10.10.10.4 1 10.1.1.21 [MPLS: Label 20 Exp 0] 272 msec 268 msec 300 msec 2 10.1.1.5 [MPLS: Label 16 Exp 0] 228 msec 228 msec 228 msec 3 10.1.1.14 92 msec * 92 msec
This document shows you how to troubleshoot the Configuring a Basic MPLS VPN document. We recommend you read this sample configuration and view the network diagram before you use this document.
Configuring a Basic MPLS VPN shows a fully functional MPLS backbone network which means provider edge (PE) routers are able to reach each other through the backbone. Refer to the MPLS Verification and Troubleshooting Support Page for information on troubleshooting an MPLS network.
Before establishing an MPLS VPN, you must be able to ping PE router A (10.10.10.4) from PE router B (10.10.10.6) and vice-versa.
Remember that VPN routing/forwarding instance (VRF) names are case sensitive, for example, Customer_A is not the same as customer_a.
Readers of this document should be familiar with:
This document is not restricted to specific software and hardware versions.
The information in this document was created from the devices in a specific lab environment. All of the devices used in this document started with a cleared (default) configuration. If your network is live, make sure that you understand the potential impact of any command.
For more information on document conventions, refer to the Cisco Technical Tips Conventions.
The show ip vrf [vrf-name] command shows a summary of all VRFs present on the current router and their associated route-distinguishers and interface(s).
Pesaro# show ip vrf Name Default RD Interfaces Customer_A 100:101 Loopback101 Loopback111 Customer_B 100:102 Loopback102
This command allows you to verify:
The configuration of VRFs (and their names).
That each route-distinguisher (RD) is the same at each concerned PE.
The show ip vrf [{detail | interfaces}] vrf-name command shows detailed configurations about the VRF.
Pesaro# show ip vrf detail Customer_A VRF Customer_A; default RD 100:101 Interfaces: Loopback101 Loopback111 Connected addresses are not in global routing table Export VPN route-target communities RT:100:1001 Import VPN route-target communities RT:100:1001 No import route-map No export route-map Pesaro# show ip vrf interfaces Interface IP-Address VRF Protocol Loopback101 200.0.6.1 Customer_A up Loopback111 200.1.6.1 Customer_A up Loopback102 200.0.6.1 Customer_B up
These commands allow you to verify:
That connected addresses are not in the global routing table.
The routing attributes of each VRF. What is exported on one side should be imported somewhere else.
The interface status (and IP addresses) of interfaces.
Use the same commands you use to verify the global routing table with the extensions shown in this section to verify routing tables or routing protocol databases.
To check the routing table, Add the vrf [vrf-name] extension to the show ip route command to verify the routing table, as shown here:
Pescara# show ip route vrf Customer_A Codes: C - connected, S - static, I - IGRP, R - RIP, M - mobile, B - BGP D - EIGRP, EX - EIGRP external, O - OSPF, IA - OSPF inter area N1 - OSPF NSSA external type 1, N2 - OSPF NSSA external type 2 E1 - OSPF external type 1, E2 - OSPF external type 2, E - EGP i - IS-IS, L1 - ISIS level-1, L2 - ISIS level-2, ia - ISIS inter area * - candidate default, U - per-user static route, o - ODR P - periodic downloaded static route Gateway of last resort is not set B 200.0.6.0/24 [200/0] via 10.10.10.6, 00:42:14 B 200.1.6.0/24 [200/0] via 10.10.10.6, 00:42:14 C 200.0.4.0/24 is directly connected, Loopback101
You can also use the show ip route vrf Customer_A 1.2.3.4 command to verify the destination for a particular address.
Border Gateway Protocol (BGP) is used between the PE routers and is necessary for inter-site connectivity. In this example, we use internal BGP (iBGP). You can also use external BGP (eBGP) as an external routing protocol for PE-CE route propagation.
You can use these commands to troubleshoot BGP:
show ip bgp neighbors
show ip bgp vpnv4 all (or show ip bgp vpnv4 vrf [VRF name])
show ip bgp vpnv4 vrf VRF name tags (this command is VPN/MPLS specific)
show ip bgp vpnv4 vrf VRF name A.B.C.D
For example:
Pescara# show ip bgp vpnv4 vrf Customer_A BGP table version is 40, local router ID is 10.10.10.4 Status codes: s suppressed, d damped, h history, * valid, > best, i - internal Origin codes: i - IGP, e - EGP, ? - incomplete Network Next Hop Metric LocPrf Weight Path Route Distinguisher: 100:101 (default for vrf Customer_A) *>i200.0.6.0 10.10.10.6 0 100 0 ? *> 200.0.4.0 0.0.0.0 0 32768 ? *>i200.1.6.0 10.10.10.6 0 100 0 ?
Refer to the BGP Support Pages for more information on troubleshooting BGP issues.
If the routing protocol used on the customer side isn't BGP, you can use traditional show commands, and apply them to the correct VRF.
Use the show ip rip database vrf [VRF name] command if you use Routing Information Protocol (RIP). For example:
Alcazaba# show ip rip database vrf vrf101 0.0.0.0/0 auto-summary 0.0.0.0/0 [2] via 150.150.0.2, 00:00:12, Ethernet1/1 6.0.0.0/8 auto-summary 6.6.6.6/32 redistributed [1] via 223.0.0.21, 7.0.0.0/8 auto-summary 7.7.7.0/24 [1] via 150.150.0.2, 00:00:12, Ethernet1/1 10.0.0.0/8 auto-summary 10.0.0.0/8 redistributed [1] via 125.2.2.2, 10.0.0.0/16 [1] via 150.150.0.2, 00:00:12, Ethernet1/1 10.200.8.0/22
Use the show ip ospf [process-id area-id] database command and specify the correct process number if you use OSPF. For example:
Alcazaba# show ip ospf 2 database OSPF Router with ID (222.0.0.10) (Process ID 2) Router Link States (Area 1) Link ID ADV Router Age Seq# Checksum Link count 222.0.0.1 222.0.0.1 1364 0x80000013 0x7369 3 222.0.0.10 222.0.0.10 1363 0x80000002 0xFEFE 2 Net Link States (Area 1) Link ID ADV Router Age Seq# Checksum 150.150.0.1 222.0.0.10 1363 0x80000001 0xEC6D Summary Net Link States (Area 1) Link ID ADV Router Age Seq# Checksum 6.6.6.6 222.0.0.10 1328 0x80000001 0x4967 69.69.0.0 222.0.0.10 1268 0x80000001 0x2427 222.0.0.3 222.0.0.10 1328 0x80000001 0xEEF7 222.0.0.30 222.0.0.10 1268 0x80000001 0x7B5A
This command allows you to verify:
If the routing table is correct (from a customer point of view), or what is missing from the routing table.
That BGP is up and working (or you can see which neighbor is missing).
MPLS VPN uses a two-level label stack. One of the labels is used to identify the VRF and is set up between the two PEs. The other label (on the top of the stack) is the "backbone" label, set up by the standard MPLS network.
You can use the traceroute VRF [vrf-name] A.B.C.B command to verify transport labels.
Note: This command only works with a MPLS-aware traceroute, if the backbone routers are configured to propagate and generate IP Time to Live (TTL) information. Refer to the documentation on the mpls ip propagate-ttl command for more information.
Pesaro# traceroute vrf Customer_B 200.0.4.1 Type escape sequence to abort. Tracing the route to 200.0.4.1 1 10.1.1.21 [MPLS: Labels 25/28 Exp 0] 464 msec 280 msec 308 msec 2 10.1.1.5 [MPLS: Labels 22/28 Exp 0] 236 msec 572 msec 228 msec 3 200.0.4.1 108 msec * 100 msec
The absence of 10.1.1.14 in this traceroute is normal due to the MPLS/VPN architecture.
You can use the show ip bgp vpnv4 all tags command to obtain more precise output, like the labels table for a particular VRF, for example:
Pescara# show ip bgp vpnv4 all tags Network Next Hop In tag/Out tag Route Distinguisher: 100:101 (Customer_A) 200.0.6.0 10.10.10.6 notag/28 200.0.4.0 0.0.0.0 16/aggregate(Customer_A) 200.1.6.0 10.10.10.6 notag/29 Route Distinguisher: 100:102 (Customer_B) 200.0.6.0 10.10.10.6 notag/30 200.0.4.0 0.0.0.0 28/aggregate(Customer_B)
You can also use the traditional show ip cef command:
Pescara# show ip cef vrf Customer_B detail IP CEF with switching (Table Version 10), flags=0x0 8 routes, 0 reresolve, 0 unresolved (0 old, 0 new) 46 leaves, 51 nodes, 54640 bytes, 361 inserts, 315 invalidations 0 load sharing elements, 0 bytes, 0 references universal per-destination load sharing algorithm, id F968AD29 5 CEF resets, 38 revisions of existing leaves refcounts: 1400 leaf, 1392 node Adjacency Table has 2 adjacencies 0.0.0.0/32, version 0, receive 200.0.6.0/24, version 9, cached adjacency to Serial0/1.1 0 packets, 0 bytes tag information set local tag: VPN-route-head fast tag rewrite with Se0/1.1, point2point, tags imposed: {20 30} via 10.10.10.6, 0 dependencies, recursive next hop 10.1.1.13, Serial0/1.1 via 10.10.10.6/32 valid cached adjacency tag rewrite with Se0/1.1, point2point, tags imposed: {20 30} 200.0.4.0/24, version 6, attached, connected 0 packets, 0 bytes tag information set local tag: 28 via Loopback102, 0 dependencies valid discard adjacency tag rewrite with , , tags imposed: {} 200.0.4.0/32, version 4, receive 200.0.4.1/32, version 3, receive 200.0.4.255/32, version 5, receive 224.0.0.0/24, version 2, receive 255.255.255.255/32, version 1, receive
This command allows you to verify:
That labels are effectively used.
That a stack of (at least) two labels is used for VPN destinations.
You can use the ping command to verify that the VRF works, but if you are on a PE router, you must indicate the specific VRF name.
Pescara# ping vrf Customer_A 200.0.6.1 Type escape sequence to abort. Sending 5, 100-byte ICMP Echos to 200.0.6.1, timeout is 2 seconds: !!!!! Success rate is 100 percent (5/5), round-trip min/avg/max = 176/264/576 ms
02-18-2015 06:28 PM
Thank you Cehill!
Deverick
02-19-2015 08:20 AM
Hi,
First of all, check that all directly connected devices are being pinged successfully.
Link state protocol is must.
Enable MPLS traffic engineering tunnels command must be enabled on all routers those are on the way of VPN.
If you are running OSFP and the area of all scenerio is area zero, then give these commands on every interface of every router on configure mode.
mpls traffic engineering tunnels
ip rsvp bandwidth (number)
Now you have to run protocol.
Now you have to give commands of ospf.
router(config)# router osfp 9
router(config)#mpls traffic engineering router-id loopback 0
router(config)#mpls traffic engineering area 0
router(config)#mpls traffic engineering multicast-intact
Explicit path selection:
router(config)#ip explicit-path name (any name)
router(config)#next-address (ip address of next router)
.
.
.
.
.
give addresses of all routers
Configuration of tunnel:
interface tunnel 0
ip-unnumbered loopback 0
tunnel destination (address)
tunnel mode mpls traffic engineering
mpls traffic engineering bandwidth (number in K)
tunnel mpls traffic engineering path-option 1 explicit (name same name in explicit)
ip route (address of last router) subnet mask tunnel
Discover and save your favorite ideas. Come back to expert answers, step-by-step guides, recent topics, and more.
New here? Get started with these tips. How to use Community New member guide