IOS XRv 9000 - PCE/PCC Policy Down

[loris.marcellini]

Hey folks, I am trying to configure an intra-domain PCE/PCC setup with headend initiated ODN policy. It looks like PCEP is up and report messages gets delivered to the PCE, however it looks like the update messages aren't there and the ERO from PCC to PCE is empty. I'll try to go step by step and describe the issue.

Topology

in short: full-mesh of cisco IOS-XRv 9000 (version Cisco IOS XR Software, Version 6.5.1) and IOS XRv (version Cisco IOS XR Software, Version 6.1.3). The routers all run L2 ISIS and BGP LU with iosxrv-16 being the RR for * AFs and with 9k-2 and iosxrv-7 being PEs for a GREEN L3VPN. The traffic-eng path I am trying to setup is from 9k2 to xr17 with the PCC being iosxr-9k-2 and the PCE being iosxr-9k-3.

                                                     iosxrv-16 (RR)   Cust-GREEN1

                                                              |                       |

Cust-GREEN2 -- iosxr-9k-2 ------ iosxrv-8 ----- iosxrv-17

                                |             \     /         |           \     /        | 

                                |            /    \           |           /     \        |

                           iosxrv-7    ------ iosxrv-9 ----- iosxr-9k-3

Configurations and Outputs

PCC configuration

router bgp 100
  address-family ipv4 unicast
    allocate-label all
  !
  address-family vpnv4 unicast
  !
  address-family vpnv6 unicast
  !
  neighbor 16.16.16.16
    remote-as 100
    update-source Loopback0
    address-family ipv4 labeled-unicast
    !
    address-family vpnv4 unicast
      route-policy VRF-COLOR in
    !
    address-family vpnv6 unicast
    !
  !
  vrf GREEN
    rd 1:1
    address-family ipv4 unicast
      network 1.1.1.4/32
    ! 
    address-family ipv6 unicast
      network 2001:1:1:1::4/128
    !
  !
!
router isis 1
 is-type level-2-only
 net 49.0001.0000.0102.00
 distribute link-state instance-id 102 
 address-family ipv4 unicast
  metric-style wide level 2
  fast-reroute per-prefix priority-limit critical
  advertise passive-only
  mpls traffic-eng level-2-only
  segment-routing mpls
  spf prefix-priority critical loopbacks
 !
 address-family ipv6 unicast
  advertise passive-only
 !
 interface Loopback0
  passive
  address-family ipv4 unicast
   prefix-sid index 102
  !
  address-family ipv6 unicast
  !
 !
 interface GigabitEthernet0/0/0/0.18
  point-to-point
  address-family ipv4 unicast
   fast-reroute per-prefix
  !
  address-family ipv6 unicast
  !
 !
 interface GigabitEthernet0/0/0/0.36
  point-to-point
  address-family ipv4 unicast
   fast-reroute per-prefix
  !
  address-family ipv6 unicast
  !
 !
 interface GigabitEthernet0/0/0/0.54
  point-to-point
  address-family ipv4 unicast
   fast-reroute per-prefix
  !
  address-family ipv6 unicast
  !
 !        
!
segment-routing
 global-block 16000 23999
 traffic-eng
  !
  on-demand color 30
   dynamic
    pcep
    !
    metric
     type te
    !
   !
  !
  pcc
   pce address ipv4 103.103.103.103
    precedence 100
   !
  !
 !
!

the PCC can reach the PCE

RP/0/RP0/CPU0:iosxr-9k-2#sh route  103.103.103.103/32  detail 
Thu Feb 25 23:29:55.884 UTC

Routing entry for 103.103.103.103/32
  Known via "isis 1", distance 115, metric 20, labeled SR, type level-2
  Installed Feb 25 22:24:56.705 for 01:04:59
  Routing Descriptor Blocks
    150.180.144.2, from 103.103.103.103, via GigabitEthernet0/0/0/0.36, Protected, ECMP-Backup (Local-LFA)
      Route metric is 20
      Label: 0x3ee7 (16103)
      Tunnel ID: None
      Binding Label: None
      Extended communities count: 0
      Path id:2       Path ref count:1
      NHID:0x2(Ref:10)
      Backup path id:1
    180.144.186.2, from 103.103.103.103, via GigabitEthernet0/0/0/0.54, Protected, ECMP-Backup (Local-LFA)
      Route metric is 20
      Label: 0x3ee7 (16103)
      Tunnel ID: None
      Binding Label: None
      Extended communities count: 0
      Path id:1       Path ref count:1
      NHID:0x1(Ref:8)
      Backup path id:2
  Route version is 0x10 (16)
  Local Label: 0x3ee7 (16103)
  IP Precedence: Not Set
  QoS Group ID: Not Set
  Flow-tag: Not Set
  Fwd-class: Not Set
  Route Priority: RIB_PRIORITY_NON_RECURSIVE_CRITICAL (5) SVD Type RIB_SVD_TYPE_LOCAL
  Download Priority 1, Download Version 96
  No advertising protos. 

PCE peer is up

RP/0/RP0/CPU0:iosxr-9k-2#sh segment-routing traffic-eng pcc ipv4 peer detail 
Thu Feb 25 23:31:17.685 UTC

PCC's peer database:
--------------------

Peer address: 103.103.103.103 (best PCE)
  State up
  Capabilities: Stateful, Update, Segment-Routing, Instantiation
  PCEP has been up for: 00:06:30
  Local keepalive timer is 30 seconds
  Remote keepalive timer is 30 seconds
  Local dead timer is 120 seconds
  Remote dead timer is 120 seconds
  Statistics:
    Open messages:      rx 3        |  tx 3      
    Close messages:     rx 2        |  tx 2      
    Keepalive messages: rx 28       |  tx 27     
    Error messages:     rx 0        |  tx 0      
    Report messages:    rx 0        |  tx 7      
    Update messages:    rx 0        |  tx 0 

the SR TE Database looks ok for the end-point

RP/0/RP0/CPU0:iosxr-9k-2#sh segment-routing traffic-eng ipv4 topology | b Node$
Thu Feb 25 23:33:02.708 UTC
Node 9
  TE router ID: 17.17.17.17
  Host name: iosxrv-17
  ISIS system ID: 0001.0000.0017 level-2 domain ID: 102
  Prefix SID:
    Prefix 17.17.17.17, label 16017 (regular)

  Link[0]: local address 129.144.165.2, remote address 129.144.165.1
    Local node:
      ISIS system ID: 0001.0000.0017 level-2 domain ID: 102
    Remote node:
      TE router ID: 8.8.8.8
      Host name: iosxrv-8
      ISIS system ID: 0001.0000.0008 level-2 domain ID: 102
    Metric: IGP 10, TE 10
    Bandwidth: Total link 125000000, Reservable 0
    Admin-groups: 0x00000000
    Adj SID: 24004 (protected) 24005 (unprotected)

  Link[1]: local address 165.217.182.2, remote address 165.217.182.1
    Local node:
      ISIS system ID: 0001.0000.0017 level-2 domain ID: 102
    Remote node:
      TE router ID: 9.9.9.9
      Host name: iosxrv-9
      ISIS system ID: 0001.0000.0009 level-2 domain ID: 102
    Metric: IGP 10, TE 10
    Bandwidth: Total link 125000000, Reservable 0
    Admin-groups: 0x00000000
    Adj SID: 24002 (protected) 24003 (unprotected)

  Link[2]: local address 182.209.152.1, remote address 182.209.152.2
    Local node:
      ISIS system ID: 0001.0000.0017 level-2 domain ID: 102
    Remote node:
      TE router ID: 103.103.103.103
      Host name: iosxr-9k-3
      ISIS system ID: 0001.0000.0103 level-2 domain ID: 102
    Metric: IGP 10, TE 10
    Bandwidth: Total link 125000000, Reservable 0
    Admin-groups: 0x00000000
    Adj SID: 24000 (protected) 24001 (unprotected)

PCE configuration

pce
  address ipv4 103.103.103.103

the policy is down on the PCC side

RP/0/RP0/CPU0:iosxr-9k-2#sh segment-routing traffic-eng policy 
Thu Feb 25 23:54:50.640 UTC

SR-TE policy database
---------------------

Name: bgp_AP_2 (Color: 30, End-point: 17.17.17.17)
    Status:
      Admin: up  Operational: down for 00:47:12 (since Feb 25 23:07:38.699)
    Candidate-paths:
      Preference 100:
        Dynamic (pce 103.103.103.103) (active)
          Metric Type: Unknown,   Path Accumulated Metric: 0 
    Attributes:
      Binding SID: 24015
        Allocation mode: dynamic
        State: Programmed
        Policy selected: yes
      Forward Class: 0
      Steering BGP disabled: no
      IPv6 caps enable: yes
      Distinguisher: 0
    Auto-policy info:
      Creator: BGP

the report is received but the ERO received by the PCC is empty

RP/0/RP0/CPU0:Feb 25 23:26:20.361 UTC: pce_server[1122]: DBG-PCEP:[139880252339968] pce_process_report_msg:4738 ******* New LSP report ********
RP/0/RP0/CPU0:Feb 25 23:26:20.361 UTC: pce_server[1122]: DBG-PCEP:[139880252339968] pce_parse_lsp_obj:4254 Report: plsp_id 524290, d:1, s:0, r:0, a1, o0, c0
RP/0/RP0/CPU0:Feb 25 23:26:20.361 UTC: pce_server[1122]: DBG-PCEP:[139880252339968] pce_parse_tlvs:3382 1. Parsing TLV IPv4 LSP Identifiers (type 18, size 16)
RP/0/RP0/CPU0:Feb 25 23:26:20.361 UTC: pce_server[1122]: DBG-PCEP:[139880252339968] pce_parse_ipv4_fec_tlv:3064 Received a ipv4 lsp identifiers TLV in LSP object - (sender addr 102.102.102.102, lsp-id 3, tun-id 2, ext tunnel-id 1717986918 (102.102.102.102), tunnel ep addr 17.17.17.17)
RP/0/RP0/CPU0:Feb 25 23:26:20.361 UTC: pce_server[1122]: DBG-PCEP:[139880252339968] pce_parse_tlvs:3382 2. Parsing TLV Symbolic Path Name (type 17, size 9)
RP/0/RP0/CPU0:Feb 25 23:26:20.361 UTC: pce_server[1122]: DBG-PCEP:[139880252339968] pce_parse_symbolic_name_tlv:2940 Received a symbolic path name TLV - (bgp_AP_2).
RP/0/RP0/CPU0:Feb 25 23:26:20.361 UTC: pce_server[1122]: DBG-PCEP:[139880252339968] pce_parse_tlvs:3382 3. Parsing TLV Binding Label (type 65505, size 6)
RP/0/RP0/CPU0:Feb 25 23:26:20.361 UTC: pce_server[1122]: DBG-PCEP:[139880252339968] pce_process_report_msg:4778 LSP object received
RP/0/RP0/CPU0:Feb 25 23:26:20.361 UTC: pce_server[1122]: DBG-PCEP:[139880252339968] pce_process_report_msg:4666 obj_class = 7 obj_type = 1 obj_len = 4 rem_len = 64
RP/0/RP0/CPU0:Feb 25 23:26:20.361 UTC: pce_server[1122]: DBG-PCEP:[139880252339968] pce_parse_ero_obj:3823 pce-pcep-ero: empty ERO
RP/0/RP0/CPU0:Feb 25 23:26:20.361 UTC: pce_server[1122]: DBG-PCEP:[139880252339968] pce_process_report_msg:4928 ERO object received
RP/0/RP0/CPU0:Feb 25 23:26:20.361 UTC: pce_server[1122]: DBG-PCEP:[139880252339968] pce_process_report_msg:4666 obj_class = 9 obj_type = 1 obj_len = 20 rem_len = 60
RP/0/RP0/CPU0:Feb 25 23:26:20.361 UTC: pce_server[1122]: DBG-PCEP:[139880252339968] pce_process_report_msg:5011 LSPA object received: exc_any 0x0, inc_any 0x0, inc_all 0x0, setup 7 hold 7 frr 1
RP/0/RP0/CPU0:Feb 25 23:26:20.361 UTC: pce_server[1122]: DBG-PCEP:[139880252339968] pce_process_report_msg:4666 obj_class = 5 obj_type = 1 obj_len = 8 rem_len = 40
RP/0/RP0/CPU0:Feb 25 23:26:20.361 UTC: pce_server[1122]: DBG-PCEP:[139880252339968] pce_process_report_msg:4936 BW object received, type 1,
RP/0/RP0/CPU0:Feb 25 23:26:20.361 UTC: pce_server[1122]: DBG-PCEP:[139880252339968] pce_process_report_msg:4953 BW object received, type 1, value 0
RP/0/RP0/CPU0:Feb 25 23:26:20.361 UTC: pce_server[1122]: DBG-PCEP:[139880252339968] pce_process_report_msg:4666 obj_class = 5 obj_type = 5 obj_len = 8 rem_len = 32
RP/0/RP0/CPU0:Feb 25 23:26:20.361 UTC: pce_server[1122]: DBG-PCEP:[139880252339968] pce_process_report_msg:4936 BW object received, type 5,
RP/0/RP0/CPU0:Feb 25 23:26:20.361 UTC: pce_server[1122]: DBG-PCEP:[139880252339968] pce_process_report_msg:4953 BW object received, type 5, value 0
RP/0/RP0/CPU0:Feb 25 23:26:20.361 UTC: pce_server[1122]: DBG-PCEP:[139880252339968] pce_process_report_msg:4666 obj_class = 6 obj_type = 1 obj_len = 12 rem_len = 24
RP/0/RP0/CPU0:Feb 25 23:26:20.361 UTC: pce_server[1122]: DBG-PCEP:[139880252339968] pce_process_report_msg:5160 METRIC object received, type 2 value 0
RP/0/RP0/CPU0:Feb 25 23:26:20.361 UTC: pce_server[1122]: DBG-PCEP:[139880252339968] pce_process_report_msg:4666 obj_class = 6 obj_type = 1 obj_len = 12 rem_len = 12
RP/0/RP0/CPU0:Feb 25 23:26:20.361 UTC: pce_server[1122]: DBG-PCEP:[139880252339968] pce_process_report_msg:5164 METRIC hop count value 16
RP/0/RP0/CPU0:Feb 25 23:26:20.361 UTC: pce_server[1122]: DBG-PCEP:[139880252339968] pce_process_report_msg:5278 successfully processed report msg
RP/0/RP0/CPU0:Feb 25 23:26:20.361 UTC: pce_server[1122]: DBG-PCEP:[139880982271744] pce_update_lsp_db_with_report:1705 lsp-create sym_name bgp_AP_2, plsp_id 524290 tun_id 2 lsp_id 3
RP/0/RP0/CPU0:Feb 25 23:26:20.361 UTC: pce_server[1122]: DBG-PCEP:[139880982271744] pce_update_lsp_db_with_report:1885 tun bw (act 0 sig 0), report_info bw (act 0 sig 0)
RP/0/RP0/CPU0:Feb 25 23:26:20.361 UTC: pce_server[1122]: DBG-PCEP:[139880982271744] pce_match_initiated_tun:2983 PCReport: tun 0x55a69dbd6da8, peer 102.102.102.102, srp-id 0
RP/0/RP0/CPU0:Feb 25 23:26:20.361 UTC: pce_server[1122]: DBG-PCEP:[139880982271744] pce_match_initiated_tun:2989 tun bgp_AP_2 D1 C0
RP/0/RP0/CPU0:Feb 25 23:26:20.361 UTC: pce_server[1122]: DBG-PCEP:[139880982271744] pce_match_initiated_tun:2991 tunnel not inititated
RP/0/RP0/CPU0:Feb 25 23:26:20.361 UTC: pce_server[1122]: DBG-PCEP:[139880982271744] pce_path_dump_rro_list:551 RRO list is empty

Questions

1. are there bad configs in my setup?
2. is the design bad and hence the config simply does not work?

Considerations 

1. The policy stays down and shows "metric: unknown" even if I try to setup a PCE initiated policy
2. A standalone (no PCEP) policy setup works and is shown as Up/Up but forwarding plane does not work
3. the routers between the 9k PEs are IOS XRv but not 9k.

I am going through the official SR book and from the look of it this config should work but I am sure I might be missing something, I really need some help here

Thanks in advance.

L.

[Harold Ritter]

Hi Loris,

Could you run the following command on the PCE to see if the PCE can calculate the path from headend to tailend PEs:

sh pce ipv4 path source 102.102.102.102 destination 17.17.17.17

Regards,

[loris.marcellini]

Hi Harold, glad to hear from you.

Here's the requested output

RP/0/RP0/CPU0:iosxr-9k-3#sh pce ipv4 path source 102.102.102.102 destination 1$
Fri Feb 26 23:31:58.671 UTC
No path exists between the specified source and destination.

it looks like the pce cannot calculate the path unfortunately, any idea why that is?

Interestingly enough, topology data is also not there

RP/0/RP0/CPU0:iosxr-9k-3#sh pce ipv4 topology 
Fri Feb 26 23:33:15.774 UTC
RP/0/RP0/CPU0:iosxr-9k-3#sh pce ipv4 topology traffic-eng 
Fri Feb 26 23:33:24.477 UTC
RP/0/RP0/CPU0:iosxr-9k-3#sh pce ipv4 topology traffic-eng 102.102.102.102
Fri Feb 26 23:33:43.920 UTC
RP/0/RP0/CPU0:iosxr-9k-3#

Thanks, L.

[Harold Ritter · Accepted solution]

Hi Loris,

Hope you are doing well.

You are probably just missing the "distribute link-state" statement under router isis on the PCE. You need this statement for the topology to be imported in the traffic-eng database. I see you configured this statement on the PCC, but it is not required there unless you are going to do any path calculation locally on the PCC. If all PCC will use the PCE for path calculation, you just need to configure this statement on the PCE.

After configuring the "distribute link-state" on the PCE, you should immediately start seeing the topology information when you do a "show pce ipv4 topology".

Regards,

[loris.marcellini]

Harold, that worked as a charm thanks!

I have added the "distribute link-state" to the PCE config and kept the PCC with a PCEP policy and added a simple ODN policy as well. The topology became immediately visible on the PCE and here's the policy status on the PCC now:

RP/0/RP0/CPU0:iosxr-9k-2#sh segment-routing traffic-eng policy                 
Sun Feb 28 00:48:15.601 UTC

SR-TE policy database
---------------------

Name: pce (Color: 30, End-point: 17.17.17.17)
  Status:
    Admin: up  Operational: up for 00:01:26 (since Feb 28 00:46:49.448)
  Candidate-paths:
    Preference 10:
      Dynamic (pce 103.103.103.103) (active)
        Metric Type: TE,   Path Accumulated Metric: 20 
          16009 [Prefix-SID, 9.9.9.9]
          16017 [Prefix-SID, 17.17.17.17]
  Attributes:
    Binding SID: 15000
      Allocation mode: explicit
      State: Programmed
      Policy selected: no
    Forward Class: 0
    Steering BGP disabled: no
    IPv6 caps enable: no

Name: bgp_AP_8 (Color: 30, End-point: 17.17.17.17)
    Status:
      Admin: up  Operational: up for 00:01:27 (since Feb 28 00:46:48.273)
    Candidate-paths:
      Preference 100:
        Dynamic (active)
          Metric Type: TE,   Path Accumulated Metric: 20 
            16009 [Prefix-SID, 9.9.9.9]
            16017 [Prefix-SID, 17.17.17.17]
    Attributes:
      Binding SID: 15000
        Allocation mode: dynamic
        State: Programmed
        Policy selected: yes
      Forward Class: 0
      Steering BGP disabled: no
      IPv6 caps enable: yes
      Distinguisher: 0
    Auto-policy info:
      Creator: BGP

Name: bgp_AP_9 (Color: 30, End-point: 103.103.103.103)
    Status:
      Admin: up  Operational: up for 00:01:27 (since Feb 28 00:46:48.273)
    Candidate-paths:
      Preference 100:
        Dynamic (active)
          Metric Type: TE,   Path Accumulated Metric: 20 
            16009 [Prefix-SID, 9.9.9.9]
            16103 [Prefix-SID, 103.103.103.103]
    Attributes:
      Binding SID: 24037
        Allocation mode: dynamic
        State: Programmed
        Policy selected: yes
      Forward Class: 0
      Steering BGP disabled: no
      IPv6 caps enable: yes
      Distinguisher: 0
    Auto-policy info:
      Creator: BGP

The only concern I have left is, why do explicit-paths don't work at the forwarding plane? I have tried to make PCE calculate a policy with explicit path or have it configure it locally on the headend:

segment-routing
 global-block 16000 23999
 traffic-eng
  segment-list manual
   index 10 mpls label 16009
   index 20 mpls label 16017
  !
  policy manual
   binding-sid mpls 15103
   color 30 end-point ipv4 103.103.103.103
   candidate-paths
    preference 100
     explicit segment-list manual

although the policy successfully applies to BGP VPNV4 routes routes and looks up/up, the traceroute does not work:

RP/0/RP0/CPU0:iosxr-9k-2#sh bgp vpnv4 un 
Status codes: s suppressed, d damped, h history, * valid, > best
              i - internal, r RIB-failure, S stale, N Nexthop-discard
Origin codes: i - IGP, e - EGP, ? - incomplete
   Network            Next Hop            Metric LocPrf Weight Path
Route Distinguisher: 1:1 (default for vrf GREEN)
*>i1.1.1.1/32         17.17.17.17              0    100      0 i
*>i1.1.1.2/32         3.3.3.3                       100      0 400 i
*>i1.1.1.3/32         1.1.1.1                       100      0 400 300 i
*> 1.1.1.4/32         0.0.0.0                  0         32768 i
*>i1.1.1.5/32         103.103.103.103 C:30
                                               0    100      0 i

RP/0/RP0/CPU0:iosxr-9k-2#sh segment-routing traffic-eng policy name manual 
Sun Feb 28 01:21:49.896 UTC

SR-TE policy database
---------------------

Name: manual (Color: 30, End-point: 103.103.103.103)
  Status:
    Admin: up  Operational: up for 00:07:18 (since Feb 28 01:14:31.501)
  Candidate-paths:
    Preference 100:
      Explicit: segment-list manual (active)
        Weight: 1, Metric Type: IGP
          16009 [Prefix-SID, 9.9.9.9]
          16017
  Attributes:
    Binding SID: 15103
      Allocation mode: explicit
      State: Programmed
      Policy selected: yes
    Forward Class: 0
    Steering BGP disabled: no
    IPv6 caps enable: no

RP/0/RP0/CPU0:iosxr-9k-2#sh segment-routing traffic-eng forwarding policy name$
Sun Feb 28 01:16:51.119 UTC
Policy        Segment         Outgoing    Outgoing            Next Hop        Bytes       
Name          List            Label       Interface                           Switched    
------------- --------------- ----------- ------------------- --------------- ------------
manual        manual          16009       Gi0/0/0/0.18        186.150.180.2   0           (!)
                 Label Stack (Top -> Bottom): { 16009, 16017 }
                 Path-id: 1 (Pure-Backup), Weight: 1
                 Packets Switched: 0
                              16017       Gi0/0/0/0.54        180.144.186.2   64 <-- counters increase when attempting traceoute         
                 Label Stack (Top -> Bottom): { 16017 }
                 Path-id: 2 (Protected), Backup-path-id: 1, Weight: 1
                 Packets Switched: 2
   Local label: 24043 
   Packets/Bytes Switched: 25/800
   (!): FRR pure backup

might this be a limitation of iosxrv in the middle? Or am I doing something wrong again here?

Thanks, L.

[Harold Ritter]

Hi Loris,

Glad that it worked for you.

For the explicit path scenario to work, you need to add the label of the egress PE to the segment list.

segment-routing
 traffic-eng
  segment-list manual
   index 10 mpls label 16009
   index 20 mpls label 16017
   index 30 mpls label 16103 
  !
!

This should fix it.

Regards,

[loris.marcellini]

That's it, it worked, thanks again!

RP/0/RP0/CPU0:iosxr-9k-2#sh segment-routing traffic-eng policy 
Sun Feb 28 22:39:26.054 UTC

SR-TE policy database
---------------------

Name: manual (Color: 30, End-point: 103.103.103.103)
  Status:
    Admin: up  Operational: up for 00:27:44 (since Feb 28 22:11:41.975)
  Candidate-paths:
    Preference 100:
      Explicit: segment-list manual (active)
        Weight: 1, Metric Type: IGP
          16009 [Prefix-SID, 9.9.9.9]
          16017
          16103
  Attributes:
    Binding SID: 15103
      Allocation mode: explicit
      State: Programmed
      Policy selected: yes
    Forward Class: 0
    Steering BGP disabled: no
    IPv6 caps enable: no

RP/0/RP0/CPU0:iosxr-9k-2#trace vrf GREEN 1.1.1.5
Sun Feb 28 22:39:35.804 UTC

Type escape sequence to abort.
Tracing the route to 1.1.1.5

 1  180.144.186.2 [MPLS: Labels 16017/16103/24003 Exp 0] 477 msec  21 msec  17 msec 
 2  165.217.182.2 [MPLS: Labels 16103/24003 Exp 0] 23 msec  27 msec  26 msec 
 3  182.209.152.2 28 msec  *  18 msec