BGP Next-hop conflict with MPLS Label.

[maritpra]

Hi, Experts

Equipment: Cisco ASR9922, IOS-XR 4.3.2

Issue: I have problem that my RR do the next-hop-self by using route-policy for client routers, the next-hop is changed as intended but the MPLS label doesn’t changed to reflect the new next-hop.

What I would like to achieve: I would like RR to set next-hop-self only for selected prefixes(172.168.0.0/24, 0.0.0.0/0) but maintain original next-hop for the rest, I do this by using route-policy.

Detail:

I have routers running MPLS infrastructure with ASR9922 as an RR. RN router is in neighbor-group RN and CPE-xx routers are in neighbor-group AN.

• •-       Every routers are in same BGP AS64549.
• •-       RN sends prefixes 0.0.0.0/0 and 172.168.0.0/24 to RR.
• •-       CPE-25 sends prefix 192.168.25.1/32 to RR.

Neighbor-group AN has the route-policy AN-OUT2 to set next-hop of prefix 172.168.0.0/24 and 0.0.0.0/0 to RR#loopback1 before send out update to CPE routers. Below is BGP and RPL configuration at RR.

router bgp 64549

nsr

bgp graceful-restart

ibgp policy out enforce-modifications

address-family vpnv4 unicast

  additional-paths receive

  additional-paths send

  additional-paths selection route-policy ADD-PATH-iBGP

  retain route-target all

!

neighbor-group AN

  remote-as 64549

  cluster-id 172.16.1.11

  update-source Loopback1

  address-family vpnv4 unicast

   route-reflector-client

   route-policy AN-OUT2 out

   soft-reconfiguration inbound

  !

!

route-policy AN-OUT2

  if destination in DEFAULT or destination in RNC then

    set next-hop 192.168.10.11

  else

    pass

  endif

end-policy

This is what RR advertises to CPE-24

RP/0/RP0/CPU0:RR#show bgp vpnv4 unicast neighbors 192.168.10.24 advertised-routes

Fri Dec 20 15:23:14.931 BKK

Network            Next Hop        From            AS Path

Route Distinguisher: 64549:3339

0.0.0.0/0          192.168.10.11   172.16.1.1      ?

                                   172.16.1.2      ?

172.168.0.0/24     192.168.10.11   172.16.1.1      ?

                                   172.16.1.2      ?

192.168.0.1/32     192.168.10.11   192.168.10.24   i

192.168.0.26/32    192.168.10.26   192.168.10.26   i

192.168.25.1/32    192.168.10.25   192.168.10.25   i

192.168.211.8/30   192.168.10.22   192.168.10.22   i

The IP part works as intended but MPLS Label doesn’t work as intended. Please take a look at RN who is originates 172.168.0.0/24, label 16025 is locally assigned.

RP/0/RP0/CPU0:RN1#show bgp vpnv4 unicast labels

Status codes: s suppressed, d damped, h history, * valid, > best

              i - internal, r RIB-failure, S stale, N Nexthop-discard

Origin codes: i - IGP, e - EGP, ? - incomplete

   Network            Next Hop        Rcvd Label      Local Label

Route Distinguisher: 64549:3339 (default for vrf VLAN3339)

*> 0.0.0.0/0          0.0.0.0         nolabel         16025          

* i                   172.16.1.11     16068           16025          

* i                   172.16.1.13     16033           16025          

*> 172.168.0.0/24     0.0.0.0         nolabel         16025          

* i                   172.16.1.11     16059           16025          

* i                   172.16.1.13     16024           16025          

172.168.0.0/24 at RR, label 16059 is locally assigned, label 16025 is receive from RN router. It should send 172.168.0.0/24 with label 16059 to CPE-24 to reflect next-hop changed.

RP/0/RSP0/CPU0:RR#show bgp vpnv4 unicast labels

Status codes: s suppressed, d damped, h history, * valid, > best

              i - internal, r RIB-failure, S stale, N Nexthop-discard

Origin codes: i - IGP, e - EGP, ? - incomplete

   Network            Next Hop        Rcvd Label      Local Label

Route Distinguisher: 64549:3339

*>i0.0.0.0/0          172.16.1.1      16025           16068          

* i                   172.16.1.2      16007           16068          

*>i172.168.0.0/24     172.16.1.1      16025           16059          

* i                   172.16.1.2      16007           16059          

*>i192.168.0.1/32     192.168.10.24   131070          16060          

*>i192.168.25.1/32    192.168.10.25   131070          16062          

*>i192.168.211.8/30   192.168.10.22   131070          16065          

What I found at CPE-24 which is Alcatel router is that, RR send prefix 172.168.0.0/24, nh 192.168.10.11 with label 16025 which is incorrect.

A:CPE-24# show router bgp routes vpn-ipv4 172.168.0.0/24

===============================================================================

BGP Router ID:192.168.10.24    AS:64549       Local AS:64549     

===============================================================================

Legend -

Status codes  : u - used, s - suppressed, h - history, d - decayed, * - valid

Origin codes  : i - IGP, e - EGP, ? - incomplete, > - best, b - backup

===============================================================================

BGP VPN-IPv4 Routes

===============================================================================

Flag  Network                                            LocalPref   MED

      Nexthop                                            Path-Id     VPNLabel

      As-Path                                                       

-------------------------------------------------------------------------------

u*>?  64549:3339:172.168.0.0/24                          100         0

      192.168.10.11                                      None        16025

      No As-Path                                                     

-------------------------------------------------------------------------------

Routes : 1

===============================================================================

On RR If I just remove the policy and do the next-hop-self under vpv4 address family, CPE-24 will get corrent nh with correct label(16059) but that won’t achieve our requirement to change nh only on selected prefixes. Is this software problem? Or is there any solution to work around?

Regard,

Marit

[Rivalino Tamaela]

Hello Marit,

As you have opened TAC case, I will follow up on that case.

thanks,

Rivalino

[maritpra]

Hi Rivalino,

I just see your message here.

Thanks.

[Rivalino Tamaela · Accepted solution]

Hello Marit,

I am able to recreate this in the lab, and unfortunately this scenario is not supported. BGP does not advertise allocated label if we set nexhop using route policy. The only way is by next-hop-self configured on RR, and yes it eventually will applies to all prefixes advertised to neighbor-group AN. Currently i do not have workaround available.

Below is the capture of what i have tested in the lab:

The topology:

CRS-4-02 ---------- CRS-8-01 ------------ ASR-9006-1

CRS-8-01 is Route-reflector of CRS-4-02 and ASR-9006-1.

CRS-4-02 advertise some prefixes.

This issue occurs when RR have route-policy toward ASR-9006-1, where it assign incorrect label. But it assign correct label if CRS-8-01 use next-hop-self.

Below is the test done in the lab if RR use next-hop-self:

RP/0/RP0/CPU0:CRS-4-02#show run router bgp

Tue Jan  7 08:16:18.945 UTC

router bgp 1

bgp router-id 172.16.4.1

ibgp policy out enforce-modifications

address-family ipv4 unicast

!

address-family vpnv4 unicast

!

neighbor 172.16.8.3

  remote-as 1

  update-source Loopback0

  address-family ipv4 unicast

  !

  address-family vpnv4 unicast

   route-policy PASS in

   route-policy PASS out

  !

!

vrf RTAMAELA

  rd 100:1

  address-family ipv4 unicast

   redistribute connected

  !

!

!

RP/0/RP0/CPU0:CRS-4-02#show bgp vpnv4 unicast advertised  summary

Tue Jan  7 08:16:29.001 UTC

Network            Next Hop        From             Advertised to

Route Distinguisher: 100:1

78.22.11.2/32      172.16.4.1      Local            172.16.8.3

78.22.11.3/32      172.16.4.1      Local            172.16.8.3

93.22.15.61/32     172.16.4.1      Local            172.16.8.3

RP/0/RP0/CPU0:CRS-4-02#

RP/0/RP0/CPU0:CRS-4-02#show bgp vpnv4 unicast labels

Tue Jan  7 08:16:53.655 UTC

BGP router identifier 172.16.4.1, local AS number 1

BGP generic scan interval 60 secs

BGP table state: Active

Table ID: 0x0

BGP main routing table version 57

BGP scan interval 60 secs

Status codes: s suppressed, d damped, h history, * valid, > best

              i - internal, r RIB-failure, S stale

Origin codes: i - IGP, e - EGP, ? - incomplete

   Network            Next Hop        Rcvd Label      Local Label

Route Distinguisher: 100:1 (default for vrf RTAMAELA)

*>i22.51.32.77/32     172.16.8.3      16056           nolabel

*> 78.22.11.2/32      0.0.0.0         nolabel         16003

*> 78.22.11.3/32      0.0.0.0         nolabel         16003

*> 93.22.15.61/32     0.0.0.0         nolabel         16003

Processed 4 prefixes, 4 paths

RP/0/RP0/CPU0:CRS-4-02#

RP/0/RP1/CPU0:CRS-8-01#show run router bgp

Wed Jan  8 11:07:05.436 UTC

router bgp 1

bgp graceful-restart

ibgp policy out enforce-modifications

address-family ipv4 unicast

  allocate-label all

!

address-family vpnv4 unicast

  retain route-target all

!

neighbor-group AN

  remote-as 1

  update-source Loopback0

  address-family vpnv4 unicast

   route-reflector-client

   next-hop-self                              <-- use next-hop-self toward ASR-9006-1

   soft-reconfiguration inbound

  !

!

neighbor-group RN

  remote-as 1

  update-source Loopback0

  graceful-restart

  address-family vpnv4 unicast

   route-reflector-client

   next-hop-self

   soft-reconfiguration inbound

  !

!

neighbor 10.10.10.10

  remote-as 1

  address-family ipv4 unicast

  !

!

neighbor 72.15.48.5

  use neighbor-group AN

!

neighbor 172.16.4.1

  use neighbor-group RN

!

!

RP/0/RP1/CPU0:CRS-8-01#show bgp vpnv4 unicast labels

Wed Jan  8 11:07:09.091 UTC

BGP router identifier 172.16.8.3, local AS number 1

BGP generic scan interval 60 secs

BGP table state: Active

Table ID: 0x0   RD version: 344169

BGP main routing table version 92

BGP scan interval 60 secs

Status codes: s suppressed, d damped, h history, * valid, > best

              i - internal, r RIB-failure, S stale

Origin codes: i - IGP, e - EGP, ? - incomplete

   Network            Next Hop        Rcvd Label      Local Label

Route Distinguisher: 100:1

*>i22.51.32.77/32     72.15.48.5      16000           16056

*>i78.22.11.2/32      172.16.4.1      16003           16053

*>i78.22.11.3/32      172.16.4.1      16003           16054

*>i93.22.15.61/32     172.16.4.1     16003           16055

Processed 4 prefixes, 4 paths

RP/0/RP1/CPU0:CRS-8-01#

RP/0/RSP1/CPU0:ASR-9006-01#show run router bgp

Wed Jan  8 17:02:02.796 UTC

router bgp 1

bgp router-id 72.15.48.5

bgp graceful-restart

ibgp policy out enforce-modifications

address-family ipv4 unicast

!

address-family vpnv4 unicast

  retain route-target all

!

neighbor-group RR

  remote-as 1

  update-source Loopback0

  graceful-restart

  address-family vpnv4 unicast

   route-reflector-client

   soft-reconfiguration inbound

  !

!

neighbor 172.16.8.3

  use neighbor-group RR

!

neighbor 192.169.1.2

  remote-as 1

  update-source Loopback0

  address-family vpnv4 unicast

   route-policy PASS in

   route-policy PASS out

  !

!

vrf RTAMAELA

  rd 100:1

  address-family ipv4 unicast

   redistribute connected

  !

!

!

RP/0/RSP1/CPU0:ASR-9006-01#show bgp vpnv4 unicast labels

Wed Jan  8 17:02:04.381 UTC

BGP router identifier 72.15.48.5, local AS number 1

BGP generic scan interval 60 secs

BGP table state: Active

Table ID: 0x0   RD version: 253825

BGP main routing table version 126

BGP scan interval 60 secs

Status codes: s suppressed, d damped, h history, * valid, > best

              i - internal, r RIB-failure, S stale

Origin codes: i - IGP, e - EGP, ? - incomplete

   Network            Next Hop        Rcvd Label      Local Label

Route Distinguisher: 100:1 (default for vrf RTAMAELA)

*> 22.51.32.77/32     0.0.0.0         nolabel         16000

*>i78.22.11.2/32      172.16.8.3      16053           nolabel          <== 172.16.8.3 is the loopback address of CRS-8-01

*>i78.22.11.3/32      172.16.8.3      16054           nolabel

*>i93.22.15.61/32     172.16.8.3      16055           nolabel

Processed 4 prefixes, 4 paths

RP/0/RSP1/CPU0:ASR-9006-01#

From output above we can see that ASR-9006-01 received correct label for each prefix.

Below is the output with route-policy configured and ASR-9006-01 receive incorrect label:

RP/0/RP1/CPU0:CRS-8-01#show run router bgp

Wed Jan  8 11:04:46.310 UTC

router bgp 1

bgp graceful-restart

ibgp policy out enforce-modifications

address-family ipv4 unicast

  allocate-label all

!

address-family vpnv4 unicast

  retain route-target all

!

neighbor-group AN

  remote-as 1

  update-source Loopback0

  address-family vpnv4 unicast

   route-reflector-client

   route-policy RTAMAELA out

   soft-reconfiguration inbound

  !

!

neighbor-group RN

  remote-as 1

  update-source Loopback0

  graceful-restart

  address-family vpnv4 unicast

   route-reflector-client

   next-hop-self

   soft-reconfiguration inbound

  !

!

!

neighbor 72.15.48.5

  use neighbor-group AN

!

neighbor 172.16.4.1

  use neighbor-group RN

!

!

RP/0/RP1/CPU0:CRS-8-01#show run route-policy RTAMAELA

Wed Jan  8 11:16:06.847 UTC

route-policy RTAMAELA

  if destination in RNC then

    set next-hop 172.16.8.3

  else

    pass

  endif

end-policy

!

RP/0/RP1/CPU0:CRS-8-01#show run prefix-set RNC

Wed Jan  8 11:16:12.099 UTC

prefix-set RNC

  78.22.11.3/32

end-set

!

RP/0/RP1/CPU0:CRS-8-01#show bgp vpnv4 unicast labels

Wed Jan  8 11:04:33.512 UTC

BGP router identifier 172.16.8.3, local AS number 1

BGP generic scan interval 60 secs

BGP table state: Active

Table ID: 0x0   RD version: 344013

BGP main routing table version 92

BGP scan interval 60 secs

Status codes: s suppressed, d damped, h history, * valid, > best

              i - internal, r RIB-failure, S stale

Origin codes: i - IGP, e - EGP, ? - incomplete

   Network            Next Hop        Rcvd Label      Local Label

Route Distinguisher: 100:1

*>i22.51.32.77/32     72.15.48.5      16000           16056

*>i78.22.11.2/32      172.16.4.1      16003           16053

*>i78.22.11.3/32      172.16.4.1      16003           16054

*>i93.22.15.61/32     172.16.4.1      16003           16055

Processed 4 prefixes, 4 paths

RP/0/RP1/CPU0:CRS-8-01#

RP/0/RSP1/CPU0:ASR-9006-01#show run router bgp

Wed Jan  8 16:59:41.601 UTC

router bgp 1

bgp router-id 72.15.48.5

bgp graceful-restart

ibgp policy out enforce-modifications

address-family ipv4 unicast

!

address-family vpnv4 unicast

  retain route-target all

!

neighbor-group RR

  remote-as 1

  update-source Loopback0

  graceful-restart

  address-family vpnv4 unicast

   route-reflector-client

   soft-reconfiguration inbound

  !

!

neighbor 172.16.8.3

  use neighbor-group RR

!

neighbor 192.169.1.2

  remote-as 1

  update-source Loopback0

  address-family vpnv4 unicast

   route-policy PASS in

   route-policy PASS out

  !

!

vrf RTAMAELA

  rd 100:1

  address-family ipv4 unicast

   redistribute connected

  !

!

!

RP/0/RSP1/CPU0:ASR-9006-01#show bgp ipv4 unicast labels

Wed Jan  8 16:59:52.173 UTC

RP/0/RSP1/CPU0:ASR-9006-01#show bgp vpnv4 unicast labels

Wed Jan  8 17:00:00.457 UTC

BGP router identifier 72.15.48.5, local AS number 1

BGP generic scan interval 60 secs

BGP table state: Active

Table ID: 0x0   RD version: 253701

BGP main routing table version 123

BGP scan interval 60 secs

Status codes: s suppressed, d damped, h history, * valid, > best

              i - internal, r RIB-failure, S stale

Origin codes: i - IGP, e - EGP, ? - incomplete

   Network            Next Hop        Rcvd Label      Local Label

Route Distinguisher: 100:1 (default for vrf RTAMAELA)

*> 22.51.32.77/32     0.0.0.0         nolabel         16000

*>i78.22.11.2/32      172.16.4.1      16003           nolabel

*>i78.22.11.3/32      172.16.8.3      16003           nolabel   <-- It receive label 16003, which is wrong. it should receive label 16054.

*>i93.22.15.61/32     172.16.4.1      16003           nolabel

Processed 4 prefixes, 4 paths

RP/0/RSP1/CPU0:ASR-9006-01#

Rivalino

[maritpra]

Hi Rivalino,

That's answer the issue i'm faceing, anyway i'm looking into the work around..

If we go back one step further, the reason I do this is

AN-AN communication: I would like it to take shortest path in golbal-routing instead going back and forth between         AN-RR-AN. This is actually X2 interface for mobile node while AN is actually access node that eNodeB connected to.

AN-RN communication through RR: PN is inline-RR where there is MPLS-TE configure in network like this.

[AN]---mpls-te---[RR]---mpls-te---[RN]

[AN]---isis_b-----[RR]---isis_a-----[RN]

I would like to drive traffic from AN-to-RN trough tunnel-te that why i need to set next-hop-self. This is actually S1 traffic and RN is a router that EPC connected to.

Work Around:

Is there anyway to drive traffic from AN to RN through tunnel-te without sacrifying AN-AN communication? UMMT3.0 say that AN should do BGP with pre-aggregate node(RR) in the same ring where next-hop-self won't hurt cuz it is in same ring. In my case AN do BGP with aggregate node NOT in the same ring and it is far away.

[Rivalino Tamaela · Accepted solution]

The MP-BGP vpnv4 unicast AF is to advertised the VRF prefixes. But eventually it is the IGP that will route the packet to destination.

For vrf-RN <--> vrf-AN:

With the logical topology, the AN will advertise its VRF prefixes to RN with AN's loopback as nexthop. And then the IGP will take the packet from RN to AN's loopback and vice versa. Looking at the physical topology, we can say that the packet should transit via RR, as this is the only way to reach AN from RN.

for vrf-AN <--> vrf-AN:

Each AN will advertise its loopback as the nexthop for each VRF prefixes. And RR will reflect this route advertisement to other AN router without changing the next hop address. This means that AN1 will receive VRF prefixes from AN2 with AN2's loopback as its nexthop. After this the IGP will be responsible to carry the packet from AN1 to AN2, and IGP will take shortest path. If you run ISIS between AN1 and AN2 on its directly connected link, then the packet will not go through RR, but it takes shortest path which is directly connected link.

rivalino