PBR with MPLS

[Plamen Mladenov]

Hello group,

I'm struggling to make the PBR working on Nexus7010 (with SUP2,N7K-M132XP-12L and NX-OS 7.3.3 D1)

The setup is the following small MPLS topology:

<Customer CE router> --- <Nexus7K MPLS PE> --- <MPLS P router> --- <Egress MPLS PE router1 & Egress MPLS PE router 2>

Nexus 7K is acting as MPLS PE device.

The customer is physically terminated on a routed port on main interface of N7K-M132XP-12L card and is running eBGP with the Nexus7K in the global routing table (customer is NOT in VRF).

<Nexus7K MPLS PE>'s default route (via IBGP) is pointing to <Egress MPLS PE router1>'s loopback0 address and we have a label switched path to that IP

I would like to force the traffic from that specific customer to use <Egress MPLS PE router2> which is also connected to <MPLS P router> just like <Egress MPLS PE router1>  (there's also a LSP to <Egress MPLS PE router2> already up and running). When I try to accomplish this with simply making the default route originated from <Egress MPLS PE router2> to be more preferable - everything is working. The problem though is that way I'm forcing the traffic from all customers and I would like to do this only for that specific customer.

My idea is to use a PBR for that purpose - to match the traffic in an ACL based on the customer's source IP addresses and to simply "set ip next-hop <Egress MPLS PE router2>'s loopback IP" and to apply that route-map on Customer facing interface on <Nexus7K MPLS PE>. For some reason that's not working. I see with "show route-map <name> pbr-statistics" that there're matches, but traffic is getting blackholed. My suspicion is that it might be not be adding a MPLS label.

Here's the sample config and outputs:

N7K_PE# show ip policy
Interface                 Route-map            Status     VRF-Name
Ethernet1/10              PBR         Active     default

N7K_PE# sh route-map PBR
route-map PBR, permit, sequence 10
Match clauses:
    ip address (access-lists): PBR-ACL
Set clauses:
    ip next-hop <Egress MPLS PE Router2>

N7K_PE# sh route-map PBR pbr-statistics
route-map PBR, permit, sequence 10
  Policy routing matches: 110 packets
  Default routing: 2210168 packets


N7K_PE# sh ip route <Egress MPLS PE Router2> detail
<Egress MPLS PE Router2>/32, ubest/mbest: 1/0
    *via <MPLS P>, EthX/Y, [110/100], 19w4d, ospf-1, intra (mpls)
         MPLS[0]: Label=330032 E=0 TTL=255 S=0, LDP

I'm a newbie with NX-OS (especially used as a MPLS PE) but my assumption for this to work is:

1) Traffic is received from Customer as a normal IP traffic on N7K PE interface

2) Traffic source is matched by ACL

3) Next hop is set to <Egress MPLS PE Router2>

4) LFIB table is examined for <Egress MPLS PE Router2> IP address and a proper MPLS label is added.

Is this logic correct? Should I look for any bugs related to PBR and that old NX-OS? If my logic is not correct (especially point 4 - then I'm thinking of MPLS-TE with.... PBR again to force the traffic to use MPLS-TE tunnel on N7K PE), but if there's a problem with PBR (bug or config issue) most likely it won't work over MPLS-TE as well ..and I'll have to prepare the whole infra for TE (RSPV, Opaque LSAs,etc which will be pointless)

I tried the same setup with IOS-XE virtual routers and all works perfectly fine (with set ip next-hop recursive <Egress MPLS PE Router2> and I did come packet captures to verify there's a proper MPLS label added.

Regards,

Plamen