Understanding of Policy Based routing

[tamimkushjar]

Hi,

I have a scenario for Policy based routing and I would like to have a opinion how the traffic should flow.

PBR is active on the LAN and is filtering traffic with the help of route-map which refers to a ACL. (standard configuration)

The route-map has a recursive next hop of it's neighbor which is not a direct hop but known via BGP.

Suppose that this recursive next hop is not seen anymore in BGP routing table, how should the router behave?

a) The packets are matched no matter if the recursive next hop is seen or not and is routed out over the PBR WAN interface.

b) The packets are matched even if the recursive next hop is not seen, but it sends the packets over the default WAN interface.

c) The packets are not matched due to the fact that recursive next hop is not seen in the BGP table and is routed directly over the default WAN interface.

Looking forward to hearing from someone what the right scenario should be.

[cadet alain]

Hi,

The answer is b)

Regards

Alain

Don't forget to rate helpful posts.

[tamimkushjar]

Hi Alain,

That's what I would have answered too, but I have placed test ACL's on the outgoing default WAN interface and I don't see the traffic passing by

lincol01-nm#sh ip access-lists TESTdefault-out

Extended IP access list TESTdefault-out

    10 permit tcp host 172.25.32.194 host 172.27.96.10 eq www  <== No matches

    20 permit tcp host 172.25.32.194 host 172.27.96.10 eq 8080

    30 permit tcp host 172.25.32.194 host 172.27.96.10 eq 1352

    40 permit ip any any (781301895 matches)

Extended IP access list TESTOFFLOAD-out

    10 permit tcp host 172.25.32.194 host 172.27.96.10 eq www (393 matches)

    20 permit tcp host 172.25.32.194 host 172.27.96.10 eq 8080 (206 matches)

    30 permit tcp host 172.25.32.194 host 172.27.96.10 eq 1352

    40 permit ip any any (241177730 matches)

[cadet alain · Accepted solution]

Hi,

I suppose you have multiple outgoing interfaces ? Have you got 2 paths installed in the RIB and are you doing load-sharing ?

Regards

Alain

Don't forget to rate helpful posts.

[tamimkushjar]

Hi Alain,

Well the routes concerning the actual offload ACL is known via the offloading and the default WAN interfaces (BGP)

The only thing which is learned extra from the offloading interface are the recursive next-hop addressess.

[tamimkushjar]

Hi Paul,

I have reviewed your answer and it turns out that the right answer was also listed in your reply!

"Now to answer you question regards recursive next hop - if the destination specified isn't available I am on the understanding it will route via default route in the rib unless otherwise stated say via a set interface null 0 value"

thanks for your posts as well as Alain!

[paul driver · Accepted solution]

Hello

I hope I can explain this too you and at the same time give it some justice, if not I am sure someone from this forum will correct me.

Policy routing as I see it is a " posh form of static routing, where as static routing is via per destination only policy routing can be source and destination or other forms of matching value to name a few to force traffic to a different path other than  the routing protocols are dictating.

Standard acl - source address only

Exceeded acl's - source and destination

Metrics value

Packet lengths

You can manipulate traffic based on some  kind of POLICY you set in route-maps  to effect traffic

Originating from the router - local policy routing - Control Plane

Traversing the router from a particular interface. -  Data Plane

Now to answer you question regards recursive next hop - if the destination specified isn't available I am on the understanding it will route via default route in the rib unless otherwise stated say via a set interface null 0 value.

Please see below is taken from  txt file I have complied mostly plagiarized from either cco  documentation or some other routing text book.

Policy routing:
###############

PBR can contain multiple set commands within a single route-map sequence number.
If multiple set commands are configured, the order that PBR will attempt them in is:

1. set ip next-hop verify-availability  – with object tracking only

2. set ip next-hop - either alone or with CDP verification

3. set ip next-hop recursive

4. set interface

5. Route table

6. set ip default next-hop - either alone or with CDP verification

7. set default interface

Recursive Next-Hop
######################
set ip next-hop recursive allow multiple interfaces or next-hop addresses to be listed in a single set statement.
If the 1st interface (or interface associated with the next-hop address, if a next-hop is used) is down or L2 address is unknown,
the rest of the interfaces/addresses in the list are tried in order

The infrastructure provided by CEF or process switching
performs the recursion to the next-hop IP address. The configuration sequence, which affects routing, is as follows:

1. Next-hop = Attempts PBR first

2. Next-hop recursive = Attempts PBR first

3. Interface = Attempts PBR first

4. Default next-hop = checks rib first

5. Default interface = check rib first

If both a next-hop and a recursive next-hop IP address are present in the same route-map entry,
the next-hop is used.

If the next-hop is not available, the recursive next-hop is used.

If the recursive next-hop is not available and no other IP address is present,
the packet is routed using the default routing table; it is not dropped.

If the packet is supposed to be dropped, use the set next-hop recursive command followed by a
set interface null0 configuration.

Hope this helps

res

Paul

Please don't forget to rate any posts that have been helpful.

Thanks.

[tamimkushjar]

Hi Paul,

Thank you for you're extensive mail what do you mean with:

"If the recursive next-hop is not available and no other IP address is present,
the packet is routed using the default routing table; it is not dropped."

Because I have for each remote site a seperate route-map sequence which has it's own recursive next hop.

I would assume that when ever the recursive next hop statmenet for that particular route-map sequence is not available it would sent the traffic over the default WAN interface which doesn't seem to be the case as shown in the reply to Alain.

[cadet alain]

Hi,

I have tested it with only one route-map statement with recursive next-hop, I shut down the interface which was the next-hop and here is what I got:

Mar  1 04:35:55.830: IP: s=1.1.1.1 (local), d=22.22.22.22, len 100, policy match
*Mar  1 04:35:55.834: IP: route map test, item 10, permit
*Mar  1 04:35:55.834: IP: s=1.1.1.1 (local), d=22.22.22.22, len 100, policy rejected -- normal forwarding.

Regards

Alain

Don't forget to rate helpful posts.

[tamimkushjar]

Hi Alain,

Yes so we can conclude that the traffic is going always through the  route-map of PBR no matter if the PBR link is valid or not meaning that the recursive-next-hop seen or not....

And now for the next step where does the traffic go to?

Should you expect the traffic to go over the default WAN interface or PBR WAN interface?

This is what I got during my test.....

1) I can see my traffic is going to the PBR ACL which is set on my inbound LAN interface

nm#clear ip access-list counters

nm#sh ip access-lists traffic-offload-acl-Cher

Extended IP access list traffic-offload-acl-Cher

    7 permit tcp host 172.25.32.194 host 172.27.96.10 eq www (381 matches)

    8 permit tcp host 172.25.32.194 host 172.27.96.10 eq 8080

    9 permit tcp host 172.25.32.194 host 172.27.96.10 eq 1352

2)  When performing a debug  I see the following output on that same router

nm#debug ip policy dynamic

Dynamic PBR debugging is on

nm#ter mon

Nov 12 19:36:52.301 UTC: IP: s=172.25.32.194 (GigabitEthernet0/1), d=172.27.96.10, len 48, FIB policy match

Nov 12 19:36:52.301 UTC: IP: s=172.25.32.194 (GigabitEthernet0/1), d=172.27.96.10, g=134.222.194.1, len 48, FIB policy routed

Nov 12 19:36:52.301 UTC: IP: s=172.25.32.194 (GigabitEthernet0/1), d=172.27.96.10, len 48, FIB policy match

Nov 12 19:36:52.301 UTC: IP: s=172.25.32.194 (GigabitEthernet0/1), d=172.27.96.10, g=134.222.194.1, len 48, FIB policy routed

Nov 12 19:36:52.301 UTC: IP: s=172.25.32.194 (GigabitEthernet0/1), d=172.27.96.10, len 48, FIB policy match

and sometimes I see the following but the bottom line is that my ping test are not succesfull! when disabling the PBR link on the other end....

Nov 12 19:36:52.305 UTC: IP: s=172.25.32.194 (GigabitEthernet0/1), d=172.27.96.10, g=134.222.194.1, len 48, FIB policy routed

Nov 12 19:36:52.305 UTC: IP: s=172.25.32.194 (GigabitEthernet0/1), d=172.27.96.10, len 48, FIB policy match, len 40, FIB policy rejected(no match) - normal forwarding

3) Also when I tried to find out to which WAN interface my packets were leading,  I didn't see any packet increase on both the outbound ACL of my WAN interfaces....

nm#sh ip access-lists TESTOFFLOAD-out

Extended IP access list TESTOFFLOAD-out

    10 permit tcp host 172.25.32.194 host 172.27.96.10 eq www

    20 permit tcp host 172.25.32.194 host 172.27.96.10 eq 8080

    30 permit tcp host 172.25.32.194 host 172.27.96.10 eq 1352

    40 permit ip any any (747 matches)

nm#sh ip access-lists TESTdefault-out

Extended IP access list TESTdefault-out

    10 permit tcp host 172.25.32.194 host 172.27.96.10 eq www

    20 permit tcp host 172.25.32.194 host 172.27.96.10 eq 8080

    30 permit tcp host 172.25.32.194 host 172.27.96.10 eq 1352

    40 permit ip any any (17496 matches)

So I wonder where my traffic is going since I don't see anything increase are you able to verify if you actually see your traffic going over the WAN interface?

[paul driver]

Hello

Then I would expect route would try to go via the next hop route path dictated by the routing protocol and then the next-hop interface.

So in this case it looks like Cadet Alain post is from 2-4 in the below sequence.

1. Next-hop = Attempts PBR first

2. Next-hop recursive = Attempts PBR first

3. Interface = Attempts PBR first

4. Default next-hop = checks rib first

5. Default interface = check rib first

what's your routing table showing regards the next hop for these routes?

Paul

[tamimkushjar]

Hi Paul,

Well my routing table showing nothing once the BGP is timed out.

Maybe I should go 1 step back in order to clear out something.

A few monthns ago I also had opened a topic that PBR is not working continously as it should or missing packets,  maybe you can check my history posts for that in order to clear out the situation for you...

https://supportforums.cisco.com/message/3985581#3985581

So during that time I found out that my recursive-next-hop was learned via the Default as well as the PBR WAN interfaces and when I tried to check the routing table for that specific recurrsive-next-hop route I noticed it continiously was resetting, a normal route would have stay stable for at least couple of hours but these routes where reset every 10 minutes or something like that.  Then I thought maybe I should block my recursive next hop addresses to go out the default WAN interfaces. And that solved the problem but I think maybe blocking the next-hop routes are working now against me...or maybe it's just coincidence

[tamimkushjar]

just a sidenote before my last change 5months ago, the PBR mechanism went flawless!

[paul driver]

Hello

It seems to me reading you posts that the root cause of this issue is the reclusive next hop address and how the router is seeing it valid path towards it, however saying that you say this what a stable setup 5 months back and after some amendments you started having issues.

What were the additional rules you implemented?

Have you tried specifying multiple next hop ip addresses instead and set the verify-availability option also to track these next hop address for added stability?

res

Paul

Please don't forget to rate any posts that have been helpful.

Thanks.

[tamimkushjar]

Hi Paul,

I am not sure what you mean with the root cause or which root cause you refer to?

The only thing what I did 5months ago in order to stabilize the PBR mechanism (meaning letting it work continuously instead occasionally) is adding a route-map in BGP, in order to prevent the recursive-next-hop to be advertised over the default WAN interface. Which made sure that recursive-next-hops to stay stable.

And before the change I also used recursive-next-hop

The reason why I use recursive-next-hop, is because my next-hop neighbor is not directly adjacent to me. So basically the recursive-next-hop is making sure that 2 routers are aware of each other even if they are not directly connected with each other.

next-hop can only be used if your neighbor is in the same subnet which is not the case in my situation and the only option is to use recursive-next-hop.

If my next-hop would be directly adjacent to me than I would use next-hop instead, at least that is my understanding between those 2 forms of next hop.

And yes I even tried reachability tracking and insertion of a default route but nothing helped here as well.

               ---------------

router1<                                         >router2

               -----------------