PBR ECMP using Recursive Next Hop on Catalyst 3650

RameshPVN8915 · ‎08-02-2019

Hi,

I was trying to do PBR ECMP on Cat3650 using recursive next hop and running into issues that the RouteMap has unsupported options. Software Version was 3.7.5E.

I created a small topology of 2 L3 Ports (20.1.1.1/30 20.1.1.5/30) having one L3 next hop (20.1.1.2, 20.1.1.6) each on them (directly connected).

I created a recursive next hop 30.1.1.1 which is reachable via 20.1.1.2 and 20.1.1.6 (equal cost). The route-map has set ip next hop recursive as 30.1.1.1 and I expect traffic to be load balanced between the 2 next hops.

This is the switch:-

Switch Ports Model SW Version SW Image Mode

------ ----- ----- ---------- ---------- ----

* 1 28 WS-C3650-24PD 03.07.05E cat3k_caa-universalk9 INSTALL

This is the syslog seen :-

*Aug 2 05:58:16.207: %PLATFORM_PBR-3-UNSUPPORTED_RMAP: Route-map pbrmap2 has unsupported options for Policy-Based Routing. It has been removed from interface, if applied.

My question is whether its not supported by this platform or in this software release in this platform.

Or am I doing fundamentally something wrong here :-). Any pointers will help.

Thanks,

Ramesh

RameshPVN8915 · ‎08-02-2019

Here is the output from switch:

cisco1#sh route-map pbrmap2

route-map pbrmap2, permit, sequence 10

Match clauses:

ip address (access-lists): pbr1

Set clauses:

ip next-hop recursive 30.1.1.1

Nexthop tracking current: 30.1.1.1

30.1.1.1, fib_nh:3CD08684,oce:38175DF4,status:1

Policy routing matches: 0 packets, 0 bytes

cisco1#sh ip route 30.1.1.1 255.255.255.255

Routing entry for 30.1.1.1/32

Known via "static", distance 1, metric 0

Routing Descriptor Blocks:

* 20.1.1.6

Route metric is 0, traffic share count is 1

20.1.1.2

Route metric is 0, traffic share count is 1

cisco1#

cisco1#sh run

cisco1#sh running-config in

cisco1#sh running-config interface vl

cisco1#sh running-config interface vlan 10

Building configuration...

Current configuration : 60 bytes

!

interface Vlan10

ip address 10.10.1.1 255.255.255.0

end

cisco1#sh logging

*Aug 2 17:11:20.433: PBR Nexthop Callback invoked: 3A5FA628, (30.1.1.1) tableid 0, status: 2,type: SET NEXTHOP RECURSIVE

*Aug 2 17:11:20.433: map: pbrmap2, sequence: 10

PBR Control Plane Notification: 30.1.1.1 PBR_CP_SET_NEXTHOP_RECURSIVE

*Aug 2 17:11:20.433: PBR CP Notification sent: Type:SET NEXTHOP RECURSIVE, 30.1.1.1SW_OBJ_TYPE: 1D, SW_HANDLE: 3D2D41B0

*Aug 2 17:11:26.899: %LINK-3-UPDOWN: Interface GigabitEthernet1/0/10, changed state to up

*Aug 2 17:11:27.897: %LINEPROTO-5-UPDOWN: Line protocol on Interface GigabitEthernet1/0/10, changed state to up

*Aug 2 17:11:27.902: PBR Nexthop Callback invoked: 3A5FA628, (30.1.1.1) tableid 0, status: 2,type: SET NEXTHOP RECURSIVE

*Aug 2 17:11:27.902: map: pbrmap2, sequence: 10

PBR Control Plane Notification: 30.1.1.1 PBR_CP_SET_NEXTHOP_RECURSIVE

*Aug 2 17:11:27.903: PBR CP Notification sent: Type:SET NEXTHOP RECURSIVE, 30.1.1.1SW_OBJ_TYPE: 15, SW_HANDLE: 38175DF4

*Aug 2 17:14:17.542: %PLATFORM_PBR-3-UNSUPPORTED_RMAP: Route-map pbrmap2 has unsupported options for Policy-Based Routing. It has been removed from interface, if applied.

Giuseppe Larosa · ‎08-02-2019

Hello Ramesh,

the feature looks like unsupported on your switch with your current software image.

Check the current SDM template (the way the TCAM is used on the switch ) using

show sdm prefer

to see if there is a more advanced routing SDM template that could be used.

However, for the way a multilayer switch works PBR can work in hardware only if the TCAM can support it.

With a standard set ip next-hop command the TCAM can be programmed with a pointer to the next-hop for each CEF entry that will be processed by PBR.

The recursive next-hop option requires more intelligence as the next-hop has to be resolved in multiple ECMP next-hops by recursion. This capability may be supported on higher end switches like C6500, C6800.

We should check with the feature navigator.

In practice you can easily create a workaround for this:

use L3 port-channel with two member links instead of two routed links you can achieve link redundancy and flow based load balancing over the etherchannel the L3 next-hop for PBR will be the other end of the L3 port channel. In this way you don't need to use the recursive option.

Edit:

link to feature navigator

https://cfn.cloudapps.cisco.com/ITDIT/CFN/jsp/by-feature-technology.jsp

I used as filter the word PBR.

There is an entry for PBR support for IPV6 recursive next-hop that points to the following link:

https://www.cisco.com/c/en/us/td/docs/ios-xml/ios/iproute_pi/configuration/xe-3s/iri-xe-3s-book/iri-pbr-rec-next-hop-support.html

IOS XE 3 likely on ASR 1000. However, it provides an explanation of how PBR set next-hop recursive is implemented.

Hope to help

Giuseppe

RameshPVN8915 · ‎08-02-2019

Thanks for your response. I already went through these parts.
The 3650 documentation says the support was added for this feature in 3.6. and I have 3.7 - so it should work and unfortunately it does not.
Will check if the newer catalyst 9000s support it.

https://www.cisco.com/c/en/us/td/docs/ios-xml/ios/iproute_pi/configuration/xe-3e/iri-xe-3e-book/iri-pbr-rec-next-hop-support.pdf