Multi-vrf and policy-based routing on Catalyst 3750

[net-harry]

Hi,

We are trying to implement multi-vrf and policy based routing on a Catalyst 3750-X. We have tested the same configuration on a Catalyst 6500 where it works without problems.

However, on the Catalyst 3750 if we try the following configuration and apply "ip policy route-map VPN-TEST" to the interface we get the error message "%PLATFORM_PBR-3-UNSUPPORTED_RMAP: Route-map VPN-TEST not supported for Policy-Based Routing".

route-map VPN-TEST permit 10
match ip address VPN-TEST
set vrf VRF1

If we instead use the "set ip vrf" command in the route-map we do not get any error message and the route-map seems to be applied successfully.

route-map VPN-TEST permit 10
match ip address VPN-TEST
set ip vrf VRF1 next-hop 172.17.1.1

However, when we do "debug ip policy" we get the the following log entries, that indicate that the normal routing table is used instead:

IP: s=172.16.99.11 (Vlan221), d=192.168.1.211, len 92, policy match
IP: route map VPN-TEST, item 10, permit
IP: s=172.16.99.11 (Vlan221), d=192.168.1.211, len 92, policy rejected -- normal forwarding

Strangely, if we set the default router of the global routing table to a non-reachable IP address or we do not define a default gateway at all, PBR seems to work fine and traffic follows the desired path.

Does anyone have any clues why we have this behaviour and if there is a good solution to fix it?

Please note that we have changed the sdm prefer to routing and are running the ip services image 12.2(58)SE2.

Thanks in advance for your help!

Best regards,

Harry

[Giuseppe Larosa]

Hello Harry,

even if you have the right SDM template the platform has some limitations

see unspupported commands in your image at:

http://www.cisco.com/en/US/docs/switches/lan/catalyst3750/software/release/12.2_58_se/configuration/guide/swuncli.html#wp1093298

you should review your design to see if you can avoid to use PBR to send traffic in the VRF

PBR based VRF selection has been introduced on cisco 12000 first

http://www.cisco.com/en/US/docs/ios/mpls/configuration/guide/mp_vpn_vrf_select_ip.html#wp1078057

later it has been ported to SW based routers down  to C7200

see feature navigator

www.cisco.com/go/fn

search by feature

MPLS VPN - VRF Selection based on Source IP Address

it is not supported on C3750

Hope to help

Giuseppe

[net-harry]

Hi Giuseppe,

Thank you very much for the information!

However, in the list of unsupported commands I only see the following route-map commands unsupported:

match route-type for policy-based routing (PBR)

set as-path {tag | prepend as-path-string}

set automatic-tag

set dampening half-life reuse suppress max-suppress-time

set default interface interface-id [interface-id.....]

set interface interface-id [interface-id.....]

set ip default next-hop ip-address [ip-address.....]

set ip destination ip-address mask

set ip next-hop verify-availability

set ip precedence value

set ip qos-group

set metric-type internal

set origin

set metric-type internal

set tag tag-value

The two set commands "set vrf" and "set ip vrf NAME next-hop ADDRESS" are not included here. As I mentioned I can see the log message that "set vrf" is not supported, but the other command does not give any error messages and strangely we are able to get the VRF selection through the route-map working perfectly when we do not have a reachable default route in the global routing table. When we add the default route we again get the "policy rejected -- normal forwarding" message.

I was hoping that someone else had seen this issue and had a workaround to get it working!

Best regards,

Harry

[net-harry]

Hi again,

After further troubleshooting we discovered that the policy-based routing is working perfectly as long as the IP address that we are trying to reach does not exist in the global routing table.

So with the configuration below if we are trying to reach 10.10.10.10 from 172.16.1.10 (through vlan 221) it will be forwarded to vrf VRF1 as long as no ip route (including the default) that covers 10.10.10.10 exists in the global routing table.

ip access-list extended VRF1

permit ip 172.16.1.0 0.0.0.255 any

route-map VPN-TEST permit 10
match ip address VRF1
set ip vrf VRF1 next-hop 172.17.1.1

interface Vlan221

ip address 192.168.1.1 255.255.255.0

ip policy route-map VPN-TEST

Ideally, the PBR should be independent on whether the route exists in the global routing table or not.

Does anyone know if this is a bug or if it is just because PBR with "set ip vrf" is not fully supported in version 12.2(58)SE2?

Best regards,

Harry

[Giuseppe Larosa]

Hello Harry,

according to FN the feature is not  supported on the platform, your findings are interesting from a technical point of view, but MPLS VPN selection based on source address per feature navigator is not supported on C3750.

It may be an error in  feature navigator documentation this sometimes happens, but notice that not even C6500 is listed in the supported platforms and this is meaningful.

So you need to take a decision on how to go on:

- keep the current design knowing that if something happens in the future Cisco might tell you in a TAC case that it is not supported ufficially  ( dangerous isn't it ?)

- perform a design review to see if you can avoid to use this PBR

Keeping the traffic within a VRF is for sure supported on the C3750.

Hope to help

Giuseppe

[net-harry]

Hi Giuseppe,

Thanks for your answer!

The reason why we need the policy-based routing is that we need to seperate customers coming in through a shared VPN solution and based on their source address send them to the correct VRF.

If we cannot get this working on a Catalyst 3750 I will try to open a TAC case to see what Cisco says and if necessary use a router or higher end switch instead of the 3750.

Please note that we are not running MPLS VPN, but instead VRF-lite.

Best regards,

Harry

[Marwan ALshawi]

if you are runing VRF-lite then the next hope is router per VRF subinterface maybe

you can do PBR based on the rourse and send it to the respective router subinterface in that router you can have multiple VRFs associate with GRE tunnel interfaces per VRF ( logical separation )

VPN tunnel ----- Switch VLan interface with PBR set nex hope 1.1.1.1  ------ router suinterface 1.1.1.1 VRF1 tunnel1 VRF1--------other end of the tunnel VRF1

HTH

[Giuseppe Larosa]

Hello Harry,

yes VRF lite I agree

However, if the different customers need to reach different destinations in different VRFs you might be able to use an extranet making the Shared VRF to exchange routes with other VRFs in a controlled way using an export-map.

this means going via MP BGP on the device itself and playing with route targets

I did this in the past in SW based routers I'm not sure this can work on C3750

example:

access-list 21 permit 192.168.11.0 0.0.0.255

access-list 22 permit 192.168.12.0 0.0.0.255

route-map partial-extranet permit  10

match ip address 21

set ip extcommunity rt 2:1

route-map partial-extranet permit  20

match ip address 22

set ip extcommunity rt 2:2

.....

route-map partial-extranet permit  2000

!

ip vrf SHARED

export-map partial-extranet

route-target import 2:1

route-target import 2:2

....

see

http://www.cisco.com/en/US/docs/ios/mpls/command/reference/mp_e1.html#wp1012602

you need also the BGP configuration with per VRF address-families in order to use route targets

this creates also the capability to forward traffic.

Hope to help

Giuseppe