Per-VLAN PBR overriding static routes

[Mario123]

In an older Cisco 3550 I have a bunch of L3 VLAN interfaces (10.100.x.1/24) and the switch has 2 uplinks. VLAN 10 is connected to ISP1  slow) : Vlan10 192.168.198.122 and VLAN 112 goes to ISP2 (fast). Vlan112 192.168.2.2

The default route points to ISP1  S* 0.0.0.0/0 [10/0] via 192.168.198.3

There is a bunch of VMs on a server connected to a trunk port on the switch on these 10.100.x.0/24 subnets and most need to route via ISP1 except a few on 10.100.9.0/24 which need faster connectivity via ISP2. I am trying to override via PBR this default route and it does not seem to work. Configuration very similar to the one here 

https://community.cisco.com/t5/routing/per-vlan-routing/td-p/3071748

Cisco3550-SMI(config)#
ip access-list extended vlan9-to-int
 10 permit ip 10.100.9.0 0.0.0.255 10.0.0.0 0.255.255.255
 20 permit ip 10.100.9.0 0.0.0.255 192.168.0.0 0.0.255.255

ip access-list extended okd-to-ext
 10 permit ip 10.100.9.200 0.0.0.7 any
 20 permit ip 10.100.9.16 0.0.0.7 any

route-map rtmap-okd-ext permit 10
  match ip address vlan9-to-int
  set ip default next-hop
!
route-map rtmap-okd-ext permit 20
  match ip address okd-to-ext
  set ip next-hop 192.168.2.1
!
interface vlan9
  ip address 10.100.9.1 255.255.255.0
  ip policy route-map rtmap-okd-ext

Actually a show route-map rtmap-okd-ext won't show any hits unless I am pinging from a VM on VLAN9 the 10.100.9.1 interface itself.

A traceroute will show that the default route ISP1 is still being used

root@ubu-v24-vlan99-tmp:~# traceroute 8.8.8.8
traceroute to 8.8.8.8 (8.8.8.8), 64 hops max
1 10.100.9.1 0.668ms 2.044ms 2.140ms
2 192.168.198.3 1.136ms 1.084ms 1.149ms

Cisco3550-SMI#sh route-map
route-map rtmap-okd-ext, permit, sequence 10
Match clauses:
ip address (access-lists): vlan9-to-int
Set clauses:
Policy routing matches: 26 packets, 2250 bytes <<== these matches are for "ping 10.100.9.1" from VM 10.100.9.18
route-map rtmap-okd-ext, permit, sequence 20
Match clauses:
ip address (access-lists): okd-to-ext
Set clauses:
ip next-hop 192.168.2.1
Policy routing matches: 0 packets, 0 bytes

I am completely confused, not sure what I am missing here. Cannot get rid of the default route but it seems the route-map is simply not matching for packets originating outside the switch and routed via the VLAN interface

Any help much appreciated

[Richard Burts]

There is much about your situation that we do not know and if we knew more we might be able to give better advice. But based on the little that we do know I offer this observation: The acl used used to identify internal traffic vlan9-to-int matches all source hosts in vlan 9. However the acl used to identify traffic going outside matches only a very few hosts

ip access-list extended oki-to-ext
10 permit ip 10.100.9.200 0.0.0.7 any (matches only 7 hosts)
20 permit ip 10.100.9.16 0.0.0.7 any (matches only 7 hosts)

I suggest that you revise the acl and try again.

[paul driver]

Hello

---

@Mario123 wrote:
There is a bunch of VMs on a server connected to a trunk port on the switch on these 10.100.x.0/24 subnets and most need to route via ISP1 except a few on 10.100.9.0/24 which need faster connectivity via ISP2.

---

Just focusing on the policy routing.

---

By default all traffic will route via ISP1 using the default static route 
So based the above and not including any object tracking for your static default routes a basic ACL/PBR policy should look something like -

route-map rtmap-okd-ext 
match ip address xxxxx
set ip next-hop 192.168.2.1

interface vlan9
ip policy route-map rtmap-okd-ext

Note:  set ip default next-hop   - is ONLY used if/when no valid route exists in the route table.

[Mario123]

The root cause of the problem seems to be the fact that "ip policy route-map" on a vlan interface although accepted is silently ignored. 

Cisco3550-SMI#conf t
Enter configuration commands, one per line. End with CNTL/Z.
Cisco3550-SMI(config)#int vlan9
Cisco3550-SMI(config-if)#ip policy route-map rtmap-okd-ext
Cisco3550-SMI(config-if)#exit
Cisco3550-SMI(config)#do sh run int vlan9
Building configuration...

Current configuration : 60 bytes
!
interface Vlan9
ip address 10.100.9.1 255.255.255.0
end

This is bizarre . The "match" counters actually increase only when the route-map is applied with ip local policy route-map and pinging the VLAN interface itself.  Otherwise the route-map setting on interface won't even appear in a sh run.

[paul driver]

Hello
As suggested by @Richard Burts you need to review the match acl

[Richard Burts]

I still maintain that the logic of the ACL is flawed. But @Mario123 presents an interesting aspect in the recent post, which seems to show that the policy based routing was not present on the interface. But the OP clearly shows PBR was working

Cisco3550-SMI#sh route-map
route-map rtmap-okd-ext, permit, sequence 10
Match clauses:
ip address (access-lists): vlan9-to-int
Set clauses:
Policy routing matches: 26 packets, 2250 bytes

So I am quite confused. If we are to sort this out we need more/better information from the OP.

[Mario123]

The origin of 26 matches is the test done with "ip local policy route-map . . . " and pinging the vlan interface itself. This worked.

But ip policy route-map on a vlan interface definitely seems to be silently ignored (being an old IOS 12.2 ipservices might be "normal" to hit such a bug) Might need totry on unetlab/eve-ng another image but this would not address the real issue.

Cisco3550-SMI#conf t
Enter configuration commands, one per line. End with CNTL/Z.
Cisco3550-SMI(config)#int vlan9
Cisco3550-SMI(config-if)#ip policy route-map vlan9-test
Cisco3550-SMI(config-if)#end
Cisco3550-SMI#sh run int vlan9
!
interface Vlan9
ip address 10.100.9.1 255.255.255.0
end

Cisco3550-SMI#sh ip policy
Interface Route map

Cisco3550-SMI#sh route-map vlan9-test
route-map vlan9-test, permit, sequence 10
Match clauses:
ip address prefix-lists: Vlan9
Set clauses:
ip next-hop 192.168.2.1
Policy routing matches: 0 packets, 0 bytes

!
Cisco3550-SMI#sh ip policy
Interface Route map

Cisco3550-SMI#

It is an old box, agree, but still ipservices and 12.2 ios

Cisco3550-SMI#sh vers
Cisco IOS Software, C3550 Software (C3550-IPSERVICESK9-M), Version 12.2(55)SE7, RELEASE SOFTWARE (fc1)
Technical Support: http://www.cisco.com/techsupport
Copyright (c) 1986-2013 by Cisco Systems, Inc.
Compiled Mon 28-Jan-13 11:46 by prod_rel_team
Image text-base: 0x00003000, data-base: 0x01606C54

ROM: Bootstrap program is C3550 boot loader

Cisco3550-SMI uptime is 32 weeks, 8 hours, 7 minutes
System returned to ROM by power-on
System restarted at 12:50:17 UTC Sat Oct 5 2024
System image file is "flash:c3550-ipservicesk9-mz.122-55.SE7/c3550-ipservicesk9-mz.122-55.SE7.bin"

. . .

Cisco3550-SMI#show sdm prefer
The current template is the default template.
The selected template optimizes the resources in
the switch to support this level of features for
8 routed interfaces and 1K VLANs.

number of unicast mac addresses: 5K
number of igmp groups: 1K
number of qos aces: 1K
number of security aces: 1K
number of unicast routes: 8K
number of multicast routes: 1K

The template stored for use after the next reload
is the routing template.
The selected template optimizes the resources in
the switch to support this level of features for
8 routed interfaces and 1K VLANs.

number of unicast mac addresses: 5K
number of igmp groups: 1K
number of qos aces: 512
number of security aces: 512
number of unicast routes: 16K
number of multicast routes: 1K