topic Re: PBR doesn't work on Catalyst 9300 ( VLAN Interface) in Routing and SD-WAN

PBR doesn't work on Catalyst 9300 ( VLAN Interface)

Ryan YC — Sat, 11 May 2019 16:06:53 GMT

Hi Guys,

I'm suffering a problem that about PBR and VLAN Interface on the Catalyst C9300-24UX, Cisco IOS XE Software, Version 16.06.03.

My question is, I created a route-map and access-list as my PBR configuration, and then applied on VLAN 1, but the PBR seems doesn't work, I even didn't see any hit count when I show route-map and didn't see the relative messages from the 'debug ip policy'

---------------The following is the part of configurations -----------------

!
interface Vlan1
description Server network subnet
ip address 10.99.99.10 255.255.255.240 secondary
ip address 192.168.34.252 255.255.255.0
ip policy route-map PBR_testing
!

access-list 110 permit ip 10.99.99.0 0.0.0.255 192.168.56.0 0.0.0.255
!
route-map PBR_testing permit 10
match ip address 110
set ip next-hop 192.168.34.245

---------------The following is my topology -----------------

The purpose that I want to make the client (on the left bottom side) can access the server (on the right bottom side). I'm sure the routing table has the PBR's next-hop existing.

Please help me if you have any idea, please. I spent almost two days trying to solve this issue.

Re: PBR doesn't work on Catalyst 9300 ( VLAN Interface)

Georg Pauwen — Sat, 11 May 2019 18:19:50 GMT

Hello,

post the full configuration of the switch. Do you have 'ip routing' enabled ?

Re: PBR doesn't work on Catalyst 9300 ( VLAN Interface)

Ryan YC — Sat, 11 May 2019 18:57:17 GMT

Please let me know if any information is needed for this troubleshooting, thank you!

------------------------------- The following is the configuration ---------------------------------
!
version 16.6
no service pad
service timestamps debug datetime localtime show-timezone
service timestamps log datetime localtime show-timezone
service password-encryption
no platform punt-keepalive disable-kernel-core
!
hostname xxxxxxx
!
!
vrf definition Mgmt-vrf
!
address-family ipv4
exit-address-family
!
address-family ipv6
exit-address-family
!
enable secret 5 xxxxxxxxxxxxxxxx
!
aaa new-model
!
!
aaa authentication login default local
aaa authentication enable default enable
aaa authorization exec default local
!
!
!
!
!
!
!
ip routing
!
!
!
!
!
!
!
!
!
!
!
cpp system-default
!
!
diagnostic bootup level minimal
!
spanning-tree mode pvst
spanning-tree extend system-id
file privilege 5
!
username admin secret 5 xxxxxxxxxxxxxxxxxx
!
redundancy
mode sso
!
lldp run
!
!
!
!
!
!
!
!
!
!
!
!
!
!
!
interface Port-channel1
switchport mode access
!
interface Port-channel2
switchport mode access
!
interface Port-channel3
switchport mode access
!
interface Port-channel4
switchport mode trunk
!
interface Port-channel5
switchport mode trunk
!
interface Port-channel6
switchport mode trunk
!
interface Port-channel7
switchport mode trunk
shutdown
!
interface Port-channel8
switchport mode access
!
interface Port-channel9
switchport mode access
!
interface Port-channel12
switchport mode trunk
!
interface Port-channel16
switchport mode access
!
interface Port-channel18
switchport mode access
!
interface Port-channel19
switchport mode trunk
!
interface GigabitEthernet0/0
vrf forwarding Mgmt-vrf
no ip address
shutdown
speed 1000
negotiation auto
!
interface TenGigabitEthernet1/0/1
switchport mode access
channel-group 1 mode on
!
interface TenGigabitEthernet1/0/2
switchport mode trunk
!
interface TenGigabitEthernet1/0/3
switchport mode access
channel-group 2 mode on
!
interface TenGigabitEthernet1/0/4
switchport mode trunk
channel-group 6 mode on
!
interface TenGigabitEthernet1/0/5
switchport mode access
channel-group 3 mode on
!
interface TenGigabitEthernet1/0/6
switchport mode trunk
channel-group 19 mode on
!
interface TenGigabitEthernet1/0/7
switchport mode trunk
channel-group 4 mode on
!
interface TenGigabitEthernet1/0/8
switchport mode trunk
!
interface TenGigabitEthernet1/0/9
switchport mode access
!
interface TenGigabitEthernet1/0/10
switchport mode trunk
shutdown
channel-group 7 mode on
!
interface TenGigabitEthernet1/0/11
switchport mode access
channel-group 8 mode on
!
interface TenGigabitEthernet1/0/12
switchport mode access
channel-group 9 mode on
!
interface TenGigabitEthernet1/0/13
switchport mode access
channel-group 18 mode on
!
interface TenGigabitEthernet1/0/14
switchport mode access
channel-group 18 mode on
!
interface TenGigabitEthernet1/0/15
switchport mode trunk
channel-group 12 mode on
!
interface TenGigabitEthernet1/0/16
switchport mode trunk
!
interface TenGigabitEthernet1/0/17
switchport mode access
channel-group 16 mode on
!
interface TenGigabitEthernet1/0/18
switchport mode access
channel-group 16 mode on
!
interface TenGigabitEthernet1/0/19
switchport mode access
spanning-tree portfast
!
interface TenGigabitEthernet1/0/20
switchport mode access
!
interface TenGigabitEthernet1/0/21
switchport mode trunk
!
interface TenGigabitEthernet1/0/22
service-policy output IP_Precedence_For_MPLS_QoS
!
interface TenGigabitEthernet1/0/23
switchport mode access
!
interface TenGigabitEthernet1/0/24
switchport mode access
!
interface GigabitEthernet1/1/1
!
interface GigabitEthernet1/1/2
!
interface GigabitEthernet1/1/3
!
interface GigabitEthernet1/1/4
!
interface TenGigabitEthernet1/1/1
switchport mode trunk
load-interval 30
channel-group 5 mode on
!
interface TenGigabitEthernet1/1/2
!
interface TenGigabitEthernet1/1/3
!
interface TenGigabitEthernet1/1/4
!
interface TenGigabitEthernet1/1/5
!
interface TenGigabitEthernet1/1/6
!
interface TenGigabitEthernet1/1/7
!
interface TenGigabitEthernet1/1/8
!
interface FortyGigabitEthernet1/1/1
!
interface FortyGigabitEthernet1/1/2
!
interface TenGigabitEthernet2/0/1
switchport mode access
channel-group 1 mode on
!
interface TenGigabitEthernet2/0/2
switchport mode trunk
!
interface TenGigabitEthernet2/0/3
switchport mode access
channel-group 2 mode on
!
interface TenGigabitEthernet2/0/4
switchport mode trunk
channel-group 6 mode on
!
interface TenGigabitEthernet2/0/5
switchport mode access
channel-group 3 mode on
!
interface TenGigabitEthernet2/0/6
switchport mode trunk
channel-group 19 mode on
!
interface TenGigabitEthernet2/0/7
switchport mode trunk
channel-group 4 mode on
!
interface TenGigabitEthernet2/0/8
switchport mode access
spanning-tree portfast
spanning-tree bpduguard enable
!
interface TenGigabitEthernet2/0/9
switchport mode access
shutdown
!
interface TenGigabitEthernet2/0/10
switchport mode trunk
shutdown
channel-group 7 mode on
!
interface TenGigabitEthernet2/0/11
switchport mode access
shutdown
channel-group 8 mode on
!
interface TenGigabitEthernet2/0/12
switchport mode access
shutdown
channel-group 9 mode on
!
interface TenGigabitEthernet2/0/13
switchport mode access
shutdown
channel-group 18 mode on
!
interface TenGigabitEthernet2/0/14
switchport mode access
shutdown
channel-group 18 mode on
!
interface TenGigabitEthernet2/0/15
switchport mode trunk
channel-group 12 mode on
!
interface TenGigabitEthernet2/0/16
switchport mode access
shutdown
!
interface TenGigabitEthernet2/0/17
switchport mode access
shutdown
channel-group 16 mode on
!
interface TenGigabitEthernet2/0/18
switchport mode access
shutdown
channel-group 16 mode on
!
interface TenGigabitEthernet2/0/19
switchport mode access
spanning-tree portfast
!
interface TenGigabitEthernet2/0/20
switchport mode access
shutdown
!
interface TenGigabitEthernet2/0/21
switchport mode trunk
shutdown
!
interface TenGigabitEthernet2/0/22
description DO_NOT_CONFIGURE
shutdown
!
interface TenGigabitEthernet2/0/23
switchport mode access
spanning-tree portfast
!
interface TenGigabitEthernet2/0/24
switchport mode trunk
spanning-tree portfast
!
interface GigabitEthernet2/1/1
!
interface GigabitEthernet2/1/2
!
interface GigabitEthernet2/1/3
!
interface GigabitEthernet2/1/4
!
interface TenGigabitEthernet2/1/1
switchport mode trunk
load-interval 30
channel-group 5 mode on
!
interface TenGigabitEthernet2/1/2
shutdown
!
interface TenGigabitEthernet2/1/3
shutdown
!
interface TenGigabitEthernet2/1/4
switchport mode access
shutdown
!
interface TenGigabitEthernet2/1/5
shutdown
!
interface TenGigabitEthernet2/1/6
shutdown
!
interface TenGigabitEthernet2/1/7
shutdown
!
interface TenGigabitEthernet2/1/8
shutdown
!
interface FortyGigabitEthernet2/1/1
!
interface FortyGigabitEthernet2/1/2
!
interface Vlan1
description Server network subnet
ip address 10.99.99.10 255.255.255.240 secondary
ip address 192.168.34.252 255.255.255.0
ip policy route-map PBR_testing

!
ip forward-protocol nd
ip http server
ip http authentication local
ip http secure-server
ip route 0.0.0.0 0.0.0.0 192.168.34.245
ip route 10.0.0.0 255.255.255.0 192.168.34.245
ip route 10.40.0.0 255.255.0.0 192.168.34.245
ip route 10.40.10.0 255.255.254.0 192.168.34.245
ip route 10.75.254.0 255.255.255.0 192.168.34.245
ip route 10.75.255.0 255.255.255.0 192.168.34.245
ip route 192.168.0.0 255.255.224.0 192.168.34.247
ip route 192.168.52.0 255.255.252.0 192.168.34.247
ip route 192.168.56.0 255.255.255.0 192.168.34.247
ip route 192.168.128.0 255.255.128.0 192.168.34.247
ip route 172.24.16.0 255.255.252.0 192.168.34.245
ip route 172.24.20.0 255.255.255.0 192.168.34.245
ip route 172.24.74.11 255.255.255.255 192.168.34.245
ip route 192.168.2.32 255.255.255.240 192.168.34.247
!
ip ssh version 2
!
access-list 110 permit ip 10.99.99.0 0.0.0.255 192.168.56.0 0.0.0.255
!
route-map PBR_testing permit 10
match ip address 110
set ip next-hop 192.168.34.245
!
!
!
control-plane
!
!
line con 0
exec-timeout 30 0
logging synchronous
stopbits 1
line vty 0 4
exec-timeout 60 0
logging synchronous
transport input all
line vty 5 15
exec-timeout 60 0
logging synchronous
transport input all
!
ntp server 192.168.34.12
!
!
end

Re: PBR doesn't work on Catalyst 9300 ( VLAN Interface)

Georg Pauwen — Sat, 11 May 2019 21:38:08 GMT

Re: PBR doesn't work on Catalyst 9300 ( VLAN Interface)

Georg Pauwen — Sat, 11 May 2019 21:48:21 GMT

Hello,

The wildcard used in your access list 110 does not match the subnet mask of the secondary IP address on the Vlan interface. Try and change it to:

access-list 110 permit ip 10.99.99.0 0.0.0.15 192.168.56.0 0.0.0.255

Re: PBR doesn't work on Catalyst 9300 ( VLAN Interface)

Ryan YC — Sun, 12 May 2019 08:57:28 GMT

Hi Georg,

I changed the wildcard mask in the access list, but the issue still same 😞

This's my current only ACL in the route-map on the Interface VLAN 1:

'access-list 110 permit ip 10.99.99.0 0.0.0.15 192.168.56.0 0.0.0.255'

However after I applied the route-map onto the Interface, I can see a lot of messages from 'debug ip policy' those are reporting:

May 12 10:48:59 UTC: IP: s=192.168.34.76 (Vlan1), d=192.168.34.255, len 692, policy rejected -- normal forwarding
May 12 10:48:59 UTC: IP: s=192.168.34.15 (Vlan1), d=192.168.34.255, len 78, policy rejected -- normal forwarding
May 12 10:49:00 UTC: IP: s=192.168.34.15 (Vlan1), d=192.168.34.255, len 78, policy rejected -- normal forwarding
May 12 10:49:01 UTC: IP: s=192.168.34.20 (Vlan1), d=192.168.34.255, len 229, policy rejected -- normal forwarding
May 12 10:49:01 UTC: IP: s=192.168.34.15 (Vlan1), d=192.168.34.255, len 78, policy rejected -- normal forwarding
May 12 10:49:02 UTC: IP: s=192.168.34.15 (Vlan1), d=192.168.34.255, len 78, policy rejected -- normal forwarding
May 12 10:49:02 UTC: IP: s=192.168.34.15 (Vlan1), d=192.168.34.255, len 78, policy rejected -- normal forwarding
May 12 10:49:03 UTC: IP: s=192.168.34.15 (Vlan1), d=192.168.34.255, len 78, policy rejected -- normal forwarding
May 12 10:49:04 UTC: IP: s=192.168.34.15 (Vlan1), d=192.168.34.255, len 78, policy rejected -- normal forwarding
May 12 10:49:05 UTC: IP: s=192.168.34.15 (Vlan1), d=192.168.34.255, len 78, policy rejected -- normal forwarding
May 12 10:49:06 UTC: IP: s=192.168.34.15 (Vlan1), d=192.168.34.255, len 78, policy rejected -- normal forwarding
May 12 10:49:07 UTC: IP: s=192.168.34.15 (Vlan1), d=192.168.34.255, len 78, policy rejected -- normal forwarding
May 12 10:49:07 UTC: IP: s=192.168.34.15 (Vlan1), d=192.168.34.255, len 78, policy rejected -- normal forwarding
May 12 10:49:08 UTC: IP: s=192.168.34.15 (Vlan1), d=192.168.34.255, len 78, policy rejected -- normal forwarding
May 12 10:49:08 UTC: IP: s=192.168.34.35 (Vlan1), d=192.168.34.255, len 229, policy rejected -- normal forwarding
May 12 10:49:08 UTC: IP: s=192.168.34.51 (Vlan1), d=192.168.34.255, len 78, policy rejected -- normal forwarding
May 12 10:49:08 UTC: IP: s=192.168.34.51 (Vlan1), d=192.168.34.255, len 78, policy rejected -- normal forwarding
May 12 10:49:09 UTC: IP: s=192.168.34.15 (Vlan1), d=192.168.34.255, len 78, policy rejected -- normal forwarding
May 12 10:49:10 UTC: IP: s=192.168.34.15 (Vlan1), d=192.168.34.255, len 78, policy rejected -- normal forwarding

I was continuing to ping a server in 192.168.56.0/24 subnet by using 10.99.99.1, but I didn't see any message about 10.99.99.0 subnet on the Core switch.

Re: PBR doesn't work on Catalyst 9300 ( VLAN Interface)

Ryan YC — Sun, 12 May 2019 09:11:22 GMT

Re: PBR doesn't work on Catalyst 9300 ( VLAN Interface)

Giuseppe Larosa — Sun, 12 May 2019 09:28:35 GMT

Hello Yung,

from the output that you have provided we can say the PBR is configured and actually skipped for all source addresses in the primary IP subnet.

Could you try to use a test in which you match a specific IP address in the primary subnet

try to use an ACL like the following

access-list 111 permit ip 192.168.34.x 0.0.0.0 192.168.56.0 0.0.0.255

I wonder if the issue is related to secondary addresses.

In old times to make PBR working in some multilayer platforms we needed to change the SDM template. I don't think it is your case as the debug lines show that the PBR is considered before using normal forwarding.

Hope to help

Giuseppe

Re: PBR doesn't work on Catalyst 9300 ( VLAN Interface)

Georg Pauwen — Sun, 12 May 2019 11:25:27 GMT

Hello,

what if you make the secondary the primary and the primary the secondary IP address on your Windows NIC ?

Re: PBR doesn't work on Catalyst 9300 ( VLAN Interface)

Sebastian Fernandez — Mon, 13 May 2019 01:54:25 GMT

Hi @Ryan YC

Your PBR configuration is correct and should work appropriately. The only think that I can think of is packets not getting to the right router. Can you make sure that you are using 10.99.99.10 as the gateway for the host in 10.99.99.0/28 subnet.

Also it would be helpful to apply below ACL on the interface and make sure that packets are indeed received on the right interface.

access-list 199 permit icmp host 10.99.99.1 192.168.56.0 0.0.0.255 log
access-list 199 permit ip any any 
!
interface vlan 1 
 ip access-group 199 in

If the hit count on the first entry is seen and you've verified in logs that the destination is correct. Then PBR should work fine.

*** Please mark this post helpful if it contributed to your query ***

Sebastian

Re: PBR doesn't work on Catalyst 9300 ( VLAN Interface)

pieterh — Mon, 13 May 2019 09:40:49 GMT

for PBR to work, I should expect to have multiple routes, where you want to force/prefer a specific path.

you are not using a routing protocol, and you are using a single static route to access 192.168.56.0 255.255.255.0 network on the 9300.

also you need to check both MPLS and VPN routers know the route to the 192.168.56.0 subnet?

Re: PBR doesn't work on Catalyst 9300 ( VLAN Interface)

Alex Pfeil — Mon, 12 Aug 2019 20:25:54 GMT

Were you able to get this issue resolved? I am seeing a similar problem.

Re: PBR doesn't work on Catalyst 9300 ( VLAN Interface)

paul driver — Mon, 12 Aug 2019 22:33:06 GMT

Hello
Regards your ping my understanding the primary interface is always used as source even when you have secondary addressing assign to the same interface.

Just for clarity - you really don’t need these static routes as they are captured by the default static route

No ip route 10.0.0.0 255.255.255.0 192.168.34.245
No ip route 10.40.0.0 255.255.0.0 192.168.34.245
No ip route 10.40.10.0 255.255.254.0 192.168.34.245
No ip route 10.75.254.0 255.255.255.0 192.168.34.245
No ip route 10.75.255.0 255.255.255.0 192.168.34.245
No ip route 172.24.16.0 255.255.252.0 192.168.34.245
No ip route 172.24.20.0 255.255.255.0 192.168.34.245

Lastly could you try the following:
no access-list 110 permit ip 10.99.99.0 0.0.0.255 192.168.56.0 0.0.0.255
no route-map PBR

access-list 110 permit ip 192.168.34.0 0.0.0.255 192.168.56.0 0.0.0.255

route-map PBR_testing deny 10
match ip address 110

route-map PBR_testing permit 99
set ip next-hop 192.168.34.245

Re: PBR doesn't work on Catalyst 9300 ( VLAN Interface)

Richard Burts — Wed, 14 Aug 2019 14:39:53 GMT

We have a post that is several months old describing an issue with PBR that received a number of suggestions and has no clear indication of whether the issue was solved or not. I found it odd in the original post that the PBR was specifying the secondary address and no mention of the primary address. And I wonder if the issue was that the device was sending using the primary address and not the secondary address. Now @Alex Pfeil says he has the same issue but provides no details. Perhaps if we get the details we might have some suggestions about his issue.

HTH

Rick