Re: Problem with PBR and static routes

Ercole77 · ‎01-17-2019

Hi all

sorry for this newbie question :-)

I have a C3850 switch (03.06.04.E cat3k_caa-universalk9) and currentlly there is a route map with an ACL

to obtain this result:

ACL permitted networks are routed to 10.242.160.251 as default gateway for Internet and to reach some subnets 10.0.0.0/8

route map:

route-map TO-JPN, permit, sequence 10
Match clauses:
ip address (access-lists): TO-JPN
Set clauses:
ip default next-hop 10.242.160.251
Policy routing matches: 11317531 packets, 998890960 bytes

ACL

Extended IP access list TO-JPN
180 permit ip 10.242.105.0 0.0.0.255 any (336 matches)
190 permit ip 10.242.155.0 0.0.0.255 any (3198791 matches)
200 permit ip 10.243.5.0 0.0.0.255 any (189 matches)
210 permit ip 10.242.157.0 0.0.0.255 any
220 permit ip 10.242.158.0 0.0.0.255 any
230 permit ip 10.242.159.0 0.0.0.255 any
240 permit ip 10.243.155.0 0.0.0.255 any (653 matches)
250 permit ip 10.243.205.0 0.0.0.255 any (1662 matches)
260 permit ip 10.242.205.0 0.0.0.255 any (15 matches)
270 permit ip 10.243.253.0 0.0.0.255 any (706 matches)
280 permit ip 10.243.254.0 0.0.0.255 any
290 permit ip 10.243.161.0 0.0.0.255 any
300 permit ip 10.242.11.0 0.0.0.255 any (9120 matches)
310 permit ip 10.242.111.0 0.0.0.255 any (183850 matches)
320 permit ip 10.242.211.0 0.0.0.255 any (91 matches)
330 deny ip any host 195.5.239.229
340 deny ip any host 10.243.252.73

additionaly, i have these static routes

S* 0.0.0.0/0 [1/0] via 10.242.160.2
10.0.0.0/8 is variably subnetted, 31 subnets, 3 masks
S 10.0.0.0/8 [1/0] via 10.242.160.251
S 10.242.5.0/24 [1/0] via 10.242.160.254
S 10.242.11.0/24 [1/0] via 10.242.160.254
S 10.242.21.0/24 [1/0] via 10.242.160.2
S 10.242.105.0/24 [1/0] via 10.242.160.254
S 10.242.111.0/24 [1/0] via 10.242.160.254
S 10.242.155.0/24 [1/0] via 10.242.160.254
S 10.242.156.0/24 [1/0] via 10.242.160.254
S 10.242.157.0/24 [1/0] via 10.242.160.254
S 10.242.158.0/24 [1/0] via 10.242.160.254
S 10.242.159.0/24 [1/0] via 10.242.160.254
C 10.242.160.0/24 is directly connected, Vlan1
L 10.242.160.1/32 is directly connected, Vlan1
S 10.242.161.0/24 [1/0] via 10.242.160.254
S 10.242.205.0/24 [1/0] via 10.242.160.254
S 10.242.211.0/24 [1/0] via 10.242.160.254
S 10.243.5.0/24 [1/0] via 10.242.160.254
S 10.243.50.0/24 [1/0] via 10.242.160.254
S 10.243.51.0/24 [1/0] via 10.242.160.254
S 10.243.52.0/24 [1/0] via 10.242.160.254
S 10.243.53.0/24 [1/0] via 10.242.160.254
S 10.243.54.0/24 [1/0] via 10.242.160.254
S 10.243.55.0/24 [1/0] via 10.242.160.254
S 10.243.56.0/24 [1/0] via 10.242.160.254
S 10.243.155.0/24 [1/0] via 10.242.160.254
S 10.243.161.0/24 [1/0] via 10.242.160.254
S 10.243.205.0/24 [1/0] via 10.242.160.254
S 10.243.251.0/24 [1/0] via 10.242.160.2
S 10.243.252.0/24 [1/0] via 10.242.160.254
S 10.243.253.0/24 [1/0] via 10.242.160.254
S 10.243.254.0/24 [1/0] via 10.242.160.254
S 172.16.0.0/12 [1/0] via 10.242.160.251
S 192.168.0.0/16 [1/0] via 10.242.160.254
S 192.168.30.0/24 [1/0] via 10.242.160.160
192.168.33.0/24 is variably subnetted, 2 subnets, 2 masks
C 192.168.33.0/24 is directly connected, Vlan33
L 192.168.33.254/32 is directly connected, Vlan33
192.168.34.0/24 is variably subnetted, 2 subnets, 2 masks
C 192.168.34.0/24 is directly connected, Vlan34
L 192.168.34.254/32 is directly connected, Vlan34
S 192.168.181.0/24 [1/0] via 10.242.160.251
S 192.168.200.0/24 [1/0] via 10.242.160.251
S 192.168.222.0/24 [1/0] via 10.242.160.251
S 192.168.223.0/24 [1/0] via 10.242.160.251

The problem:

I am requested to change ip default next-hop to 10.242.160.250

Final goal is: set default gw for subnets in ACL to 10.242.160.250 and set routes to specific networks with static routes to 10.242.160.251.

The problem is: PBR is matched but ALL the traffic is routed to next hop and static routes are ignored.

I my understanding, this shouldn happen because static routes to these networks are present (ip default next-hop)

Im sure im missing something...

Thanks

Richard Burts · ‎01-17-2019

Am I correct in assuming that this route map and PBR have been in place for a while and have been working as intended? And am I correct in assuming that you have made changes but the changed configuration does not work as intended? If so please post the original config for interface, route map, acl, and static routes and post what you changed them to. If I am not correct about something then please clarify.

HTH

Rick

HTH

Rick

Ercole77 · ‎01-17-2019

Hi Richard

thank you very much for your kind reply.

I have inherited this config but i suspect it never worked as it should.

Im saying this because originally 10.242.5.251 was the only router for all, internet and other networks.

So, probably, it wasnt a problem if static routes never worked.

Now, i need to mantain 251 for some networks (with static routes) and to route internet traffic to 250 and the problem appears.

Looks like it doenst care of the statics

paul driver · ‎01-18-2019

Hello

Just like to add..
PBR logic differs on what command you apply for it.

set default ip next hop/ default interface will ONLY policy route your acl traffic if there isn't an active entry in the route table, meaning it checks the route table prior to decided weather to policy route or not, So if the router sees an entry in the route table then policy routing isnt activated and so the PBR acl traffic is routed normally.

set ip next hop/interface - will NOT check the route table and will attempt to policy route straight away.

As rick stated - please post your configuration so it can be reviewed.

Please rate and mark as an accepted solution if you have found any of the information provided useful.
This then could assist others on these forums to find a valuable answer and broadens the community’s global network.

Kind Regards
Paul

Ercole77 · ‎01-20-2019

Hi all

thank you Paul for this useful explanation.

This is the switch config

Current configuration : 14165 bytes
!
! Last configuration change at 19:24:52 GMT Wed Jan 16 2019 by ufficioced
! NVRAM config last updated at 19:49:01 GMT Wed Jan 16 2019 by ufficioced
!
version 15.2
no service pad
service tcp-keepalives-in
service tcp-keepalives-out
service timestamps debug datetime msec localtime show-timezone
service timestamps log datetime msec localtime show-timezone
service password-encryption
service compress-config
service sequence-numbers
!
hostname SWI-IDC
!
boot-start-marker
boot-end-marker
!
!
vrf definition Mgmt-vrf
!
address-family ipv4
exit-address-family
!
address-family ipv6
exit-address-family
!
!

aaa new-model
!
!
!
!
!
!
!
!
aaa session-id common
clock timezone GMT 1 0
clock summer-time GMT recurring last Sun Mar 2:00 last Sun Oct 2:00
switch 1 provision ws-c3850-24t
switch 2 provision ws-c3850-24t
!
!
!
!
!
no ip source-route
ip routing
!
!
!
login on-failure log
login on-success log
qos queue-softmax-multiplier 100
vtp domain idc
vtp mode transparent
!
crypto pki trustpoint TP-self-signed-3984151465
enrollment selfsigned
subject-name cn=IOS-Self-Signed-Certificate-3984151465
revocation-check none
rsakeypair TP-self-signed-3984151465
!
!
crypto pki certificate chain TP-self-signed-3984151465
certificate self-signed 01
3082022B 30820194 A0030201 02020101 300D0609 2A864886 F70D0101 05050030
31312F30 2D060355 04031326 494F532D 53656C66 2D536967 6E65642D 43657274
69666963 6174652D 33393834 31353134 3635301E 170D3137 30323133 30353133
35375A17 0D323030 31303130 30303030 305A3031 312F302D 06035504 03132649
4F532D53 656C662D 5369676E 65642D43 65727469 66696361 74652D33 39383431
35313436 3530819F 300D0609 2A864886 F70D0101 01050003 818D0030 81890281
8100BE4F C654E136 C2C48332 2ACC6CE5 6046B80B 708C1B54 7EE7CE44 F65C4E6A
894D2339 F2F50078 063EDD5B 5F6E01C4 10AA48E2 7775D331 FDA0DBA9 A8B47D08
730B0EE5 938C1024 05909107 17E60F71 5F2D212D 3C37FEF5 F7C9E721 F092FF08
8029F122 56618C4A 33060416 AA8937FA 348C6ECC A18326E1 C374E583 28968DA0
692B0203 010001A3 53305130 0F060355 1D130101 FF040530 030101FF 301F0603
551D2304 18301680 14B650B3 EA019A79 5DEED9AF E1E41936 5F5028A8 09301D06
03551D0E 04160414 B650B3EA 019A795D EED9AFE1 E419365F 5028A809 300D0609
2A864886 F70D0101 05050003 81810004 125F290A 1B25355D 2ECDBF9C 78351648
4AB3C677 BA6195AA 3FA0CDEB 3C537158 48A1F71C 2F9DF488 7A3FE726 5C400382
87F18E81 9A91DC9A B88887C7 0101B206 75561B68 28AAD570 E87A0FA4 964C966A
F6A30E79 0FAF311D C83D269B 8DB275CE 3A086FE0 E15C9C74 95E1449F 5A13F3D6
E04DC0EA 62524C01 57A2F44F 45B432
quit
diagnostic bootup level minimal
port-channel load-balance src-dst-ip
!
spanning-tree mode pvst
spanning-tree extend system-id
spanning-tree vlan 1-1000 priority 24576
hw-switch switch 1 logging onboard message level 3
hw-switch switch 2 logging onboard message level 3
!
redundancy
mode sso
!
!
vlan 2
name OUTSIDE_FW
!
vlan 3
name DMZ
!
vlan 4
name FAILOVER_FW
!
vlan 5
name OUTSIDE_BT
!
vlan 33
name VMOTION
!
vlan 34
name VSAN
!
!
class-map match-any non-client-nrt-class
!
policy-map port_child_policy
class non-client-nrt-class
bandwidth remaining ratio 10
!
!
!
!
!
!
!
!
!
!
!
!
!
!
interface Port-channel1
description LAG ToR Switch
switchport mode trunk
logging event trunk-status
!
interface Port-channel10
description IDC-VIOS1
switchport mode trunk
!
interface Port-channel11
description IDC-VIOS2
switchport mode trunk
!
interface Port-channel30
description IBMFlex-ModuleGbit-01
switchport mode trunk
shutdown
spanning-tree portfast trunk
!
interface Port-channel31
description IBMFlex-ModuleGbit-02
switchport mode trunk
shutdown
spanning-tree portfast trunk
!
interface GigabitEthernet0/0
vrf forwarding Mgmt-vrf
no ip address
negotiation auto
!
interface GigabitEthernet1/0/1
description ASA Inside
switchport mode access
!
interface GigabitEthernet1/0/2
description ASA DMZ
switchport access vlan 3
!
interface GigabitEthernet1/0/3
description ASA OUTSIDE
switchport access vlan 2
switchport mode access
!
interface GigabitEthernet1/0/4
description ASA FAILOVER
switchport access vlan 4
!
interface GigabitEthernet1/0/5
description ASA OUT-BT
switchport access vlan 5
!
interface GigabitEthernet1/0/6
description Power8 HMC1
switchport mode access
spanning-tree portfast
!
interface GigabitEthernet1/0/7
description Power8-AS-C8-Eth0
switchport mode trunk
channel-protocol lacp
channel-group 10 mode active
spanning-tree portfast
spanning-tree bpduguard enable
!
interface GigabitEthernet1/0/8
description Power8-AS-C10-Eth0
switchport mode trunk
channel-protocol lacp
channel-group 11 mode active
spanning-tree portfast
spanning-tree bpduguard enable
!
interface GigabitEthernet1/0/9
description Internet BT
switchport access vlan 5
speed 100
duplex full
!
interface GigabitEthernet1/0/10
description ROUTER BT MPLS BACKUP
speed 100
duplex full
!
interface GigabitEthernet1/0/11
description ROUTER BT MPLS BACKUP
speed 100
duplex full
!
interface GigabitEthernet1/0/12
description INTERNET-PUBBLICO
switchport access vlan 5
!
interface GigabitEthernet1/0/13
description SwitchFiber-01-mgmt
!
interface GigabitEthernet1/0/14
description SAW2 internet
switchport access vlan 5
switchport mode access
spanning-tree portfast
!
interface GigabitEthernet1/0/15
description technician available
switchport mode access
!
interface GigabitEthernet1/0/16
description V7000 CTRL 1
!
interface GigabitEthernet1/0/17
description DD2500 porta integrata 0
!
interface GigabitEthernet1/0/18
description SAW2 LAN
!
interface GigabitEthernet1/0/19
description iDRAC
switchport mode access
!
interface GigabitEthernet1/0/20
description PER MIGRAZIONE - TEMPORANEO
switchport mode trunk
spanning-tree portfast trunk
!
interface GigabitEthernet1/0/21
description iDRAC
switchport mode access
!
interface GigabitEthernet1/0/22
description iDRAC
switchport mode access
!
interface GigabitEthernet1/0/23
description KDDI Linea COLT Master
!
interface GigabitEthernet1/0/24
description internet da deconfigurare
switchport access vlan 5
switchport mode access
spanning-tree portfast
spanning-tree bpduguard enable
spanning-tree guard root
!
interface GigabitEthernet1/1/1
!
interface GigabitEthernet1/1/2
!
interface GigabitEthernet1/1/3
!
interface GigabitEthernet1/1/4
!
interface TenGigabitEthernet1/1/1
!
interface TenGigabitEthernet1/1/2
!
interface TenGigabitEthernet1/1/3
description UPLINK ToR Switch
switchport mode trunk
logging event trunk-status
channel-protocol lacp
channel-group 1 mode active
!
interface TenGigabitEthernet1/1/4
description UPLINK ToR Switch
switchport mode trunk
logging event trunk-status
channel-protocol lacp
channel-group 1 mode active
!
interface GigabitEthernet2/0/1
description ASA Inside
switchport mode access
!
interface GigabitEthernet2/0/2
description ASA DMZ
switchport access vlan 3
!
interface GigabitEthernet2/0/3
description ASA OUTSIDE
switchport access vlan 2
switchport mode access
!
interface GigabitEthernet2/0/4
description ASA FAILOVER
switchport access vlan 4
!
interface GigabitEthernet2/0/5
description ASA OUT-BT
switchport access vlan 5
!
interface GigabitEthernet2/0/6
description Power8 HMC2
switchport mode access
spanning-tree portfast
!
interface GigabitEthernet2/0/7
description Power8-AS-C8-Eth1
switchport mode trunk
channel-protocol lacp
channel-group 10 mode active
spanning-tree portfast
spanning-tree bpduguard enable
!
interface GigabitEthernet2/0/8
description Power8-AS-C10-Eth1
switchport mode trunk
channel-protocol lacp
channel-group 11 mode active
spanning-tree portfast
spanning-tree bpduguard enable
!
interface GigabitEthernet2/0/9
description Internet BT
switchport access vlan 5
speed 100
duplex full
!
interface GigabitEthernet2/0/10
!
interface GigabitEthernet2/0/11
description ROUTER BT MPLS PRIMARIO
speed 100
duplex full
!
interface GigabitEthernet2/0/12
!
interface GigabitEthernet2/0/13
description SwitchFiber-02-mgmt
!
interface GigabitEthernet2/0/14
switchport access vlan 5
switchport mode access
spanning-tree portfast
!
interface GigabitEthernet2/0/15
!
interface GigabitEthernet2/0/16
description V7000 CTRL 2
!
interface GigabitEthernet2/0/17
description DD2500 porta esterna 0
!
interface GigabitEthernet2/0/18
!
interface GigabitEthernet2/0/19
description iDRAC
switchport mode access
!
interface GigabitEthernet2/0/20
description PER MIGRAZIONE - TEMPORANEO
switchport mode trunk
spanning-tree portfast trunk
!
interface GigabitEthernet2/0/21
description iDRAC
switchport mode access
!
interface GigabitEthernet2/0/22
description iDRAC
switchport mode access
!
interface GigabitEthernet2/0/23
description KDDI Linea RETELIT Backup
!
interface GigabitEthernet2/0/24
!
interface GigabitEthernet2/1/1
!
interface GigabitEthernet2/1/2
!
interface GigabitEthernet2/1/3
!
interface GigabitEthernet2/1/4
!
interface TenGigabitEthernet2/1/1
!
interface TenGigabitEthernet2/1/2
!
interface TenGigabitEthernet2/1/3
description UPLINK ToR Switch
switchport mode trunk
logging event trunk-status
channel-protocol lacp
channel-group 1 mode active
!
interface TenGigabitEthernet2/1/4
description UPLINK ToR Switch
switchport mode trunk
logging event trunk-status
channel-protocol lacp
channel-group 1 mode active
!
interface Vlan1
ip address 10.242.160.1 255.255.255.0
ip policy route-map TO-JPN
!
interface Vlan33
ip address 192.168.33.254 255.255.255.0
!
interface Vlan34
ip address 192.168.34.254 255.255.255.0
!
ip forward-protocol nd
ip http server
ip http authentication local
ip http secure-server
ip route 0.0.0.0 0.0.0.0 10.242.160.2
ip route 10.0.0.0 255.0.0.0 10.242.160.251
ip route 10.242.5.0 255.255.255.0 10.242.160.254
ip route 10.242.11.0 255.255.255.0 10.242.160.254
ip route 10.242.21.0 255.255.255.0 10.242.160.2
ip route 10.242.105.0 255.255.255.0 10.242.160.254
ip route 10.242.111.0 255.255.255.0 10.242.160.254
ip route 10.242.155.0 255.255.255.0 10.242.160.254
ip route 10.242.156.0 255.255.255.0 10.242.160.254
ip route 10.242.157.0 255.255.255.0 10.242.160.254
ip route 10.242.158.0 255.255.255.0 10.242.160.254
ip route 10.242.159.0 255.255.255.0 10.242.160.254
ip route 10.242.161.0 255.255.255.0 10.242.160.254
ip route 10.242.205.0 255.255.255.0 10.242.160.254
ip route 10.242.211.0 255.255.255.0 10.242.160.254
ip route 10.243.5.0 255.255.255.0 10.242.160.254
ip route 10.243.50.0 255.255.255.0 10.242.160.254
ip route 10.243.51.0 255.255.255.0 10.242.160.254
ip route 10.243.52.0 255.255.255.0 10.242.160.254
ip route 10.243.53.0 255.255.255.0 10.242.160.254
ip route 10.243.54.0 255.255.255.0 10.242.160.254
ip route 10.243.55.0 255.255.255.0 10.242.160.254
ip route 10.243.56.0 255.255.255.0 10.242.160.254
ip route 10.243.155.0 255.255.255.0 10.242.160.254
ip route 10.243.161.0 255.255.255.0 10.242.160.254
ip route 10.243.205.0 255.255.255.0 10.242.160.254
ip route 10.243.251.0 255.255.255.0 10.242.160.2
ip route 10.243.252.0 255.255.255.0 10.242.160.254
ip route 10.243.253.0 255.255.255.0 10.242.160.254
ip route 10.243.254.0 255.255.255.0 10.242.160.254
ip route 172.16.0.0 255.240.0.0 10.242.160.251
ip route 192.168.0.0 255.255.0.0 10.242.160.254
ip route 192.168.30.0 255.255.255.0 10.242.160.160
ip route 192.168.181.0 255.255.255.0 10.242.160.251
ip route 192.168.200.0 255.255.255.0 10.242.160.251
ip route 192.168.222.0 255.255.255.0 10.242.160.251
ip route 192.168.223.0 255.255.255.0 10.242.160.251
!
ip access-list extended TO-JPN
deny ip any host 192.203.234.151
deny ip any host 193.41.198.151
deny ip host 10.242.155.180 any
deny ip any host 212.45.144.88
deny ip any host 212.45.144.3
deny ip host 10.243.205.25 any
deny ip any host 185.15.129.3
deny ip any host 158.102.161.215
deny ip host 10.243.205.24 any
deny ip host 10.243.254.217 any
deny ip any host 193.204.114.232
deny ip any host 193.204.114.233
deny ip any host 89.118.11.123
deny ip host 10.243.253.245 any
deny ip 10.242.156.0 0.0.0.255 any
permit ip host 10.242.5.166 any
deny ip any 10.243.251.0 0.0.0.255
permit ip 10.242.105.0 0.0.0.255 any
permit ip 10.242.155.0 0.0.0.255 any
permit ip 10.243.5.0 0.0.0.255 any
permit ip 10.242.157.0 0.0.0.255 any
permit ip 10.242.158.0 0.0.0.255 any
permit ip 10.242.159.0 0.0.0.255 any
permit ip 10.243.155.0 0.0.0.255 any
permit ip 10.243.205.0 0.0.0.255 any
permit ip 10.242.205.0 0.0.0.255 any
permit ip 10.243.253.0 0.0.0.255 any
permit ip 10.243.254.0 0.0.0.255 any
permit ip 10.243.161.0 0.0.0.255 any
permit ip 10.242.11.0 0.0.0.255 any
permit ip 10.242.111.0 0.0.0.255 any
permit ip 10.242.211.0 0.0.0.255 any
deny ip any host 195.5.239.229
deny ip any host 10.243.252.73
deny ip any host 10.242.160.23
ip access-list extended TO-JPN-NET
permit ip host 10.242.105.69 10.0.0.0 0.255.255.255
ip access-list extended TO-JPN-VPN
permit ip host 10.242.105.69 any
deny ip any any
!
ip sla enable reaction-alerts
logging facility local0
logging host 10.242.5.59
!
route-map TO-JPN permit 10
match ip address TO-JPN
set ip default next-hop 10.242.160.251
!
snmp-server community public RO
!
!
!
!
line con 0
exec-timeout 60 0
stopbits 1
line aux 0
stopbits 1
line vty 0 4
exec-timeout 60 0
privilege level 15
logging synchronous
transport input telnet
line vty 5 15
exec-timeout 60 0
privilege level 15
logging synchronous
transport input telnet
!
ntp server 193.204.114.232 prefer
ntp server 193.204.114.233
wsma agent exec
profile httplistener
profile httpslistener
!
wsma agent config
profile httplistener
profile httpslistener
!
wsma agent filesys
profile httplistener
profile httpslistener
!
wsma agent notify
profile httplistener
profile httpslistener
!
!
wsma profile listener httplistener
transport http
!
wsma profile listener httpslistener
transport https
!
ap group default-group
end

YKK-SWI-IDC#term len 256

Richard Burts · ‎01-21-2019

Thank you for posting the config of the switch. I see that PBR is configured on interface vlan 1 and that it uses an acl and does set ip default. But the logic of your PBR is a bit difficult to understand. The subnet of vlan 1 is 10.242.160.0 and typically the acl for PBR would have entries for this as the source address. But the only entry I find in the acl is a deny for a host in that subnet. And I find permit statements for many subnets such as 10.242.105.0, 10.242.157.0, 10.242.158.0, 10.242.159.0. Are these subnets reached through vlan 1? I am wondering if the issue may be that the policy is configured on the interface that is the exit interface while it should be configured on the entering interface. Perhaps you might provide some information on the topology of your network and where these subnets are located?

It would also be helpful if you would post the output of traceroute from some device that you expect to be policy routed showing how it goes through your network.

HTH

Rick

HTH

Rick