PBR not Routing on FTD (with FDM)

kmcgraw · ‎01-30-2025

We have two internet providers. Trying to route all outbound Internet traffic to the second ISP but PBR is not working after applying the configuration on FP1140.

After enabling the route map the device matches to the correct next ip hop as configured but it ignores the setting and uses the default route on E1/1(outside) and not E1/4(outside-ISP2).

I have configured an additional NAT for traffic heading to ISP2.

I don't know what what I am missing.

Thanks

Here is the config:

access-list PBR_ACL line 1 extended deny ip 172.16.5.0 255.255.255.0 10.0.0.0 255.0.0.0 log disable
access-list PBR_ACL line 2 extended deny ip 172.16.5.0 255.255.255.0 172.16.0.0 255.240.0.0 log disable
access-list PBR_ACL line 3 extended deny ip 172.16.5.0 255.255.255.0 192.168.0.0 255.255.0.0 log disable
access-list PBR_ACL line 4 extended permit ip 172.16.5.0 255.255.255.0 any4 log disable

route-map RouteMap_PBR, permit, sequence 10

Match clauses:

ip address (access-lists): PBR_ACL

Set clauses:

ip next-hop xxx.xxx.81.49

community none

interface Ethernet1/1
nameif outside
cts manual
propagate sgt preserve-untag
policy static sgt disabled trusted
security-level 0
ip address 10.33.3.31 255.255.255.0
!
interface Ethernet1/2
nameif inside
cts manual
propagate sgt preserve-untag
policy static sgt disabled trusted
security-level 0
ip address 172.16.2.10 255.255.255.0
policy-route route-map RouteMap_PBR
!
interface Ethernet1/3
nameif dmz
security-level 0
ip address 192.168.0.1 255.255.255.0
!
interface Ethernet1/3.1
vlan 50
nameif extranet
security-level 0
ip address 172.16.1.1 255.255.255.0
!
interface Ethernet1/4
nameif outside-ISP2
security-level 0
ip address xxx.xxx.81.50 255.255.255.240

Packet Tracer results:

Phase: 1

Type: PBR-LOOKUP

Subtype: policy-route

Result: ALLOW

Elapsed time: 48640 ns

Config:

route-map RouteMap_PBR permit 10

match ip address PBR_ACL

set ip next-hop xxx.xxx.81.49

set community noneAdditional Information:

Matched route-map RouteMap_PBR, sequence 10, permit

Phase: 2

Type: INPUT-ROUTE-LOOKUP

Subtype: Resolve Egress Interface

Result: ALLOW

Elapsed time: 3584 ns

Config:

Additional Information:

Found next-hop 10.33.3.1 using egress ifc outside(vrfid:0)

Phase: 3

Type: ACCESS-LIST

Subtype: log

Result: ALLOW

Elapsed time: 6144 ns

Config:

access-group NGFW_ONBOX_ACL global

access-list NGFW_ONBOX_ACL advanced permit object-group |acSvcg-268435519 ifc inside any ifc outside any rule-id 268435519 event-log both

access-list NGFW_ONBOX_ACL remark rule-id 268435519: ACCESS POLICY: NGFW_Access_Policy

access-list NGFW_ONBOX_ACL remark rule-id 268435519: L5 RULE: Allow ICMP Out

object-group service |acSvcg-268435519

service-object icmp

Additional Information:

This packet will be sent to snort for additional processing where a verdict will be reached

Forward Flow based lookup yields rule:

in id=0x14c189e11c10, priority=12, domain=permit, deny=false

hits=8437, user_data=0x14c1a4c36080, cs_id=0x0, use_real_addr, flags=0x0, protocol=1

src ip/id=0.0.0.0, mask=0.0.0.0, icmp-type=0, tag=any, ifc object-group id 13688

dst ip/id=0.0.0.0, mask=0.0.0.0, icmp-code=0, tag=any, ifc object-group id 19353, vlan=0,

src nsg_id=none, dst nsg_id=none

dscp=0x0, input_ifc=any, output_ifc=any

Phase: 4

Type: NAT

Subtype:

Result: ALLOW

Elapsed time: 6144 ns

Config:

nat (inside,outside) after-auto source dynamic Inside_Network |10.33.3.251 interface

Additional Information:

Dynamic translate 172.16.5.82/0 to 10.33.3.251/11427

Forward Flow based lookup yields rule:

in id=0x14c18c7a09a0, priority=6, domain=nat, deny=false

hits=273834, user_data=0x14c18c7a8b40, cs_id=0x0, flags=0x0, protocol=0

src ip/id=172.16.0.0, mask=255.240.0.0, port=0, tag=any

dst ip/id=0.0.0.0, mask=0.0.0.0, port=0, tag=any

src nsg_id=none, dst nsg_id=none

dscp=0x0, input_ifc=inside(vrfid:0), output_ifc=outside(vrfid:0)

Phase: 5

Type: NAT

Subtype: per-session

Result: ALLOW

Elapsed time: 6144 ns

Config:

Additional Information:

Forward Flow based lookup yields rule:

in id=0x14c1b9f85430, priority=0, domain=nat-per-session, deny=true

hits=684919, user_data=0x0, cs_id=0x0, reverse, use_real_addr, flags=0x0, protocol=0

src ip/id=0.0.0.0, mask=0.0.0.0, port=0, tag=any

dst ip/id=0.0.0.0, mask=0.0.0.0, port=0, tag=any,

src nsg_id=none, dst nsg_id=none

dscp=0x0, input_ifc=any, output_ifc=any

Phase: 6

Type: IP-OPTIONS

Subtype:

Result: ALLOW

Elapsed time: 6144 ns

Config:

Additional Information:

Forward Flow based lookup yields rule:

in id=0x14c1bfb2d080, priority=0, domain=inspect-ip-options, deny=true

hits=330898, user_data=0x0, cs_id=0x0, reverse, flags=0x0, protocol=0

src ip/id=0.0.0.0, mask=0.0.0.0, port=0, tag=any

dst ip/id=0.0.0.0, mask=0.0.0.0, port=0, tag=any

src nsg_id=none, dst nsg_id=none

dscp=0x0, input_ifc=inside(vrfid:0), output_ifc=any

Phase: 7

Type: INSPECT

Subtype: np-inspect

Result: ALLOW

Elapsed time: 19968 ns

Config:

class-map inspection_default

match default-inspection-traffic

policy-map global_policy

class inspection_default

inspect icmp

service-policy global_policy global

Additional Information:

Forward Flow based lookup yields rule:

in id=0x14c1bdc568b0, priority=70, domain=inspect-icmp, deny=false

hits=8590, user_data=0x14c1bdb6cbf0, cs_id=0x0, use_real_addr, flags=0x0, protocol=1

src ip/id=0.0.0.0, mask=0.0.0.0, icmp-type=0, tag=any

dst ip/id=0.0.0.0, mask=0.0.0.0, icmp-code=0, tag=any,

src nsg_id=none, dst nsg_id=none

dscp=0x0, input_ifc=inside(vrfid:0), output_ifc=any

Phase: 8

Type: INSPECT

Subtype: np-inspect

Result: ALLOW

Elapsed time: 3584 ns

Config:

Additional Information:

Forward Flow based lookup yields rule:

in id=0x14c1bdc6b410, priority=70, domain=inspect-icmp-error, deny=false

hits=8590, user_data=0x14c1bdb6faf0, cs_id=0x0, use_real_addr, flags=0x0, protocol=1

src ip/id=0.0.0.0, mask=0.0.0.0, icmp-type=0, tag=any

dst ip/id=0.0.0.0, mask=0.0.0.0, icmp-code=0, tag=any,

src nsg_id=none, dst nsg_id=none

dscp=0x0, input_ifc=inside(vrfid:0), output_ifc=any

Phase: 9

Type: VPN

Subtype: ipsec-tunnel-flow

Result: ALLOW

Elapsed time: 1024 ns

Config:

Additional Information:

Forward Flow based lookup yields rule:

in id=0x14c18cebf3f0, priority=13, domain=ipsec-tunnel-flow, deny=true

hits=313386, user_data=0x0, cs_id=0x0, flags=0x0, protocol=0

src ip/id=0.0.0.0, mask=0.0.0.0, port=0, tag=any

dst ip/id=0.0.0.0, mask=0.0.0.0, port=0, tag=any

src nsg_id=none, dst nsg_id=none

dscp=0x0, input_ifc=inside(vrfid:0), output_ifc=any

Phase: 10

Type: NAT

Subtype: rpf-check

Result: ALLOW

Elapsed time: 6144 ns

Config:

nat (inside,outside) after-auto source dynamic Inside_Network |10.33.3.251 interface

Additional Information:

Forward Flow based lookup yields rule:

out id=0x14c18c1dd010, priority=6, domain=nat-reverse, deny=false

hits=311130, user_data=0x14c18c7b9060, cs_id=0x0, use_real_addr, flags=0x0, protocol=0

src ip/id=172.16.0.0, mask=255.240.0.0, port=0, tag=any

dst ip/id=0.0.0.0, mask=0.0.0.0, port=0, tag=any

src nsg_id=none, dst nsg_id=none

dscp=0x0, input_ifc=inside(vrfid:0), output_ifc=outside(vrfid:0)

Phase: 11

Type: NAT

Subtype: per-session

Result: ALLOW

Elapsed time: 34304 ns

Config:

Additional Information:

Reverse Flow based lookup yields rule:

in id=0x14c1b9f85430, priority=0, domain=nat-per-session, deny=true

hits=684921, user_data=0x0, cs_id=0x0, reverse, use_real_addr, flags=0x0, protocol=0

src ip/id=0.0.0.0, mask=0.0.0.0, port=0, tag=any

dst ip/id=0.0.0.0, mask=0.0.0.0, port=0, tag=any,

src nsg_id=none, dst nsg_id=none

dscp=0x0, input_ifc=any, output_ifc=any

Phase: 12

Type: IP-OPTIONS

Subtype:

Result: ALLOW

Elapsed time: 512 ns

Config:

Additional Information:

Reverse Flow based lookup yields rule:

in id=0x14c1be169540, priority=0, domain=inspect-ip-options, deny=true

hits=318483, user_data=0x0, cs_id=0x0, reverse, flags=0x0, protocol=0

src ip/id=0.0.0.0, mask=0.0.0.0, port=0, tag=any

dst ip/id=0.0.0.0, mask=0.0.0.0, port=0, tag=any

src nsg_id=none, dst nsg_id=none

dscp=0x0, input_ifc=outside(vrfid:0), output_ifc=any

Phase: 13

Type: FLOW-CREATION

Subtype:

Result: ALLOW

Elapsed time: 14336 ns

Config:

Additional Information:

New flow created with id 898972, packet dispatched to next module

Module information for forward flow ...

snp_fp_inspect_ip_options

snp_fp_snort

snp_fp_inspect_icmp

snp_fp_translate

snp_fp_adjacency

snp_fp_fragment

snp_fp_tracer_drop

snp_ifc_stat

Module information for reverse flow ...

snp_fp_inspect_ip_options

snp_fp_translate

snp_fp_inspect_icmp

snp_fp_snort

snp_fp_adjacency

snp_fp_fragment

snp_fp_tracer_drop

snp_ifc_stat

Phase: 14

Type: EXTERNAL-INSPECT

Subtype:

Result: ALLOW

Elapsed time: 28160 ns

Config:

Additional Information:

Application: 'SNORT Inspect'

Phase: 15

Type: SNORT

Subtype: firewall

Result: ALLOW

Elapsed time: 169260 ns

Config:

Network 0, Inspection 0, Detection 0, Rule ID 268435519

Additional Information:

Starting rule matching, zone 1 -> 2, geo 0 -> 0, vlan 0, src sgt: 0, src sgt type: unknown, dst sgt: 0, dst sgt type: unknown, user 9999997, no url or hos

t, no xff

Matched rule ids 268435519 - Allow

Phase: 16

Type: SNORT

Subtype: appid

Result: ALLOW

Elapsed time: 13740 ns

Config:

Additional Information:

service: ICMP(3501), client: (0), payload: (0), misc: (0)

Phase: 17

Type: INPUT-ROUTE-LOOKUP-FROM-OUTPUT-ROUTE-LOOKUP

Subtype: Resolve Preferred Egress interface

Result: ALLOW

Elapsed time: 9728 ns

Config:

Additional Information:

Found next-hop 10.33.3.1 using egress ifc outside(vrfid:0)

Phase: 18

Type: ADJACENCY-LOOKUP

Subtype: Resolve Nexthop IP address to MAC

Result: ALLOW

Elapsed time: 2560 ns

Config:

Additional Information:

Found adjacency entry for Next-hop 10.33.3.1 on interface outside

Adjacency :Active

MAC address a44c.11e5.b500 hits 3133786 reference 5904

Result:

input-interface: inside(vrfid:0)

input-status: up

input-line-status: up

output-interface: outside(vrfid:0)

output-status: up

output-line-status: up

Action: allow

Time Taken: 380120 ns

Joshqun Ismayilov · ‎01-30-2025

It looks like your PBR matches traffic correctly, but the packets still use the default route instead of being forwarded to ISP2.
You can check NAT/

# nat (inside,outside-ISP2) after-auto source dynamic Inside_Network interface

Thanks,
Joshqun Ismayilov

kmcgraw · ‎01-30-2025

Yes I have got that. I don't get why it is ignoring the PBR.

Manual NAT Policies (Section 3)
1 (inside) to (outside) source dynamic Inside_Network |10.33.3.251 interface
translate_hits = 521520, untranslate_hits = 8927
2 (inside) to (outside-isp2) source dynamic Inside_Network |xxx.xxx.81.62
translate_hits = 0, untranslate_hits = 0

Joshqun Ismayilov · ‎01-30-2025

It looks like your PBR is not working /

Verify that the Access List used in PBR is correctly matching the traffic you expect to go via ISP2.

Would you like to share your route-map and ACL configuration?

Regards,
Joshqun Ismayilov

MHM Cisco World · ‎01-30-2025

Show asp table classify domain pbr

Show policy-route

Show route-map

Debug policy-route

Share output of above

Thanks

MHM

kmcgraw · ‎01-30-2025

Using the CLI console:

Show asp table classify domain pbr
This command is not supported.

Show policy-route
Interface Route map
Ethernet1/2 RouteMap_PBR

Show route-map

route-map RouteMap_PBR, permit, sequence 10
Match clauses:
ip address (access-lists): PBR_ACL

Set clauses:
ip next-hop xxx.xxx.81.49
community none

Debug policy-route
This command is not supported.

Thanks for your help.

MHM Cisco World · ‎01-30-2025

Show asp table classify ? <<- check what option you will get

MHM

kmcgraw · ‎01-30-2025

I figured it out. Typo in next Hop IP address.

Thanks for your help.