Re: 2 Gateways of last resort for unknown traffic (PBR): Cisco 3750x switch - Page 2

VRAI · ‎08-24-2020

Hi,

I have some new VLANs on my 3750x IOS:15.0(2)SE C3750E-UNIVERSALK9-M (SDM-Prefere routing enabled) switch that I need to direct to a different IP address/gateway for the purposes of internet access than the rest of the existing VLANs. Both sets of VLANs (Old and New) are still required to talk to each other.

Looking through some similar scenarios on this discussion forum I came across a couple of possible solutions. One of them being PBR which seemed simple enough but for some reason when I tried it I could only get the traffic to hit the policy (the hit count goes up on the route-map/ACL) but the policy still routes it out of the pre-existing Gateway #1

Initially, I created the PBR and added a new default gateway (Gateway#2) with a higher metric so it does not initiate Equal cost load balancing because that would defeat the purpose of what I am trying to do. When I put the higher metric on gateway#2 noticed that the traffic from the new VLAN source was still taking the old gateway (Gateway#1).

my aims:

Route traffic from new VLANs destined for unknown destination based on source IP subnet to a different gateway
Continue to have pre-existing VLANs use the old gateway
Allow existing and NEW VLANs to still communicate with each other

What I want to know is:

Is my assumption correct that you can have 2 gateways of last resort for unknown traffic where one of the gateways is used only for source-based traffic matching a policy and is destined for internet?
Does the "set ip next-hop" command force all PBR matched traffic to use that next-hop IP for ALL traffic no matter if it is internet bound /unknown or not? (not what I want) or will it send it if there is no matching destination?
Can both old and new VLANS still talk to each other

Example config:

interface vlan500

Description NEW_VLAN

IP address 192.168.50.254 255.255.255.0

IP policy route-map PBR

!

access-list 101 permit ip 192.168.50.0 0.0.0.255 any

!

route-map PBR permit 10

match ip address 101

set ip next-hop 192.168.50.253

!

ip route 0.0.0.0 0.0.0.0 192.168.1.253

ip route 0.0.0.0 0.0.0.0 192.168.50.253 10

!

I want to avoid having to do VRF if at all possible but depending on your answers I may not have a choice. In which case I will come back with a follow up on how I will do that.

Thanks!

VRAI · ‎09-06-2020

When I try to apply the route-map to the VLAN SVIs they don't show up under the "show run interface x.y.z". I checked the logging and I see this message which is a bit shocking to be honest....

%PLATFORM_PBR-3-UNSUPPORTED_RMAP: Route-map NEW_MAP not supported for Policy-Based Routing

please don't tell me that after all this the switch is not capable of doing the job... I supplied the switch model number and IOS etc in my original post if that helps...

actually come to think of it I have a copy of the config from a few days ago when I was testing and I was able to get the "IP policy route-map" statement to stick to the interface... what is going on here I wonder?

EDIT: It looks like that I can't use the "IP default next-hop" approach on this switch (correct me if I am wrong). I may be able to use the "set IP next-hop" instead? I think this would mean I would have to add deny statements in the ACL to all other vlans/subnets??

Richard Burts · ‎09-06-2020

I have seen several instances where there are platform differences in what is supported on some platforms and not supported on other platforms. ip set default next-hop vs ip set next-hop is an example of that. You should be able to get it to work with set ip next-hop if you make appropriate changes in the acl used for pbr.

HTH

Rick

VRAI · ‎09-06-2020

Delete

VRAI · ‎09-06-2020

From what I understand per your instructions for the "IP next-hop" method:

Deny traffic for sources I want normal forwarding to operate (Deny traffic from New_VLANs to old vlans)
1. Does this mean I have to create ACL ACE for each New VLAN source to EVERY OLD VLAN on the switch (there are many)
Permit all other traffic (permit NEW_VLAN any)

example:

IP access-list extended New_VLAN_ACL

deny IP 192.168.50.0 0.0.0.255 any

deny IP 10.1.1.1 0.0.0.255 any

permit IP any any

am I on the right track?

VRAI · ‎09-06-2020

I am getting ACL matches/hits but they are still being routed to the old default gateway.

Richard Burts · ‎09-06-2020

There are a couple of ways that we can try to work this out. You could post the configuration that you have developed and we can try to figure out why it is not working. Or we can make suggestions about how to do it and you can apply those suggestions to your configuration. I am happy to do it either way that you would like.

In a couple of posts you have asked questions that are essentially do you need to identify individually each new vlan and each old vlan. That is an important question and the answer is that no you do not need to identify each individual vlan. It is quite ok to summarize the addresses. For example if all of the old vlan are in the range of 192.168.0.0 and all of the new vlans are in the range of 10.0.0.0 then the access list might be something like this:

access-list 101 deny ip 10.0.0.0 0.255.255.255 192.168.0.0.0.0.255.255

access-list 101 permit ip 10.0.0.0 0.255.255.255 any

you would then use this acl in the route map which would match the acl and have set ip next-hop 192.168.50.250

HTH

Rick

VRAI · ‎09-07-2020

There are Old and New VLANS in the 10.0.0.0/8 range and 192.168.0.0/16

so there is a mix of both old and new in the same ranges. They are all /24 subnets within those class A and C networks

I will work on getting the config condensed to post it.

Richard Burts · ‎09-07-2020

You have asked if you could use summary addresses in the access lists because you do not want to have to list each vlan subnet individually. I suggested an approach to summarizing assuming that old and new could be separated by range. But I now realize that my logic was not correct. What we need to deny is really anything from new vlan to inside (both to old vlan and to new vlan not just from new vlan to old vlan). You probably will need a couple of statements for each new vlan which would look something like this

access-list 101 deny ip 192.168.1.0 0.0.0.255 10.0.0.0 0.255.255.255

access-list 101 deny ip 192.168.1.0 0.0.0.255 192.168.0.0 0.0.255.255

HTH

Rick

VRAI · ‎08-31-2020

Hi I will try:

Source subnet{New VLANs} Destination: Unknown/internet-------------------> Use new default gateway [192.168.50.250]
Source subnet{old VLANS} Destination: Unknown/internet--------------------> Use old gateway of last resort [192.168.1.250]
Both Old VLAN/s{192.168.1.0} and new VLAN/s{10.1.1.1.0/24} must still be able to talk to each other on the 3750x
New DMZ Vlan will have the Firewalls/PE IPs as their default gateway

The only thing I want to be changed is the gateway of last resort decision via PBR.

VRAI · ‎09-06-2020

as per my switch log:

%PLATFORM_PBR-3-UNSUPPORTED_RMAP: Route-map NEW_MAP not supported for Policy-Based Routing

I don't think this switch can do the "IP default next-hop" process so I am trying the "IP next-hop" method instead...

I need some help with how the ACL to match to the policy will look like:

ip access-list extended NEW_VLAN_ACL

deny ip 192.168.50.0 0.0.0.255 any
deny ip 192.168.1.0 0.0.0.255 any
deny ip 10.1.1.0. 0.0.0.255 any
permit IP any any??

Giuseppe Larosa · ‎09-07-2020

Hello @VRAI ,

as explained in the example made by Richard for using the set ip next-hop approach you need to deny traffic from New Vlans to Old vlans.

You have said tha you have both new and old Vlans/subnets in 10/8 and in 192.168.x.0/24 ranges.

You cannot use a line like the following:

>> deny ip 192.168.50.0 0.0.0.255 any

because in that any is matched also traffic to the internet that should use the new default gateway.

You should write something like:

ip access-list extended NEW_VLAN_ACL

deny ip 192.168.50.0 0.0.0.255 10.0.0.0 0.255.255.255

deny ip 192.168.50.0 0.0.0.255 192.168.0.0 0.0.255.255

deny ip 192.168.1.0 0.0.0.255 10.0.0.0 0.255.255.255

deny ip 192.168.1.0 0.0.0.255 192.168.0.0 0.0.255.255

deny ip 10.1.1.0 0.0.0.255 10.0.0.0 0.255.255.255

deny ip 10.1.1.0 0.0.0.255 192.168.0.0 0.0.255.255

permit ip any any

You will need two deny statements for each new subnet VLAN if you want to create a single route-map to be applied inbound to each of them

Because in ACLs the order of statement is very important only traffic to the internet will be a match for the final permit statement and only that will use the new gateway2 as next-hop.

With your current ACL configuration all traffic from new Vlans is routed via GW1 because of wrong configuration of deny statements.

Hope to help

Giuseppe

VRAI · ‎09-07-2020

This is my current config.

I am using a machine in the 10.2.1.0/24 subnet but my traffic is still getting routed out of default gateway#1 not the 192.168.50.250 as set in the route-map "IP next-hop"

ip access-list extended Deny_New_to_Old_ACL
deny ip 192.168.50.0 0.0.0.255 10.0.0.0 0.255.255.255
deny ip 192.168.50.0 0.0.0.255 192.168.0.0 0.0.255.255
deny ip 192.168.50.0 0.0.0.255 172.16.0.0 0.15.255.255

deny ip 192.168.60.0 0.0.0.255 10.0.0.0 0.255.255.255
deny ip 192.168.60.0 0.0.0.255 192.168.0.0 0.0.255.255
deny ip 192.168.60.0 0.0.0.255 172.16.0.0 0.15.255.255

deny ip 10.1.1.0 0.0.0.255 10.0.0.0 0.255.255.255
deny ip 10.1.1.0 0.0.0.255 192.168.0.0 0.0.255.255
deny ip 10.1.1.0 0.0.0.255 172.16.0.0 0.15.255.255

deny ip 10.2.1.0 0.0.0.255 10.0.0.0 0.255.255.255
deny ip 10.2.1.0 0.0.0.255 192.168.0.0 0.0.255.255
deny ip 10.2.1.0 0.0.0.255 172.16.0.0 0.15.255.255

permit ip any any

route-map PBR_RMAP permit 10
match ip address Deny_New_to_Old_ACL
set ip next-hop 192.168.50.250

interface Vlan1231
ip address 10.1.1.254 255.255.255.0
ip policy route-map PBR_RMAP

interface Vlan1232
ip address 10.2.1.254 255.255.255.0
ip policy route-map PBR_RMAP

interface Vlan1226
ip address 192.168.1.254 255.255.255.0
ip policy route-map PBR_RMAP

interface Vlan1227
ip address 192.168.60.254 255.255.255.0
ip policy route-map PBR_RMAP

ip route 0.0.0.0 0.0.0.0 192.168.1.250 (Old gateway)

Giuseppe Larosa · ‎09-07-2020

Hello @VRAI ,

now the configuration looks like correct .

How do you make the test ?

Do you send traffic sourced from host 10.2.1.x to a public address ?

access-list 121 remark for debug

access-list permit ip host 10.2.1.x any

debug ip packet det 121

and/or debug ip policy

Hope to help

Giuseppe

VRAI · ‎09-07-2020

I sent ping and tracert from host in 10.2.1.x/24 to www.google.com and other web address (also tried via IP)

here is the ACL debug output:

Sep 7 18:20:26.541: IP: s=10.2.1.1 (Vlan1232), d=8.8.8.8, len 92, policy match
Sep 7 18:20:26.541: ICMP type=8, code=0
Sep 7 18:20:26.541: IP: route map PBR_RMAP, item 10, permit
Sep 7 18:20:26.541: IP: s=10.2.1.1 (Vlan1232), d=8.8.8.8, len 92, policy rejected -- normal forwarding
Sep 7 18:20:26.541: ICMP type=8, code=0
Sep 7 18:20:26.541: IP: s=10.2.1.1 (Vlan1232), d=8.8.8.8, len 92, input feature
Sep 7 18:20:26.541: ICMP type=8, code=0, Policy Routing(74), rtype 0, forus FALSE, sendself FALSE, mtu 0, fwdchk FALSE
Sep 7 18:20:26.541: FIBipv4-packet-proc: route packet from Vlan1232 src 10.2.1.1 dst 8.8.8.8
Sep 7 18:20:26.541: FIBfwd-proc: packet routed by adj to Vlan1212 192.168.1.250
Sep 7 18:20:26.541: FIBipv4-packet-proc: packet routing succeeded
Sep 7 18:20:26.541: IP: tableid=0, s=10.2.1.1 (Vlan1232), d=8.8.8.8 (Vlan1212), routed via FIBpak 634617C consumed in input feature , packet consumed, MCI Check(80), rtype 0, forus FALSE, sendself FALSE, mtu 0, fwdchk FALSE
Sep 7 18:20:26.541: IP: s=10.2.1.1 (Vlan1232), d=8.8.8.8, len 92, policy match
Sep 7 18:20:26.541: ICMP type=8, code=0
Sep 7 18:20:26.541: IP: route map PBR_RMAP, item 10, permit
Sep 7 18:20:26.541: IP: s=10.2.1.1 (Vlan1232), d=8.8.8.8, len 92, policy rejected -- normal forwarding
Sep 7 18:20:26.550: ICMP type=8, code=0
Sep 7 18:20:26.550: IP: s=10.2.1.1 (Vlan1232), d=8.8.8.8, len 92, input feature
Sep 7 18:20:26.550: ICMP type=8, code=0, Policy Routing(74), rtype 0, forus FALSE, sendself FALSE, mtu 0, fwdchk FALSE
Sep 7 18:20:26.550: FIBipv4-packet-proc: route packet from Vlan1232 src 10.2.1.1 dst 8.8.8.8
Sep 7 18:20:26.550: FIBfwd-proc: packet routed by adj to Vlan1212 192.168.1.250
Sep 7 18:20:26.550: FIBipv4-packet-proc: packet routing succeeded
Sep 7 18:20:26.550: IP: tableid=0, s=10.2.1.1 (Vlan1232), d=8.8.8.8 (Vlan2121), routed via FIBpak 6344AE8 consumed in input feature , packet consumed, MCI Check(80), rtype 0, forus FALSE, sendself FALSE, mtu 0, fwdchk FALSE
Sep 7 18:20:26.550: IP: s=10.2.1.1 (Vlan1232), d=8.8.8.8, len 92, policy match
Sep 7 18:20:26.550: ICMP type=8, code=0
Sep 7 18:20:26.550: IP: route map PBR_RMAP, item 10, permit
Sep 7 18:20:26.550: IP: s=10.2.1.1 (Vlan1232), d=8.8.8.8, len 92, policy rejected -- normal forwarding
Sep 7 18:20:26.550: ICMP type=8, code=0
Sep 7 18:20:26.550: IP: s=10.2.1.1 (Vlan1232), d=8.8.8.8, len 92, input feature
Sep 7 18:20:26.550: ICMP type=8, code=0, Policy Routing(74), rtype 0, forus FALSE, sendself FALSE, mtu 0, fwdchk FALSE
Sep 7 18:20:26.550: FIBipv4-packet-proc: route packet from Vlan1232 src 10.2.1.1 dst 8.8.8.8
Sep 7 18:20:26.558: FIBfwd-proc: packet routed by adj to Vlan1212 192.168.1.250
Sep 7 18:20:26.558: FIBipv4-packet-proc: packet routing succeeded

I also attached a file with a load of output from debug IP policy. The routed by adjacent vlan output is interesting...this is the old gateway vlan/ip

VRAI · ‎09-07-2020

What else is strange is that when I change the IP next hop statement to a different IP (10.2.1.250) it looks like it may be getting routed like it is supposed to according to policy:

Sep 7 19:12:39.636: IP: s=10.2.1.1 (Vlan1232), d=8.8.8.8, len 92, policy match
Sep 7 19:12:39.636: ICMP type=8, code=0
Sep 7 19:12:39.636: IP: route map PBR_RMAP, item 10, permit
Sep 7 19:12:39.636: IP: s=10.2.1.1 (Vlan1232), d=8.8.8.8 (Vlan1227), len 92, policy routed
Sep 7 19:12:39.636: ICMP type=8, code=0
Sep 7 19:12:39.636: IP: Vlan1232 to Vlan1231 10.1.1.250
Sep 7 19:12:39.636: IP: s=10.2.1.1 (Vlan1232), d=8.8.8.8 (Vlan1227), len 92, input feature
Sep 7 19:12:39.636: ICMP type=8, code=0, Policy Routing(74), rtype 2, forus FALSE, sendself FALSE, mtu 0, fwdchk FALSE pak 6342FD0 consumed in input feature , packet consumed, MCI Check(80), rtype 2, forus FALSE, sendself FALSE, mtu 0, fwdchk FALSE
Sep 7 19:12:39.644: IP: s=10.2.1.1 (Vlan1232), d=8.8.8.8, len 92, policy match
Sep 7 19:12:39.644: ICMP type=8, code=0
Sep 7 19:12:39.644: IP: route map PBR_RMAP, item 10, permit
Sep 7 19:12:39.644: IP: s=10.2.1.1 (Vlan2132), d=8.8.8.8 (Vlan1227), len 92, policy routed
Sep 7 19:12:39.644: ICMP type=8, code=0
Sep 7 19:12:39.644: IP: Vlan1231 to Vlan1227 10.1.1.250
Sep 7 19:12:39.644: IP: s=10.2.1.1 (Vlan1231), d=8.8.8.8 (Vlan1227), len 92, input feature
Sep 7 19:12:39.644: ICMP type=8, code=0, Policy Routing(74), rtype 2, forus FALSE, sendself FALSE, mtu 0, fwdchk FALSE pak 6346600 consumed in input feature , packet consumed, MCI Check(80), rtype 2, forus FALSE, sendself FALSE, mtu 0, fwdchk FALSE
Sep 7 19:12:39.653: IP: s=10.2.1.1 (Vlan1231), d=8.8.8.8, len 92, policy match
Sep 7 19:12:39.653: ICMP type=8, code=0
Sep 7 19:12:39.653: IP: route map DR_VDOM_RMAP, item 10, permit
Sep 7 19:12:39.653: IP: s=10.2.1.1 (Vlan1231), d=8.8.8.8 (Vlan1227), len 92, policy routed
Sep 7 19:12:39.653: ICMP type=8, code=0
Sep 7 19:12:39.653: IP: Vlan1231 to Vlan1227 10.1.1.250
Sep 7 19:12:39.653: IP: s=10.2.1.1 (Vlan1231), d=8.8.8.8 (Vlan1227), len 92, input feature

2 Gateways of last resort:-Route based on source subnet to unknown destinations (PBR): Cisco 3750x switch