PBR can't override connected routes

[Netnux]

Hello Everybody.
I'm configuring a little PBR on a Cisco 6504.

I have a VLAN with a SVI with an IP Address of 172.32.41.1 that works as the gateway for the 172.32.41.0/24 subnet

I want all the traffic from this subnet to all the 192.168.X.Y networks to go towards the 172.32.20.12 address, instead of going through the normal routing.
I configured the PBR (ACL, route-map and policy) and it works, but... if I have a connected network on the 192.18.X.Y range, my router ignores the PBR.
If I try from the 172.32.41.0/24 subnet to reach the 192.168.2.1 address, the router goes to another SVI interface with that address configured.

Also, if I run the "deb ip policy" command, I see nothing.
Even when the PBR is surely working (I verify I reach the intended network).

What am I doing wrong?

Many thanks for your kind help

[Richard Burts]

I am surprised that PBR does not seem to work when the destination is a connected subnet. Would you post the output of show ip interface brief, your PBR ACL, and your PBR route map?

HTH

Rick

[Netnux]

Hello Richard-

Thank you for your kind reply.

I'm also surprised and so I hope I made some silly error somewhere

Here are my show outputs. I just removed the many physical interfaces from the output of the "sh ip int brief", for more confortable reading.

The interested SVI are VLAN2 and VLAN214 (in red).

Many thanks for your help.

Giovanni

-----------------------------------

CS-6500#
CS-6500#sh ip int brief
Interface              IP-Address      OK? Method Status                Protocol
[...]
Vlan1                  172.17.2.65     YES NVRAM  up                    up
Vlan2                  192.168.2.1     YES NVRAM  up                    up
Vlan5                  192.168.5.1     YES NVRAM  up                    up
Vlan10                 192.168.17.14   YES NVRAM  up                    up
Vlan11                 192.168.17.20   YES NVRAM  up                    up
Vlan22                 192.168.22.1    YES manual up                    up
Vlan23                 172.23.10.1     YES manual up                    up
Vlan24                 unassigned      YES manual down                  down
Vlan28                 unassigned      YES manual down                  down
Vlan29                 172.23.250.1    YES manual up                    up
Vlan31                 192.168.31.1    YES NVRAM  up                    up
Vlan32                 192.168.32.1    YES NVRAM  up                    up
Vlan33                 192.168.33.1    YES NVRAM  up                    up
Vlan35                 192.168.35.1    YES NVRAM  up                    up
Vlan61                 192.168.70.1    YES NVRAM  up                    up
Vlan62                 192.168.62.1    YES manual up                    up
Vlan63                 192.168.63.1    YES manual up                    up
Vlan90                 192.168.90.1    YES NVRAM  up                    up
Vlan107                unassigned      YES NVRAM  up                    up
Vlan110                unassigned      YES NVRAM  administratively down down
Vlan111                192.168.111.1   YES NVRAM  up                    up
Vlan113                192.168.113.1   YES NVRAM  up                    up
Vlan115                192.168.115.1   YES NVRAM  up                    up
Vlan116                192.168.116.1   YES NVRAM  up                    up
Vlan117                192.168.117.1   YES NVRAM  up                    up
Vlan118                192.168.118.1   YES NVRAM  up                    up
Vlan119                192.168.119.1   YES NVRAM  up                    up
Vlan120                192.168.120.1   YES NVRAM  up                    up
Vlan121                192.168.121.1   YES manual up                    up
Vlan122                192.168.122.1   YES manual up                    up
Vlan123                192.168.123.1   YES manual up                    up
Vlan124                192.168.124.1   YES manual up                    up
Vlan192                unassigned      YES manual administratively down down
Vlan199                192.168.199.1   YES manual up                    up
Vlan202                172.23.2.1      YES manual up                    up
Vlan214                172.32.41.1     YES manual up                    up
Vlan220                172.23.20.1     YES manual administratively down down
Vlan222                172.23.22.1     YES manual up                    up
Vlan240                172.23.240.1    YES manual up                    up
Vlan245                192.168.245.1   YES manual down                  down
Vlan250                192.168.250.1   YES manual up                    up
Vlan253                172.23.253.1    YES manual up                    up
Vlan990                192.168.80.1    YES manual up                    up
Vlan991                unassigned      YES unset  administratively down down
Vlan992                unassigned      YES unset  administratively down down
Vlan993                192.168.93.1    YES manual up                    up
Vlan994                unassigned      YES unset  administratively down down

CS-6500#sh access-lists pbr-NET_it-192.168
Extended IP access list pbr-NET_it-192.168
    10 permit ip any 192.168.0.0 0.0.255.255 (21 matches)
CS-6500#

CS-6500#sh route-map pbr-NET_it-192.168
route-map pbr-NET_it-192.168, permit, sequence 10
  Match clauses:
    ip address (access-lists): pbr-NET_it-192.168
  Set clauses:
    ip next-hop 172.23.2.21
  Policy routing matches: 21 packets, 2226 bytes
CS-6500#

CS-6500#sh run int vlan 214
Building configuration...

Current configuration : 137 bytes
!
interface Vlan214
 ip address 172.32.42.1 255.255.255.0
 ip policy route-map pbr-NET_it-192.168
end

CS-6500#

[Richard Burts]

Thank you for posting the information that I requested. It seems appropriate and looks like it does work some of the time. I have an additional test that I would ask you to perform:

- pick a host in the 172.32.42 subnet that you will test with.

- post the output of ipconfig and of route print from that host (if the output is extensive I particularly want to see anything that relates to 192.168.x.x)

- post the output of show access-list pbr-NET_it-192.168 and show route-map pbr-NET_it-192.168 (to establish a base line of how many matches).

- attempt access from the test host to an address in 192.168.2

- post the output of show access-list pbr-NET_it-192.168 and show route-map pbr-NET_it-192.168

HTH

Rick

[Netnux]

Hello Richard.

I surely will perform this test and post the results as soon as I can (not at work, now), but I have performed this test already.

The 172.32.41.0/24 network is "fresh new", so the 21 matches you can see on the posted "sh acl" are because of my tests from a fresh installed test-laptop on this network.
Currently, there is no other traffic on this subnet.

From this test-laptop on the 172.32.41.0 network, when I ping to a destination on a 192.168.X.X network I don't have already assigned to a SVI on my 6504, PBR works and I can see the match counter to increase.
When I ping the 192.168.2.1, or any other address on the 192.168.X.X range I already assigned to a SVI... it doesn't work.

The next hop in the route-map of my PBR is a CE of an MPLS network.
So, with a tracert (windows) from the test-laptop to a 192.168.X.X I can see the packets go through the intended next hop (CE) and then on the MPLS network till the destination (I'm on the customer side).

If I try to reach the 192.168.2.1 on the MPLS network, I can only see one line in my tracert output: the destination SVI reached on my 6504.

Also, I tryed to shut the VLAN 2 interface and... yes.
The tracert to the 192.168.2.1 from the test laptop tells I go through the PBR intended next hop, and after some hops fails, because I don't really have an host with 192.168.2.1 address on the MPLS network
When I turn the VLAN2 interface back on, ping from the test laptop goes fine ad tracert tells me I stop at 6504.

I missed to say that the 6504 switch, really are a couple of twin 6504 in VSS, but I don't really know if that matter, in this case.

Many thanks for your precious help.

Giovanni

[Richard Burts]

Giovanni

Thanks for the explanation. Given what you have told me I suspect we can predict what we will see in the test results. But I would still appreciate your executing the test that I suggest and post the results.

If the next hop in the route map is the CE for MPLS then I wonder about the logic of sending all traffic for 192.168.0.0/16 to it when there are several 192.168 networks connected on your side of the MPLS.

HTH

Rick

[Netnux]

Hello richard.
Sorry for the late, I had to work elsewhere, yesterday.
Yes, we have some connected local networks on the 192.168.X.X prefix, as you can see.

The need is due to a company merge.
The new company also has many remote sites on the 192.168.X.X prefix.
We need to reach those sites through a MPLS link, from the new network created for the new company.
I don't care to reach the local 192.168.X.X networks, from that subnet.

I performed the tests and I have found something is changed (I did nothing).

Here are the ipconfig and route print of the test-laptop.
I just skipped the non-significant parts.

Ethernet adapter Local Area Connection:

   Connection-specific DNS Suffix  . : XXXXX.local
   Link-local IPv6 Address . . . . . : fe80::38bf:7c1f:bc37:7a22%11
   IPv4 Address. . . . . . . . . . . : 172.32.41.11
   Subnet Mask . . . . . . . . . . . : 255.255.255.0
   Default Gateway . . . . . . . . . : 172.32.41.1

IPv4 Route Table
===========================================================================
Active Routes:
Network Destination        Netmask          Gateway       Interface  Metric
          0.0.0.0          0.0.0.0      172.32.41.1     172.32.41.11     20
        127.0.0.0        255.0.0.0         On-link         127.0.0.1    306
        127.0.0.1  255.255.255.255         On-link         127.0.0.1    306
  127.255.255.255  255.255.255.255         On-link         127.0.0.1    306
      172.32.41.0    255.255.255.0         On-link      172.32.41.11    276
     172.32.41.11  255.255.255.255         On-link      172.32.41.11    276
    172.32.41.255  255.255.255.255         On-link      172.32.41.11    276
        224.0.0.0        240.0.0.0         On-link         127.0.0.1    306
        224.0.0.0        240.0.0.0         On-link      172.32.41.11    276
  255.255.255.255  255.255.255.255         On-link         127.0.0.1    306
  255.255.255.255  255.255.255.255         On-link      172.32.41.11    276
===========================================================================
Persistent Routes:
  None

I cannot figure out.
When I wrote my first post, ping from the test-laptop to 192.168.2.1 was successful and the answer was from local VLAN2 interface.
Now the same ping is unsuccessful! I cannot ping 192.168.2.1 anymore.

However, if I try a "tracert" of the 192.168.2.1 from the test-laptop I see the following.
-----
Tracing route to 192.168.2.1 over a maximum of 30 hops

  1    <1 ms    <1 ms    <1 ms  192.168.2.1

Trace complete.
----

If I try the same "tracert" after shutting down the VLAN2 interface on the Cisco 6504, I see the correct path:
-----
Tracing route to 192.168.2.1 over a maximum of 30 hops

  1    <1 ms    <1 ms    <1 ms  172.32.41.1
  2    <1 ms    <1 ms    <1 ms  172.23.2.22
  3     2 ms     2 ms     2 ms  AAAAAAAA.BBBB.somedomain.fr [A.B.C.D]
  4     3 ms     3 ms     2 ms  10.205.100.1
  5     4 ms     3 ms     3 ms  10.205.100.2
  6  10.205.100.2  reports: Destination host unreachable.

Trace complete.
-----

OK, maybe there is nothing on the 192.168.2.1, but with the VLAN2 interface shut down, the path is correct.

If I try the "tracert" of an address I don't have I see the following:
-----
Tracing route to 192.168.135.1 over a maximum of 30 hops

  1    <1 ms    <1 ms    <1 ms  172.32.41.1
  2    <1 ms    <1 ms    <1 ms  172.23.2.22
  3     2 ms     2 ms     2 ms  AAAAAAAA.BBBB.somedomain.fr [A.B.C.D]
  4    65 ms    74 ms    38 ms  CCCCCCCC.DDDD.somedomain.fr [A.B.C.D]
  5    21 ms    21 ms    21 ms  192.168.135.1

Trace complete.
-----

Since I don't have anything on 192.168.135.0, and even I don't have this network in my routing table, I'm quite sure that PBR is working.
Also, the second hop is the expected router configured by the route-map.
However... if I take a show of the involved ACL before and after del test, I always see the same number of matches:
-----
CS-6500#sh access-lists pbr-NET_it-192.168
Extended IP access list pbr-NET_it-192.168
    10 permit ip any 192.168.0.0 0.0.255.255 (15 matches)
CS-6500#

CS-6500#sh route-map pbr-NET_it-192.168
route-map pbr-mfe2_it-192.168, permit, sequence 10
  Match clauses:
    ip address (access-lists): pbr-NET_it-192.168
  Set clauses:
    ip next-hop 172.23.2.21
  Policy routing matches: 21 packets, 2226 bytes
6500#
-----

Moreover, if I try a tracert from the test-laptop to an address I have both assigned to a local SVI and on the assigned toa host in the MPLS VPN, I see:

-----
Tracing route to 192.168.245.1 over a maximum of 30 hops

  1    <1 ms    <1 ms    <1 ms  172.32.41.1
  2    <1 ms    <1 ms    <1 ms  172.23.2.22
  3     2 ms     2 ms     2 ms  AAAAAAAA.BBBB.somedomain.fr [A.B.C.D]
  4    65 ms    74 ms    38 ms  CCCCCCCC.DDDD.somedomain.fr [A.B.C.D]
  5    21 ms    21 ms    21 ms  192.168.245.1

Trace complete.
-----

So I could say that PBR now definetly works, but I cannot figure out the behaviour of the 192.168.2.1 I have been using for tests at the beginning.

Thank you for your kind help

Giovanni

[Richard Burts · Accepted solution]

Giovanni

I am glad to know that PBR now does seem to be working. I believe that there is an explanation for the behavior of tracert to 192.168.2.1. Your test PC sends the first tracert packet with TTL = 1. It gets to the 6504 which decrements the TTL and determines that it should discard the packet and send an ICMP response. Note that it is not trying to route this packet (which would have gotten it to PBR) but discards the packet. The 6504 generates an ICMP packet indicating TTL expired and my guess is that the source of the ICMP is the 192.168.2.1 address. When the ICMP gets to your test PC it interprets it as a successful completion of the trace route.

HTH

Rick

[Richard Burts]

Giovanni

This has been a very interesting discussion and I am glad that my explanations have been helpful. Thank you for using the rating system to mark this question as answered. This will help other readers in the forum to identify discussions which have helpful information.

HTH

Rick

[Netnux]

I don't get the answer from 172.32.41.1 on the first line, which would be the normal intended gateway, but from 192.168.2.1.
I think you are right.
Great.

I still miss some things, however
I cannot figure out why I don't have the same behaviour if I point to an address the "PBR next hop" knows about.
If I point the trace to 192.168.135.0/24 (I don't have it in routing table) or to 192.168.245.0/24 (I also have it in routing table) I get the answer I wait for aspect
At least the 192.168.245.0/24 should behave the same as the 192.168.2.0/24, but it's not.

Above all, I cannot understand why just after the configuration of the PBR and still some hours after, the ping to the 192.168.2.1 was successful, of course from the SVI interface on the 6504.
The day after, the ping to 192.168.2.1 was unsuccessful, after have done just nothing.
This behaviour is repeated each time i modify the routemap.
I have to sit and wait till the day after to have it working.

I'll try to investigate more, with more calm, since my users can work.

Many thanks for you help!

Regards,
Giovanni

[Richard Burts]

Giovanni

I am thinking about your comment about the ping to 192.168.2.1. Am I correct that you are talking about pinging that address from the interface of the 6504? Unless you have configured PBR for local as well as for the interface then PBR should not impact ping from the switch interface. Can you clarify whether there is any PBR configured in addition to vlan 214?

HTH

Rick