I have an ASA on which I'm trying to use PBR to route to one of two ISPs which
I'll call "slow" and "fast."
The interface to the slow ISP is connected to a subnet on which the next-hop
address is clearly in the subnet and it would count as "directly connected."
The interface to the fast ISP is connected via pppoe. The interface address
is on a different subnet than the next-hop address so it would not be directly
The default route is to the slow ISP.
When I create route maps to send traffic to the slow ISP, I see the next-hop
address being selected and the egress interface selected in the first phase
of the packet trace. That tells me that my rules are working.
When I switch the map's next-hop address to be the next-hop address of the fast
ISP interface, PBR is selecting the right next-hop address, but it leaves the
egress interface decision to the next processing step, which always selects
the slow ISP interface. Using the recursive next-hop address selection in
the route map doesn't correct the problem.
Any suggestions on how to fix this? The only thing I can think of is to set
the default route to the fast ISP and use PBR to route to the exceptions
that need to go over the slow ISP instead of now where the exceptions
are to route to the fast ISP.
I assume that you're using ASA version 9.4 minimum (http://www.cisco.com/c/en/us/td/docs/security/asa/asa94/config-guides/cli/general/asa-94-general-config/route-policy-based.html)
the ip next-hop is working only if next-hop IP is directly connected.
You might try with set ip default next-hop or set interface.
could you give the output of debug ip policy?
Yes, ASA 9.5(1) on a 5506-X. I have tried ip next-hop, default next-hop, recursive next-hop, and set interface. I think the debug output confirms that the problem is with this ppoe configured interface, since my default route is out the slow interface, there is no route to the outside world on the fast interface but I thought that PBR would fix that. Here's the output:
pbr: policy based route lookup called for 192.168.20.22/45321 to 22.214.171.124/0 proto 1 sub_proto 8 received on interface inside
pbr: First matching rule from ACL(15)
pbr: route map SingleHostTest, sequence 50, permit; proceed with policy routing
pbr: evaluating recursive next-hop 126.96.36.199
pbr: no route to next-hop 188.8.131.52 found
pbr: evaluating interface fast
pbr: no route to 184.108.40.206 found on interface fast
pbr: policy based routing could not be applied; proceeding with normal route lookup
<uses default route out slow interface>
Here is the policy map with recursive next hop and set interface configured as was the case when I tested:
route-map SingleHostTest, permit, sequence 50
ip address (access-lists): test_route_maps
ip next-hop recursive 220.127.116.11
I'm missing something on your explanation.
Could you send a quick sketch of your infrastructure? And sending out interfaces, routes and route-map configuration?
PBR is a simple thing that's working quite in the same way as router, I've never done any testing with pppoe interface but I'm certainly missing something on your explanation but with a design and some config output it would be clear.
Would you mind to add the output of sh int ip brief ?
Here it is, addresses have been changed for anonymity.
I am convinced that PBR is working fine but that the problem is basic routing. I don't know how to set a route that would direct any traffic reaching the fast interface out to any IP address. And that is complicated by it being a pppoe-defined interface without a directly connected next-hop IP address.
So no matter what I do in PBR, the route lookup will always send out the default (slow) interface.
Thanks so much for helping… this is not my "day job" so I'm over my head.
I don't have any pppoe connection right here. I'm trying to mount my Cisco Virl lab up and simulate a pppoe connection in order to test.
Maybe some issues with ASA and pppoe. Have you raised a TAC case?
I've never tried with pppoe connection. and even if you are doing a set route for the pppoe, as you already have a default route it will not be installed.
Try to see with TAC and in the mean time, I'll try to build up my lab soon.
I've an issue with my virl server. Waiting for Cisco feedback and then I'll build up the lab to test. In the mean time, are you able to configure a fix IP for test pppoe connection? You can set a old router as pppoe server.
Thanks so much for taking an interest in this. I'm starting think that this is a problem with pppoe and routing and not PBR.
I trie switching the default route to the fast ISP / pppoe connection and found problems with the traffic PBR routed out the slow interface, so I tried switching back. ASDM would not let me set the default route to the slow interface because of a "conflict with existing routes." The state was no static route at all so I essentially had a brick but I can configure it inbound over the fast interface. I think that there is an implicit default route out the fast/pppoe interface now but it doesn't show up in the static IP list (doing this through ASDM)
I'll raise a TAC case if nothing in this situation sounds familiar to anyone.
I've tried it in my virtual lab (Cisco VIRL) and I'm facing the exact same issue when my interface out is configured as PPPOE. Be careful, virtual is not the same as reality, however as we have exact same issue and we've not copied our configs, I think you need to raise a TAC case.
Please come back to us with TAC solution if possible.
Thanks so much
Wow, thanks for re-creating the problem.
I see what happened when I removed the default route to the slow ISP, the route out the pppoe interface became the default route because of "ip address pppoe set route." The other thing that happened is that I believe (and I didn't watch for long before I tried restoring the default route) that inbound traffic to the DMZ over the slow interface was being routed out (and dropped) on the fast interface.
To your knowledge, should PBR be able to override the default route? It seems like if you are setting the next-hop address you are declaring that you know what route to use and the ASA shouldn't reevaluate what route the packet takes. Clearly if I can get packets out to the next-hop addresses defined by PBR, the upstream routers will know what to do with them.
The goal of pbr is to forward specific traffic out to a different next hop without taking into account the default route. However, the next hop has to be reachable.
PBR will not erase the default route because others will use it if not matching pbr acl.
in your case asa seems to not be able to get out to pppoe interface and lokking towards the routing table. If you test with 2 fixed ip addresses (no pppoe) it will work.
generally we use pbr to forward specific traffic based on source ip and destination port (https, https,....) out to a specific next hop. For example, you want specific traffic to be redirected out to a proxy; in this case pbr is usefull. This is the test I've done but asa don't work as expected, maybe a limitation (but not seen on release notes). TAC should debug it in details if a bug or they will say that's a limitation.
It's along time I'm using pbr but never with pppoe ISP. I never gate such specific case.
Keep us informed about TAC solution.
First word back from TAC is that PBR+pppoe as the secondary route is a known limitation but not documented as such.
We are supposed to be able to allow the pppoe interface to have the primary route and use the static (slow in this example) be a backup route with a higher metric.
When I get this going I will report back.
Ok thanks. This is what I've configured yesterday but not yet tested (run out of time, sorry).
But quite sure that it will works in that way
I have it configured and running with great help from TAC.
As I noted earlier, the pppoe interface cannot be the secondary interface, most likely because it sets a default route with metric 1 when it finds out what the next hop is. This also makes some engineering sense to me because if I configure PBR to set the next hop out pppoe, and the upstream provider changes the next hop address, my configuration would be broken.
The solution is to allow pppoe (fast in the diagram) to set the default route (ip address pppoe setroute), and use PBR to send any exceptions to the static (slow) interface. This is straightforward because the rules you're creating are just the inverse of using PBR to route out the fast interface. With the fast interface as the default, I just don't bother assigning a next hop for anything I want to go out that route.
The non-intuitive part for me is this: I would think that by setting next hop in PBR, the ASA would leave the route alone because clearly if I set the next hop I know where the packet should be routed. But after setting the next hop to be the slow interface, and the next-hop address is correctly set, the routing tables are still consulted and the packet is sent out the fast interface because it has the default route.
So you need to have a secondary default route with a higher metric to allow packets with next-hop set through PBR to go out the slow interface. So based on the updated diagram attached, I have:
route slow 0.0.0.0 0.0.0.0 xxx.xxx.xxx.73 2