Question regarding difference between Cisco policy based routing and linux port forwarding

[TouFueVue]

I am working a new firewall project that requires policy based routing to route all port 80 and 443 traffic to the load balancer which the filters are connected too.  Our Cisco router is unable to do PBR because of IOS version.  We have a Linux configured as router and port forwarding.  When port forwarding is turned enabled the port 80 and 443 traffic is hitting the load balancer but does not seem to hit the filter.  Our vendor who sold us the firewall says that they need to traffic to hit load balancer box via PBR and via port forwarding.  Is there difference between the two. If there is a difference can someone please help explain the difference.  

Thanks,

Tou

[Peter Paluch · Accepted solution]

Tou,

PBR and port forwarding are indeed different techniques.

PBR is a way of routing traffic based on other criteria than just a destination, but the original sender and destination addresses and ports are not changed. With PBR, you decide where to route packets based on, say, the sender address, or the the destination port, but you never change the addressing in these packets.

Port forwarding is a technique of rewriting the addresses and ports in packets so that after the rewriting, packets either appear to come from a different source, or they are delivered to a different destination. Port forwarding does not perform any path selection (routing). Port forwarding simply rewrites source IP and port in one packet flow direction, and destination IP and port in the opposite packet flow direction. However, how the packets are routed before/after their addressing is rewritten is not handled by the port forwarding but rather by the routing, optionally even PBR.

Would you perhaps mind to post how the PBR would be configured if you were able to configure it on your router, and how your Linux is currently configured? By seeing the PBR config (even though it is not deployed), we could perhaps understand what you intended to achieve, and by comparing it to your current Linux config, we would be able to find out the differences.

Also, is it possible to post a diagram of your network to understand how the load balancers and firewalls are connected, and where the Linux router is placed?

Best regards,
Peter

[TouFueVue]

Peter,

Thanks for the fast response and very good explanation.  I greatly appreciate your time in helping me solve this issue.  I am currently running an old 7603 with old code but there lots of  policy maps for QoS so I am not sure why my PBR does not work unless I am configuring it wrong but with your assistance I maybe able to get it to work.  Attached and posted below is my configs for my Cisco router and the Linux box.  Attached is the topology diagram.  Please ask any questions for clarification so you can help me solve this issue.

Cisco PBR config: 

route-map smoothwall permit 10
 match ip address smoothwall
 set ip next-hop 10.189.100.7

ip access-list extended smoothwall
 deny   ip host 10.189.100.2 any
 deny   ip host 10.189.100.3 any
 deny   ip host 10.189.100.4 any
 deny   ip any 10.0.0.0 0.255.255.255
 deny   ip any 172.16.0.0 0.15.255.255
 deny   ip any 192.168.0.0 0.0.255.255
 permit tcp any any eq www
 permit tcp any any eq 443
!

interface GigabitEthernet2/47
 description Link to ISP
 ip policy route-map smoothwall
 ip address 192.168.189.14 255.255.255.248

Linux config:

# Exempt the DMZ and other private addresses that are not filtered /sbin/iptables -t nat -A PREROUTING -p all -s 0.0.0.0/0 -d 172.16.0.0/16 -j ACCEPT /sbin/iptables -t nat -A PREROUTING -p all -s 0.0.0.0/0 -d

192.168.0.0/16 -j ACCEPT

/sbin/iptables -t nat -A PREROUTING -p all -s 10.189.100.0/24 -d

0.0.0.0/0 -j ACCEPT

# Enable the primary redirect to the load balancers in vlan10 # Test config - normally this would not be source limited to one machine /sbin/iptables -t nat -A PREROUTING -p tcp -i eth0 -s 10.189.16.101 --dport 80 -j DNAT --to 10.189.100.7 /sbin/iptables -t nat -A PREROUTING -p tcp -i eth0 -s 10.189.16.101 --dport 443 -j DNAT --to 10.189.100.7

# Allow everything from vlan10 to pass to the web.

/sbin/iptables -I INPUT-FORWARD-CHAIN -p tcp -s 10.189.100.0/24 -d

0.0.0.0/0 -j ACCEPT

This is how it looks in iptables when it is actually live - hope this is

not just a confused jumble due to line wrapping etc.

pkts bytes target  prot opt in     out     source               destination

0     0 ACCEPT     all  --  *      *       0.0.0.0/0

172.16.0.0/16

0     0 ACCEPT     all  --  *      *       0.0.0.0/0

192.168.0.0/16

0     0 ACCEPT     all  --  *      *       10.189.100.0/24  0.0.0.0/0

0     0 DNAT       tcp  --  eth0   *       10.189.16.101

0.0.0.0/0           tcp dpt:80 to:10.189.100.7

0     0 DNAT       tcp  --  eth0   *       10.189.16.101

0.0.0.0/0           tcp dpt:443 to:10.189.100.7

0     0 ACCEPT     tcp  --  *      *       10.189.100.0/24      0.0.0.0/0

[Peter Paluch]

Tou,

Thanks for the information.

So let me ask you this: Your 7600 is not able to perform PBR, correct? That is why you want the Linux to do something to achieve the PBR, right?

If my understanding as summarized above is correct then we have a problem. Your 7600 does not support PBR but it still operates as a router. That means that it is making the routing and forwarding choices on its own, independently of any other device in the network, just like normal IP routing rules dictate (every router is completely autonomous in its routing decisions). The Linux cannot do anything to make the 7600 suddenly do different routing decisions. In other words, you cannot move a PBR functionality one hop back before the router that was actually the proper place for doing the PBR, and hope that just because you made a specific routing choice one hop before, the subsequent router will do a consistent choice as well.

NAT/port forwarding isn't really a solution here, either, because when you change the original destination or source, packets never arrive to their proper destination or place of origin. Specifically, I see that the Linux is configured to do DNAT toward the load balancer - but in the process of doing this, it makes the load balancer to be the end receiver, not just a transit device. It's logical, then, that the packets never really make it where they originally were meant to be delivered.

If the PBR is to be done on Linux then first, we need to create a direct connection between the Linux and the load balancer. This can be technically accomplished by different means:

• Creating a dedicated physical connection between the Linux and the load balancer
• Creating a separate VLAN on the 7600 dedicated to interconnecting just the Linux and the load balancer. This VLAN will be purely switched by the 7600 - there will be no routed ports. The Linux and load balancer can be connected either via their own subinterfaces if they support them (7600 would be then using trunks), or via dedicated ports (7600 would be then using access ports)
• Creating a tunnel (GRE or IPIP) between the Linux and the load balancer.

By having this direct connection between the Linux and the load balancer, we can then configure the PBR on the Linux and have it deliver the packets directly to the load balancer.

Can any of this be accomplished in your network?

Best regards,
Peter

[Jon Marshall]

Peter / Tou

Without wishing to confuse the issue the 7600 does support PBR.

I notice the PBR has been applied to the interface connecting to the ISP which doesn't look right.

If the traffic that is meant to be sent to the load balancer is from the schools then it needs to be applied to the interface on the 7600 that the schools WAN link connects to assuming that is a L3 port or the SVI for that vlan.

Apologies if I have just added to the confusion.

Jon

[TouFueVue]

Not drawn in the topology, the WAN link is a port channel interface with 3-1 gig interface.  Do i apply that to the port channel config.  Also, what about the traffic that is from the local network and not drawn in the topology is the traffic from the wirreless controller.  The reason i applied in the interface going to isp is because i wanted it to route all 80 and 443 traffic.  Please assist me more to solve this issue.   Thanks, Tou

[Jon Marshall · Accepted solution]

Tou

You have to apply PBR to the L3 interface the traffic arrives on.

You cannot apply it to the interface connecting to the ISP because, if I understand correctly, that is the not the right interface.

So for the WAN traffic if the port channel is a L3 etherchannel then apply the PBR to the 7600 port channel. If it is a L2 etherchannel it must be associated with a vlan so apply PBR to the L3 vlan interface (SVI) for that vlan.

For your local traffic you do the same ie. presumably you have vlans so you apply the PBR to the SVIs for those vlans.

Like I say it must be applied to the L3 interface on the 7600 that the traffic arrives on and you can apply the same route map to multiple L3 interfaces whether they are physical L3 ports or SVIs.

Edit - you should also remove the PBR from the ISP link.

Jon

[Jon Marshall]

I just noticed in your original post you said your router won't do PBR because of the IOS version (sorry to both of you for not noticing that !).

I assumed because it was in the configuration that it did.

Or is that configuration not actually applied to the 7600 ?

Are you sure or are you saying it doesn't do it because your PBR didn't work ?

Jon

[TouFueVue]

Jon,

Sorry for the confusion.  Based on my configuration that I posted, I could not get it to work so I assume that the 7603 did not support it.  All along it may have been me mis-configuring it.  I will take your advice and apply it to correct interface.  Based on what I have, is my configuration correct?  Please ask any questions for clarification so I can get PBR working.  I greatly appreciate it.

Tou 

[Jon Marshall]

Tou

It depends on what you are trying to do with your acl in the route map.

It's not clear what each line of your route map acl is meant to be doing so perhaps you could explain ?

You could try applying your PBR to one L3 interface to see if it works.

If you do before you apply it take a note of the CPU on the router and then after applying check it again.

Which IOS version are you running ?

Jon

[TouFueVue]

I currently running old code of: 12.2(17d)SXB11a.  

Besides checking the CPU load, what is another way to check to see PBR is working correctly?

ACL:

ip access-list extended smoothwall
 deny   ip host 10.189.100.2 any- content filters in the vlan configure for them within 7603
 deny   ip host 10.189.100.3 any- content filters in the vlan configure for them within 7603
 deny   ip host 10.189.100.4 any- content filters in the vlan configure for them within 7603
 deny   ip any 10.0.0.0 0.255.255.255- not PBR any port 80 to 443 to these internal networks.
 deny   ip any 172.16.0.0 0.15.255.255- not PBR any port 80 to 443 to these internal networks.
 deny   ip any 192.168.0.0 0.0.255.255- not PBR any port 80 to 443 to these internal networks.
 permit tcp any any eq www- route any port 80 to the content filter virtual IP (10.189.100.7)
 permit tcp any any eq 443- route any port 80 to the content filter virtual IP (10.189.100.7)

[Jon Marshall]

Okay, not sure you need the first 3 lines because you won't be applying PBR to the 10.189.100.x IP subnet or at least I don't think you will.

Are you thinking of applying PBR to that IP subnet ?

The other lines make sense.

Some devices do not like using deny lines in a PBR acl so as I say keep an eye on your CPU just in case.

Edit - ideally you could use the "set ip default next-hop ..." command and then you would only need your last two lines in your acl but I just checked the 7600 configuration guide and it says that command is not supported or rather it would mean all traffic is process switched which you don't want.

So probably best to stick with what you have in terms of the acl except the lines for the content filters vlan as above.

Jon