Routing Issue

[jmbattlemotors]

I'm having an issue with routing I can't seem to figure out. We have a Cisco 3850 switch stack that operates at the gateway for all devices on our network. The 3850 then routes traffic to our firewall (10.51.5.1) before going out to the internet. We're in the process of replacing this firewall with a new one, which is currently on 10.51.5.245. Our current routing table is below.

Gateway of last resort is 10.51.5.1 to network 0.0.0.0

S* 0.0.0.0/0 [1/0] via 10.51.5.1
10.0.0.0/8 is variably subnetted, 23 subnets, 3 masks
S 10.51.0.0/16 [1/0] via 10.51.5.1
C 10.51.5.0/24 is directly connected, Vlan5
L 10.51.5.254/32 is directly connected, Vlan5
C 10.51.10.0/24 is directly connected, Vlan10
L 10.51.10.254/32 is directly connected, Vlan10
C 10.51.11.0/24 is directly connected, Vlan11
L 10.51.11.254/32 is directly connected, Vlan11
C 10.51.15.0/24 is directly connected, Vlan15
L 10.51.15.254/32 is directly connected, Vlan15
C 10.51.17.0/24 is directly connected, Vlan17
L 10.51.17.254/32 is directly connected, Vlan17
C 10.51.19.0/24 is directly connected, Vlan19
L 10.51.19.254/32 is directly connected, Vlan19
C 10.51.30.0/24 is directly connected, Vlan30
L 10.51.30.254/32 is directly connected, Vlan30
C 10.51.35.0/24 is directly connected, Vlan35
L 10.51.35.254/32 is directly connected, Vlan35
C 10.51.40.0/24 is directly connected, Vlan40
L 10.51.40.254/32 is directly connected, Vlan40
C 10.51.50.0/24 is directly connected, Vlan50
L 10.51.50.254/32 is directly connected, Vlan50
S 10.51.150.0/24 [1/0] via 10.51.5.250
S 10.99.0.0/16 [1/0] via 10.51.5.245
S 192.168.20.0/24 [1/0] via 10.51.5.245
S 192.168.39.0/24 [1/0] via 10.51.5.1
C 192.168.240.0/22 is directly connected, Vlan240
192.168.240.0/32 is subnetted, 1 subnets
L 192.168.240.254 is directly connected, Vlan240

So there's a route in here for 10.51.0.0 sending all traffic to 10.51.5.1, which is our firewall. All our devices are on this 10.51.0.0 network, depending on the VLAN they're on. If I change this route to go to 10.51.5.245 (our new firewall), traffic still gets routed through the old firewall. If I have 10.51.0.0/16 routing to 10.51.5.245, shouldn't my internet traffic go through that new firewall if I'm on a device with an IP of 10.51.X.X? I know there's a blackhole route in here sending all traffic to 10.51.5.1, but that should only apply if there's not a specific route in the table for my network, no?

[Richard Burts]

Probably the most important entry in your routing table is this

S* 0.0.0.0/0 [1/0] via 10.51.5.1

which is what determines how your traffic will get to the Internet. You need to change that to the new gateway.

I am somewhat surprised by the route 

S 10.51.0.0/16 [1/0] via 10.51.5.1

The /16 indicates that this is a summary route. You have many more specific routes for that address range (each of your vlan subnets). This route says that anything for 10.51.x.x that is not in a connected subnet should go to the gateway. But traffic would do that anyway. I believe that if you removed that static route that the behavior of your network would not change.

[jmbattlemotors]

OK, that makes sense. My thought was that the 10.51.0.0/16 route would take precedent over the 0.0.0.0 route as long as you were on that 10.51.0.0 network, but it doesn't sound like that's the case. I think you're right in saying it's not doing anything, because like I said, I changed that route to go to the new firewall and it did absolutely nothing.

I'm not quite prepared to change the 0.0.0.0 route yet because there are some services not built on the new firewall. Is there a way I can route the individual VLANs we have established? If I can route the individual VLANs to the new firewall for now that would be enough to get me by.

[johnd2310]

Hi,

Normally a router will route based on the destination address. In your case the default route (0.0.0.0/0 via 10.51.5.1) is used because the destination networks ( i.e. Internet) are not in the routing table. If you need to route based on the source addresses , you will need to implement policy-based-routing.

Have a look at the following docs on how to configure policy-based-routing on 3850:

https://www.cisco.com/c/en/us/td/docs/switches/lan/catalyst3850/software/release/16-9/configuration_guide/rtng/b_169_rtng_3850_cg/b_169_rtng_3850_cg_chapter_01010.html

https://www.cisco.com/c/en/us/td/docs/ios-xml/ios/iproute_pi/configuration/xe-3se/3850/iri-xe-3se-3850-book/iri-pbr.pdf

Thanks

[paul driver]

Hello
That static isnt required as the default static will take care of "any other non specifc routes" not residng on/off  or connected to that L3 switch and TBH your getting away with it at present because you have much longer connected prefixes wthin that 10.51.0.0/16 static address that the L3 switch is accepting.

[jmbattlemotors]

I guess what I don't understand is that we have two other static routes (10.99.0.0/16 and 192.168.20.0/24) that are going to the new firewall. Those work perfectly fine and the traffic is going where it's supposed to. So why would those work normally, but if something like 10.51.0.0/16 is added it does nothing and the 0.0.0.0 route takes over?

[Richard Burts]

The difference is that 10.99.0.0/16 and 192.168.20.0/24 do not have any more specific routes in the routing table and 10.51.0.0/16 does have more specific entries. And is IOS the more specific route is preferred.

I suggest this as a test of my explanation:

- change the static route for S 10.51.0.0/16 [1/0] via 10.51.5.1 to use the new firewall.

- configure a new vlan with a new subnet in that range, perhaps 10.51.25.0/24.

-connect a PC in that new vlan and configure it with an IP in that subnet (and appropriate mask and gateway).

- generate some traffic from that PC to some remote destination. You should see that this traffic is forwarded to the new firewall.

[Richard Burts]

I realize that the suggestion that I made is not correct. I suggested creating a new subnet and generating traffic from that subnet. But the issue is not about traffic from the subnets of 10.51.0.0/16. The issue is about traffic to those subnets. So you do not need to create a new vlan and subnet. Just generate traffic to a subnet in that range that is not present in the routing table. Perhaps tracert 10.51.26.26. You should see that it will go to the new firewall (as long as you change the static route for S 10.51.0.0/16 [1/0] via 10.51.5.1 to use the new firewall).

[Richard Burts]

You asked " we have two other static routes (10.99.0.0/16 and 192.168.20.0/24) that are going to the new firewall. Those work perfectly fine and the traffic is going where it's supposed to. So why would those work normally, but if something like 10.51.0.0/16 is added it does nothing". But it is not doing nothing. If you had some subnet of 10.51.0.0 that was somewhere remote then this route would be used. Perhaps the real issue is that 10.51.0.0 should all be local. Your static route is to catch incorrect traffic and send to firewall.

[Richard Burts]

Here is another thought about the question. If the 10.51.0.0/16 has the same next hop as the default route then yes it is not making a difference and is not required. But not needed does not mean that it is wrong. Think about this: if subnet 10.51.26.0/24 did exist somewhere in your network not on this 3850 and you wanted to forward traffic to it using firewall 2 (as you do for 192.168.20.0/24) then this static route would be a way to accomplish it and would make a difference.

Having said that I acknowledge that if your goal is to have a single firewall then the static for 10.51.0.0/16 does not serve a useful purpose and could be removed without impact to the network. And many of us would call that good housekeeping.

[jmbattlemotors]

The 10.51.0.0/16 does have the same hop as the default route right now, but that's kind of the issue, I don't want it to. I basically want anything from a 10.51.0.0/16 network going to our new firewalls and anything not on those going to the default route hitting the old firewall. The reason for this is because there are some services I'm not ready to have on the new firewalls yet, but I'd like everything on that 10.51.0.0 subnet to go to the new one.

From what it sounds like this isn't happening because the VLANs are on this network? VLAN 11 for example is our main wired network, 10.51.11.0. I'd like this to go to the new firewall. However, I don't see anything in the VLAN config that would specifically send traffic to the old firewall other than that default route. If there's a way I can configure the VLANs to have specific routes to the new firewall, that would work just as well.

[Richard Burts]

Thank you for the additional information. The issue is now more clear. The route for 10.51.0.0/16 works on those addresses when they are the destination of the packet. But you want to make a forwarding decision when they are the source of the packet. To make forwarding decisions based on the source address you need to implement Policy Based Routing. For PBR you need these steps:

- configure an access list that identifies the traffic to be forwarded (basically anything where 10.51.0.0/16 is the source address).

- configure a route map for PBR. The route map would have a match statement which specifies the acl and a set statement which specifies the next hop address.

- on each vlan interface with addresses in 10.51.0.0/16 an ip policy statement which specifies the route map.