Purpose of using a Switch as a default gateway as opposed to the Router it’s connected to

[fbeye]

Hello

This is sort of a generic question and don’t really have a good reason as to what my meaning is but…

I have a Router connected to a Switch.. On the Switch I have 3 vlans, all coming from different Routers. 
Ive noticed that any device connected to the Switch can either #1 see each other if they use the Switch gateway (but no internet access) or #2 connect to the internet by using their default routers gateway )but can not connect to other devices in the switch).

I’ve eliminated this issue by directly connect the other 2 routers to 2 interfaces in my default router (bypassing switch) and then their ip routes are in the ASA (router) and everything works fine.

I just don’t want to use 2 of the 8 only Interfaces on the ASA so that everyone can talk.

Would ‘ip routes’ on each Router to each other work instead of having a common gateway kn the switch?

[Joseph W. Doherty]

What you likely need is additional route statements, on the L3 switch, to also route to networks "beyond" your "off-site" routers or run a dynamic routing protocol between the L3 switch and those "off-site" routers.

[fbeye]

I do not think I am able to a dynamic routing protocol as I am limited to what I am able to configure in the 2 Routers (not the Cisco). 

I assume I would need to keep the 0.0.0.0 0.0.0.0 192.168.0.1 for the vlan 1 to have its internet access through the L3 but how do other subnets know which routes they are to use? 

I will have to look more into ip routing but I just can’t wrap my mind around it. What would I even tell the 10.0.2.0 subnet to look for to obtain internet access, doesn’t it already know it’s own Gateway (10.0.2.1)? 
I do not want the subnets to gain access to the net on another subnets [outside] IP so I can’t do 0.0.0.0 0.0.0.0 10.0.2.1 alongside the 192. 
I will have to really look into this from a top view to see who needs to talk take what. 

[Richard Burts]

What I think I understand is that you have 3 routers, each router has a "Lan" subnet and each router connects to a Catalyst switch which is configured with 3 vlans (one vlan for each of the 3 router subnets). The hosts in each Lan subnet are connected to switch ports which are configured as access ports in the appropriate vlan. The issue is that if those hosts are configured with a default gateway using the router address then the hosts have Internet access but can not access the other subnets. And if the hosts are configured with a default gateway which is the switch vlan interface of their subnet (and if ip routing is enabled) then hosts are able to communicate with hosts in the other Lan subnets but are not able to access the Internet. If something I have described is not correct please provide clarification.

The issue here is fairly simple. The routers do not know about the Lan subnets of the other routers. So they can provide Internet access to their connected Lan hosts but can not route to the other Lan subnets. The Catalyst knows about all 3 Lan subnets, and (if ip routing is enabled) can provide Lan to Lan communication but does not have a default route so the hosts do not have Internet access.

I believe that there are 2 possible solutions:

1) Have each host configured with the router as their default gateway and on each router configure static routes for the other Lan subnets.

2) Have each host configured with the switch as their default gateway. On the Catalyst (assuming that it has correct licensing to support the protocol) configure Policy Based Routing. The access list in PBR applied to each vlan interface would deny any traffic from hosts in that subnet to the other 2 subnets on the switch (which would allow normal ip routing for the local subnets) and would set the next hop for all other traffic as the appropriate router interface. This would allow inter vlan routing on the Catalyst and would forward all Internet traffic to the appropriate router interface.

[fbeye]

What you have recited back to me is quite accurate so that is awesome.

Until I can get home and verify I have PBR as a function let us go with #1, enabling ip routing and then for example on 10.0.1.1 Router I would have 10.0.2.0 255.255.255.0 10.0.1.5, 192.168.1.0 255.255.255.0 10.0.1.5 and the same (but relative to each router) on the other 2 Routers? And also leave the Gateways as their own router (.1 for each) for the Internet access?

[Richard Burts]

Thanks for confirming that my understanding is correct. I believe that your understanding of #1 is correct.

[fbeye]

Awesome, I will give it a try and let you know.

Also, would the PBR route, if available, be the more appropriate path to what I am trying to do? 

[Richard Burts]

You ask an interesting question about which approach would be more appropriate. And I think that to some degree the answer might depend on the particular situation. 

For example in a previous response you said " I am limited to what I am able to configure in the 2 Routers (not the Cisco)." If configuring additional static routes on those routers is difficult or not possible then obviously PBR is better.

Or if it turns out that your switch does not support PBR then additional static routes are better.

But if we consider the question without considering those possible factors I would say that I believe that PBR would be the better approach. My choice is based on these factors:

- you are making changes on a single device rather than on multiple routers.

- you are making the change closer to the origin of the packet and so potentially using a shorter path. With PBR the path for vlan to vlan would be source to switch to destination while with static routes on router the path would be source to switch, to router, back to switch, to destination.

[fbeye]

Correct, I am quite certain I am unable to fully configure Routers 2 and 3 (10.0.1.0/10.0.2.0) but I can on the ASA (192.68.1.0).

I did verify I have PBR ability on the switch,

Switch(config-if)#ip policy ?
route-map Policy route map.

I will have to look into that and see exactly what my goals are. 10.0.1.0,10.0.2.0 and 192.168.1.0 are to all use their Routers Gateway for internet but are able to communicate with each other’s subnets to access data on each subnet. 

[Joseph W. Doherty]

Hmm, limited to what you may config on the routers, eh?

Well, assuming someone has full control of them, you should be able to discuss this situation with them and either work together to unify the routing topology or at least ascertain, from them, what networks are on the "other side" of those router (so that you can route to them, specifically).

Rick suggests PBR as a possible solution, and it very well may be, but PBR has it's own considerations a couple being (at least, I recall [?] being true some years ago) handling fall over failure situations is a bit complex and certain PBR actions, especially on some switches might not be directly supported on the switch hardware).

So, another approach might be to route to other routers for the private address space blocks, e.g. all of 10.0.0.0/8, etc.

[fbeye]

Well I suooose I could have just explained it all from the start. 10.0.2.0 is in the apartment next door. It is a DDWRT Router and he is quite happy how he has it set up and doesn’t want me to mess with it too much. I connect via a hole we knocked in the wall between apartments. The purpose of this is we access each other’s NAS Servers.

As I mentioned we currently have all we need set up perfectly but I am utilizing an Interface on my 8 Port ASA so I wanted to free one up and move the access from ASA to the Switch (I can control). Problem is when I move it to switch, I all of a sudden can either access the NAS and no No internet or no Internet and the NAS.

My only option on 10.0.2.0 is to do the Static Routes

10.0.1.0 255.255.255.0 10.0.2.5

192.168.1.0 255.255.255.0 10.0.2.5 (.5 is the IP of the Interface on the ASA). When I disable that on the ASA and create a vlan 11 interface 10.0.2.5 I then lose one or the other (access or Net). The vlan11 was already being utilized for the 10.0.2.0 access but I made a vlan 11 interface to allow routing.

So I am clearly missing the big picture.  

[fbeye]

It just occurred to me, but I may be uncertain, but I think maybe I forgot to add a 10.0.1.0 and 192.168.1.0 Interface on the Switch!! Maybe that is why when I tried all this it didn’t work, but could Ping cause I’ve learned that pinging doesn’t mean access. I was so focused on 10.0.2.0 that I may have other looked the “routing” to other Subnets having to have an interface with an IP to route to. 

[Richard Burts]

Thanks for the additional information. Some parts I now understand better and some parts are more confused than before. Perhaps you can provide some clarification about these points:

- the earlier part of the discussion was dealing with your ASA and 2 routers. The recent posts have been about your ASA and the router in the apartment next door. What happened to the other router?

- in the description of the original environment you said that the switch had 3 vlans (and therefore 3 subnets) and that hosts in each vlan could reach other vlans/other subnets if they were configured with the switch SVI as their gateway. This could only be the case if the switch had configured the 3 vlans and 3 SVIs with appropriate IP addresses. Now you suggest that perhaps you did not have 10.0.1.0 and 192.168.1.0 addresses on the switch?

Am I correct in understanding that in the apartment next door there are hosts connected in 10.0.2.0 (probably on some switch) and the DDWRT Router? My preference for the PBR solution was based in part on thinking that it would produce a shorter path. And for the inter vlan traffic that is true. But in both networks there is probably more traffic that is not inter vlan (either host to host in the same vlan or host to Internet) and for host to Internet traffic the PBR path would in fact be longer. I believe that I will change my suggestion and say that I would suggest that you use the static route on the router/ASA approach.

Joseph mentions some aspects of PBR but I do not see them as applicable in your situation. It is true that some switches do not implement the full range of PBR options. But it seems to me that your PBR would be quite simple and straight forward. There would be a unique PBR route map for each vlan interface. Each route map would match an access list which would deny traffic with source address in the local subnet and destination in the other 2 subnets) and then permit traffic with source address in the local subnet to any destination. The route map would then set the next hop address. I do not believe that there is any switch that supports PBR that would not support this. Also Joseph mentions complications in doing failover. In reading the discussion I do not see anything that suggests that you want to do failover (if the Internet connection in the apartment next door fails do you want their hosts to use your Internet connection?) And if you do desire to implement failover it introduces new complications in configuring the 2 routers and your ASA - especially in configuring address translation.

[Joseph W. Doherty]

"In reading the discussion I do not see anything that suggests that you want to do failover . . ."

True, and as you also note Rick, topology and situation is a bit unclear.  What I had in mind though, was perhaps if a path to/through one router failed with the assumption other other might support an alternative path.  (BTW, reason I think that might be the case, if I understand, correctly, part of what has been described, if those two routers, as gateways, are able to get all local traffic to where it needs to go.)  My (dated) experience with PBR is it can be a bit difficult to define that (fail over) in PBR maps, although, I recall, often not impossible.

My intent was just to note PBR can be more problematic, in some situations, over static routing or dynamic routing, both in dealing with "corner cases" and/or features issues on some platforms (like L3 switches).

[fbeye]

Hello sorry for the delay. Ran 15 miles of fiber aerial through the hills and heat. Anyway…

Yes my initial description still stands true. 3 Routers; ASA 192.168.1.0, R1 10.0.1.0 and R2 10.0.2.0. 
In a later description I reduced it to 2 Routers (ASA and R2) simply because of the response I got as to why I could not fiddle with R2 Config so I reduced the narrative, but the 3 Router still stands true. I figure I fix 1 issue then I can resolve to all. Anyway, I was able to finally config 2 Routes to R2;

192.168.1.0 255.255.255.0 10.0.2.5

10.0.1.0 255.255.255.0 10.0.2.5.

On the ASA (let’s exclude 10.0.1.0) I have a route;

10.0.2.0 255.255.255.0 192.168.1.5

I use 1.5 cause that is the IP Address vlan1 is set to on the Catalyst Switch.

On the Switch, I have ‘up routing’ configured as well as 10.0.2.5 IP Address vlan 3.

At this point everyone sees everyone and can PING from each subnet to the other etc, but anything on 192.168.1.0 can not ACCESS any devices on the 10.0.2.0 (specifically 10.0.2.126, 10.0.2.111).

So to test my theory of my Mia configuration, I put everything back as it was and making an interface on the ASA 10.0.2.5 and everything works fine.

It is clearly an ip route concept I am missing.