Purpose of using a Switch as a default gateway as opposed to the Router it’s connected to

[fbeye]

Hello

This is sort of a generic question and don’t really have a good reason as to what my meaning is but…

I have a Router connected to a Switch.. On the Switch I have 3 vlans, all coming from different Routers. 
Ive noticed that any device connected to the Switch can either #1 see each other if they use the Switch gateway (but no internet access) or #2 connect to the internet by using their default routers gateway )but can not connect to other devices in the switch).

I’ve eliminated this issue by directly connect the other 2 routers to 2 interfaces in my default router (bypassing switch) and then their ip routes are in the ASA (router) and everything works fine.

I just don’t want to use 2 of the 8 only Interfaces on the ASA so that everyone can talk.

Would ‘ip routes’ on each Router to each other work instead of having a common gateway kn the switch?

[Richard Burts]

Thanks for the clarification. I certainly understand (and approve) taking one router out of the discussion so that we can focus better on the remaining router. And I agree that if we solve the issue for one router that should make it easier to solve the third router.

I am a bit confused about the routes that you mention. So let me ask a couple of questions:

- you have these routes

192.168.1.0 255.255.255.0 10.0.2.5

10.0.1.0 255.255.255.0 10.0.2.5

which device are these on? Probably not on R2 since the next hop address is an address of the R2 subnet. If these are on the ASA then why is there a route for the ASA subnet? And if on R1 then why is there a route for the R1 subnet?

[fbeye]

Hello.

Yes, Router 2 (10.0.2.1) has the 192.168.1.0 255.255.255.0 10.0.2.5 and 10.0.1.0 255.255.255.0 10.0.2.5 (10.0.2.5 is vlan 3 IP on the common Switch) because I thought that in order for the network to see the other network it would need a route to the furthermost IP of its subnet that touches the next.

The ASA (192.168.1.1) connects to the common Switch (among the different subnets) with 192.168.1.5 as its vlan 1 IP and has a route of 10.0.2.0 255.255.255.0 192.168.1.5 as its subnets furthermost IP touching the next subnet. I then assumed the Gateway with ‘up routing’ would allow the subnets to talk.

I appear to be misunderstanding the concept of ip routing 8(

[Richard Burts]

My apologies. Big mistake on my part. I assumed that 10.0.2.5 was the router address. If this is the Catalyst address then it makes sense, and is what I was suggesting in my follow up message. I was so focused on the addressing aspect of the post that I did not pay enough attention to another paragraph of your post;

At this point everyone sees everyone and can PING from each subnet to the other etc, but anything on 192.168.1.0 can not ACCESS any devices on the 10.0.2.0 (specifically 10.0.2.126, 10.0.2.111).

If hosts in each subnet can ping hosts in the other 2 subnets then the routing issue is solved. If 192.168.1.0 can not access 10.0.2.126 then we are looking at something different from a routing issue. 

First let me verify an assumption that I am making - some host in 192.168.1.0 is successful in ping to 10.0.2.126 but is not successful when it attempt to access some service provided by 10.0.2.126?

Can you verify for this host that its default gateway is the ASA address and not the Catalyst address?

When you attempt this access and are not successful can you look in the ASA logs and see if there are any related messages?

Is it possible that there is some security policy on the ASA that is preventing this access? Perhaps some details of the ASA config might be helpful.

[fbeye]

Hello.

The Gateway for 10.0.2.126 is 10.0.2.1 and the Gateway for 10.0.2.111 was blank and I changed to 10.0.2.1 during testing. I suppose there could be an ACL of sorts but I do have same-security access enabled. When I get home I will go through it all again but what you say makes sense that it’s more of an access /permission issue rather than Routing as they can all ping as you said. 
The accesses needed would be Samba /NFS but never needed this when the ASA was having an interface dedicated to 10.0.2.5 rather than the Catalyst. 

[Richard Burts]

It is interesting, but not necessarily significant, that the gateway for 10.0.2.111 was blank. Depending on the behavior of the OS of the server and on how the local router responds it might work just fine with a blank default gateway. The really important thing is whether your host in 192.168.1.0 is successful in ping to the server. If ping is successful then routing and IP connectivity are working and the problem is something else - probably access permissions.

I had wondered briefly about same-security. Glad to confirm that same-security is enabled. But if it had not been enabled then ping would not have worked.

I am wondering about this statement "never needed this when the ASA was having an interface dedicated to 10.0.2.5 rather than the Catalyst." Can you provide some clarification? What changed from the ASA dedicated interface to using the Catalyst?

[fbeye]

You seem to be spot on with where I am at thus far.

The initial post was about a way to eliminate the usage of 1 of my 8 Interfaces on the ASA (5508-X) that was used as a single Interface with an IP of 10.0.2.5 that the ASA could then route my 192.168.1.0 to. At that role, because that IP was on the ASA, the ASA had no up route to the 10.0.2.0 aside from 10.0.2.5 being on an interface. Everything worked like a charm.

In the event of me wanting to free an ASA Interface, I made a vlan 3 interface 10.0.2.5  (and set aside 3 GE Interfaces) in vlan 3 on the Catalyst. Being that the ASA no longer sees 10.0.2.0 I made an ip route 10.0.2.0 255.255.255.0 192.168.1.5 for the ASA to see 10.0.2.0 and I used 192.168.1.5 cause that was the Catalysts vlan 1 IP address and ASA was hosting the 192.168.1.0 subnet.

Long story short, when the ASA had 10.0.2.5 it all worked flawlessly. Now that I “moved” 10.0.2.5 to the Catalyst, PING works flawlessly but now no “data” transmits I.E I can not access 10.0.2.111 GUI or it’s data via Samba.

So I am not sure it is a permission either as I did no such thing in my original configuration. Maybe as you say the OS on the .111 simply doesn’t like how I have it. 
I also want to leave everyone’s default Gateways as their original gateways cause then I have data access but no internet.  

[Richard Burts]

Glad to hear that now I am spot on (and sorry for a major mistake on my part earlier in the discussion). I am still not clear about some of the changes. But I think I understand that at the beginning 10.0.2.5 was connected on an ASA interface, where it worked. And then in an attempt to conserve interfaces you moved 10.0.2.5 to the Catalyst and now reach 10.0.2.0 through the 192.168.1.0. After moving 10.0.2.0 you are able to ping hosts in that subnet but not able to have data access. 

I have wondered if perhaps the 10.0.2.5 had some security level when it was on an ASA interface but now that it is reached via 192.168.1.0 it is in a different security level. But if that were the case I would expect it to impact ping. And ping works.

As long as ping works then I believe that security levels and ip routing are working ok. If ping works but data access does not then it seems that there must be some security policy on the ASA that is impacting that traffic.

[Richard Burts]

I am finding parts of this discussion to be confusing. So I would like to start over again. Starting from basics and hopefully ending with a solution that works. The original post describes 3 layer 3 devices each with its own subnet (ASA 192.168.1.0, R1 10.0.1.0, and R2 10.0.2.0) and all 3 devices connected to a Catalyst switch. The intent is that each L3 device provide Internet access for its own devices and that hosts in all 3 subnets should be able to communicate with each other. This would not provide failover - if failover is desired that needs to be a separate discussion after we have resolved the original question.

In considering this there are several inter related questions to address:

1) where would routing between the subnets take place?

2) where would routing to the Internet take place?

3) what would be the default gateway for the individual hosts in the subnets.

Originally I suggested that the Catalyst switch do the inter vlan routing, and for that the default gateway for the hosts in each subnet would be the Catalyst interface in that vlan address. This was based, in part, on the assumption that most of the hosts were connected to interfaces on the Catalyst switch as shown in the diagram of the original post. But this approach turns out to be problematic. One problem is that Internet routing would be more complicated (requiring PBR).  And as the discussion provided more details I think it is likely that most hosts in the 10.0.1.0 and 10.0.2.0 subnets may not be connected to the Catalyst switch ports but may be connected in their own apartments.

So I now think that the solution would be that each host would be configured with a default gateway that is the L3 device in its subnet (ASA for 192.168.1.0, R1 for 10.0.1.0, and R2 for 10.0.2.0). The L3 device have a default route to forward traffic from its own subnet to the Internet. And each L3 device would have 2 static routes (one for each of the other 2 subnets). The static routes on the L3 device would specify the next hop as the Catalyst switch interface to which it is connected. The Catalyst switch needs to have ip routing enabled, but would not need any routing protocol and not need any static or default route. The Catalyst only would route for its connected interfaces/subnets.

I believe this approach is pretty simple and would work.

[nagrajk1969]

Hi

Iam kind of mentioning the below config steps which maybe a repetition of what you would have obviously applied as such, and therefore may seem redundant or unneccessarily LONG

BUT iam mentioning the steps/config anyways so that it allows us to analyze/follow the logical flow of this network deployment

One very important point that seems to have NOT been raised at all in all of the previous posts is

a) IF you have your own Internet-Link on ASA, AND
b) If you are connecting to the offsite router1/router2 to primarily access the NAS-servers/Other-lan-resources, AND
c) Obviously you are owning the Cisco-Catalyst switch and the Hosts connected to Cisco-Catalyst-switch

- then, WHY CANT YOU ROUTE INTERNET-TRAFFIC FROM HOSTS CONNECTED TO CISCO-CATALYST VIA YOUR OWN INTERNET-LINK ON CISCO-ASA?
- you have hosts in 10.0.1.x(vlan2) and 10.0.2.x(vlan3) subnets ONLY to access the NAS-server/Other-Server in the same subnet BUT connected to Offsite-Routers
- You could simply route internet traffic from 10.0.1.x and 10.0.2.x hosts on Catalyst-switch via your Internet-link on Cisco-ASA

---------------------------
Existing Config Applied
------------------------
1. On Cisco Catalyst:

Configure as below:

vlan1 192.168.1.5/24
vlan2 10.0.1.5/24
vlan3 10.0.2.5/24

- Default-route to Internet via Cisco-ASA-Router
ip route 0.0.0.0 0.0.0.0 192.168.1.1

2. On Cisco-ASA router:
- add the below static route

ip route 10.0.1.0 255.255.255.0 192.168.1.5
ip route 10.0.2.0 255.255.255.0 192.168.1.5

3. On Router1:

- apply the below routes:

ip route 192.168.1.0 255.255.255.0 10.0.1.5
ip route 10.0.2.0 255.255.255.0 10.0.1.5

4. On Router2:

- apply the below routes:

ip route 192.168.1.0 255.255.255.0 10.0.2.5
ip route 10.0.1.0 255.255.255.0 10.0.2.5

5. Say on the Cisco-Catalyst switch you have connected 3 hosts/servers and configured them as below:

Host1 in vlan1 with ipaddr 192.168.1.2/24 (Def-Gw 192.168.1.5)

Host2 in vlan2 with ipaddr 10.0.1.2/24 (Def-Gw 10.0.1.5)

Host3 in vlan3 with ipaddr 10.0.2.2/24 (Def-Gw 10.0.2.5)

6. And becos as you mentioned you are connected to Router1 offsite-router to access its "LAN-Inside Network Resources" such as say NAS1-server

- Lets assume that the lan-interface on Router1 (in vlan2) is configured with ipaddr 10.0.1.1/24

- And the NAS1-server is connected to Router1 either directly to one of the router1 ports OR it could be connected via another lan-switch

- lets assume that the NAS1-server is configured with ipaddr as 10.0.1.9/24 (Def-Gw 10.0.1.1)

7. And becos as you mentioned you are also connected to Router2 offsite-router to access its "LAN-Inside Network Resources" such as say NAS2-server

- Lets assume that the lan-interface on Router2 (in vlan3) is configured with ipaddr 10.0.2.1/24

- And the NAS2-server is connected to Router2 either directly to one of the router2 ports OR it could be connected via another lan-switch

- lets assume that the NAS2-server is configured with ipaddr as 10.0.2.9/24 (Def-Gw 10.0.2.1)

8. Now my thoughts are:

a). You have been allowed access to the offsite-routers Router1/Router2 to primarily share access to the "NAS-servers" connected in the respective lan-networks connected to Router1 and Router2 respectively

b). And if you have your own Internet-Link on Cisco-ASA, it is my assumption that obviously the admin of Router1/Router2 would NOT ALLOW THEIR INTERNET-LINKS TO BE SHARED WITH THE HOSTS/SERVERS CONNECTED TO YOUR CISCO-CATALYST SWITCH

c). So if the point-8b is correct, then you should simply configure as in Points-1-thru-5 above (with the "default-route" 0.0.0.0/0 pointed to Cisco-ASA on the Cisco-Catalyst)

AND

d) For Internet Access for hosts connected to Cisco-Catalyst switch, in Phase-1 now, route the Internet traffic for the hosts in vlan2/vlan3 thru the Cisco-ASA by doing NAT/MASQUERADE on ASA for 10.0.1.0/24 and 10.0.2.0/24 and ofcourse 192.168.1.0/24 networks

e) Once this above stabilizes, IF YOU AGREE WITH THE ADMIS OF ROUTER1 AND ROUTER2 TO SHARE THE INTENET-ACCESS TOO, THEN MAYBE IN PHASE-2

- you could delete the default-route on Cisco-Catalyst (ip route 0.0.0.0/0 to 192.168.1.1) and
- you could configure OSPF dynamic routing on Cisco-Catalyst (on the vlan1/vlan2/vlan3 L3-interfaces) and Cisco-ASA/Router1/Router2 and configure some failovers/load-balance for "Internet-Traffic ONLY" AND

- The inter-vlan traffic will continue to remain the same as configured above

[fbeye]

Whew that is a whole lot of something something. I will print that out and look over it all as it seems pretty complicated to be viewing on my iPhone. 
From what I read you are spot on to all I am trying to achieve. 
I want to mention (affirm I suppose) that I only want DATA (NAS/Samba) to be shared but each vlan (subnet/network) use their own routers Internet address. I mean I suppose that’s not an issue but may be a security issue. So that was my goal, LAN share no Internet. But I feel you already assumed that so I’m just confirming. 
The Catalyst I have at this point (other than interfaces dedicated to their respective vlans (1-10 vlan 2, 11-20 vlan 1 and 21-24 vlan 3) really serves no intricate purpose and can be configured just as a gateway ain’t the LAN’s. I can divide up the catalyst in any fashion if need be.

As I said what you wrote is a lot to digest on my phone so tonight I’ll go over it all so thank you. 

[Richard Burts]

There is certainly a lot in the response from @nagrajk1969, especially if you are trying to read it on iPhone. Print out or reading on a computer will certainly be much better.

@nagrajk1969 provides a very complete explanation of routing configuration. My understanding is that at this point any device in any vlan/subnet is successful in ping to devices in the other subnets. So the routing logic is working correctly. The issue at this point seems not to be in routing but in data access to the servers. 

If I am understanding the suggestions correctly @nagrajk1969 is suggesting that the Catalyst would route all 3 vlans/subnets to the Internet using the link on ASA. It was my understanding of your requirements that any device in 10.0.1.0 should access Internet through R1 and any device in 10.0.2.0 should access Internet through R2. Which do you want to be the case?

[fbeye]

I can see the 2 options before me but what I’d prefer is 10.0.1.0 use it’s ISP IP for Internet, 10.0.2.0 use it’s ISP IP for Internet and 192.168.1.0 (ASA) use it’s x.x.x.182 ISP IP for Internet, but on the Catalyst they can crossover and route to each other for Data purposes. 

[Richard Burts]

I thought that was what you wanted. In that case there should be no default route on the Catalyst switch. Each host (no matter whether connected on Catalyst switch or connected to something else) should have its default gateway not as the Catalyst switch address in that subnet but as the router/ASA connected for that subnet. The 2 static routes configured on ASA/routers would forward inter vlan traffic to the Catalyst which would forward traffic between vlans (but not forward anything for Internet).

[fbeye]

Alright I will have to go over all of this again…. I feel I tried this before but can’t factually say whether or not I had a default route on the Catalyst, and if I did that may have been the issue, but I feel I did this snd could ping everything but not access 10.0.2.111 but could PING it. 
Also again it may be something on the host OS but I will have to look it all over. I may just default everything and recreate it all again step at a time. 

[Richard Burts]

First let me clarify my comment about not have a default route on the Catalyst. This does not have anything to do with vlan to vlan access. This has to do with making sure that all devices in a subnet use the right device (R1, R2, ASA) to get to the Internet. The Catalyst will see the 3 vlan subnets as directly connected subnets. The Catalyst would need a default route only if the Catalyst were to route packets destined for the Internet. You want Internet routing done only on R1, R2, ASA and so no default route on Catalyst.

You certainly could default everything and start over fresh. And that might be a good learning experience. But at this point I believe that we have the routing working as you want. So do you want to spend the time to start over fresh?

To verify that routing is working I would use these steps for verification:

- a device in any vlan/subnet can ping devices in both of the other subnets. (so 192.168.1.0 can ping 10.0.1.0 and 10.0.2.0, 10.0.1.0 can ping 192.168.1.0 and 10.0.2.0, and 10.0.2.0 can ping 192.168.1.0 and 10.0.1.0)

- a device in any vlan/subnet can ping addresses in the Internet (perhaps ping 8.8.8.8).

- a traceroute (or tracert) to that Internet address goes out through the correct gateway (R1, R2, ASA)

Once you have verified those things I would suggest that you ping from your PC to 10.0.2.111. That should succeed. Then attempt data access, which probably will not work. That would demonstrate that we have correct routing and IP connectivity but not have data access.