Default gateway for multiple vlans

[kcross]

So a bit stuck and confused here, I have a firewall and a cisco 9300 connected to it. On the firewall I have multiple vlans setup and the same vlans setup on the switch. I can get VLAN 5 to reach the internet since the default gateway for the switch is set to the same subnet as it but the other VLANs don't communicate with the firewall and I believe this is the reason why. Would I need to route those vlans to the trunk port connected to the firewall so it can communicate? My gap in knowledge is here, I will attach a config for review as well. Any suggestions would be greatly appreciated! 

[Reza Sharifi]

If you are going to use the switch as layer-2 devices then simply create all the vlans on the switch and trunk them to the firewall. Now to be able to manage the switch, you need to create one SVI for one of the vlans with an IP. You also need to configure a default gateway on the switch to point to the firewall IP. The rest of the vlans don't need any IPs on the switch. The ips should be configured on the firewall.

Example: 

vlan 10 (this is the vlan you want to use to manage the switch)

Interface vlan 10

ip address 192.168.10.2/24

no shut

ip default-gateway 192.168.10.1 (192.168.10.1/24 should be the IP address configured on the firewall.

HTH

[Jan Kolecek]

Hello Reza,

As a beginner, I have a similar question regarding the multiple VLANs on the switch. Recently I purchased a Cisco CBS350 16port switch and NetGate 6100 router with pfSense system. (I would have bought a Cisco router if they had not been so expensive for home users.). On swiitch and also router, i have following VLANs:

VLAN 1: 192.168.98.0/24

VLAN 10: 192.168.10.0/24

VLAN 20: 192.168.98.20.0/24

The switch is connected to the router via LAG (3x 1Gbit/s link) configured as trunk where VLAN1 is native and VLAN 10 and VLAN 20 are tagged.

All VLANs on the switch have SVI configured to 192.168.xxx.253)

All interfaces on the router have assigned the last usable IP address (192.168.xxx.254). The switch has a default gateway configured to IP 192.168.98.254 (VLAN 1 GW)

Everything works fine with my current setup and configuration but I am not sure whether the configuration is ideal. My concern is connected with the question of where to do local inter-VLAN routing.

If I am not mistaken it can be done on the router or on the switch.

In case I set to client's default GW to 192.168.xxx.254, routing will be done on the router. In case I set to client's default GW to 192.168.xxx.253, routing will be done on the switch.

I have read that L3 managed switches have "wire-speed routing" capability, that's why I prefer routing on the switch instead on the router, even though I have a router with routing speed (even through firewall) >1Gbit/s. This is also why I could not afford to buy Cisco router as one of my requests was firewall speed > 1Gbit/s as my whole network is using 1Gbit/s links. I did not want any bottleneck in my whole network. That's why I decided for pfSense (5 times chapter compared to the quotation for a similar router from Cisco and without extra super-expensive licenses (which cost more than the router itself + some of them are paid on regular basic and contract with Cisco is needed).

But back to my concern. Let's imagine that a client connected to the access port on the switch on VLAN 20 is sending the packet to the internet (or any other remote network behind the router). If my understanding is correct, in my case switch will re-send this packet over the trunk to the router using the VLAN1 to VLAN1 GW. And this is exactly where I am not sure whether this behavior is correct from a networking practices point of view. My opinion is that traffic on each VLAN on the switch should use its respective GW on the router to avoid a "mixture" of the local traffic. I am pretty sure that in case I would create a firewall rule on LAGG0 interface on the router (LAGG0 = 192.168.98.0/24 network) which would allow only traffic originating from VLAN1 clients, it would block outbound traffic of all clients on VLAN 10 and VLAN 20, as this rule would block such traffic.

In the case of routing on the router, this would not happen. As traffic on all VLANs would be 100% isolated on the switch.

What is the best practice in this scenario? Use a router to do Inter-VLAN routing or use the switch to do this routing and accept that VLAN 10 and VLAN 20 outbound traffic is entering the VLAN1 interface on the router.

I have tried to assign a default gateways for all VLANs. It added static routes, but also it added "default" to the first line in the forwarding table. I have not tried behaving of this setup.

Could you explain to me the relationship between static routes and forwarding table? Is there some priority? These terms are from GUI of my switch. I can re-do the configuration and post a GUI screenshot if needed.

Thank for your advices.

Jan

[paul driver]

Hello
It sounds like the Fw is performing the interlvan routing and possibly network translation/dhcp allocation?
If so you would not require those additional L3 svis on the 9300.
Try disable any ip routing on the switch and put all edge ports in a administrative mode of access

Lastly if the FW is performing the routing functions you need to make sure its setup for the other subnets you have shown

9300
conf t
no ip routing

int x/x
Description access port
switchport host

[kcross]

So yes our FW handles DCHP allocation and NAT. How would the firewall know that certain traffic belongs to certain VLANS if I don't have that setup on the switch? The subnets shown in the config are also present on the firewall. Would it be easier to redirect traffic from each VLAN to the trunk port connected to the firewall? 

[Reza Sharifi]

interface Vlan5
ip address 10.20.1.10 255.255.255.0

ip default-gateway 10.20.1.1

Based on the config you posted, you are basically using vlan 5 as the management vlan. So, you don't need the other SVIs for all the other vlans on the switch. Once you add all the vlans to the trunk port connecting the switch to the firewall and configure the IPs on the firewall, all the routing between vlans and out to the Internet will be done by the firewall.

HTH

[kcross]

So in theory all I would need to do is remove the SVI's but keep the VLANs set for the respective ports needed? Once that is done add the vlans to the trunk port and it should all function?

[Reza Sharifi]

That is correct. You only create the layer-2 vlans, which you already have e.g 

vlan x

description data vlan

vlan y

description phone vlan

etc.

and then you remove all the SVIs except for vlan 5 and let the firewall do all the routing.

Again, make sure all the vlans are added to the trunk port between the switch and the firewall.

HTH

[MHM Cisco World]

to solve this issue we start from DHCP server or local 
DHCP send GW to host, host use this GW to connect to outside it subnet. 
OpA- DHCP send SVI of VLAN as GW to host 
here the SW must L3 SW and it must have default route toward FW interface 
FW must have config with NAT overload 
OpB- DHCP send sub-interface of FW as GW to host 
here you dont need to make SW as L3SW it can L3SW or L2 SW, but what is important is checking 
L2 link between SW and FW it must be trunk and allow all VLAN you add in SW. 

[KJK99]

@Jan Kolecek 

“What is the best practice in this scenario? Use a router to do Inter-VLAN routing or use the switch to do this routing and accept that VLAN 10  and VLAN 20 outbound traffic is entering the VLAN1 interface on the router.”

Since you already have a capable routing switch, I think you should do inter-VLAN routing on that switch. This way your local traffic will not be affected by the availability and performance of your Internet gateway or the link between the gateway and the switch.

I used to have a NETGATE pfSense+ box and did inter-VLAN routing on it. Initially I was very impressed with it, but later I started to have doubts about my choice.

You have a fast NETGATE box that comes with a robust firewall and several additional firewall as well as network monitoring tools. However the use of that software is going to lower the performance of the box. That may be still okay for Internet traffic, but it may not be so for inter-VLAN one. Also, if you create a trunk link between the gateway and the switch, that link may turn out to be an unnecessary bottleneck. Not only Internet and inter-VLAN traffic will share that link, but also inter-VLAN traffic will always flow twice through it. I guess you can have only 1Gb link with that hardware and the good routing performance of your NETGATE box will not make it any better.

I also had a bad experience with my NETGATE pfSense+. It just happened that my Internet connection became unreliable. It took me a while to figure out the cause of it (bad coax cable) and in the meantime my NETGATE box was becoming extremely slow and required frequent reboots. That, of course, had serous negative consequences for my inter-VLAN traffic. Even the whole local network was affected because the DHCP server was not available, either.

[Richard Burts]

Jan

I have a similar opinion as Kris K and suggest doing the inter vlan routing on the switch. Do not have all vlans on the router/firewall. Enable routing on the switch, configure vlans and vlan interfaces on the switch. Configure a default route on the switch with the router/firewall as the next hop (you might use vlan 1 to connect switch to router or you might configure another vlan dedicated to forwarding traffic to router). Configure the router/firewall with routes for the subnets on the switch. Let the switch handle local traffic and let the router/firewall concentrate on address translation and security policies.

With a 16 port switch and 3 vlans the amount of traffic will not be large and you would be quite fine routing on either device. And as I think some more about your question we do not know what you are planning for the 3 vlans and what kind of traffic they will generate. If there might be some desire to separate traffic for one of the vlans (a guest vlan or something like that which you would not want to be able to access the other vlans) then I would change my suggestion and say that routing on the router/firewall would be better.

[KJK99]

Rick is certainly right with his advice to use the firewall for inter-VLAN routing if traffic inspection and control is of primary importance. pfSense is pretty good in doing it and easy to use. I do miss pfSense when it come to that. It took me quite a bit of time and effort as well as different way of thinking to implement a similar functionality without it. That’s mainly because of lack of stateful pocket inspection in general, but also CBS350 switches lack some useful ACL features that can be found on enterprise class switches.

[Jan Kolecek]

Hello @KJK99 and @Richard Burts,

thank you for your advice. Now I really thinking of rebuilding my scenario to do Inter-VLAN routing on the switch. Maybe based on the HW I described one could assume this is set up for some small business company, but this is really a home network where two computers and two smartphones are being used for most of the time. I am fully aware that this is overkill and I have a lot of unusable power. I am a beginner network enthusiast, that's the only reason I bought such HW. Therefore I do not need any advanced security polices. But most likely I will try to implement some of them just for fun.

Is it OK, to create VLAN 98 (192.168.98.0/24) as a replacement for the actual VLAN 1 and change VLAN 1 range to 192.168.1.0/30 and use this VLAN to handle the traffic between the switch and router?

On the router, I can remove all VLAN interfaces and replace them with only one 192.168.1.0/30 interface right? This solution will force me to add static routes to those VLANs on the router because, in my current solution where I have created interfaces for all local VLANs on the router, the router created the routes automatically.

I think this solution can maximize the power utilization of my HW. The Switch can use whole power for inter-VLAN routing and the router's power can be utilized for the implementation of some security policies like traffic monitor etc.

@KJK99, glad to hear you have experience with pfSense routers. Could you please share certain models you owned and when it was (how old they are now)? Hopefully, my box will not cause random internet drops like in your case. What was the replacement HW in your case? What other platform you would recommend to me instead of pfSense?

Thank you guys for your posts here. I am glad that I have heard that pfSense is not so bad, even on the Cisco forum :-).

Regards,

Jan

[KJK99]

@Jan Kolecek 

My NETGATE pfSense+ was 3100. It looks like NETGATE has already discontinued it. I think your 6100 is a good router and you should keep it. It’s also a better router than the one I own now.

If you do inter-VLAN routing on the switch, you do not use trunk ports on the link between the switch and the router. You use either access or routed ports for that. You can certainly create that VLAN 98 on the switch and use it, as you are planning, as a transition VLAN. In that case, you define the involved ports as access. Traffic flowing through those port will be untagged. And yes, two IP addresses in that transition VLAN is all you need. In that case, create also a transition VLAN on the router.

[Richard Burts]

Jan

Thank you for the clarification that this is a home network and that a major reason for getting this hardware was to be a learning experience. With that in mind I certainly endorse the suggestion from Kris K that you use vlan 1 as a transit subnet to connect to the router/firewall and create a new vlan and subnet for locally connected devices. This would be a good design and would give you good experience.