Network design query

[darren-carr]

Hi All,

I have a query that I hope you can help me with? I am in the process of planning our new network. Our business is changing from hosting its own data centre, to moving it to a professional facility. We have 120 users, over 100 servers (physical and virtual) and three sites (main premise, data centre, dr site). The new network will connect all three.

Our new WAN links are almost ordered. We will be making use of a managed MPLS IP VPN, with a 100M access rate at each site.

I am currently focusing on the desing of the network at the main business premise. We have a significant investment in Cisco 2960 & 3750 switches and Fortinet firewall appliances. I plan to re-use these in the design.

Our current LAN is very flat and I want to segment the network. My plan is to create a number of VLANs, enable the Inter VLAN routing on the 3750 and then attach the 3750 to the Fortinet appliance which will provide stateful firewalling and traffic policin based on the VLAN (subnet) addresses. It is important that the traffic be routed as quickly as possible from this site to our prod and dr data centres.

The 2960's act as the access layer, the 3750 as the distribution layer. The 2960's will connect via port channels (layer 2) to the 3750's and the VLAN interfaces will be configured on the 3750.

I was then planning on creating a VLAN on the 3750 to connect to the Fortigate appliance with a /29 address to limit the addresses used whilst also providing some flexibility for any future design changes.

I want to implement a little security between the VLANs on the 3750 switches. I have a question about this coming up.

I then plan to use the Fortigate appliance to do basic traffic policing based on source/destination addresses.

The WAN routers will connect to the Fortinet appliance on a Gigabit copper interface. The WAN routers will run HSRP between themselves and only one router will be active at any one time.

The failover will be managed by the Fortigate and Cisco routers.

I plan to define those addresses hosted at the other data centres and associate them with the interface associated with the WAN.

I will then define the routing on the firewall for the two other data centres through summary routes for each of the sites. We will run static routing from the Cisco 3750 to the Fortigate and Fortigate to WAN router. We have no other networks/sites and won't have any others in the future.

Does this design sound reasonable? I am looking for some feedback. I can provide a drawing tomorrow if this would prove to be useful.

Thanks,

Darren

[darren-carr]

Hi Ed,

My absolute main goal is to make this network design as simple as possible. I don't want to create something that is difficult to manage/administer and every day I work on this I remind myself of that.

With respect to the Fortigate appliance we have a pair of 1000A devices. These are massively over spec for a company of our size (120 users), and were purchased by my predecessor. I've looked at the performance stats and I have little concern about the throughput of these devices. I like the idea of managing the separation of the VLANS through the firewall as it presents a simple GUI, it makes it easier for me to also train those who are working on the appliance. I've worked with ACL's in the past, and although they are effective trying to explain these to others can be quite challenging.

I'm confident that with using the Fortigates as the local VLAN gateway, for each of the vlans provides for a simple design. The WAN will be a managed WAN service so it is what it is. I am trying to make accessing this, via the Fortigates as simple as possible.

I plan to patch 2 x 1G ports from switch 1 in the stack to the ACTIVE Fortigate (Gi 1/0/1, 1/0/2) and 2 x 1G ports from switch 2 in the stack to the PASSIVE Fortigate (Gi 1/0/1, 1/0/2) . I plan to enable port monitor on the aggregated interface.

To review the potential failures:

1. If switch 1 in the stack fails I am confident that the Fortigate cluster will failover to the PASSIVE appliance and traffic will still be routed via the HSRP virtual IP address as the port monitor will detect this

2. If one of the patched interfaces from switch 1 to the ACTIVE Fortigate traffic will continue to route and the cluster will not failover

3. If an upstream connection to the MPLS cloud fails on the ACTIVE Cisco router it will hand the role of the ACTIVE router to the current STANDBY router, this should be seamless to the Fortigate

I have two others sites that will use 10.2.x.x and 10.3.x.x. I plan to create a summary route for each of these networks (10.2.0.0/16 & 10.3.0.0/16) and will route them via the HSRP virtual IP address.

I hope this makes sense?

Thanks,

Darren

[campbell · Accepted solution]

Hey Darren,

Yes I see how you are doing the failover. The fortigate will support this without issue. You should look at trying to allow the active firewall to always be online, so perhaps this may help:

Fw0-int-0 -> sw0

Fw0-int-1 -> sw1

Fw1-int-0 -> sw0

Fw1-int-1 -> sw1

If you replicate this for outside and inside firewall, it should survive one side of your network going down and still route to the HSRP IP with the active firewall still active. The stack will allow this configuration, it is a standard configuration I use for existing clients. a switch failure will mean nothing to the firewall as it will still have an active interface on the inside and out. Just test the aggregate failover feature, to make sure it doesn't fail the firewall too if one interface of the group fails. You should be able to control how this happens anyway.

You are correct about the 1000 series firewalls, they are overkill but that puts you in an even better situation. The performance will not be a problem at all between networks, but just be aware of how many vlans you put on one interface. If there are 10 vlans, then they all share the one gigabit port etc, you get the idea.

What routing protocol had you planned to use to advertise your summary routes? Or were you just planning to use static? Since there is only one way out of the network, static should be sufficient.

Don't forget to do a good diagram too, so that others are aware how things are connected and how they fail over.

Regards,

Ed

Sent from Cisco Technical Support iPad App

[darren-carr]

Hi Ed,

Just to clarify something here....

Lets say my appliances are FW1 and FW2 and I have SWITCH-1 (first switch in stack) and SWITCH-2 (second switch in stack)

I currently only have three interfaces available until I move the data centre off site, those interfaces are port6, port7 and port9

What I was thinking was creating a 802.3ad interface made up of port6 & 7 and use port9 for the WAN interface (10.1.255.1/29)

Lets also say I have switchports Gi 1/0/42 & Gi 1/0/43 available on SWITCH-1 and Gi 2/0/42 & Gi 2/0/43 available on SWITCH-2

If I therefore create a port-channel made up of Gi 1/0/42 & Gi 2/0/42 and physically patch these interfaces to FW1 port6 & 7 this would be the optimal configuration. This should ensure that if SWITCH-1 or SWITCH-2 fails that FW1 remains active. I'm pretty sure that this is what you were stating? Whilst using 1/0/43 and 2/0/43 for FW2 port6 & 7.

With respect to the WAN interface (port9) I was planning on configuring this physical inteface with ip address 10.1.255.1/29 patching this interface (from FW1 & FW2) into VLAN 255 that is configured on two Cisco 2960 layer 2 switches that are connected using ports Gi 0/47 & 48. I was then planning on patching the 'inside' facing interface of each of the Cisco routers into the same VLAN. Allocating the ISP the IP address 10.1.255.2/29 for the virtual IP address and .3 and .4 for the physical IP addresses.

I would then route anything off the local VLANs, with a static route, to 10.1.255.2?

Does this sound reasonable?

I do like to back all these things up with a diagram, it is one of my key KPI's so I have to!!!

Cheers,

Darren

[campbell]

Hey Darren,

Yes, that's exactly what I was saying.

Also, if possible, configure your wan interface as an aggregate of one, this will allow you to add another later when you have freed an interface after your move to the datacenter, and will give you the same level of redundancy as the inside.

The rest should be plain sailing.

Regards,

Ed

Sent from Cisco Technical Support iPad App

[darren-carr]

I see. Good point regarding the WAN I'll do just that.

I should also clarify that I am only looking to deploy around six to seven VLANs at the business premise.

1 x General user (workstations, printers)

1 x Server VLAN (very few on site)

1 x IT Admin

1 x IT Developer

1 x Management

All excluding the 'General User' will be of a low utilisation.

IT Admin and developers are separated as we have issues with how our developers work.

We  will have a small server presence made up of a domain controller, print  server, DFS host (tightly configured to reduce replication), SCCM  deployment host.

The management VLAN will control access to the inband management of servers, switches, etc.

Hope this makes sense?

Thanks again for all your help. Really appreciate the time.

Cheers

Darren

[campbell]

Hey Darren,

I doubt you will find that the developers are easy users, any I know all are heavy users especially those that have access to the Internet, but you can always hope that they are not in your organisation. hehe ;-)

Since most attacks come from within, because user's go to unassuming sites, you will need a good policy between vlans. Have allow rules that allow between vlans, deny all else between vlans, before allowing specifics to the internet and finally dropping all else.

Have AV/AS/IPS between vlans especially coming off the developers and users. The 1000 series supports this, so there should not be a problem.

Deny all to the management LAN obviously and where practical, only allow local access (i.e console) to the management hosts, or use a different host to manage from, one that is not on the users' or developers' lan.

Best of luck with it.

Regards,

Ed

[campbell]

Hi Mohamed,

Thanks for the clarification.

However it is a trivial point I think because the firewalls do not give you a secondary interface to route to, unless you create another vlan interface and use it as your secondary route. In active/active you have one shared virtual IP and in active/standby you have one IP that's fails over to the other node. There is no need for a secondary route as you suggest.

On the security point you make, there is a whole lot of difference. One is a switch with no security features at all and the other is a firewall that checks each packet with stateful inspection and provides a whole range of security features that the switch can't.

I think you are missing the point, the design should be simple and easy to support. Should Darren leave the company tomorrow, then his underlings will still need to understand the design. I would think that the switch Option here would complicated design when there no need.

Ed

Sent from Cisco Technical Support iPad App

[Jon Marshall]

Ed / Darren

I was hoping to keep a low profile after my HSRP fiasco

The switches in between the firewalls and the WAN routers, are they a switch stack in terms of 3750s ?

I only ask as if they are not can the fortigate support 2 interfaces from the same IP subnet on the same device ?

In reference to Mohammed's reply. Ed you are a security consultant so i think you come at this from a different perspective. I understand what you are saying about keeping it simple but personally if i walked into a network with a L3 switch and a firewall i would expect the L3 switch to be routing the vlans otherwise why bother with a L3 switch at all.

If inter-vlan security is paramount then yes it makes sense to route the vlans on the fortigate but i wasn't aware that was the issue. And if it's not then why complicate matters by sending internal traffic to the firewall when it could simply be routed by the switch.

Having said that i readily admit i haven't used fortigate firewalls so they may be a lot easier to configure than i suppose.

Jon

[campbell]

Hi Jon,

I'm a network and security consultant actually .. hehe

I saw what Mohamed was saying and do agree with what he is stating, but as you have picked up on, most network designs also have to encompass a degree of security as well as manageability. Routing everything through the switch is great for performance, I would do this too, but only where there is no need for security between the VLAN's and by security I mean application aware security. However, I would then ask the question, why would you create a layer 3 network with no need for security, when a layer 2 network will give you better performance considering that packets are switched, or more precisely not processed through a route process, but who is counting bits eh? ;-)

If there is a need for security, don't forget that most hack attempt actually come from within, then it is vital for the rest of the network to be protected. Look at the make-up of hosts, management, developer and normal users. These are the greatest risk to any network with Developers creating the greatest risk of all because of their need to download stuff. I have successfully compromised networks where they are using layer 3 switches. A layer 3 network only gives another point on the network to attack, not withstanding the fact that a layer 2 network already poses this issue, why make it easier.

In theory a layer 3 switched network is all good, but in practice, it isn't very secure. Ok, I realise you can use ACL's etc, mac security and so on, but it doesn't give you granular control and for an IT dept. that doesn't understand ACL's etc. this would be a major cause for concern. We have to look at the whole pie before we consume it. :-) cheesy I know ...

Regards,

Ed

[Jon Marshall]

Ed

Yes, pretty cheesy

Apologies, didn't mean to suggest you didn't know about networks.

One thing though. L3 switches are not slower than L2 switches to be honest. The packets on a L3 switch do not go through the route processer as such, or rather the hardware switched packets don't which is the vast majority of packets. If they do go to the RP then yes, you do get a performance hit but as i say it's the exception rather than the norm. But L3 switches, as with L2 switches, have dedicated ASICs to forward packets.

I do agree about the security or lack of it with no stateful acls and also that the majority of attacks come from within. The problem with this approach is that if followed to it's conclusion every company should have every vlan firewalled which just isn't pratical. Firewall key vlans absolutely but a one size fits all just isn't practical.

I suspect we are saying pretty much the same thing. In the networks i have worked on/designed it is always a tradeoff between security and flexibility/useablilty and in this instance because there are so few vlans in use then using the fortigates does make sense.  However in a large network, firewalling every vlan just does not scale and really does add a very large overhead and complexity that generally is not needed for the majority of vlans.

One question for my clarification. I may have misunderstood what you were saying about the fortigates but when you say -

Fw0-int-0 -> sw0

Fw0-int-1 -> sw1

Fw1-int-0 -> sw0

Fw1-int-1 -> sw1

are the interfaces on each firewall in an etherchannel ?  If so the stack of switches they go to would have to be 3750/6500/Nexus - anything else wouldn't work.

If they are not etherchannel then how does this work ie. on most devices i have worked with you cannot give 2 interfaces on the same device an IP from the same subnet.

Jon

[darren-carr]

Hi Jon,

You are correct, the switches are indeed a stack of 3750 switches. A combination of two interfaces (from each of the different switches) has been bundled to form an etherchannel that is patched to two x 1G ports on the Fortigate appliance, ports that have also been bundled into a 802.3ad interface.