Layer 3 Switch at Remote Locations

[mjensen323]

Hello,

I have a 8 locations in my organization, a few months ago, we did a large WAN upgrade.

We have a Layer 3 core switch at the central hub datacenter, and each remote site is connected via Fiber or Motorola PTP Wireless (150 meg connection / layer 2 bridge). All WAN connections start from the core Layer 3 switch.  At each location the bridge/fiber terminates at a Layer 2 switch.

Each location has between 140-500 connections. We are rolling out VOIP (120-150 handsets) over the next 6-18 months and we have VLAN's setup to separate each location.

My questions are:

Do I need layer 3 switches at each remote site? Would there be any gains from this? Right now the gateway for each building, resides at the datacenter. Would it be better if each vlan (3-4 per site, data, wifi, etc..) terminated at their site and then a single "WAN VLAN" connected the hub to the remote?

Thanks in advance.

[Edison Ortiz]

Extending your L2 domain over a WAN connection is not considered best practice. In your situation, you are planning to have 8 locations sharing the same broadcast domain?

I would suggest terminating your Vlans at the local Layer3 switch and route the traffic. It is a more scalable design and it will preserve valuable bandwidth on the WAN links. A broadcast storm from any of the remotes can cause a total outage for all remote and central location.

Regards,

Edison

[mjensen323]

Each building has its own vlan / ip / subnet and the bridge is connected to a port on the layer 3, each building has a similar design (all over the WAN). The gateway for each building resides at the central hub.

So a typical remote is like this:

Remote ------------------Layer 2 Bridge -----------Hub ----------------Port 1-----------> To Internet on VLAN900

Layer 2 switch                                         Layer 3 Switch

ip range 192.168.40.2-254                         Vlan 40 is defined with 192.168.40.1
                                                                Port 4 on the switch is untagged vlan 40

Then the same with ports 2,3, etc..

All remotes share internet connection from single location and all servers reside at hub.

[Edison Ortiz]

I understand your setup better now. The WAN connection represents a single Layer2 domain, right?

If that's the case, there isn't much you can do to control a broadcast storm if occurs so the IP addressing does not make much difference.

[mjensen323]

You are correct.  Would I gain anything by putting layer 3 switches at the head in's at each remote, terminating all their local VLAN's,  etc.. ? Then the only data carried over the WAN would be the dhcp addressing and connections to servers, internet, but not local resources?

[Edison Ortiz]

You edited your previous response after I entered my reply. I'm confused again on your setup.

Do you have a diagram that can somewhat be readable? I'm trying to understand your setup but the information from the ASCII is kind of confusing.

[mjensen323]

Try this.

Thanks for your help.

[Edison Ortiz · Accepted solution]

Thanks for the diagram, much clearer now. I'm going back to my initial assessment. Broadcast is constantly being flooded via the WAN links.

That's going to take away bandwidth from data and voice traffic. How are you planning to do QoS over the WAN? Are you providing the phone traffic some kind of SLA?

I recommend breaking the layer2 domain at the remote locations and place routers (2900/3900) at the WAN edge.

Classify your voice traffic for QoS at the access layer and prioritize the traffic as it goes to the DC.

If your setup was just data - this design may be doable but not recommended. Since you mentioned voice over IP, you don't have any tools to perform the above tasks.

Regards,

Edison

[mjensen323]

If the connections to remotes are via Fiber, would you still recommend routers? Or would you do layer 3 switching at those sites? I'm running 2 x 1gb fiber trunks.

[Edison Ortiz]

On the fiber situation, I would terminate the fiber to a cheap layer 2 switch and then connect the switch to the WAN edge router (2900/3900).

The goal is to limit the broadcast over the WAN and provide QoS for your voice traffic.

For traffic shaping and bandwidth guarantee services, you need to buy a very expensive switch. A router option is a cheaper alternative.

[mjensen323]

Last question! I promise! Thanks for all your help.

With the 2900 series routers, and the throughput is 100-200mbps, am I losing anything with the 1 gig fiber trunks? Or will the data forward / transfer to the layer 2 switch (where the fiber would terminate) it would not make a difference?

[Edison Ortiz]

You will lose the line rate feature with those routers, the 3900 series can do up to 500Mbps but not 1Gbps.

In your initial post, you mentioned 150Mbps WAN. I'm assuming the handoff is 1Gbps fiber but subrated to 150Mbps?

If so, your Layer2 switch will accept the 1Gbps fiber handoff and the router can be configured with QoS on the interface

facing the Layer2 switch and shape down to 150Mbps on egress towards the WAN.

[mjensen323]

The WAN at 150mbps is wireless point to point to 3 sites, I have 3 other sites connected using fiber tunneled between buildings (2 x 1 GB connections per building).

[Edison Ortiz]

For those locations, I recommend a 7200VXR with NPE-G2 or ASR1k.