Datacenter Switch Design with 3750 stacks

[martens72]

We have one 3750 switch stack, with everything collapsed into it; and a second stack of switches arriving. We are in a Datacenter environment, and my concern is best practice design, and how we should prepare for switch stack 3 and 4 (we are expecting growth).

I am having a mental block about whether I should make the current stack root distribution with all the SVI's for both stacks, or if I should create SVI's on the new stack and have two distribution stacks. Any thoughts would be very helpful.

[paolo bevilacqua]

Distribution switches can work at layer 2 and in fact you could have bought layer 2 switches.

Generally, you only need a single layer 3 core.

[martens72]

Everything was ordered and in place when I started, so I am trying to work with what we have. My main concern is should I make the new switch an access layer block, and perhaps split the current stack into a distribution and access block. Either way, I have a feeling some re-wiring is in order. I do not feel comfortable with 2 switches in place that are connected layer 3 with access/dist/core collapsed into each - knowing that other switches will be on the way. Ideally, I am trying to get a seperate distribution layer and access layer out of two 3750 switch stacks.

[Reza Sharifi]

If you think you are going to be expanding more and more, make one of the stack your distro/core.  Deploy all you SVIs/route points in the core. Once the core is done, then you just add access layer switches and uplink them to your core.

HTH

Reza

[hobbe]

First some questions i think needs to be answered.

1) what type of 3750 switches ? (they do act differently in forwarding traffic things like 32/64 Gbps, local switching and so on )

2) how many are there in a stack ?

3) whats the distance between stacks ?

4) what is the purpose ? ie how many servers and serverfarms are you to serve and how many lines in (routes) are you expecting ?

5) how many racks ?

[martens72]

Thank you both for the input.

Reza:

the switch stack containing the SVI's now is maxed out at 9 switches. The stack is not wired so that an access layer can be easily split off. If I make this the distribution/core, my concern is there are many user vlans that should be in an access layer. Do you think it would make sense to make a new distribution/core out of the stack that is arriving? Perhaps order additional 3750x's specifically for that function and then transfer all the SVI's over, making the current stack all access layer?

Hobbe:

1) they are 3750-x series
2) The existing stack is maxed at 9 switches. An additional 5 have been ordered for a new stack.
3) The stacks will be about 15 ft. apart from each other - in different isles
4)We are a hosting service with colocated customer servers (hyper-v servers for VMs). We also house our own internal core servers and hyper-v.
5)we will have 8 racks soon, with expansion planned in the next 3 months to another cage.The switch stack in place currently takes one rack with patch panel.

I am particulary concerned about connecting the two cages together - I would specifically like a collapsed distro/core in each cage, then inter-connect them.

[Jon Marshall]

Michael

the switch stack containing the SVI's now is maxed out at 9 switches. The stack is not wired so that an access layer can be easily split off. If I make this the distribution/core, my concern is there are many user vlans that should be in an access layer. Do you think it would make sense to make a new distribution/core out of the stack that is arriving? Perhaps order additional 3750x's specifically for that function and then transfer all the SVI's over, making the current stack all access layer?

When you say you are concerned about the user vlans do you mean you actually have user devices connected into this switch stack ? ( as opposed to simply having the SVIs on this stack ?). So in effect is this switch stack a collapsed access/distro stack or even a collapsed access/distro/core ?

If the stack is simply maxed out because of all the end devices then it would make sense to use this as your access-layer stack and use another stack as a further access stack (simply for growth). Then the new stack coming should be a distro stack. The distro stack should only have uplinks from the access-layer and uplinks to the core (if you have one). Servers/end devices etc. should not, if you can avoid it, be directly connected to the distro switches so the distro stack would not, i wouldn't have thought need to be anywhere need 9 switches.

If you are concerned about uplink speed for the servers if you move them off the distro switches then you can either -

1) use a dedicated set of switches for the servers

2) leave them on the distro switches if needed

If you are not using VSS/Nexus which allow certain L2 advantages then generally speaking in a DC the most flexible design is L2 access to L3 distro to L3 core. So all vlans should be routed off your distro switches.

Jon

[martens72]

Jon:

I guess when I said "user vlan", I meant client servers (co-located), which are isolated to their own vlans and are on the same switch stack as our core services. I was thinking of having our core services vlans (which need connectivity to all co-located servers) connected to a distribution layer, along with wan access. You are correct also in that the single, existing switch stack of 9 switches, is acting as one core/distro/access layer switch. The new switch stack (soon to arrive) will be the second switch installed in the datacenter. The conplication with the second stack is that client servers (access layer) will need to be connected immediately due to expansion. That is why I was thinking of suggesting purchase of additional 3750x's to make a third stack of say 2-3 switches for distro purposes. That would provide ports for our core services, which we are open to moving to the new isle - away from the current stack.

As far as I can tell, this can get messy real quick, so I appreciate everyone's input again!

[Jon Marshall · Accepted solution]

martens72 wrote:
Jon:
I guess when I said "user vlan", I meant client servers (co-located), which are isolated to their own vlans and are on the same switch stack as our core services. I was thinking of having our core services vlans (which need connectivity to all co-located servers) connected to a distribution layer, along with wan access. You are correct also in that the single, existing switch stack of 9 switches, is acting as one core/distro/access layer switch. The new switch stack (soon to arrive) will be the second switch installed in the datacenter. The conplication with the second stack is that client servers (access layer) will need to be connected immediately due to expansion. That is why I was thinking of suggesting purchase of additional 3750x's to make a third stack of say 2-3 switches for distro purposes. That would provide ports for our core services, which we are open to moving to the new isle - away from the current stack.
As far as I can tell, this can get messy real quick, so I appreciate everyone's input again!

Michael

Generally speaking you wouldn't want any vlans with devices in them connected to your distro switches even your core services. If you had the core services on their own access-layer switch(es) there is no reason why they couldn't route to all the other vlans. However it's not the end of the world if you do this to be honest.

If i understand correctly, you are saying that your current stack is fully populated. You are getting a 2nd stack because you need that growth immediately for new client servers. And you are proposing to purchase an addtional 3rd stack just for distribution and so relegate the existing maxed out stack to access. Then repatch core servers into the new distro. Have i understood correctly ?

If so this would require a certain amount of downtime as you need to provision a new distro layer remove all the distro config off the maxed out stack. It would certainly work better in the lon run as far as i can see as their is no point having a 9 stack switch as your distro if it is only to be used for uplinks and core servers.

If you could not get the immediate downtime you could just use the 2nd stack as an access-layer stack and uplink it to the 9 stack distro, if you have space for the uplinks on the 9 stack but that would only be a temporary solution.

Jon

[martens72]

Jon:

Thank you for your input. I suppose the best short term solution is to make the new stack an access-layer block and keep the 9-stack as is. Are you agreeing that a long term solution would be to introduce a new stack as being only distribution, with only the access layers uplinked? If so, would the 3750 be the best platform for this? I am stressing over this design, because there is limited rack space in this cage, and will be no immediate redundant distro/core.

[Leo Laohoo]

Do you really need to stack more than 7 switches in a stack?

I've always found difficulty maintaining a stack in excess of 7 switches.  And I've also made it a rule that if the stack needs to go more than 7, then I'd create a different stack or get a 6509 chassis.