Design/Best Practice Question

[DuncanM2008]

Hi,

I have an situation that I need advice/input on.

There are 25 switches in a building all Cisco.

At the core of the network is a stack of 7 Catalyst 3750's which have EtherChannel uplinks to all the outer switches.

The core switch also serves as the default gateway for all servers and pc's via the Vlan1 SVI (ip routing is enabled), the 3750 only has a single SVI (the default Vlan1 interface).

Some of the edge switches also perform L3 routing between two SVI's (1 is the default Vlan1 interface the other is a SVI for a voice vlan, vlan xx).

All of the switches apart from 3 are in the same VTP domain and are running PVST (configuration is default, priorities are default).

The switches at the edge of the network are connected as below based on being grouped in seperate rooms/floors:

Room 1

Switch 1 -> Core

Switch 1 -> Switch 2

Switch 2 -> Switch 3

Switch 3 -> Core

In terms of STP the Core switch is the STP Root Bridge for all VLAN's apart from Vlan1, the Vlan 1 root bridge is a switch hanging off the Core that is in a seperate VTP domain but has a lower mac-address among VLAN1 switches so naturally has been elected root bridge.

I had an issue the other day with a switch that was part of a configuration similar to my Room 1 description above, I rebooted the switch and during the reboot it seemed to cause an STP Loop as the entire network went offline for the best part of a few minutes.

Is this likely to be a symptom of the way the network is configured in terms of STP?

From what I understand the core switch should be the STP Root Bridge when all other switches link back to it, in terms of PVST is there a set criteria for designating a root bridge for each vlan or should it just be the core switch as root for every Vlan?

If any of the above doesn't make sense which is quite likely please ask me for clarification:

Kind Regards,

Duncan.

[Jon Marshall]

Duncan

There are a number of things you could do to improve the network.

1) Currently apart from the voice vlan you seem to have everything else in vlan 1. This means as soon as you reboot a switch, any switch, you create an STP convergence in your network because all switches are members of vlan 1 and are therefore running an STP instance for it.

Cisco recommend you don't use vlan 1 anyway but even if you do you would want your servers in a dedicated vlan apart from your clients. And on the client switches you would not need to have the server vlan and hence if you rebooted a client switch this would not affect the server vlan.

If you do use more vlans then your uplinks from the access-layer switches may well need to become trunks but you want to try and avoid having every vlan on every switch.

2) routing on some switches and not others. It would make sense to decide on one or the other ie. have your core do all the inter-vlan routing or have your access-layer switches do the routing between vlans on those switches and then route to other vlans across the core stack.

Either will work but be aware if you choose routed then each vlan becomes isolated to that particular access-layer switch ie. you cannot have a vlan spanning multiple access-layer switches.

Having a mix of the two makes it messy and causes more chance of misconfiguration.

3) STP root - make your 3750 core the STP root. You should not leave it to chance. What would happen for example if the access-layer switch you wanted to reboot was the STP root for all vlans ?

I appreciate the above cannot be done overnight and needs careful planning. In order of preference i would do 3) first. You will need an outage for this but it will make your L2 topology a lot more predictable.

1) would be next. Break up vlan 1 into more vlans and if possible don't use vlan 1 at all. Only allow vlans on trunk links to access-layer switches that need to be on the access-layer switches. How you setup 1) does depend on 2) because it makes a difference whether you have a routed or L2 access-layer but you could still use more vlans with the mixture you have.

2) is more long-term. You can get by with your current mix but i would recommend looking at why you are both routing and switching from the access-layer and if it actually makes sense or whether it is something that has just evolved.

Jon

[DuncanM2008]

Jon,

That's a fantastic post and very helpful I've just got a lot to think about now but it's all for the better .

Just to summarise if I understand correctly,

Change the Core Stack (3750's) to be the root bridge for all Vlan's which is likely to be the first change.

Second looking at doing all the inter-vlan routing at the core and exchanging only the neccesary vlan's with the access-layer switches via the existing EtherChannel links.

Third, probably the most time consuming and complicated change would be to move everything off Vlan1 and create new vlan's for the servers and clients alongside the existing vlan's for the ip phones/voice.

Assuming the above would you still leave Vlan1 as the native vlan/management vlan for the switches?

Duncan.

[Jon Marshall]

DuncanM2008 wrote:
Jon,
That's a fantastic post and very helpful I've just got a lot to think about now  but it's all for the better .
Just to summarise if I understand correctly,
Change the Core Stack (3750's) to be the root bridge for all Vlan's which is likely to be the first change.
Second looking at doing all the inter-vlan routing at the core and exchanging only the neccesary vlan's with the access-layer switches via the existing EtherChannel links.
Third, probably the most time consuming and complicated change would be to move everything off Vlan1 and create new vlan's for the servers and clients alongside the existing vlan's for the ip phones/voice.
Assuming the above would you still leave Vlan1 as the native vlan/management vlan for the switches?
Duncan.

Duncan

Thanks for the kind words.

You've summarised it very well. The only thing i would add is to be sure you understand why you have L3 from some access-layer switches ie. is it needed or was it just something added for no real reason ? You need to know this otherwise moving everything to L2 could create problems.

Also be aware that L3 from the access-layer is also a valid design choice in a campus/building setup but you do need access-layer switches that are all capable of L3 routing.

But standardising on one or the other would be the way to go.

As for vlan 1 - well couple more things for you to think about

1) the native vlan should not be vlan 1. It should be a vlan that is unused for anything else eg vlan 999. You don't need a L3 SVI for vlan 999 because you never need to route the native vlan and you should not allocate any end user devices into this vlan.

2) the switch management vlan should be a separate vlan with no end user devices in it ie. you only have switches in it. This vlan will need a L3 SVI on the L3 core and then each access-layer switch simply needs a L3 SVI in that vlan with it's default-gateway set to the IP address assigned to the management vlan on the 3750 core.

Vlan 1 is then not used to either manage the switches or for end user devices. Shutdown vlan 1 interface on all switches, obviously after you have migrated all devices off it. You can also clear vlan 1 off all the trunk links.

Note for your info, even after clearing vlan 1 off the trunks and shutting down the L3 SVI Cisco switches still use vlan 1 for certain management protocols such as CDP/PagP/VTP etc. There is nothing you can do about this and it's not an issue.

Jon