Create a single VLAN on multiple switches

[Phill Hodges]

Sorry I am pretty new to Cisco networking, I've managed to pick up quite a bit, but I am a little puzzled by this scenario.

My setup includes a "Core" switch (3750-E) which is acting like a router for each of my (6) floor switches. The ports that link between the core and the floor switches are routed (no switchport), not access or trunk ports. Each floor switch has its own data and voice vlans which is defined locally per switch.

I am building a WiFi network, which I want to be on its own separate vlan (actually I want 3 vlans for 3 different SSIDs, but for simplicity I will refer to just one). I have one AP per floor that will be plugged into the switch on that floor. I want this vlan to be present on every switch, so no matter which AP you are conencted to, you are on the same VLAN.

I've managed to create the vlan on a single switch by doing the following:

Flr1 (config)# vlan 60
Flr1 (config)# int vlan 60
Flr1 (config-if)# ip address 10.45.60.1 255.255.252.0
Flr1 (config-if)# ip helper-address 10.34.24.10

Core (config)# ip route 10.45.60.0 255.255.252.0 10.255.255.41

Which works because the IP of the vlan interface is the Gateway IP that DHCP gives. If I configure another switch the same way, but give the itnerface a different IP, I can get a DHCP response, but cannot access anything because the gateway is wrong. I then attempted to assign the "gateway" ip address to the vlan interface on the "core" switch, and set no IP address on the floor switches, but this also would not work.

I know how to create the VLAN on each switch, but I am unsure as to what IP to give the interface (if any) and where to define the default gateway. Please advise. Thanks,

[Jon Marshall · Accepted solution]

Tom

The problem is you have routed connections between your switches so you cannot have the same vlan (or more specifically subnet) across multiple switches. I'm assuming that the data and voice vlans on each switch are routed by that switch locally and not on the core switch ?

Put simply you cannot route from a subnet on one switch to the same subnet on another switch, you can only L2 switch. This is one of the main limitations of running L3 from the access-layer to the core/distro. It works fine unless you need the same vlan (subnet) on multiple switches which is exactly your problem. You have a couple of options -

1)  redesign your network so the connections from the floor switches to the core switch are trunk ports and then you can have the same vlan on multiple switches and you route all vlans off the core switch.  This is a fair bit of reconfig and you would need downtime. Plus there may be other complications depending on your setup.

2) a simpler solution if you have spare fibres and ports is to run another connection from each floor switch back to the core switch and make it a L2 access port in the WiFi network.

Edit - re option 1). You don't actually have to route all vlans off the core switch as i previously said. You can still route the existing vlan(s) off the floor switches. What you would do is - 

i) make the connextion from the floor switch to the core switch a trunk port.

ii) allow 2 vlans only on that trunk port. One vlan would be the WiFi vlan.

3) the second vlan would be a new vlan used purely for routing between the floor switch and the core.

So lets say you currently have something like this -

WiFi vlan = 21

new routing vlan = 22

core switch 1

==========

int gi0/1 <-- connects to floor switch 1

no switchport

ip address 192.168.5.1 255.255.255.252

floor switch 1

==========

int gi0/1 <-- connects to core switch

no switchport

ip address 192.168.5.2 255.255.255.252

you would change this to -

core switch 

==========

int gi0/1

switchport mode trunk

switchport trunk encapsulation dot1q

switchport trunk allowed vlan 21,22

int vlan 22

ip address 192.168.5.1 255.255.255.252

floor switch 1

==========

same port config as above.

int vlan 22

ip address 192.168.5.2 255.255.255.252

Your routing should still work okay because you have simply migrated the same IPs from the actual ports to the vlan 22 SVIs. Vlan 22 will be used when your data/voice vlans need to route off the floor switch to the core. Vlan 21 can then be added to other switches.

Note that this all assumes you are currently routing the data/voice vlans on each floor switch on that switch but if you are using routed connections then you must be.

Jon

[Phill Hodges]

Thanks for that information Jon, although not exactly what I wanted to hear.

jon.marshall wrote:
I'm assuming that the data and voice vlans on each switch are routed by that switch locally and not on the core switch ?

As far as this goes, the static routes for all the networks are defined on the core switch. I'm not sure if that matters at all though.

[Jon Marshall]

Tom

I've just updated my post. The redesign is not as bad as it first sounded although there would still be some downtime while you changed the uplinks from routed ports to trunks.

Have a read of my updated post and see if it all makes sense. If in doubt about your config can you post the core switch config and one of the floor switches.

There may be statics on the core switch but the key question is still does each floor switch have 2 SVIs ie. one for the data vlan and one for the voice vlan. It sounds as though they do but just need to make sure.

Jon

[Phill Hodges]

Yes each floor switch has an SVI for each VLAN.

The redesign idea does not sound as though it would be that involved, and I may end up going through this route. I think it will benefit in the long run to do it this way. Are there any other considerations

Thank you very much for this insight.

[Jon Marshall]

Tom

It's actually not that complicated to change but as i said you will obviously need downtime because you are having to change the uplinks. It's difficult to say what other considerations there are because i don't know why the network was designed that way in the first place. Note that i'm not saying there is anything wrong with your setup, it is a perfectly valid L3 design.

However there is one general consideration and that is STP. You would need to make the core switch the STP root for the WiFi vlan and the new dedicated vlan as well. Note that the dedicated routing vlan (vlan 22 in my example) should be different per switch because it is purely to route for the local floor switch.

Whether STP has any more impact depends on some more info i need from you. What is the core comprised of ie. is it just one 3750, a stack of 3750s or a pair interconnected. If it is a pair interconnected then what is that interconnection ie. L2 or L3.  And again if it is a pair how do the floor switches connect ie. do you uplink to both switches.

Apologies for all the questions but we need to be sure of the impact before making any changes.

Jon

[Phill Hodges]

I am also not sure why it is set up this way. The previous admin had a tendancy to do some wacky stuff that I now have to weed through.

The core is comprised of a stack of two 3750's and there is only 1 link to each floor switch.

I assume this means I'll need to enable STP on all of the interfaces, but I am not sure what it means to make the core the STP root.

[Jon Marshall]

Okay, if it is a stack and you only have one connection per floor switch we should be fine. So basically each floor switch is STP root for the local vlans ie. the data and voice. Then the 3750 stack is STP root for the dedicated vlan per floor switch and the WiFi vlans. Just to reiterate it is important to make sure when you configure the trunk links you add the allowed vlans and these vlans should be just the dedicated routing vlan (the new dedicated vlan) and the WiFi vlan.

You don't need to enable STP on the interfaces. Unless your admin delibarately turned off STP, and he shouldn't have, it will be running anyway. To make a switch root for STP you would use this command -

switch(config)# spanning-tree root root

I have included this link to the config guide for STP on 3750E switches -

http://www.cisco.com/en/US/customer/docs/switches/lan/catalyst3750e_3560e/software/release/12.2_55_se/configuration/guide/swstp.html#wp1039614

It says in the config guide that by default it runs PVST+ and not Rapid PVST+ so you may want to consider changing to Rapid PVST+ but if you only have one uplink per floor switch and a stack in the core then you have no L2 loops as far as i can see so it's not such an issue.

To be fair to the previous admin the setup you have is actually a valid Cisco design and i implemented something similiar in my last job but we did not need WiFi there so it worked for us. But as i said before it is unfortunately not as flexible as a L2 setup. Using routed links from the access switches to the core/distro switches  removed the reliance on STP so it was quite attractive but with the advent of MEC (multichassis etherchannel), VSS etc. the reliance on STP has been lessened anyway.

Wihout wanting to confuse you if you implement the solution i suggested you will be in a sort of halfway house between L2 and L3 ie. some vlans are routed locally and some are routed off the core. There is nothing wrong per se with this but it could be confusing for future admins.

What is more concerning is the one uplink. If this is a single link rather than an etherchannel then obviously this is a single point of failure. Because the 3750s support MEC you can use an etherchannel trunk link between each floor switch and the core stack and spread the etherchannel ports across both members of the stack. This way if one of the ports in the etherchannel fails you still have other ports to use.  Using an etherchannel trunk would not affect the solution proposed but the trunk link config would be slightly different.

Finally, a more common design with the equipment you have is to route all vlans on the core switches using etherchannel trunks from each floor switch. However this involves quite a bit more config and if there is QOS config for example on the voice vlan this would also probably need changing. And it may be that there was a good reason for routing vlans locally on the switches that i am not aware of.

Hope i haven't confused the issue too much.

Jon

[Phill Hodges]

No, you haven't confused me at all, you've done quite the opposite. I see where you are coming from and I would definitely consider eventually moving all of the vlans off of the individual switches and onto the core. This is less of a concern right now, but once I have the links set up as trunks, it will be easier to implement at a later date.

I'm definitely going to have a talk with my maintenace team to see if we can get and run some more fiber to the floor closets. I would love to have the failover/redundancy of the etherchannel link, but right now the physical constraints of only have 1 fiber drop is holding me back.

I can't thank you enough for this information

[Jon Marshall]

Tom

No problem, glad to have helped. I hope the interim solution goes well if you decide to implement it and any problems with it don't hesitate to come back.

Jon

[Phill Hodges]

I just wanted to follow up on this post- I followed the advice Jon gave and performed the network redesign and everything worked perfectly! Thanks again Jon.

[Phill Hodges]

I seem to be missing an integral part of this set up...

After I created the new VLAN, I cannot access anything outside of the core and floor switches. I can reach all of the networks on the floors switches, but I can't get out to the Internet or access my servers. All of the vlans on the floor switches can access the Internet and Servers. I'm not really sure how to proceed from here, any ideas?

[glen.grant]

  More than likely you have a FW for internet access , did you add rules and routes for the new subnets on FW so it knows where to send packets for those new subnets ?   Where are your servers located ?

[Phill Hodges]

DUH! I forgot to add the route to the firewall.

Thanks Glen.

[Jon Marshall]

Tom

Thanks for coming back and letting us know it all worked, And glad Glen was around to sort out the firewall issue

Jon