03-16-2010 10:20 AM - edited 03-06-2019 10:10 AM
We are currently creating all our VLANS on our core (6513 sup2 720). One of our VLANS has over 19 class c subnets on it. Last week after a power outage, the core's processor went up to 92% until I looked on the wire and saw that DHCP was going mad! We restarted the DHCP service and all was well again.
The DHCP server is in the VLAN that has all those subnets. I'm trying to convince my boss that it is wise to move all the vlans out to the edge onto the distribution switches and have a set of VLANS per building.
I'm sure my preferred scenario would have stopped this problem, which I think was a massive broadcast storm.
Would you all agree?
Jamie
Solved! Go to Solution.
03-17-2010 04:36 AM
jhancockuwic wrote:
Hi Jon,
Yeah that makes sense to me. That is the scenario I would like to put in place, it's just convincing my colleague now!!
Jamie
So, 1 /24 VLAN per floor, then you split that VLAN into 2 /25. I missing something again here, if you create the vlan on the 6500 "ip address 192.168.1.1 255.255.255.0. How are you then splitting this up into 2 /25?
Sorry to be a pain.
Jamie
Jamie
Jamie
Sorry, posted before you edited this post.
You don't create the vlan ip address of 192.168.1.1 255.255.255.0. Instead you would have for example 2 vlans per floor - vlan 10 & 11. On the 6500 switch
int vlan 10
ip address 192.168.1.1 255.255.255.128
int vlan 11
ip address 192.168.1.129 255.255.255.128
Don't get too sidetracked with /24 vs /25. There is nothing wrong with a /24 and a lot of networks use this as standard.
Jon
03-16-2010 10:27 AM
jhancockuwic wrote:
We are currently creating all our VLANS on our core (6513 sup2 720). One of our VLANS has over 19 class c subnets on it. Last week after a power outage, the core's processor went up to 92% until I looked on the wire and saw that DHCP was going mad! We restarted the DHCP service and all was well again.
The DHCP server is in the VLAN that has all those subnets. I'm trying to convince my boss that it is wise to move all the vlans out to the edge onto the distribution switches and have a set of VLANS per building.
I'm sure my preferred scenario would have stopped this problem, which I think was a massive broadcast storm.
Would you all agree?
Jamie
Jamie
It would be a very good idea not to have such large vlans ie. 19 class C subnets, as that is the root cause of your problem and not where the inter-vlan routing is being done.
However If you have multiple buildings interconnected via the core 6500(s) then yes it would be a good idea to have each buliding use it's distribution switch(s) to do the inter-vlan routing assuming of course that you have L3 capable distribution switches in each building ?
Jon
03-16-2010 10:31 AM
Hi Jon,
Yeah I thought so, unfortunately I'm the new kid so it's not always easy to get my point across.
Going to try and put my point across tomorrow.
Oh, and yeas we have layer 3 capable switches at the distribution.
Jamie
03-16-2010 10:39 AM
Hello Jamie,
having 19 IP subnets /24 sharing the same L2 broadcast domain is a bad idea as you have seen on that outage.
When a broadcast is sent it needs to be processed by all devices including those that actually are not interested on it.
also cpu usage on user PCs suffer from this.
There is a relationship between number of hosts and broadcast level in a vlan/L2 broadcast domain.
You should have a one to one corrispondence between one IP subnet and one vlan/L2 broadcast domain.
if the total number of vlans is high and you cannot scale moving the L3 boundary towards access is a wise step.
if you have only few vlans you can create more vlans easily and you move users to them
Hope to help
Giuseppe
03-16-2010 10:42 AM
Hi Giuseppe
Thanks for your help.
We have over 20 vlan's per site.
Can you explaine this point please "if you have only few vlans you can create more vlans easily and you move users to them"?
Jamie
03-16-2010 10:58 AM
Hello Jamie,
you are right I haven't explained what is behind that sentence:
low end cisco switches have limitations in the number of STP instances that they can run and also limits on the max limit of Vlans that can be present at the same time in the vlan database.
These two limits can bwe different.
For example some device like a C2950 could have a limit of 64 STP instances and 128 vlans in VTP DB.
This fact causes a scalability limit it is not possible to have more then 128 vlans in VTP database.
STP limits can be handled with appropriate manual control of list of vlans on each trunk link both sides using allowed vlan command
switchport trunk allowed vlan x,y,z
so moving from 20 vlans to 400 vlans may be not possible in a campus if some devices don't support so many vlans will:
- move to VTP transparent mode automatically to protect themselves
- disable STP instance for vlans in excess of first limit.
the second can be handled with manually configured trunks, the first has impact only if you use VTP.
Moving routing to distribution is wise (if a distribution layer exists in your campus or if you introduce it), moving it to access can be a solution too nowdays.
Hope to help
Giuseppe
03-17-2010 01:38 AM
Thanks for your help guy's.
I take it from reading various threads that one thing I should change as soon as possible is the 19 secondary IP address we have in a particular VLAN.
I take it the easiest way is to create a VLAN and use a subnet mask like 192.168.0.0 /21 which would give us 2048 IP address.
Would this have stopped the problem of the CPU overload?
Jamie.
03-17-2010 02:04 AM
jhancockuwic wrote:
Thanks for your help guy's.
I take it from reading various threads that one thing I should change as soon as possible is the 19 secondary IP address we have in a particular VLAN.
I take it the easiest way is to create a VLAN and use a subnet mask like 192.168.0.0 /21 which would give us 2048 IP address.
Would this have stopped the problem of the CPU overload?
Jamie.
Jamie
Probably not because the issue is with that many clients on the same vlan all broadcasting at roughly the same time. However you should still look to get rid of the secondary IP addresses if you can.
As Giuseppe noted having a vlan that size can cause many problems, you really need to try and get down to a vlan per class C or thereabouts if possible.
Jon
03-17-2010 02:12 AM
Hi Jon,
The trouble is having over 3000 pc's + peripheral devices it would mean having 12 VLANS all for the same purpose, or am I missing something?
Plus, administering that would be a nightmare.
The only thing I can see that would have helped with this problem would have been having the VLANS distributed to the edge. So if the problem was coming from a particular VLAN it would have only affected that building.
Also, having the DHCP servers in the server farm would have helped, I think?
Does this make sense? Would the links still have got saturated?
03-17-2010 02:37 AM
Jamie
The trouble is having over 3000 pc's + peripheral devices it would mean having 12 VLANS all for the same purpose, or am I missing something?
Plus, administering that would be a nightmare.
Put bluntly, yes you are missing something Best practices says you should use a /24 per vlan or perhaps a /23 if needed. I use /25s often. It doesn't matter if they are all for the same purpose, having that many devices in one vlan means a huge amount of broadcast traffic within that vlan. Every time a broadcast happens in that vlan every single device has to process that broadcast.
If i started at a new company and found a vlan with 3000 devices in it i would view that as a quick win to split it up, wouldn't matter whether i was the new boy or not. And you may well find that performance improves for the end users. From my experience you will find that using /24's etc. for vlans is common practice and what you have is the exception.
As for adminstering it and it being a nightmare, i can't see how. 12 vlans is nothing in terms of administration as most of the devices will be using DHCP to obtain their IP addresses. You may need to readdress some devices with static IPs but once done that is it.
Jon
03-17-2010 02:39 AM
Jamie
A quick follow up -
The only thing I can see that would have helped with this problem would have been having the VLANS distributed to the edge. So if the problem was coming from a particular VLAN it would have only affected that building.
This seems to contradict what you are saying because here you seem to be suggesting breaking up the one vlan into many or am i misreading ?
The key point is the size of the vlan not the location. Moving it out of the core as it is will protect the core but not where you move it to.
Jon
03-17-2010 02:53 AM
Hi Jon,
Thanks for your help with this.
My point about the VLAN administration is this. If we have lots of /24 VLANS, it would mean changing all the switch ports to suit. How would we mange what switch ports had what VLANS? If we set more than 255 switch ports for a particular VLAN we could run out of I address.
For instance, in Block A we have 2 wiring closets with 3 access layer switches in each closet. So, total number of ports would be 288, would we assign 255 ports to VLAN 2 and the rest to VLAN 3? Can you see my point, or am I still missing something?
Does that make sense?
Jamie
03-17-2010 03:53 AM
Jamie
My point about the VLAN administration is this. If we have lots of /24 VLANS, it would mean changing all the switch ports to suit. How would we mange what switch ports had what VLANS? If we set more than 255 switch ports for a particular VLAN we could run out of I address.
You allocate a /24 to each vlan but that doesn't mean you then go and allocated 254 ports to that vlan on a switch at the same time. A way to separate it up is to have vlans per floor of a building unless of course you have 3000 users on one floor. There is no requirement to fill every single vlan, you can actually have more vlans than you need so there is some slack in each vlan for new users.
Last place i worked we had multiple buildings within a MAN. An average building had 5 floors so we allocated a /24 to each floor and then split the subnet into 2 x /25. One side of the floor got one /25 and the other half the other /25. Then we simply allocated ports on that floors closet switches into either vlan when needed and any unallocated ports were put into a non-routed "dummy vlan" to be used if new users moved onto the floor.
The floor access-layer switches connected via fibre back to the main LAN room for that building. The uplinks were L2 trunks and the inter-vlan routing was done on a pair of 6500 switches in the main LAN room.
Basically managing ports, allocating them into vlans etc. is a standard part of a network administrators job. The setup will take some time but once setup you are really only having to do minor changes from there on in.
Jon
03-17-2010 04:24 AM
Hi Jon,
Yeah that makes sense to me. That is the scenario I would like to put in place, it's just convincing my colleague now!!
Jamie
So, 1 /24 VLAN per floor, then you split that VLAN into 2 /25. I missing something again here, if you create the vlan on the 6500 "ip address 192.168.1.1 255.255.255.0. How are you then splitting this up into 2 /25?
Sorry to be a pain.
Jamie
Jamie
03-17-2010 04:36 AM
jhancockuwic wrote:
Hi Jon,
Yeah that makes sense to me. That is the scenario I would like to put in place, it's just convincing my colleague now!!
Jamie
So, 1 /24 VLAN per floor, then you split that VLAN into 2 /25. I missing something again here, if you create the vlan on the 6500 "ip address 192.168.1.1 255.255.255.0. How are you then splitting this up into 2 /25?
Sorry to be a pain.
Jamie
Jamie
Jamie
Sorry, posted before you edited this post.
You don't create the vlan ip address of 192.168.1.1 255.255.255.0. Instead you would have for example 2 vlans per floor - vlan 10 & 11. On the 6500 switch
int vlan 10
ip address 192.168.1.1 255.255.255.128
int vlan 11
ip address 192.168.1.129 255.255.255.128
Don't get too sidetracked with /24 vs /25. There is nothing wrong with a /24 and a lot of networks use this as standard.
Jon
Discover and save your favorite ideas. Come back to expert answers, step-by-step guides, recent topics, and more.
New here? Get started with these tips. How to use Community New member guide