Network Design

[SEAN LAURIE]

I Have 5 building with 3 IDF in each building, the client access range from 100 port to 200 ports 2950 or 2960 with 1 gig uplinks to 3560-g's. my core is two 6500sup720-3b. the bulk of my users are in 3 of the build with about 200 is my largest stack. I have just upgraded my standby 6500 to IOS from catos.

my question is I have the ability to take multiple paths if i you ospf or eigrp. also does hsrp play nice between catos and ios for the short term.

[Nathan Spitzer]

First, CatOS does not support HSRP, CatOS is layer-2 only. On hybrid-mode 6500's there is a SP (Switch Processor) which runs CatOS and an RP (MSFC or Route processor) that runs IOS and does the routing. So CatOS doesnt support HSRP, however if your question is if native-mode 6500 plays nice with hybrid-mode 6500 the answer is yes.

Without seeing a diagram of your networks, it sounds like you have an access-layer of 2950 and 2960 switches (two to four per IDF) and each access switch connects to a building 3560(s) distribution switches and I assume they then plug into the 6500 cores.

Couple of questions:

• Do you route or switch from the distribution layer to the core?
• How many uplinks do you provide from the access layer to the distribution layer?
• How many distribution layer switches do you have per building?
• How many connections do you run from each 3560 to the core?
• How are VLANS/subnets assigned? One per floor? Two per floor? Do they span floors? Do they span buildings?

Once we have the answer to those questions we can better gauge whats going on.

[SEAN LAURIE]

Couple of questions:

• Do you route or switch from the distribution layer to the core?

Routing in core, Layer2 in IDF

• How many uplinks do you provide from the access layer to the distribution layer?

1 for each of 2950 or 2960 to 3560-g and 2 back to the core.

• How many distribution layer switches do you have per building?

About 20 to 30

• How many connections do you run from each 3560 to the core?

2 (not channeled)

• How are VLANS/subnets assigned? 6500 VTP Domain Server to client - 3560

One per floor? Mostly one per floor my larger closets have Two per floor? Do they span floors? NO I try to keep a single Vlan in one IDF Do they span buildings? Only one vlan does vlan 30 wifi for management

Once we have the answer to those questions we can better gauge whats going on.

[Nathan Spitzer]

Just to be clear:

You have one set of core 6500's which do all the routing, and you have 3560's acting as distribution switches (one or two per building) with 2 uplinks to the core. Off of each 3560 is a gaggle of 2650/60 access switches?

In that case your question about OSPF/EIGRP is moot since the only routers you have are your cores.  For each VLAN you will have one one root port per distribution switch to the core and the other one will be blocked by spanning-tree.

One mistake people make in this design is "load balancing" between the cores, ie one core is HSRP primary and root bridge for VLANS w-x and the other handles VLANS y-z. In this design the primary HSRP router (and it should have prempt turned on) and root bridge should be the same for all VLANS other wise traffic just ends up taking extra hops unneccessarily.

One thing I will say is that one VLAN spanning the buildings is one to many. Your design would be MUCH more robust if you turned on routing on the 3560's and used multiple routed L3 links and limited your spanning-tree domains to a single building. First, instead of having one link from the distribution layer blocked by spanning tree EIGRP or OSPF would load-balance between the equal-cost links. Second, it would prevent issues (like a (l)user plugging in a hub) that cause a spanning-tree loop from cascading throughout the network and instead would limit it to a single building.

That one VLAN spanning the buildings seriously undermines the stability of the whole network. Been there, done that, taken down the campus.

Nathan Spitzer

Sr. Network Communication Analyst

Lockheed Martin

[SEAN LAURIE]

Thanks for your reply.

You have confirmed what I was thinking of doing. I do have hsrp config with prempt on and primary and secondary root now.

I'm planning on make the changes you have suggested. When I go L3 in each 3560, I will need to config vtp domain and server for each for the vlan\vlans per stack or should I go with transparent?

Also should consider cluster the switches.

Being each closet will have one or two vlan's. would you suggest EIGRP or OSPF for load balancing? What is more popular?

When I make the changes I will no span vlans over multiple buildings. By moving the routing to the 3560 I can use access list to fix the one vlan spanning multiple buildings.

[mwhitlow]

> On hybrid-mode 6500's there is a

> SP (Switch Processor)

> which runs CatOS and an RP > (MSFC or Route processor)

>that runs IOS and does the routing.

Nathan (or anybody) - Do you know if this setup works with the WS-SUP32-10GE-3B blades?

[Nathan Spitzer]

Why would you want to run hybrid mode? As I read the product sheet you can run hybrid mode on a

SUP32  but have no idea why you would. Cat-OS is EOL in 2013 meaning there are no updates to it and support will be harder to get.

[mwhitlow]

> Why would you want to run hybrid mode?

>As I read the product sheet you can run

> hybrid mode on a

SUP32  but have no idea why you would.

> Cat-OS is EOL in 2013 meaning

> there are no updates to it and support will be harder to get.

The answer to this is political, complex, and beyond the scope of this thread. But in any case we do have a reason to do it unfortunately.  Thank you for answering.

[Nathan Spitzer]

One sign of an experienced network engineer is that they have learned to rise above the dogma that there are seven layer of the OSI model and have learned the transendental truthyness of layers 8 and 9 (politics and money)

Seeing as how your bio says you work for an institute of higher learning I would guess the former is an art form and the latter sadly lacking.

[mwhitlow]

LOL  I like your layer 8 and 9 analogy. So true.. So true...

[Nathan Spitzer]

I do not personally like using VTP to manage VLANS but thats partly personal preference. I would say that if YOU ARE D' MAN and its essentially your network it is probably ok, but if there are many cooks in the kitchen I would recommend against it. Additionally if there are "rogue" elements on you network (labs, department-specific engineers, etc) I would highly recommend against it.

OSPF is arguably a better protocol but it has a much steeper learning curve then EIGRP. Understanding why its a better protocol in your case is not worth it most likely. Additionally, it requires more planning up-front to get it right. Unless you think you might have non-cisco equipment in the future I would go with EIGRP.

As for clustering the 6500's. if you have the VSS supervisors absolutely. Read the VSS Campus design guide first but once the 6500's are in a VSS cluster you would use MEC's (multi-chassis etherchannels) between the 6500's and the 3560's.

[SEAN LAURIE]

How would you manage your vlans? if you do not use vtp with the segment(a 3650 with the 2950's or 2960's uplinked to it). In transparent mode? or Do you just use subnets and no vlans at all? I am the D'man with a 100 swiches or so. I do use ciscowork and solarwinds for management.

I no to have vss on my core I was thinking of the 3560 and access layer switchs in a cluster for a single ip management.

I have about 2 month to work this all out before I move my computer room. I will take my time the OSPF.

I have converted one 6513 to ios and 2 3560-g to config in a lab setting. I do see with the new ios you can encrypt and not see vtp password any more if you do show vtp pass in cat os you can see it not very secure like you said.

[Nathan Spitzer]

You add vlans to switches as they are needed and only where needed. For example, a "sane" VLAN topology would be something like this. If you do decide to use VTP each building would be its own VTP domain. I have done the clustering like you describe once. It works, but I dont do it myself but its certainly a valid option.

Building 1

      VLAN100 - management VLAN 192.168.10.0/24

      VLAN101 - 1'st floor access VLAN 192.168.11.0/24

      VLAN102 - 2'nd floor access VLAN 192.168.12.0/24

      VLAN103 - 3'rd floor access VLAN 192.168.13.0/24

Building 2

      VLAN200 - management VLAN 192.168.20.0/24

      VLAN201 - 1'st floor access VLAN 192.168.21.0/24

      VLAN202 - 2'nd floor access VLAN 192.168.22.0/24

      VLAN203 - 3'rd floor access VLAN 192.168.23.0/24

In this case VLANs 100-103 are defined on the Building 1 distribution 3560, VLAN 101 is only defined on the first floor switches, vlan 102 is only defined on the second floor access switch, etc

In this case VLANs 200-203 are defined on the Building 2 distribution 3560, VLAN 201 is only defined on the first floor switches, vlan 202 is only defined on the second floor access switch, etc 

If you do something like this there is no "management" of the VLANs neccessary and spanning-tree for any particular VLAN is limited to only where necessary.

As for OSPF vs EIGRP your design is simple enough that the strengths of OSPF are not really applicable but whether you do EIGRP or OSPF is up to personal preference.