VLAN Architecture for Large Networks

[fbreeze]

I am finishing up a three tier network design for a major enterprise using a pair of layer three 6509's in the core, pairs of layer 3 6509's in each of nine (growable to 32) distribution regions and dual homed layer two switch stacks ranging from a few stacks up to a maximum of 32 per distribution region. Two server farms are involved, each with a pair of 6509 layer 3 distribution switches. A layer 3 core is being used as this network will work into a planned metropolitan GigE deployment.

I am planning for a maximum of 16 VLANS per switch stack with a separate instance of STP on each to allow for load balancing between two redundant GigE backbone systems and to provide segregation of mission critical data from more hostile traffic.

The CCNP/CCDP texts make some fairly good suggestions for VLAN configuration at the network edge but tend focus on a layer two core and skip over any VLAN mapping strategy in the core. I would appreciate any thoughts or experience on successful resource based or other VLAN architectures between the core and distribution layers of a large network. Thanks Frank

[ssymonds]

Frank,

If not done so already, I'd suggest to check out the 'Multilayer' model as described at the following URL:

http://www.cisco.com/warp/public/cc/so/cuso/epso/entdes/highd_wp.htm

I have been involved in a similar design to yours recently for a large banking institution - a couple of the prime design goals were scaleability and high-availability.

Essentially, we followed the multilayer model (as you seem to be also following)

Key design points for this scenario are:

- Core and Dist layers are L3 whilst the Dist-> Access layer is L2.

- Links between all core-distribution switches each have a dedicated VLAN / subnet

- OSPF running on core-dist switches

- Single VLAN/Subnet per wiring closet ( access switches dual-homed to distribution layer)

- Dual distribution switches run HSRP (tracking uplinks to core)

- No trunking configured between core/dist/access layer switches

- No VTP

Hope this helps a bit,

best rgds

steve

[fbreeze]

Steve...

Many thanks, I'll check out the white paper... Frank

[ccie5851]

I have a few questions that I didn't see the answer to in the URL provided.

1. Why do you not recommend running VTP?

2. Why aren't you trunking between the switches?

3. Are you using small subnets on the core-distribution layer links? (/30, etc)

Thanks!

[ssymonds]

Ron,

I will try to answer your questions:

1. Why do you not recommend running VTP?

It is sometimes a favorable option when designing 'highly available' networks not to configure VTP. VTP adds extra complexity to the network, therefore, not configuring it protects against possible VTP protocol bugs, or, such well-known catastrophic operational issues as; plugging in a switch (note: VTP client or server..) to the netwotrk with higher VTP revision number than all others, and deleting all VLAN definitions. Of course, this 'no VTP' solution will not fit 'every' design, as the sheer number of VLANs and switches in the network, coupled with the VLAN 'spread' may not make this a feasible option. However, in the design that we used, individual VLANs were pretty much restricted to pairs of 'peering-switches', so the admin overhead of manual config was not an issue. One of our design goals was to restrict the 'failure-domain' as much as possible.

2. Why aren't you trunking between the switches?

In the design thatt we used, there is no need for us to trunk between switches, as each core-core or core-distrinution switch link is a single VLAN / subnet (L3 core + L3 distribution layers). Only the distribution-access switches run at layer-2... with a single VLAN per wiring closet,so no need for us to trunk here either. The rationale here is again to restrict failure domains, by utilizing the advantages of an L3 architecture down to the distribution switches as much as possible, and restricting the size or L2 domains, to just the connected L3 distribution switches.

3. Are you using small subnets on the core-distribution layer links? (/30, etc)

Yes exactly. Each core-core or core-dist link forms an L3 subnet with a /30 mask - i.e. pure L3 core-distribution layers.

I hope that I've explained clearly enough ?

thx

steve

[ccie5851]

That did the trick! Thanks!

[fbreeze]

Steve...

Thanks for the further explanation. Clear, concise and it fills in the holes in the BCMSN text that focuses on L3 distribution and L2 core architectures.

Tnx... Frank