Spanning VSS across the campus

[Keith Lawrence]

Hi,

I am currently working on a network design for a new campus and would like to get some opinions on VSS.

The campus is fairly large (45 acres) and we have the use of multiple buildings for the data centre and DR facility. As the campus is large, we are planning for an on-campus DR facility as well as an offsite facility. We are putting in a transit fibre infrastructure that will cover the entire campus with diverse paths and plenty of capacity. Despite the size, we expect the switch port density requirements to be moderate (lots of non-office space and warehouse-style areas).

As part of the onsite DR, we plan to protect our core switching as well as servers/telephony etc.

My plan so far is to deploy a single pair of 6506e chassis, one at each side of the campus, and connect the VSL using diverse 10gb links from our fibre network to form a VSS pair. Then we will be connecting our access layer stacks in each building to both chassis, using MEC.

So essentially, I will end up with a collapsed core design, utilising VSS to protect the core, with each of the chassis pair located about 300m apart on diverse paths.

Anyone implemented a similar design? Any gotchas I should be aware of?

[Bilal Nawaz]

Ensure the 10GigE optics you go for the VSL link supports the distance. Apart from this, I have also implemented similar to you in the past on OM3 fibre and all was good.

Bilal

[Jami Bailey]

Keith,

VSL can be run up to distances of 40km so from a technical perspective, you can achieve what your asking. Just ensure your design is as redundant and diverse from a path perspective as possible to avoid split brain.

[paul driver]

Hello

I did this last week basicaly just how you have mentioned it, but using 4500x, but also incopoarating DAD on the mec's and all seems to good at present.

(Dual-Active-Detection)

res
Paul

[shillings · Accepted solution]

Hi Keith,

Ensure you install plenty of fibre between the two comms rooms. We don't have a view of your entire solution but a very large campus network can require several dozen pairs between the two comms rooms once everything is totalled up.

Whilst OM3/OM4 is OK now for 10GBASE-SR, it won't be sufficient if you move to 40GBASE-SR4 in future so SMF is worth the extra cost to my mind. I'd expect the 40G transceiver cost to fall over time so don't be too put off by current pricing.

As Paul mentioned, you need some form of dual-active detection so include a VSLP keepalive link (1G is fine) and ideally ePAgP for backup dual-active detection, perhaps via one of your access layer switches/stacks, if feasible.

Try to dual home everything across the 6506 pair, ideally uplinking only to other switches, i.e. terminate firewalls and WLCs on service block switches rather than directly into the core. Use dedicated server switching for the servers. This way you reserve the collapsed core/distribution layer for routing (assuming L3 edge), avoid using the VSL for data (except during some failures), and take advantage of VSS SSO / MEC for sub-second recovery following many core-related failure scenarios (recovery from some scenarios can take just 150ms, short enough not to impact any VoIP calls in progress.) This approach will also enable you to perform sub-second ISSU on the core (ISSU process is called EFSU on a VSS-enabled 6K/Sup2T pair.) I think the ESFU outage is reported to be ~200ms per chassis.

Ensure the VSL has enough bandwidth to cater for the failure of a single connected switch. For example, if you have a 4 x 10G full mesh connecting a VSS-enabled pair of service block switches, then failure of a single service block switch will result in up to 20G of traffic needing to re-route via the VSL in order to reach the single remaining service block switch. 

Cisco recommends up to 20:1 oversubscription ratio on access-to-distribution layer uplinks, so design your access switch uplinks accordingly.

The latest (2013?) Wired LAN design guide is very good and covers much of this.

Good luck.

[Keith Lawrence]

Everyone - thanks very much for the input.

Looking forward to implementing this in the next few months. Ill drop a reply once we have it up and running.

Happy new year!

Keith