Best practice for chaining switches together?

[jmcgrady1]

I have a campus of 6 buildings. Each will be installed with two C9300-24P-A switches in a stack. Two 10gb fibres connect adjacent pairs of buildings so as to form a "ring".  The switches will present numerous vlans on their access ports. Some vlans are unique to a given building and will have their own IPv4 subnet. Other vlans must be presented in multiple buildings.

Can you advise as to what would be best practice for the topology of this scenario?  My initial thought is to aggregate each pair of 10gb fibres using one SFP from each switch in a stack. The aggregate 20gb link (LACP or native Cisco?) will be a dot1q trunk carrying numerous vlans. Rapid spanning tree would be established across the campus.   

As layer 3 routing is also required, i plan to have vlans setup as uplink subnets. EIGRP will be used to advertise. Would it be better to have a dedicated subnet between each pair of adjacent switch stacks? Or to have a single mesh subnet across the campus, with each switch stack having a single IP on that network?

[Reza Sharifi]

If you have enough fiber runs to one of the locations, it may be a better design to have star topology by uplinking each set of 9300 stack switches to a pair of core switches. So, for example, if building 3 has fiber runs from all the other 5 buildings, you could install a set of core switches like the 9600 series in that building and then make a connection from each 9300 switch stack and put them on Portchannel and connect them to both core switches. 

BLD3-9300----Core

BLD1-9300----Core

BLD2-9300----Core

and so on.

For your Portchannel, you can simply use LACP or mode on. LACP is standard and works fine in a multi-vendor environment or all Cisco.

As for routing protocol, turn it on at the core only. You can use EIGRP or OSPF, again OSPF is standard in case you need to connect to other vendor's devices.

For STP, Run RSTP between access and core switches, and since you are using Portchannels, there is no STP blocking any uplink ports.

HTH

[jmcgrady1]

Thanks for the reply Reza. Unfortunately a hub and spoke topology is not available to me. 

[balaji.bandi]

if you looking RINg deployment have a look at the document :

https://www.cisco.com/c/en/us/products/collateral/switches/catalyst-9300-series-switches/white-paper-c11-741468.html

[Reza Sharifi]

Hi,

That is fine. If you have to daisy-chain switches together and build a ring topology, the points regarding routing protocols and Portchannles are still valid. You just have to make sure you have a solid STP deployment so, there is very little chance of any broadcast storm. Obviously,  stacking and using Portchannels will eliminate a bunch of STP issues that can occur, as there will be no blocking and forwarding on those interfaces.

HTH

[Joseph W. Doherty · Accepted solution]

What I would suggest would be your dual 10g links (each to a different stack member [as you propose]) are combined using LACP.

You would configure the Etherchannel as a trunk.

Best/better practice is to avoid distributing L2 all over a campus, so only pass around VLANs that absolutely need to span more than one building.  (NB: This doesn't mean to not spread a VLAN across buildings such that you don't have a redundant L2 path.)

For all other VLANs, where you really don't need to span them beyond buildings, route between them.

The way to do both, is first, of course, spread the inter-building VLANs, across your trunks, as needed.

Second, for routing, between each building to building Etherchannel trunk, define just one VLAN shared, across that trunk, only known to the two connecting L3 switches.  Effectively, each of these unique VLANs will become a logical p2p L3 link (SVIs would using a /30 or perhaps /31 IPs).

I.e. L3a VLAN 901 <trunk> VLAN901 L3b VLAN 902 <trunk> VLAN902 L3c . . . L3# VLAN 9xx <trunk> VLAN 9xx L3a

[jmcgrady1]

Thanks Joseph. What you describe is what i have in mind. One more question; with the 9300 switch range, and using RSTP, are there limitations on ring size? If i was to have nine c9300 stacks chained together in a ring, would that be an issue?

[Joseph W. Doherty]

STP does have "diameter" size restrictions, but I think (?) RSTP isn't bound with quite the same limits.  (I also believe [?], on modern switches, you can press somewhat beyond the diameter restriction, even perhaps w/o any STP parameter adjustments.)  You might research or perhaps post this question as a new forum entry.

Also, if your 9300 support REP, that might be another and/or better option.