Fiber Ring Design Considerations

[david]

I have a customer that is interested in building a fiber ring network.  Original discussions centered around building a network with approximately 15 devices on the network.  So we sold and implemented 14 Cisco Catalyst 2960S switches and 1 Catalyst 4948E switches.  Each switch connected to the next device in the ring via 10GB Ethernet over Fiber utilizing Cisco 10GB SFP+ modules. 

We then sold the customer another similar fiber ring network.  Now the customer is coming back to us, and wants to all of the 30+ devices into a single fiber ring.  My engineer is saying that this won't work, that there is a limit of approximately 17 devices in the ring.  See link below for more details.

http://www.cisco.com/en/US/tech/tk389/tk621/technologies_tech_note09186a0080094954.shtml

My question - has anyone built a fiber ring with more devices?  Is it physically possible?  Would there be limitations?  Any gotchas to be aware of? 

Any feedback would be greatly appreciated.

David

[mdaaxcess]

Try to upload an drawing of the network, would help

Sent from Cisco Technical Support iPad App

[david]

The first image below shows how the 2 fibers exist currently.  ph0 consists of 15 Catalyst 2960S switches with 2 10GB ports (in/out on the fiber ring), and 2 Catalyst 4948M with 4 10GB ports (2 for in/out on the fiber ring, 2 for connectivity to VMware servers).  ph1 is pretty much an exact duplicate.

The second image below shows the proposed combining of the 2 existing fiber rings.  Currently these are simply flat networks, no layer 3 routing.

If we combine the 2 fiber rings with approximately 31 total switches, leaving it as a flat network, will the network work correctly?  What happens if there is a switch or fiber cable failure? 

[Amit Singh]

Hello David,

There are a lot of things to consider if you want to combine the two rings.

Is customer running STP on these rings?

Is customer running switch stacks of the 2960S?

If yes, How many switches are stacked and how they are connected?

I am curious to know if customer is running STP and this network has worked fine in the past? Ideally in a stable Layer-2 network with STP, your end switch can be just 7 hops away from the root bridge. This would make the STP behaviour predictable. If your end switch is more than 7 hops away, STP might be unpredictable.

If yes, How would the root bridge is placed within this network? If you connect these two rings on L2, where would you place the root bridge.

With the Ring topology, your STP link blocking, traffic forwarding in  case of failure is a bit unpredictable as you do not have any control  which link will be blocked by STP and how would the traffic pass over  the ring. In case the adjacent switch link next to root bridge gets  blocked, the last switch will have 14 hops to pass the traffic to the  outside network if root bridge is your L3 exit gateway for the outside  world.

Why customer cannot have a point-point network design dual connected to to a pair of aggregation switches like 4500? You are sort of making a hybrid design with but I would make it more point-point design vs sticking with the ring toplogy.

Hope this helps.

-amit singh

[david]

Unfortunately the customer has already implemented the 2 fiber rings.  And yes, they are using Spanning Tree.  To go to a hub and spoke network design at this time would be VERY expensive, as it would require different core switch, plus all new fiber cabling.  So I'm trying to figure out a way to do this with what we have.

Could we utilize some layer 3 routing?  I've also read a little on a Cisco protocol called REP, and something called Flexlink. 

I'm trying to make sure that this network will work properly, and if I need to do some redesign, I need to make it as unobtrusive as possible.

[Amit Singh]

David,

Unfortunately REP protocol is not supported on 2960S switches, which could be the best choice within this design.

Flex  links cannot be used in this design as Flex-links are primarily used in  a point-point Loop design where one interface act as a back up of  another and gives your faster recovery in case of primary link failure.

As 2960S only support basic L3 funcationality like SVI ad static route so it will be help much in this design.

Another  design that you could do is to break these big rings into small stacks  and place the 4948 M in the center of these stacks. Breaking these into  small stacks will give you STP Loop free design and more predicatble  netwotk behaviour.

Hope this helps.

Cheers,

-amit singh

[david]

Amit,

      Thanks for the feedback.  Any idea how the network will respond if we simply merge the two existing rings together?  We have a 2 hour window where we can do testing, and I'm looking for some information on what will happen when we merge the two rings.  Will the network crash?  What happens if there is a link or switch failure?  Feel free to contact me directly at david@fulcrumgroup.net.

David