Network design - Layer 3 boundary To be in IDFs or not to be?

[William Reed]

I'd like to get some input regarding network infrastructure design, and L2/L3 boundary recommendations.

Let's assume the customer has a collapsed core w/ X IDFs. The current deployment is Layer 2 everywhere, w/ all routing coming back to the core stack/chassis. (They're not currently IP voice/video, but we'd like to design a solution that will support this, and efficient inter-data/v/v routing for collaboration purposes.)  Each IDF/building will have Y endpoints.

At which point do you recommend a Layer 3 (L3) boundary at each IDF vs. an all L2 solution?  Specifically, we could position a single L3 switch at each IDF (like a Cisco 3650), with 2960X L2 switches within the building, to 1) control b/m-cast traffic, and 2) to facilitate inter-VLAN routing (between data/voice/vid VLANs) for more efficient collaboration traffic. Without a L3 switch at each IDF, all this b/m-cast and routing traffic would go back to the core.

I'm hoping/looking for design recommendations - case studies/best practices, whatever. Customer-facing, I need to present real numbers for a L3 IDF switch vs. pure L2 for whatever number of ports. (Again, keep in mind that we need to design the solution to comfortable support IP voice/video in the future.But also keep in mind 98% of the traffic goes back to 1 of 2 data centers.)

Thanks, and reference docs/links are always appreciated.

[Joseph W. Doherty]

Disclaimer

The Author of this posting offers the information contained within this posting without consideration and with the reader's understanding that there's no implied or expressed suitability or fitness for any purpose. Information provided is for informational purposes only and should not be construed as rendering professional advice of any kind. Usage of this posting's information is solely at reader's own risk.

Liability Disclaimer

In no event shall Author be liable for any damages wha2tsoever (including, without limitation, damages for loss of use, data or profit) arising out of the use or inability to use the posting's information even if Author has been advised of the possibility of such damage.

Posting

Basically, L3 at the edge is advantageous if there's routing between ports on the edge device.  The two common situations for that would be routing between VLAN ports on the edge device (but you say 98% of traffic goes to DC) or if your edge connected to multiple, different, upstream L3 devices (so multiple links might be used).  If you have only one logical or physical core device, that won't come into play.

Ideally you don't want to extend L2 VLANs beyond each edge device.  I.e. to get to another edge device, you're route via the collapsed core.  Also you're route on that core to jump between VLANs on the same edge device.

[William Reed]

Thank you for replying, we are thinking the same thing also.

However I would love for more engineers to comment to backup what we are thinking.

[Jon Marshall]

There is no specific point at which it makes more sense to use one or other, it depends on a number of things.

L3 at the edge made more sense when you did not have stacked switches or VSS because then STP had to start blocking links.

But now you don't have those restrictions and L2 will always be more flexible in terms of vlan placement ie. I have seen a number of design issues on these forums where people were using L3 from the access layer and then suddenly had a requirement to span a vlan across multiple access switches.

If, as you and Joe point out, most of the traffic is via the distribution switches then there is little to gain from going to L3 other than to isolate broadcast domains to the wiring closet but the gain is minimal to be honest.

Multicast traffic would not be helped particularly by L3 because it would still need to be sent across the links if there were interested receivers whether they were L2 or L3 and if there were no interested receivers on the access switch then there would be no need for the distribution switch to send the multicast packets assuming IGMP snooping and PIM sparse mode were in use.

Unless the source and receiver were actually on the same switch which is unlikely I would have thought.

Plus there is the additional cost of the L3 licenses for your switches.

If you look at the Cisco design guides for campus environments they are based around stacking or VSS because it gives the most flexibility with ease of deployment.

That said if you think the amount of traffic between vlans on the same switch may increase to the extent where it becomes more viable then it may be worth considering.

Not trying to put you off it :)

Jon

[William Reed]

Thanks guys!

Everyone else feel similar?