MPLS Termination at Datacenter - Router vs. L3 Switch

[L P]

Hello all,

I'm in a position where I need to make a decision regarding termination of an ethernet MPLS link at our datacenter, and I have two options -

Terminate to the current "core" switch, a 3750 stack, and run BGP.

Terminate to a 3925, run BGP, and redistribute into EIGRP or OSPF for the core switch.

I know the older wisdom held that a WAN should generally terminate to a router, but the 3750s present a potential opportunity to eliminate a piece of hardware that is less redundant than the switch stack itself.  Am I missing any caveats that I shouldn't be, in considering eliminating the router from the equation?

Any opinions and/or insight are appreciated.

Thanks!

[Jon Marshall]

Some things to consider -

1) can your 3750 handle the BGP routes ie. they often run at high CPU as it is

2) do you want to do any QOS. If so switches generally have a smaller feature set than routers

3) unlikely but if you need to do any NAT you can't with a 3750 switch

4) GRE, IPSEC etc. Again, unlikely over MPLS but worth mentioning.

In terms of eliminating a single point of failure you aren't really because if the switch the link is connected to fails then it doesn't really matter if it is part of a stack or not.

All that said it's not necessarily a bad idea to do it. If all traffic to and from the MPLS network has to go via the switch stack then it doesn't introduce any more failure points in your network.

It really depends on your requirements.

Jon

[Joseph W. Doherty]

Disclaimer

The Author of this posting offers the information contained within this posting without consideration and with the reader's understanding that there's no implied or expressed suitability or fitness for any purpose. Information provided is for informational purposes only and should not be construed as rendering professional advice of any kind. Usage of this posting's information is solely at reader's own risk.

Liability Disclaimer

In no event shall Author be liable for any damages whatsoever (including, without limitation, damages for loss of use, data or profit) arising out of the use or inability to use the posting's information even if Author has been advised of the possibility of such damage.

Posting

The L3 switch has much more raw packet forwarding performance than an ISR; how much bandwidth is a consideration.

Is the hand-off require a MPLS enabled port?  (I don't thing the LAN 3750s support that.)

Regarding redundancy, depending exactly upon hand-off configuration port requirements, it may be possible to configure a ready-for-immediate-use backup port on another switch member.  I.e. if the port hosting switch member dies, you can quickly repatch into the predefined "backup" port.

A 3750 will likely be much more BGP feature "challenged" then the router; also a 3750's has less control plane capacity than an ISR (although the ISR control plane and data plane share the CPU - i.e. in corner cases, a 3750 could have more control plane capacity than the ISR).

[Collin Clark]

The  3750 switch is an access layer switch. I would never terminate a WAN link on that or terminate it on the core. Follow best practice design as much as possible.

[Jon Marshall]

Hi Collin

Haven't spoken you to you in a while, hope you are well.

I know the 3750 is sold as an access switch but we've seen quite a few setups on these forums where people use them as a small core/distro.

I agree best practice says separate WAN connectivity but best practice often equals more money and sometimes that can be hard to justify.

I am genuinely interested to hear what the cons are against terminating it on the 3750 apart from the ones already mentioned.

Don't take this the wrong way but saying don't do it because of best practice isn't really an argument or otherwise we wouldn't really need network architects ie. we could all just download the three tier model and job done.

Honestly not trying to be argumentative, just wanted to know if there are other things I should be thinking about.

Edit - I should say to me it doesn't feel right to do it but I'm just trying to think of good reasons why not.

Jon

[Collin Clark]

Hey Jon-

A couple of (additional) things;

CPU: Stacking takes up CPU, routing on a stack takes even more and redistribution on a stack is horrible. Now you can tweak some things to make it usable, but for the most part it just consumes too much CPU.

Design: A core, distribution, and WAN on an access switch, I'd never bless that design. Functionally you're setting yourself up for failure. Can it be done? Yes, w/o a doubt, but not by most engineers. It's just too much to protect one thing from all the other things when they are all living on the same box.

Misc: Can the 3750 do the QoS I need it to do? What about security (GETVPN across MPLS, CPPr)? Can it scale (how many remotes/traffic)? What about maintenance, am I willing to bring down my entire network so I can upgrade the IOS? What other features do I need/want (Netflow, NAT, GRE)? Am I running routed ports vs SVI's (and what are the shortcomings- like MTU)? What does my spanning tree look like (and do I know how to control it)? What features will I be using that will require running in software vs hardware?

I guess I'm taking a look at it like "should I be doing this" vs "can I make this work with help from others"?

Hope you're doing well too Jon.

[Jon Marshall]

Agreed on the functionality which I did cover, although not as extensively as you, in my original response.

Not sure about upgrading IOS takes down whole site because if the core/distro is down having a router gives you nothing ie. you can't route into and out of the site anyway.

Assuming there is no need for the extra functionality my main concern would come down to the impact on CPU by running BGP on the switch and also the number of remote MPLS routes which I overlooked but you covered.

If the number of routes was large and placed an extra strain on the 3750 then yes a router would be needed although obviously you wouldn't then redistribute those routes into an IGP.

But if the 3750 could handle the additional load although it may not feel right to me i would still struggle to say why a router was needed other than to argue for future scalability and functionality but then if you need that you could always buy one then ie. it's not exactly going to require a redesign of the network.

I know best practices are there for a reason and i have used them many times but there are certain scenarios to my mind where following them to the letter means cost with no gain and i think then it becomes really hard to justify, at least for me.

Which is why i asked for your input really.

Much appreciated.

Jon