3945 Throughput Question

[55cfffb534]

Hello,

Right now we're using a metro ethernet connection with a bandwidth of 200Mbps between our office and datacenter.  Our ISP is offering us a deal to upgrade to a 1Gbps connection for basically no extra cost.  What I'm trying to figure out is if this router can even come close to moving that much traffic?  We also have an older ASA in between as well.  The topology is as follows:

LAN > CORE > ASA > 3945(gen1.0) > METRO ETHERNET > 3945(gen1.0) > CORE > LAN

I'm assuming not only are the 3945's going to be the bottle neck but also the 5520 ASA.  I can't find a concrete answer on the throughput capabilities of the 3945.  The ASA seems to have a max advertised firewall throughput of 450Mbps.  The 3945 routers have 1Gb single-mode fiber SFP modules so I would imagine they would be compatible with whatever device is on the other end.

Thanks for any advice.

Nick

[Leo Laohoo]

What I'm trying to figure out is if this router can even come close to moving that much traffic?

Not even close.   Ok, here's the low-down story ... The ISP is offering you 1 Gbps link, the question is this:  Is this a 1 Gbps upload AND download WAN link or a total of 1 Gbps upload and download. 

Next, you're looking for router which can push 1 Gbps on TWO ports (one port for your WAN and one port for your LAN), as a minimum.  

The "smallest" router that can do this would be an ASR 1002-X.  Cisco has been marketing a feature called "pay as you grow".  This basically means that if you need a bigger WAN bandwidth, you just pay for an electronic license and the appliance can process at a higher level.    If you plan to go with the ASR 1002-X then start with the basic 5 Gbps and monitor your LAN & WAN utilitization.  If you need to go higher then contact Cisco and cough up the dough and they'll give you instructions to unlock the higher bandwidth support.

[55cfffb534]

Thanks for the reply, Leo.

The way I understand it is the ISP is offering a full duplex 1Gb connection.  To complicate things further, the package actually includes redundant 1Gb connections so in essence, I would need two interfaces to handle WAN traffic plus some type of routing configuration to support them (I haven't even gotten to thinking about how we'd route traffic) and then a 3rd interface would connect to the LAN.

So if it has been established that the 3945 isn't even close to moving traffic near 1Gbps, does that mean it will drop packets if we try and push a bunch of traffic that way because of CPU utilization?  I guess I'm not exactly sure what to expect if we hook up a big pipe to with what we have?

We seem to do just fine with the 200Mbps (full duplex) connection right now.  The CPU never goes above 25% as far as I can tell on the router.  Our ASA runs about 50% all day long (I'd say we average around 100Mbps but we can max it out if we are too greedy with some of our replication traffic).  I guess the thing is, we can potentially save money if we make the switch but if there is no way for our hardware to handle these speeds safely, it doesn't make as much sense given we'd need to purchase a couple of routers and a pair of firewalls + support for it all.

Is it possible to throttle an interface down from the line-rate?

[Leo Laohoo]

To complicate things further, the package actually includes redundant 1Gb connections so in essence, I would need two interfaces to handle WAN traffic plus some type of routing configuration to support them (I haven't even gotten to thinking about how we'd route traffic) and then a 3rd interface would connect to the LAN.

Please forgive me for being a d1ck.  

It's not called "redundant" in my book because: 

1.  The second link goes to the same router.  This is your single-point-of-failure.  If your router fails, then it's Good-bye, Pittsburg!

2.  The second link comes from the same exchange.  If this exchange fails, then it's Good-bye, Seattle!

[55cfffb534]

I guess I'm not trying to split hairs on the topology as much as I'm trying to understand the implications of having two 1Gbps connections on a 3945 router which doesn't have the horsepower to utilize the available bandwidth.  I suppose it would be 3 total 1Gb interfaces if you count the one that goes to the firewall.

If we start copying a big file over the connection will it peg the CPU's on everything and start dropping packets?

Nick

[Leo Laohoo]

If we start copying a big file over the connection will it peg the CPU's on everything and start dropping packets?

I can't answer this question because I don't know the state of the CPU of the router.  

The thing to look out for is the CPU.  If the CPU starts to hit, say, 90% then it's a sure sign the router has reached the limit.

[55cfffb534]

We'll have to have more discussion with the ISP.  I can see having switches on either end but the router and to a lesser degree the ASA throw things off.

Thanks,

Nick

[Leo Laohoo]

So if it has been established that the 3945 isn't even close to moving traffic near 1Gbps, does that mean it will drop packets if we try and push a bunch of traffic that way because of CPU utilization?  I guess I'm not exactly sure what to expect if we hook up a big pipe to with what we have?

You're one lucky person.  No one gets a FREE upgrade to 1 Gbps from a provider until now.  Ok, so the router won't be able to handle 1 Gbps WAN link ... that's only for NOW.  

If I were in your shoes now, I would simply "$uck it up" and let it run with this current setup.  When your managers start questioning the benefit, then tell them that the router can't handle this much traffic and maybe (just maybe) they might cough up some money.   

But the main thing is:  You've got a 1 Gbps link and it's ready-to-go.  

Is it possible to throttle an interface down from the line-rate?

You can do some QoS stuff and you might be able to squeeze some more into it but I'm no expert in this 'black magic" called QoS.  Joseph is the guy and he normally trolls the CSC forums when he's not busy.

[55cfffb534]

Sounds good.  I guess I'll have to see what we can do.  I think the ISP is motivated to get us off of their old copper stuff and onto their fiber network more than anything else.

Thanks for the info. 

Nick

[Joseph W. Doherty]

Disclaimer

The Author of this posting offers the information contained within this posting without consideration and with the reader's understanding that there's no implied or expressed suitability or fitness for any purpose. Information provided is for informational purposes only and should not be construed as rendering professional advice of any kind. Usage of this posting's information is solely at reader's own risk.

Liability Disclaimer

In no event shall Author be liable for any damages whatsoever (including, without limitation, damages for loss of use, data or profit) arising out of the use or inability to use the posting's information even if Author has been advised of the possibility of such damage.

Posting

Here I am, trolling - laugh.

In answer to your question about rate limiting traffic, you could, but if done on the 3945 itself, that too contributes to its CPU consumption.  Doing ingress rate limiting might use less CPU than forwarding the packets; it might not too.  Ideally, if you were going to use such an approach, you would want to rate-limit (or better, shape) the traffic upstream of the 3945.  (BTW, rate limiting can be very harsh to network applications.)

On your original question about the capacity of your 3945 ...

The non-E variant of the 3945, Cisco documents as being able to forward up to 8 Gbps.  However, that's all 1500 byte packets and with a configuration that does nothing beyond forwarding.  For comparison, Cisco also documents the same router's PPS as 982 Kpps for minimum size packets, again with a configuration that only forwards.  (NB: 1 Gbps, minimum size Ethernet packets, requires 1.488 Mpps.  Keep in mind, duplex would require twice this capacity.)  From the forgoing, a 3945 isn't really a good choice for a gig link, although if your actual usage is lower, it might be used.  Leo recently posted https://supportforums.cisco.com/sites/default/files/legacy/2/7/8/139872-white_paper_c11_595485.pdf which will explain various ISR capacities, in different usage roles.

Leo's suggestion of the ASR 1002-X is great, but perhaps the "smallest" router that might deal with a gig link might be the 4451-X with its performance upgrade.  Other smaller ASRs than the 1002-X, such as the 1001, 1002 and 1001-X should all easily handle a gig link.  (We use original 1001s for MetroE gig links, and haven't seen one yet have any difficulty supporting a single gig path.)

PS:

Concerning using an ASA, I'm curious why?  Does your MetroE also provide your Internet access?  If not, generally, MetroE might be considered "private", but if you want to add a bit more security, you could do that with ACLs on the routers.  You could even create an IPSec tunnel between them.

[55cfffb534]

Hi Joseph,

Thanks for the detailed information!  Our routers are the non-E models (C3945-SPE150/K9).  The only thing they do is forward traffic from our office to the datacenter.  There are three access list entries to permit the traffic and that's it - the config is tiny.

Makes sense doing any work on the traffic upstream of the router; I was thinking about that yesterday too.  I think there are products that would handle that although I'm not sure we'd want to introduce anything like that at this time.

We haven't gotten the proposal from the ISP yet but it sounds like we'd have two 1Gbps fibers terminating in our building and at the datacenter.  For now, both would end up being plugged into a single router at the datacenter side and office side.  Yes I know the router is a SPOF but is what it is.  The third interface would also be a 1Gb interface as well and would plug into the core switches on their respective sides.

OFFICE LAN > CORE > ASA > 3945(gen1.0) > 1Gb FIBER x2 > 3945(gen1.0) > CORE > DATACENTER LAN

The ASA (transparent mode) is there to filter unwanted traffic to the datacenter from the office.  It's one of those design details that has always been part of the network at this company for better or worse.  We don't require the traffic to be encrypted at this time but it is one of those things that could become a requirement.  So the ASA simply has some ACL's and requisite ACE's to filter based on source, destination and tcp/udp port - that's it.  No VPN's or anything like that.

The link between the office and the datacenter is considered private and does not provide internet traffic.

While I'd love to get new routers on either side, that isn't an option for us as we still have some time left on the maintenance agreement.  I believe it is something we'll be able to look at more closely next year.

[Joseph W. Doherty]

Disclaimer

The Author of this posting offers the information contained within this posting without consideration and with the reader's understanding that there's no implied or expressed suitability or fitness for any purpose. Information provided is for informational purposes only and should not be construed as rendering professional advice of any kind. Usage of this posting's information is solely at reader's own risk.

Liability Disclaimer

In no event shall Author be liable for any damages whatsoever (including, without limitation, damages for loss of use, data or profit) arising out of the use or inability to use the posting's information even if Author has been advised of the possibility of such damage.

Posting

Two additional thoughts came to mind.

First, unsure about the 3945, but if you need to add any modular gig ports, I've seen other ISRs struggle with high speed module ports.  (I.e. built-in ports seem to work better.)  Otherwise, just monitor your overall CPU utilization.  As long as you generally average below 75%, you're not overrunning the router's capacity.

Second, back to the question of ingress rate limiting, if it's applied on ingress to rate adaptive traffic, and the traffic restraints itself, then it could be a safeguard to overrunning the CPU capacity of the router.  With an inbound policy, you can also be selective to what traffic it's applied to.

[55cfffb534]

Your second point is intriguing.  Perhaps ingress rate limiting would help us out.  What's interesting is when I started a large file copy across the current link, the ASA CPU ran at about 77% while the router CPU didn't hardly move - granted we're only using half of the available bandwidth in a test where we're copying a file from A to B.  I would attribute low CPU number to the router simply forwarding the traffic and not doing much else.  Just monitoring normal CPU for production traffic/load during the day, the ASA runs around 50% and the router sits between 7 and 10% most of the time.  Historically, the router is just off-idle most of the time it seems.  Perhaps 20% at most.  Unfortunately, there is really no way to know how it's going to perform until we try it.  I'm interested to hear what the ISP says other people are running for this type of connection.  

Thanks again for the info.  I'm starting to think this is do-able but the ASA might be the bigger issue in terms of performance (or lack thereof).  I just noticed it's actually a 5510 (not a 20) so has even less ability.

[Leo Laohoo]

Post removed.

[Leo Laohoo]

Other smaller ASRs than the 1002-X, such as the 1001, 1002 and 1001-X should all easily handle a gig link.

Post removed.