issues with sudden burst in bandwidth

[Sharin Taib]

Hi, I have a very unique problems that I am unable to solve as I have never seen the symptoms before + there are no errors on the affected equipment.

The layout of the equipments are as such

other equipment <--> 2911 Router <--> Ether-Switch/3925 Router <--> 7206 <--> Internet

During certain times at night, the 2911 exhibits cpu load, high packet loss and an increase in bandwidth of at least twice it's normal amount. this results in packet loss in all the other equipment. I am attempting to locate who was demanding such a high burst but so far the graphs display normal rates for all the other equipment.

Because the 2911 demands an increase from the 7206, the 7206 is able to accomodate this as it still has balance BUT my other equipment that are connected to the 7206 takes a drop in bandwidth as well. the graph shows that when the 7206 bursts higher, the rest of my other links take a slight drop in bandwidth.

Also, the graph from Ether-switch to 2911 indicates the bandwidth hike BUT the graph from 2911 to ether-switch does not display the same thing as due to the cpu load, the data is somehow not captured properly.

The 2911 has firmware c2900-universalk9-mz.SPA.150-1.M1.bin, no logs to indicate any errors, CPU processes appear normal. is there anything I can do to resolve this issue?

[lawsonad]

Upstream internet bandwidth speed?  Keep in mind the 2911 is rated for a max of 180Mbps in best case IP only traffic.  If you have IPSec, GRE (i.e. stuff running outside CEF) on the 2911 that number decreases dramatically.

http://www.cisco.com/web/partners/downloads/765/tools/quickreference/routerperformance.pdf

Copy and paste an output from "show proc cpu" during high CPU load.  What is the main process consuming cpu cycles?

My guess is the 2911 is the weak link in the chain, both the 3925 and 7206 are significantly more powerful.  What does the 2911 do that the 3925 could not? Try and remove any unnecessary equipment from the scenario.

Another suggestion is enabling NetFlow on one of the routers to determine what exactly is the spike in traffic, and if it's legitimate traffic.

[Sharin Taib]

100mb for upstream. No IPSec, GRE.

The issue is my NOC didn't capture the cpu process so will ask them to check on this the next time it happens.

The 2911 is a legacy equipment which we haven't got around to changing because too many customers on it.

Netflow doesn't indicate anything either. We can't hook it up to 3925 either unless we purchase a new one.

[sumeydrane412]

I can understand your frustration with the unique problem you're facing. It's puzzling when there are no visible symptoms or errors on the affected equipment. From the equipment layout you've provided, it seems that the 2911 router is experiencing high CPU load, packet loss, and increased bandwidth during certain times at night. While you're trying to identify the cause, the graphs for other equipment show normal rates. It's interesting to note that the 7206 router can accommodate the burst from the 2911, but other connected equipment experiences a drop in bandwidth. Additionally, the graph from the Hiking Bra Ether-switch to 2911 reflects the bandwidth hike, but not the other way around due to CPU load. Since the 2911 appears normal in terms of firmware, logs, and CPU processes, it might be helpful to consult with a network specialist or your equipment's manufacturer for further assistance in resolving this issue.