Cisco RV340 | CPU 100%

[knollknows]

I am experiencing severe network degradation on client machines and I am having extreme delays to access the admin console of the router.  From a client side perspective, some sites are appearing to load slowly and I am noticing some lag. My environment consists of ~~30 devices from Xbox, PCs and IoT devices.  I reviewed the Cisco "summary" page an I see 100% CPU utilization.  I configured  syslog to a remote server and enabled "debugging". I can share my findings as required. 

I did see some other posts of other users experiencing similar issues and I also can confirm that a reboot will resolve this issue for roughly ~~8 hrs.  I have done a router reset and reconfigured all options to rule out a stuck config from a router upgrade. I did try out the suggested "peer solutions" to lower the IPS/IDS options to "balanced" however this had no impact. 

Model: Cisco RV 340

Firmware:

Serial Number: OMITTED System Up Time: 0 Days 22 Hours 27 Minutes 11 Seconds Current Time: 2019-Dec-13, 07:49:18 PST CPU/Memory Usage: 100% / 37% PID VID: RV340-K9 V01

Firmware Information   Firmware Version: 1.0.03.16

[Sujoy Paria]

Hi,

In case IPS, Web Filtering and Antivirus options are enabled on the Router then please disable those options once and check the performance; you can start with Antivirus option.

What is the log details showing when you are facing the high CPU utilization issue?

[knollknows]

I will attempt to disable the IPS, Web Filtering and Antivirus. Should I also include the Application Visibility Control (AVC) ?

On a second thought, is there an option to manage the router via a command line (CLI) to conduct these tasks since when the 100% CPU utilization issue surfaces, the router admin console (GUI) is unstable and almost unusable.

In regards to the log details, unfortunately the logging level I had of 'error' didn't indicate or clearly define a problem. I have changed the log levels to debug for now. I will share once the issue resurfaces.  Lastly, the router has a "Export Technical Support Report". Would that help as well ?

[Sujoy Paria]

Hi,

Thank you for your reply. You may first try with the three options I have mentioned earlier.

1 RJ-45 port is for future use on this router model. The port is disable, so the CLI option to perform the mentioned task will not be feasible now.

Yes, you can capture the Technical Support Report; we may get some additional information from that.

[knollknows]

Hi Sujoy,

This issue surfaced again. I attempted to disable the AV option and/or the IPS/IDS options however the UI for the router logon console was just unstable so I had to reboot the router to restore network operational health. I have the log files as I set the logging level to debug. How would I go about sharing this information with you?

 

[LICENCJEGLOSEL33267]

Hello

I have same exact problem. The only difference is that we have around 100 PCs. At first it started to reach 100% CPU usage, later it reached up to 60-70% but the memory usage went up to even 82% and the interface was not responding. Can I do something about it? The firmware version is 1.0.03.16. We got another one of these routers and instead of copying the settings we rewrote them manually and now the router does work properly, at least for one week.

[knollknows]

This issue surfaced again and continues to occur. I took the time to do further troubleshooting. Below are my findings.

Network speed symptoms 'felt' slow. Pages took unusually longer to load. Mobile apps such as Facebook , Youtube and Netflix where the obvious ones to demonstrate an issue. The 'unusually longer' than normal examples are short 1 minutes videos with typically low res. vid/audio would not load. Instagram pictures and auto play video also didn't load.

I checked my syslog server and saw some SEV0 and SEV 1 items.
The screenshots are attached to this ticket. The SEV 0 items indicated a msm=FATAL 'Error accepting connection'.
I also looked at SEV 1 items listed in the logs and found multiple "Webfilter" line items that stated "WebFilter application damonizing" and "INFO:Updated LocalDB memory" / "INFO:Updated Freeing old LocalDB memory"

Given these items, I though perhaps the "webfilter" is the cause so I proceeded to disable that first. This has no impact as the router remains unstable and CPU utilization remained at 100% and ~~38 to 46% memory utilization.

I logged onto the Router management console.
After ~10 minutes the UI section of System summary loaded and indicated
CPU 100%/46% memory usage.
These are the actions conducted.
5:42 PM PST Disabled WebFiltering
NO Change to performance
6:01 PM disabled IPS - result - no change to CPU/Memory utilization
6:04 PM pst disabled AVC - no change to CPU/Memory utilization
Reboot was only solution.

[knollknows]

Example syslog data

[Sujoy Paria]

Hi,

Sorry for the inconvenience you are facing. Please follow the below mentioned link and raise a service request with us to further troubleshoot and proceed on that…

https://www.cisco.com/c/en/us/support/web/tsd-cisco-small-business-support-center-contacts.html

Thank you for your understanding.

[Powerwav]

Yea, me too. Basically what I found out is this router is a Turd....;

[normh]

This does sound similar to the problem I just posted in this community.  Perhaps the 1.0.03.16 firmware has a memory leak resulting in the router degrading to an unusable state over time?  It seems like I started experiencing the problem after I purchased a Security Smart License subscription and enabled its features.

[normh]

I can add one other data point...  I have two other RV-340 routers that are running 1.0.03.16 and are working fine, no slowness or high CPU.  The only real difference is that their Registration Status is 'Unregistered' and their License Authorization Status is 'Evaluation Expired'.  As I mentioned before, my home RV-340 developed this problem when I registered the device and added the 1 year 'Security-License' subscription. 

Can others experiencing this problem weigh in on whether they have an active Security-License or not?

[michaeldowns1000]

Yes, i have reported the same bugs in the RV340 firmware with the security licensed features fully enabled.

Throughput bandwidth would be significantly degraded and the admin UI was unusable until a hardware reboot.

Cisco support was unable to solve the issue or offer any suggestions other than to experiment with various settings..

In my case i had to experiment to find a workaround which involves disabling some of the security functions of the router/license.

[normh]

I have a support ticket open on this issue now.  Since opening the ticket, the support engineer had me disable the Security license and then reboot the router.  That was 18 hours ago and the router is still working normally.  One other observation I have made came from some cabling changes I made yesterday.  During that work, the router was disconnected from the Internet due to the cable modem being powered off and on.  After that, and with no reboot of the router itself, and with the Security license enabled, the problem cleared for a few hours.  This indicates that when the router loses Internet connectivity, the 100% CPU condition clears.

[michaeldowns1000]

That may be useful information to the developers. Hopefully Cisco support will eventually provide some fix for this bug.