12-13-2019 07:59 AM
I am experiencing severe network degradation on client machines and I am having extreme delays to access the admin console of the router. From a client side perspective, some sites are appearing to load slowly and I am noticing some lag. My environment consists of ~~30 devices from Xbox, PCs and IoT devices. I reviewed the Cisco "summary" page an I see 100% CPU utilization. I configured syslog to a remote server and enabled "debugging". I can share my findings as required.
I did see some other posts of other users experiencing similar issues and I also can confirm that a reboot will resolve this issue for roughly ~~8 hrs. I have done a router reset and reconfigured all options to rule out a stuck config from a router upgrade. I did try out the suggested "peer solutions" to lower the IPS/IDS options to "balanced" however this had no impact.
Model: Cisco RV 340
Firmware:
|
|
01-26-2020 10:32 AM
Latest update on the TAC Case... The support engineer has asked me to let the router run over the weekend with the Security license disabled but still on the latest firmware, 1.0.03.16. So far, I have not seen the problem. He said next actions Monday will probably be to downgrade the firmware to the previous version, 1.0.03.15, and re-enable the Security license to see if the problem was introduced in the latest firmware (regression testing anyone?). I will update again after testing this or whatever he comes up with for next steps.
01-28-2020 05:16 PM
After no callback and no updates to the case from Caleb, the Cisco support engineer either yesterday or today, I decided to go ahead and downgrade my RV340 to firmware version 1.0.3.15 and re-enable the Security license. So far it has behaved normally with no 100% CPU spikes. I will update this post if this changes.
02-06-2020 07:32 AM
I have gone back to my Netgate SG3100 pfSense firewall for now. Waiting for Cisco to come up with a fix. I am tired of dealing with the quirks of this box. Hopefully the next version of firmware will allow the use of the Security license I paid for.
01-29-2020 07:58 AM - edited 01-29-2020 08:33 AM
The latest now is that the router spontaneously rebooted again at 4:00am today and reloaded the 1.0.3.16 firmware! This is despite having auto upgrade turned off for firmware. I assume the reason why it did this is because the 1.0.3.16 firmware is loaded in the router (and was the active image even though it was booted with 1.0.3.15??). If there were a way to delete it, I would, but I can't find a way to do it. Also, I spoke with Caleb just now and he says they have multiple cases open now about this problem. He thinks it may be due to some changes made in the licensing functionality in 1.0.3.16.
01-29-2020 01:39 PM
Very disconcerting!
Since paying for the security license, and bugs in those functions causing so many problems, i was considering reverting to 1.0.3.15 firmware as a work-around.
That does not seem like a viable option.
normh, thanks for updating this thread!
02-01-2020 12:26 PM
02-01-2020 12:52 PM - edited 02-01-2020 12:54 PM
I assume you meant to say 1.0.3.15 exhibits the problem after 3-5 days, not 10.0.3.16. My router is at 3 days, 4 hours run-time since last boot and is running 1.0.3.15. No problems yet. I'll post back whether my unit starts exhibiting the problem in the next 2 days.
02-02-2020 01:58 AM - edited 02-02-2020 02:01 AM
You're correct. The version 1.0.03.15 seems to be more stable. I wrote it incorrectly. Last period without the need of restart was almost 4 days, but the load grew-up faster and more than usual so I was unable to log-in to the admin interface and reboot it and I had to manually switch-off the router. I've attached statistics for last 14 days. Periods highlighted means response that exceeds 2 ms. I used version 1.0.03.16 until 01/29. Router uptime without problems was stable usually for 8 - 48 hours. response time of the router decreased, but I was always able to log-in and reboot the device. After downgrade, the router was stable for almost 4 days, but after that the response time grew-up approx. 100x and made the admin interface inaccessible.
02-16-2020 08:13 AM
Some news: due internet provider infrastructure problems firewall was unable to re-verify license validity and last 6 days ran without advanced features (web filtering, IPS, AV, etc.) ... without any problem. Average CPU load about 3%, response about 0,8 ms. It leads to the conclusion, that a problem is really in some advanced feature (memory leak or something else). I'm gonna try to run more times with a registered/unregistered license to verify the behavior.
01-30-2020 09:04 AM
Hello everyone,
I see that this thread has gained momentum in having multiple users reporting the same if not similar issues. The last update from a Cisco 'agent' was in having me call support and make a formal troubleshooting support ticket. I have NOT done so.
Like previously mentioned, I am simply rebooting the server on a daily basis for now. Time permitting I will call support and have this 'documented'.
I can see that many of you are trying or considering downgrading the Firmware as a workaround. I can see the logic behind this but I would decline that option myself as the bug fixes enabled by the current version resolve security/performance issues. Again I am assuming that this version of firmware is the 'cause' for the CPU utilization to hit 100%
Hopefully a solution comes forward soon. I'll update this thread if further updates become available for the curious minds.
02-03-2020 07:25 PM
I am now at 5 days, 11 hours since last reboot and I have not seen any problems with the Security license and its features enabled. It looks like, in my case, downgrading the 1.0.3.15 has mitigated the 100% CPU problem. Don't get me wrong, I do not think this is a satisfactory solution, but it enables me to continue to use the router with the license until Cisco releases a fix.
02-04-2020 02:54 AM
02-06-2020 06:49 AM
Move the routing/filtering capability to lower segments (e.g. core switch) is not a system solution. I've bought the router and additional Security license primarily from the reason of single point of rules definition/VLAN management/VPN server/statistics etc., but as I can see, if I would like to have a stable device, I must move advanced (paid) features to other network segments. Nope. Another 1 day, 20 hours without problem, but after that 100% CPU load occurred again and a reboot was required. The only advice from Cisco's side is to "open a new case", but apparently I have no such permission now :( ... I wish it only SSH access working. In this case, I could enforce a device reboot based on slow response events. But this is another Cisco's "joke" - to have a console port on the device, serial-2-LAN cable included in package, BUT "CLI access disabled for feature use". What are they waiting for? Christmas?
02-06-2020 07:37 AM
I have gone back to my Netgate SG3100 pfSense firewall for now. Waiting for Cisco to come up with a fix. I am tired of dealing with the quirks of this box. Hopefully the next version of firmware will allow the use of the Security license I bought.
02-22-2020 11:08 AM
Guess What! Cisco closed my TAC Case with NO SOLUTION! Thanks Cisco:
This baby goes on eBay. I am totally disgusted with Cisco. Wasted $100 for this Security license that breaks the router and TAC closes the case when the problem has not been fixed.
Here is the email reply to my request for an update:
Hello <my CCO ID Email>,
Your email attempt to update a Service Request (SR) number or Third Party Number (TPN) had one or more failures. Details provided below.
Note update successful: NONE
Note update (and any file attachments) failed - SR Closed: 688307775
Note update failed - SR Not Found: NONE
The following file(s) could not be uploaded successfully. Failure details provided below.
NONE
You can also reach the Cisco Technical Assistance Center via the additional methods found on our Technical Support Worldwide Contacts page at:
http://www.cisco.com/warp/public/687/Directory/DirTAC.shtml
For other questions, please visit our Technical Support Website:
http://www.cisco.com/cisco/web/support/index.html
Sincerely,
Technical Assistance Center
Cisco Systems, Inc.
My email:
Hello Caleb,
I haven’t heard anything further about this. Can you give me an update on a fix for this problem?
It doesn’t appear that Cisco has much interest in fixing even serious problems with SOHO products.
Thanks
Discover and save your favorite ideas. Come back to expert answers, step-by-step guides, recent topics, and more.
New here? Get started with these tips. How to use Community New member guide