RV345 just hangs - Calling Cisco service is another challenge

[DVDW38191]

For my (small) business, I have about 15 PC's connected to an R|V345. 

Once, twice per day, the router completely stops working. The network is down. Pinging is impossible. Accessing the admin web pages is impossible. 

Seems to relate to one (new) Dell laptop. I can't imagine how a PC can break the network and the router functionality. Checkbox DoS has been enabled !? 

Latest firmware !

Decoupling the PC does not help. Only a full reboot, wasting a lot of time.

I was hoping for a brand like Cisco to deliver qualitative solutions and/or qualitative service...

Tried 4 times : 

1/ I got one sentence as an answer : "You have to call a number in the US : 1-800- ... Bye".  Hung up. 

2/ I explained in a few words the problem I was facing. Cisco just hung up. 

3/ I tried again and choose for calling me back, because heavy load on their service dept. 4 days later... still no call back !?

4/ Dialed in again and got some 'tier 1' questions... is the router powered ? Not helping me at all. 

Anyone facing the same problem ? I give it another week, and then I will send the router back and go for another brand.

Unfortunately

Dirk 

[marce1000]

 - I can also respond to the technical-side of the issue : check the logs when this happens. Better still configure a syslog-server as log collector, that way the device can be monitored in an overall way  , and keeping an eye on the logging may lead to better understanding as to what is going on.

 M.

[DVDW38191]

Hi Marc, 

Thanks for the support !! 

Unfortunately, the logs are empty when I reboot the router, which I have to, because otherwise inaccessible ! 

So, following your advice, I have now Kiwi Syslog Server running on one of the PC's. Waiting till it happens again...

Hope -for a change- this happens now soon.  

Thanks again. 

Regards

Dirk 

[DVDW38191]

Hm... nothing interesting to see : 

11-28-2020 17:42:08 Kernel.Info 192.168.1.199 1 2020-11-27T21:00:57+01:00 Router kernel - - [meta sequenceId="8"] [ 101.978418] pfe_hif_client_register
11-28-2020 17:42:08 Kernel.Info 192.168.1.199 1 2020-11-27T21:00:57+01:00 Router kernel - - [meta sequenceId="7"] [ 101.978406] hif_process_client_req: register client_id 1
11-28-2020 17:42:08 Kernel.Info 192.168.1.199 1 2020-11-27T21:00:57+01:00 Router kernel - - [meta sequenceId="6"] [ 101.977843] eth2: pfe_eth_open
11-28-2020 17:42:08 Kernel.Info 192.168.1.199 1 2020-11-27T21:00:55+01:00 Router kernel - - [meta sequenceId="5"] [ 99.940145] eth3: pfe_gemac_init
11-28-2020 17:42:08 Kernel.Info 192.168.1.199 1 2020-11-27T21:00:55+01:00 Router kernel - - [meta sequenceId="4"] [ 99.940127] hif_lib_set_rx_cpu_affinity:Client (2) and HIF Rx processing will use same cpu
11-28-2020 17:42:08 Kernel.Info 192.168.1.199 1 2020-11-27T21:00:55+01:00 Router kernel - - [meta sequenceId="3"] [ 99.940112] pfe_hif_client_register
11-28-2020 17:42:08 Kernel.Info 192.168.1.199 1 2020-11-27T21:00:55+01:00 Router kernel - - [meta sequenceId="2"] [ 99.940090] hif_process_client_req: register client_id 2
11-28-2020 17:42:08 Kernel.Info 192.168.1.199 1 2020-11-27T21:00:55+01:00 Router kernel - - [meta sequenceId="1"] [ 99.939471] eth3: pfe_eth_open

<<NETWORK DOWN / IMMEDATE FULL REPOWER OF THE ROUTER>>

11-28-2020 17:36:41 Kernel.Info 192.168.1.199 1 2020-11-28T17:36:41+01:00 router01DFA9 kernel - - [meta sequenceId="768"] [74340.160237] FIREWALL: DROP PACKET is not associated with an existing connectionsIN=eth3.1 OUT=eth0 DST_MAC=bc:4a:56:01:df:a9 SRC_MAC=:f6:17:8e:ff:19:03 src=192.168.1.196 DST=179.60.195.52 LEN=76 TOS=0x00 PREC=0x00 TTL=63 ID=60187 PROTO=TCP SPT=60823 DPT=443 WINDOW=4096 RES=0x00 ACK PSH URGP=0 MARK=0x200 
11-28-2020 17:36:41 Kernel.Info 192.168.1.199 1 2020-11-28T17:36:41+01:00 router01DFA9 kernel - - [meta sequenceId="767"] [74340.146157] FIREWALL: DROP PACKET is not associated with an existing connectionsIN=eth3.1 OUT=eth0 DST_MAC=bc:4a:56:01:df:a9 SRC_MAC=:f6:17:8e:ff:19:03 src=192.168.1.196 DST=179.60.195.52 LEN=76 TOS=0x00 PREC=0x00 TTL=63 ID=55312 PROTO=TCP SPT=60825 DPT=443 WINDOW=4096 RES=0x00 ACK PSH URGP=0 MARK=0x200 
11-28-2020 17:36:40 Kernel.Info 192.168.1.199 1 2020-11-28T17:36:40+01:00 router01DFA9 kernel - - [meta sequenceId="766"] [74339.928562] FIREWALL: DROP PACKET is not associated with an existing connectionsIN=eth3.1 OUT=eth0 DST_MAC=bc:4a:56:01:df:a9 SRC_MAC=:f6:17:8e:ff:19:03 src=192.168.1.196 DST=179.60.195.52 LEN=76 TOS=0x00 PREC=0x00 TTL=63 ID=0 DF PROTO=TCP SPT=60827 DPT=443 WINDOW=4096 RES=0x00 ACK PSH URGP=0 MARK=0x200 
11-28-2020 17:35:50 Daemon.Info 192.168.1.199 1 2020-11-28T17:35:50+01:00 router01DFA9 dnsmasq-dhcp 20096 - [meta sequenceId="765"] If IP Source Guard Enable, Add the Record with IP [192.168.1.196] and MAC [f6:17:8e:ff:19:3]
11-28-2020 17:35:50 Daemon.Info 192.168.1.199 1 2020-11-28T17:35:50+01:00 router01DFA9 dnsmasq-dhcp 20096 - [meta sequenceId="764"] DHCPACK(eth3.1) 192.168.1.196 f6:17:8e:ff:19:03 iPhonevanIanka
11-28-2020 17:35:50 Daemon.Info 192.168.1.199 1 2020-11-28T17:35:50+01:00 router01DFA9 dnsmasq-dhcp 20096 - [meta sequenceId="763"] DHCPREQUEST(eth3.1) 192.168.1.196 f6:17:8e:ff:19:03 

At 17:37 the network went down. Repowered the router immediately after it happened. Few minutes later operational again. 

Until it went down again 20 minutes later... but again no errors or warnings. 

Only errors after a day of logging : 

11-28-2020 17:42:20 User.Alert 192.168.1.199 1 2020-11-28T17:42:20+01:00 Router Webfilter - - [meta sequenceId="202"] Webfilter application damonized successfully !!!!
11-28-2020 17:42:20 User.Alert 192.168.1.199 1 2020-11-28T17:42:20+01:00 Router Webfilter - - [meta sequenceId="201"] Webfilter application damonizing !!!
11-28-2020 17:42:20 Kernel.Info 192.168.1.199 1 2020-11-28T17:42:20+01:00 Router kernel - - [meta sequenceId="200"] [ 170.736350] ct: ifindex changed 15, 14
11-28-2020 17:42:20 User.Info 192.168.1.199 1 2020-11-28T17:42:20+01:00 Router Webfilter - - [meta sequenceId="199"] Creating 4 MB cache capacity: 98304 entries
11-28-2020 17:42:20 User.Error 192.168.1.199 1 2020-11-28T17:42:20+01:00 Router Webfilter 13649 - [meta sequenceId="198"] Calling poller creation .

Dirk 

[DVDW38191]

And again...

What brand is a good alternative ?! Do not buy this POS...

11-28-2020 18:50:18 Daemon.Info 192.168.1.199 1 2020-11-28T18:50:18+01:00 router01DFA9 dnsmasq-dhcp 30583 - [meta sequenceId="268"] DHCPDISCOVER(eth3.3) d8:f1:5b:da:f4:7a 
11-28-2020 18:50:14 Daemon.Info 192.168.1.199 1 2020-11-28T18:50:14+01:00 router01DFA9 dnsmasq-dhcp 30583 - [meta sequenceId="267"] If IP Source Guard Enable, Add the Record with IP [192.168.3.171] and MAC [d8:f1:5b:e8:5:7b]
11-28-2020 18:50:14 Daemon.Info 192.168.1.199 1 2020-11-28T18:50:14+01:00 router01DFA9 dnsmasq-dhcp 30583 - [meta sequenceId="266"] DHCPACK(eth3.3) 192.168.3.171 d8:f1:5b:e8:05:7b ESP_E8057B
11-28-2020 18:50:14 Daemon.Info 192.168.1.199 1 2020-11-28T18:50:14+01:00 router01DFA9 dnsmasq-dhcp 30583 - [meta sequenceId="265"] DHCPREQUEST(eth3.3) 192.168.3.171 d8:f1:5b:e8:05:7b 
11-28-2020 18:50:14 Daemon.Info 192.168.1.199 1 2020-11-28T18:50:14+01:00 router01DFA9 dnsmasq-dhcp 30583 - [meta sequenceId="264"] DHCPOFFER(eth3.3) 192.168.3.171 d8:f1:5b:e8:05:7b 
11-28-2020 18:50:14 Daemon.Info 192.168.1.199 1 2020-11-28T18:50:14+01:00 router01DFA9 dnsmasq-dhcp 30583 - [meta sequenceId="263"] DHCPDISCOVER(eth3.3) d8:f1:5b:e8:05:7b 

<<HANG --> REPOWER>>

11-28-2020 18:14:45 Daemon.Info 192.168.1.199 1 2020-11-28T18:14:45+01:00 router01DFA9 dnsmasq-dhcp 30583 - [meta sequenceId="256"] read /etc/ethers - 0 addresses
11-28-2020 18:14:45 Daemon.Info 192.168.1.199 1 2020-11-28T18:14:45+01:00 router01DFA9 dnsmasq 30583 - [meta sequenceId="255"] read /etc/hosts - 2 addresses
11-28-2020 18:14:45 Daemon.Info 192.168.1.199 1 2020-11-28T18:14:45+01:00 router01DFA9 dnsmasq 30583 - [meta sequenceId="254"] using nameserver 195.130.131.2#53
11-28-2020 18:14:45 Daemon.Info 192.168.1.199 1 2020-11-28T18:14:45+01:00 router01DFA9 dnsmasq 30583 - [meta sequenceId="253"] using nameserver 195.130.130.2#53
11-28-2020 18:14:45 Daemon.Info 192.168.1.199 1 2020-11-28T18:14:45+01:00 router01DFA9 dnsmasq 30583 - [meta sequenceId="252"] reading /tmp/resolv.conf.auto
11-28-2020 18:14:45 Daemon.Info 192.168.1.199 1 2020-11-28T18:14:45+01:00 router01DFA9 dnsmasq-dhcp 30583 - [meta sequenceId="251"] DHCP, IP range 192.168.1.100 -- 192.168.1.198, lease time 1d
11-28-2020 18:14:45 Daemon.Info 192.168.1.199 1 2020-11-28T18:14:45+01:00 router01DFA9 dnsmasq-dhcp 30583 - [meta sequenceId="250"] DHCP, IP range 192.168.2.100 -- 192.168.2.198, lease time 1d
11-28-2020 18:14:45 Daemon.Info 192.168.1.199 1 2020-11-28T18:14:45+01:00 router01DFA9 dnsmasq-dhcp 30583 - [meta sequenceId="249"] DHCP, IP range 192.168.3.100 -- 192.168.3.198, lease time 1d
11-28-2020 18:14:45 Daemon.Info 192.168.1.199 1 2020-11-28T18:14:45+01:00 router01DFA9 dnsmasq-dhcp 30583 - [meta sequenceId="248"] DHCP, IP range 192.168.4.100 -- 192.168.4.198, lease time 1d
11-28-2020 18:14:45 Daemon.Info 192.168.1.199 1 2020-11-28T18:14:45+01:00 router01DFA9 dnsmasq 30583 - [meta sequenceId="247"] compile time options: IPv6 GNU-getopt no-DBus no-i18n no-IDN DHCP DHCPv6 no-Lua TFTP no-conntrack no-ipset no-auth no-DNSSEC loop-detect no-inotify
11-28-2020 18:14:45 Daemon.Info 192.168.1.199 1 2020-11-28T18:14:45+01:00 router01DFA9 dnsmasq 30583 - [meta sequenceId="246"] started, version 2.78 cachesize 150
11-28-2020 18:14:44 Daemon.Info 192.168.1.199 1 2020-11-28T18:14:44+01:00 router01DFA9 dnsmasq 16409 - [meta sequenceId="245"] exiting on receipt of SIGTERM
11-28-2020 18:14:43 Local0.Info 192.168.1.199 1 2020-11-28T18:14:43+01:00 router01DFA9 ddns - - [meta sequenceId="244"] wan2:DDNS service is disabled.
11-28-2020 18:13:02 User.Info 192.168.1.199 1 2020-11-28T18:13:02+01:00 router01DFA9 Webfilter - - [meta sequenceId="243"] Update to rtu version 7.581.152

[marce1000]

 - Make sure to use the latest firmware available for the RV (too)

 M.

[DVDW38191]

Hi Marc, 

I am using the very last available firmware version.

Device Model: RV345
PID VID: RV345-K9 V05
Current Firmware Version: 1.0.03.20
Last Updated: 2020-Nov-20, 17:41:14 CET
Last Version Available on Cisco.com: 1.0.03.20
Last Checked: 2020-Nov-28, 19:04:03 CET

[marce1000]

 - What OS is the PC running ? Check it's logs related to networking and others, look for out-of-band and or irregular networking behavior (e.g.)

 M.

[DVDW38191]

Hi Marc,. 

Windows 10

McAffee + Windows vulnerability checking up to date

I tried to download a DDOS tool to see if I can reproduce , but it seems impossible to download such tools (protection starts always) 

No warnings/errors in Windows logs. 

Again tried scanning for Dell Latitude updates (= new laptop) - all up to date.

I swapped the USB-C<>RJ45 DA300. Same issue.

So, clearly a laptop problem, but how this can crash the RV345 !? I don't get it. 

regards

Dirk 

[marce1000]

 - Do you have the option of further scrutinize the particular interface counters where the laptop is connected too. Look for input/output errors ,overruns, crc errors etc. But in theory that shouldn't hang-up the switch. If needed connect the laptop to a switch which can follow-up on interface counters.

 M.

[DVDW38191]

I will see if I can get hold of a hub, and to connect another laptop with WireShark. But hey... what will it help me ? 

Let us assume that indeed the network is flooded with -I don't know what !? APR requests, DHCP requests, ... -> nothing I can do about it. As said, I changed the DELL USB to RJ45 (DA300) with another one. Maybe the network controller (which is inside DA300) had an issue. Did not bring a solution. Same thing. As soon as the laptop is closed and goes to sleep, Cisco RV345 dies...

All VLAN's are down. I have to decouple the network cable to this laptop physically, and reboot the router. 

Dell suggests 'to take the laptop to another network' and see if the laptop can crash the network there too.  

[marce1000]

 > Let us assume that indeed the network is flooded with -I don't know what !? APR requests, DHCP requests, ... -> nothing I can do about it. 

  No , no , I mean just from the viewpoint as how the laptop-interface  behaves on the network, does it induce errors , in the category that  I mentioned earlier , therefore the need for port-monitoring

  >Dell suggests 'to take the laptop to another network' and see if the laptop can crash the network there too.

                                          That's at lease 'ambitious' ... 

 M.

[DVDW38191]

Hi Marc, 

I was about to give up... until I took out an ancient Cisco SR224 between the DELL laptop and Cisco SR345 router and replaced it by a more modern 1Gbit switch. And... the problem magically disappeared !? 

Then I switched them again... problem appeared. 

So, now, final situation with newer unmanaged switch : all ok !? (touch wood) 

Now a matter of keeping an eye on network availability. Very strange that unmanaged switch could cause so much problems. 

Thanks a lot for your ideas and support. You are not a Cisco employee ?!

Regards

Dirk 

[marce1000]

                  - Great to here it good resolved , no I am not a Cisco employee

 M.