Cisco reboots during night without any visible errors.

Edgardo Rodriguez · ‎02-23-2015

Hello everyone!

Well, last weeks i am observing some quite weird behaviours on one cisco 1921/K9 we have as a VPN hub. I´ve been trying different IOS versions without luck. The thing is that the router suddenly reboots (fortunately at night) without any previous advice, nor any kind of errors visible. I kept a serial console open, in orden to see all the router´s output.

3 hours ago, to be precise, this router went through this process and i´ll show you the "reload reason" command output:

router-cisco#sh ver | inc reload
System returned to ROM by reload at 12:07:07 ARG Thu Feb 19 2015
Last reload type: Normal Reload
Last reload reason: reload
router-cisco#sh ver | inc uptime
router-cisco uptime is 3 hours, 28 minutes

At console i see for example as normal operation tunnels being brought up or down, suddenly bootstrap shows up showing a normal router boot process, after that everything keeps working fine.

Is this an expected behaviour? I remeber a cisco 1811 which was in this role (before we upgraded this hardware) which was up up to 265 days running without any issues nor reboots.

Thanks BTW.

Regards

InayathUlla Sharieff · ‎02-23-2015

Upload the latest Software and check.

Check if there is no power issue at site during the time? Is the reboot happen every 3 hours or specific time?

If its at specific time do check with the onsite people about power fluctiation.

HTH

Edgardo Rodriguez · ‎02-23-2015

Hi InayathUlla ! Thanks for answering,

* No power issues (we are on a datacenter) and we also have many other devices using the same line, with no reboots.

* No specific time, perhaps router works fine for 3 days, or one week, depends on something i don´t certainly know.

* I´ve tried latest FW but no luck and now i preffer to keep using c1900-universalk9-mz.SPA.152-4.M7.bin which is the most stable at least for me.

rakeshvelagala · ‎02-23-2015

Is there any crashinfo?

Edgardo Rodriguez · ‎02-23-2015

No crashinfo :(

router-cisco#sh flash | inc crash
router-cisco#

rakeshvelagala · ‎02-23-2015

Hi

Can you check if there are any drops, collisions.... on the interface and they keep increasing?

We have seen due to too many collisions or drops the device rebooted.

Thanks

Edgardo Rodriguez · ‎02-23-2015

I´ve been working on that, previously when this began to happen and we realized, i have enabled flowcontrol on the switch, because i have had some drops on interfaces due to overflows or queue filled.

As you can see there are some pause input, but no drops.

router-cisco#sh int gi0/0
GigabitEthernet0/0 is up, line protocol is up
Hardware is CN Gigabit Ethernet, address is c067.af37.d6c0 (bia c067.af37.d6c0)
Description: hub - eth5 - lan
Internet address is 192.168.19.3/24
MTU 1500 bytes, BW 100000 Kbit/sec, DLY 100 usec,
reliability 255/255, txload 5/255, rxload 27/255
Encapsulation 802.1Q Virtual LAN, Vlan ID 1., loopback not set
Keepalive set (10 sec)
Full Duplex, 100Mbps, media type is RJ45
output flow-control is XON, input flow-control is XON
ARP type: ARPA, ARP Timeout 04:00:00
Last input 00:00:00, output 00:00:00, output hang never
Last clearing of "show interface" counters never
Input queue: 0/2048/0/34 (size/max/drops/flushes); Total output drops: 0
Queueing strategy: fifo
Output queue: 0/512 (size/max)
5 minute input rate 10878000 bits/sec, 1603 packets/sec
5 minute output rate 2017000 bits/sec, 1266 packets/sec
19592766 packets input, 2173731972 bytes, 0 no buffer
Received 524 broadcasts (0 IP multicasts)
0 runts, 0 giants, 0 throttles
0 input errors, 0 CRC, 0 frame, 0 overrun, 0 ignored
0 watchdog, 0 multicast, 12 pause input
16319281 packets output, 2982027630 bytes, 0 underruns
0 output errors, 0 collisions, 0 interface resets
0 unknown protocol drops
0 babbles, 0 late collision, 0 deferred
0 lost carrier, 0 no carrier, 0 pause output
0 output buffer failures, 0 output buffers swapped out
router-cisco#sh int gi0/1
GigabitEthernet0/1 is up, line protocol is up
Hardware is CN Gigabit Ethernet, address is c067.af37.d6c1 (bia c067.af37.d6c1)
Description: hub - eth24 - wan
Internet address is 190.224.163.136/28
MTU 1500 bytes, BW 100000 Kbit/sec, DLY 100 usec,
reliability 255/255, txload 31/255, rxload 7/255
Encapsulation ARPA, loopback not set
Keepalive set (10 sec)
Full Duplex, 100Mbps, media type is RJ45
output flow-control is XON, input flow-control is XON
ARP type: ARPA, ARP Timeout 04:00:00
Last input 00:00:00, output 00:00:00, output hang never
Last clearing of "show interface" counters never
Input queue: 0/300/0/41 (size/max/drops/flushes); Total output drops: 0
Queueing strategy: fifo
Output queue: 0/160 (size/max)
5 minute input rate 3102000 bits/sec, 1415 packets/sec
5 minute output rate 12404000 bits/sec, 2075 packets/sec
18990366 packets input, 535861003 bytes, 0 no buffer
Received 209 broadcasts (0 IP multicasts)
0 runts, 0 giants, 0 throttles
0 input errors, 0 CRC, 0 frame, 0 overrun, 0 ignored
0 watchdog, 0 multicast, 27 pause input
26950829 packets output, 261674749 bytes, 0 underruns
0 output errors, 0 collisions, 0 interface resets
0 unknown protocol drops
0 babbles, 0 late collision, 0 deferred
0 lost carrier, 0 no carrier, 0 pause output
0 output buffer failures, 0 output buffers swapped out

Whats more confusing is that never happens on heavy loads, during night when CPU won´t go further than 8% load average and interfaces input traffic are not higher than ~5MBps.