Cisco reboots during night without any visible errors.

[Edgardo Rodriguez]

Hello everyone!

Well, last weeks i am observing some quite weird behaviours on one cisco 1921/K9 we have as a VPN hub. I´ve been trying different IOS versions without luck. The thing is that the router suddenly reboots (fortunately at night) without any previous advice, nor any kind of errors visible. I kept a serial console open, in orden to see all the router´s output.

3 hours ago, to be precise, this router went through this process and i´ll show you the "reload reason" command output:

router-cisco#sh ver | inc reload
System returned to ROM by reload at 12:07:07 ARG Thu Feb 19 2015
Last reload type: Normal Reload
Last reload reason: reload
router-cisco#sh ver | inc uptime
router-cisco uptime is 3 hours, 28 minutes

At console i see for example as normal operation tunnels being brought up or down, suddenly bootstrap shows up showing a normal router boot process, after that everything keeps working fine.

Is this an expected behaviour? I remeber a cisco 1811 which was in this role (before we upgraded this hardware) which was up up to 265 days running without any issues nor reboots.

Thanks BTW.

Regards

[InayathUlla Sharieff]

Upload the latest Software and check.

Check if there is no power issue at site during the time? Is the reboot happen every 3 hours or specific time?

If its at specific time do check with the onsite people about power fluctiation.

HTH

[Edgardo Rodriguez]

Hi InayathUlla ! Thanks for answering, 

* No power issues (we are on a datacenter) and we also have many other devices using the same line, with no reboots.

* No specific time, perhaps router works fine for 3 days, or one week, depends on something i don´t certainly know.

* I´ve tried latest FW but no luck and now i preffer to keep using c1900-universalk9-mz.SPA.152-4.M7.bin which is the most stable at least for me.

[rakeshvelagala]

Is there any crashinfo?

[Edgardo Rodriguez]

No crashinfo :(

router-cisco#sh flash | inc crash
router-cisco#

[rakeshvelagala]

Hi

Can you check if there are any drops, collisions.... on the interface and they keep increasing?

We have seen due to too many collisions or drops the device rebooted.

Thanks 

[Edgardo Rodriguez]

I´ve been working on that, previously when this began to happen and we realized, i have enabled flowcontrol on the switch, because i have had some drops on interfaces due to overflows or queue filled.

As you can see there are some pause input, but no drops.

router-cisco#sh int gi0/0
GigabitEthernet0/0 is up, line protocol is up
  Hardware is CN Gigabit Ethernet, address is c067.af37.d6c0 (bia c067.af37.d6c0)
  Description: hub - eth5 - lan
  Internet address is 192.168.19.3/24
  MTU 1500 bytes, BW 100000 Kbit/sec, DLY 100 usec,
     reliability 255/255, txload 5/255, rxload 27/255
  Encapsulation 802.1Q Virtual LAN, Vlan ID  1., loopback not set
  Keepalive set (10 sec)
  Full Duplex, 100Mbps, media type is RJ45
  output flow-control is XON, input flow-control is XON
  ARP type: ARPA, ARP Timeout 04:00:00
  Last input 00:00:00, output 00:00:00, output hang never
  Last clearing of "show interface" counters never
  Input queue: 0/2048/0/34 (size/max/drops/flushes); Total output drops: 0
  Queueing strategy: fifo
  Output queue: 0/512 (size/max)
  5 minute input rate 10878000 bits/sec, 1603 packets/sec
  5 minute output rate 2017000 bits/sec, 1266 packets/sec
     19592766 packets input, 2173731972 bytes, 0 no buffer
     Received 524 broadcasts (0 IP multicasts)
     0 runts, 0 giants, 0 throttles
     0 input errors, 0 CRC, 0 frame, 0 overrun, 0 ignored
     0 watchdog, 0 multicast, 12 pause input
     16319281 packets output, 2982027630 bytes, 0 underruns
     0 output errors, 0 collisions, 0 interface resets
     0 unknown protocol drops
     0 babbles, 0 late collision, 0 deferred
     0 lost carrier, 0 no carrier, 0 pause output
     0 output buffer failures, 0 output buffers swapped out
router-cisco#sh int gi0/1
GigabitEthernet0/1 is up, line protocol is up
  Hardware is CN Gigabit Ethernet, address is c067.af37.d6c1 (bia c067.af37.d6c1)
  Description: hub - eth24 - wan
  Internet address is 190.224.163.136/28
  MTU 1500 bytes, BW 100000 Kbit/sec, DLY 100 usec,
     reliability 255/255, txload 31/255, rxload 7/255
  Encapsulation ARPA, loopback not set
  Keepalive set (10 sec)
  Full Duplex, 100Mbps, media type is RJ45
  output flow-control is XON, input flow-control is XON
  ARP type: ARPA, ARP Timeout 04:00:00
  Last input 00:00:00, output 00:00:00, output hang never
  Last clearing of "show interface" counters never
  Input queue: 0/300/0/41 (size/max/drops/flushes); Total output drops: 0
  Queueing strategy: fifo
  Output queue: 0/160 (size/max)
  5 minute input rate 3102000 bits/sec, 1415 packets/sec
  5 minute output rate 12404000 bits/sec, 2075 packets/sec
     18990366 packets input, 535861003 bytes, 0 no buffer
     Received 209 broadcasts (0 IP multicasts)
     0 runts, 0 giants, 0 throttles
     0 input errors, 0 CRC, 0 frame, 0 overrun, 0 ignored
     0 watchdog, 0 multicast, 27 pause input
     26950829 packets output, 261674749 bytes, 0 underruns
     0 output errors, 0 collisions, 0 interface resets
     0 unknown protocol drops
     0 babbles, 0 late collision, 0 deferred
     0 lost carrier, 0 no carrier, 0 pause output
     0 output buffer failures, 0 output buffers swapped out

Whats more confusing is that never happens on heavy loads, during night when CPU won´t go further than 8% load average and interfaces input traffic are not higher than ~5MBps.