SPA112 goes catatonic, crashes, no dialtone - Page 8

sgtpanties · ‎10-26-2012

Hi, I am a home user and ex-programmer. I'm using the Cisco SPA112 with the latest firmware 1.2.1 (004). Every two or three days the ATA crashes. Here are the symptoms:

a) no dial tone

b) line 1 LED is ON (as normal)

c) ATA is PINGable

d) IVR is unresponsive

e) web setup URL does not respond, can not get to the setup page

f) The reset button does not respond.

My only recourse now is the check if I have a dialtone every morning and, if not, pull and reinsert the power plug. The cold boot of course results in the local log being lost. Here is the log right after boot up.

Jan 1 00:00:04 SPA112 kern.warning [ 0.000000] CPU: ARM926EJ-S [41069265] revision 5 (ARMv5TEJ), cr=00053177

Jan 1 00:00:04 SPA112 kern.warning [ 0.000000] Machine: NXP PNX8181

Jan 1 00:00:04 SPA112 kern.warning [ 0.000000] Memory policy: ECC disabled, Data cache writeback

Jan 1 00:00:04 SPA112 kern.warning [ 0.000000] CPU0: D VIVT write-back cache

Jan 1 00:00:04 SPA112 kern.warning [ 0.000000] CPU0: I cache: 32768 bytes, associativity 4, 32 byte lines, 256 sets

Jan 1 00:00:04 SPA112 kern.warning [ 0.000000] CPU0: D cache: 32768 bytes, associativity 4, 32 byte lines, 256 sets

Jan 1 00:00:04 SPA112 kern.warning [17179569.184000] Built 1 zonelists in Zone order, mobility grouping on. Total pages: 7874

Jan 1 00:00:04 SPA112 kern.warning [17179569.184000] PID hash table entries: 128 (order: 7, 512 bytes)

Jan 1 00:00:04 SPA112 kern.warning [17179569.184000] Console: colour dummy device 80x30

Jan 1 00:00:04 SPA112 kern.warning [17179569.260000] Mount-cache hash table entries: 512

Jan 1 00:00:04 SPA112 kern.warning [17179569.264000] Board HW MODEL : 0x3

Jan 1 00:00:04 SPA112 kern.warning [17179569.336000] squashfs: LZMA suppport for slax.org by jro

Jan 1 00:00:04 SPA112 kern.warning [17179569.384000] [ip3912] : Bridge Mode...

Jan 1 00:00:04 SPA112 kern.err [17179569.408000] physmap-flash physmap-flash.0: map_probe failed

Jan 1 00:00:04 SPA112 kern.warning [17179569.416000] Using Full Image\'s RootFS

Jan 1 00:00:04 SPA112 kern.warning [17179569.420000] Using static partition definition

Jan 1 00:00:04 SPA112 kern.warning [17179569.424000] !!! do adler32 checksum !!!

Jan 1 00:00:04 SPA112 kern.warning [17179571.260000] File system image checksum OK

Jan 1 00:00:04 SPA112 kern.err [17179571.344000] ksz8873 0-005f: failed with status -1

Jan 1 00:00:04 SPA112 kern.warning [17179571.348000] ksz8873: probe of 0-005f failed with error -1

Jan 1 00:00:04 SPA112 kern.warning [17179571.352000] PNX8181 watchdog timer: timer margin 16 sec

Jan 1 00:00:04 SPA112 kern.warning [17179571.372000] GACT probability on

Jan 1 00:00:04 SPA112 kern.warning [17179571.376000] Mirror/redirect action on

Jan 1 00:00:04 SPA112 kern.warning [17179571.380000] u32 classifier

Jan 1 00:00:04 SPA112 kern.warning [17179571.384000] Performance counters on

Jan 1 00:00:04 SPA112 kern.warning [17179571.388000] input device check on

Jan 1 00:00:04 SPA112 kern.warning [17179571.392000] Actions configured

Jan 1 00:00:04 SPA112 kern.warning [17179571.396000] Netfilter messages via NETLINK v0.30.

Jan 1 00:00:04 SPA112 kern.warning [17179571.400000] nf_conntrack version 0.5.0 (1024 buckets, 4096 max)

Jan 1 00:00:04 SPA112 kern.warning [17179571.412000] ipt_time loading

Jan 1 00:00:04 SPA112 kern.warning [17179571.464000] VFS: Mounted root (squashfs filesystem) readonly.

Jan 1 00:00:04 SPA112 kern.warning [17179576.144000] ***** LED_DRV init *****

Jan 1 00:00:04 SPA112 kern.warning [17179576.148000] ***** LED_DRV end *****

Jan 1 00:00:04 SPA112 kern.warning [17179576.176000] *** sys event driver initialized ***

Jan 1 00:00:04 SPA112 kern.err [17179579.184000] br0: Dropping NETIF_F_UFO since no NETIF_F_HW_CSUM feature.

Jan 1 00:00:04 SPA112 kern.warning [17179580.880000] Empty flash at 0x00069454 ends at 0x00069600

Jan 1 00:00:05 SPA112 daemon.err dnsmasq[136]: failed to load names from /etc/hosts: No such file or directory

Jan 1 00:00:23 SPA112 kern.err [17179600.844000] br0: Dropping NETIF_F_UFO since no NETIF_F_HW_CSUM feature.

Jan 1 00:00:27 SPA112 daemon.err system[1]: notify add wan1 interface(br0)

Jan 1 00:00:27 SPA112 daemon.err system[1]: start 0 vlan-id... ok

Jan 1 00:00:39 SPA112 daemon.err dnsmasq[136]: failed to load names from /etc/hosts: No such file or directory

Here are some things I have tried on my own, to no effect:

a) Tried running with Provisioning/Firmware Upgrade/Upgrade set to false.

b) Reloaded the firmware with IE. Before, I did it with FireFox.

c) Did a factory reset and re-entered the settings suggested for my ATA from Callcentric.

How should I proceed?

Patrick Born · ‎02-11-2013

Hi Shar,

1.3.1(003) is available on Cisco.com (since 1/10/13) and is the current release for the ATA and addresses the title issue of this thread.

Regards,

Patrick---

triplef111 · ‎04-24-2013

well when you go to cisco... you see FW is 1.3.1 003 but inside the zip file is

payton_1.0.2_006_102411_1744.bin and that fails....

1.0.2 ?" wthell ?

cindy toy · ‎04-24-2013

Hi Frank,

I just dowloaded the file from here:

http://software.cisco.com/download/release.html?mdfid=283998771&flowid=30521&softwareid=282463187&release=1.3.1&relind=AVAILABLE&rellifecycle=&reltype=latest

The file I see looks correct: SPA112_SPA122_1.3.1_003.bin

Regards,
Cindy Toy
Cisco Small Business Community Manager
for Cisco Small Business Products
www.cisco.com/go/smallbizsupport
twitter: CiscoSBsupport

Regards, Cindy If my response answered your question, please mark the response as answered. Thank you!

Daniele Giordano · ‎04-26-2013

We have about 2000 SPA2102 CPEs and they are very stable.

Recently we started to deploy the SPA122 model.

Actually we have about 50 new CPEs installed with firmware 1.3.1.

Every day our contact center receive call from customers due to freezing of SPA122.

The WEB GUI is unreachable and we have no response to PING.

If the device works in bridge mode, the below equipment works fine.

The only way to solve this issue is an hardware reload of SPA122.

We have tested also the beta firmware 1.3.2(009) but it doesn't fix this issue.

News from Cisco?

Patrick Born · ‎04-26-2013

Hi Daniele,

Please open a support ticket with Cisco Support in order to get this issue properly tracked and addressed. This community supported by volunteers that are not part of Cisco Support so while this is a great mechanism for sharing information, it is not an appropriate way for you to receive the expeditious support you need.

This thread was related to a memory leak issue in earlier code which we've resolved.

It seems like you are very experienced with the SPA2102. The SPA122 is similar in operation, but significantly different in configuration.

The issues you're describing sound like something we've not previously seen or heard of. I'm very interested in working on this with you so that we can understand and address the root cause. I'm going to need to understand how the devices are provisioned and deployed by you. It appears that you have at least two ways of deploying 1. router and 2. bridge mode. I'll need a sample configuration from each type of deployment. It will also be helpful if you can enable debug and capture a Wireshark trace on a device that is freezing each day.

If you're willing, please contact me at paborn@cisco.com so that I can work directly with you. I'll update the thread once we've resolved the problems your company is experiencing.

NOTE: I'm not a support person and cannot replace the support you'll receive from Cisco's Support team.

Regards,

Patrick---

Daniele Giordano · ‎04-30-2013

Hi, sorry for late.

My SPA122 are configured in two ways:

1) ROUTER

In this case the SPA122 works as a router with NAT. The network scheme is below.

2) BRIDGE

In this case the SPA122 works as a bridge with VLANs: a voice VLAN and a data VLAN. This config was inherited by old Cisco ATA188 installation. The SPA122 works on a voice VLAN while data VLAN is switched behind toward PC or Router. The network scheme is the following:

The provisioning system is the same for both cases. The SPA load spa122config_b.xml file attached via TFTP server. The firmware is loaded via HTTP server.

We are currently capturing logs form SPA to see what happens before the block.

Daniele Giordano · ‎04-30-2013

Interesting log from SPA122:

ip3912: Allocating new skb in RX IRQ handler failed, losing one DMA descriptor for eth0!

swapper: page allocation failure. order:0, mode:0x20

syncrotechnology · ‎04-30-2013

Have you tried to watch memory usage with SNMP? It may be helpful. I use PRTG "% free memory" sensor.

fredk2500 · ‎05-25-2013

I have a problem with my spa122 with Voip.ms. Voice is cutting ramdomly for few seconds each time. When this happens, I have that in my log:

<3>May 25 14:09:29 SPA122 [17184444.020000] voice_kmode_thread_send(0): sock_sendmsg() failed witherror -101
<3>May 25 14:09:29 SPA122 [17184444.048000] voice_kmode_thread_send(0): sock_sendmsg() failed witherror -101
<3>May 25 14:09:29 SPA122 [17184444.056000] voice_kmode_thread_send(0): sock_sendmsg() failed witherror -101
<3>May 25 14:09:29 SPA122 [17184444.076000] voice_kmode_thread_send(0): sock_sendmsg() failed witherror -101

I enabled debug log. The problem a happen at 14:09:12:

May 25 14:04:40 SPA122 kern.warning [17184155.816000] #### RTP STOP Flag set in this channel break ####

May 25 14:04:40 SPA122 kern.warning [17184155.816000] ###### sock_sendmsg return 172

May 25 14:05:55 SPA122 syslog.notice syslog-ng[2436]: STATS: dropped 0

May 25 14:08:02 SPA122 kern.warning [17184357.052000] In cordless Driver Codec 0 and str PCMU/8000 chan 0

May 25 14:08:02 SPA122 kern.warning [17184357.052000] In cordless Driver Codec 18 and str G.729/8000 chan 0

May 25 14:08:02 SPA122 kern.warning [17184357.052000] In cordless Driver Codec 2 and str G.726/8000 chan 0

May 25 14:08:02 SPA122 kern.warning [17184357.052000] In cordless Driver Codec 8 and str PCMA/8000 chan 0

May 25 14:08:02 SPA122 kern.warning [17184357.052000] In cordless Driver Codec 100 and str NSE/8000 chan 0

May 25 14:08:02 SPA122 kern.warning [17184357.052000] In cordless Driver Codec 112 and str encaprtp/8000 chan 0

May 25 14:08:04 SPA122 kern.warning [17184358.760000] ###### sock_sendmsg return 172

May 25 14:08:04 SPA122 kern.warning [17184358.764000]

May 25 14:08:04 SPA122 kern.warning [17184358.764000] #### RTP STOP Flag set in this channel break ####

May 25 14:08:04 SPA122 kern.warning [17184358.776000] In cordless Driver Codec 0 and str PCMU/8000 chan 0

May 25 14:08:04 SPA122 kern.warning [17184358.776000] In cordless Driver Codec 18 and str G.729/8000 chan 0

May 25 14:08:04 SPA122 kern.warning [17184358.776000] In cordless Driver Codec 2 and str G.726/8000 chan 0

May 25 14:08:04 SPA122 kern.warning [17184358.776000] In cordless Driver Codec 8 and str PCMA/8000 chan 0

May 25 14:08:04 SPA122 kern.warning [17184358.776000] In cordless Driver Codec 100 and str NSE/8000 chan 0

May 25 14:08:04 SPA122 kern.warning [17184358.776000] In cordless Driver Codec 112 and str encaprtp/8000 chan 0

May 25 14:09:12 SPA122 kern.err [17184427.056000] voice_kmode_thread_send(0): sock_sendmsg() failed witherror -101

May 25 14:09:12 SPA122 kern.err [17184427.080000] voice_kmode_thread_send(0): sock_sendmsg() failed witherror -101

May 25 14:09:12 SPA122 kern.err [17184427.096000] voice_kmode_thread_send(0): sock_sendmsg() failed witherror -101

May 25 14:09:12 SPA122 kern.err [17184427.116000] voice_kmode_thread_send(0): sock_sendmsg() failed witherror -101

May 25 14:09:12 SPA122 kern.err [17184427.136000] voice_kmode_thread_send(0): sock_sendmsg() failed witherror -101

May 25 14:09:12 SPA122 kern.err [17184427.156000] voice_kmode_thread_send(0): sock_sendmsg() failed witherror -101

May 25 14:09:12 SPA122 kern.err [17184427.176000] voice_kmode_thread_send(0): sock_sendmsg() failed witherror -101

May 25 14:09:12 SPA122 kern.err [17184427.196000] voice_kmode_thread_send(0): sock_sendmsg() failed witherror -101

May 25 14:09:12 SPA122 kern.err [17184427.216000] voice_kmode_thread_send(0): sock_sendmsg() failed witherror -101

May 25 14:09:12 SPA122 kern.debug [17184427.228000] sysevt_comm_sendto: (16, rc)=>

May 25 14:09:12 SPA122 kern.err [17184427.240000] voice_kmode_thread_send(0): sock_sendmsg() failed witherror -101

May 25 14:09:12 SPA122 kern.debug [17184427.248000] sysevt_comm_sendto: (480, rc)=>

May 25 14:09:12 SPA122 kern.err [17184427.256000] voice_kmode_thread_send(0): sock_sendmsg() failed witherror -101

May 25 14:09:12 SPA122 kern.err [17184427.276000] voice_kmode_thread_send(0): sock_sendmsg() failed witherror -101

May 25 14:09:12 SPA122 kern.err [17184427.300000] voice_kmode_thread_send(0): sock_sendmsg() failed witherror -101

May 25 14:09:12 SPA122 kern.err [17184427.316000] voice_kmode_thread_send(0): sock_sendmsg() failed witherror -101

May 25 14:09:12 SPA122 kern.err [17184427.336000] voice_kmode_thread_send(0): sock_sendmsg() failed witherror -101

May 25 14:09:12 SPA122 kern.err [17184427.356000] voice_kmode_thread_send(0): sock_sendmsg() failed witherror -101

May 25 14:09:12 SPA122 kern.err [17184427.376000] voice_kmode_thread_send(0): sock_sendmsg() failed witherror -101

May 25 14:09:12 SPA122 kern.err [17184427.396000] voice_kmode_thread_send(0): sock_sendmsg() failed witherror -101

May 25 14:09:12 SPA122 kern.err [17184427.416000] voice_kmode_thread_send(0): sock_sendmsg() failed witherror -101

May 25 14:09:12 SPA122 kern.err [17184427.436000] voice_kmode_thread_send(0): sock_sendmsg() failed witherror -101

May 25 14:09:12 SPA122 kern.err [17184427.456000] voice_kmode_thread_send(0): sock_sendmsg() failed witherror -101

May 25 14:09:12 SPA122 kern.warning [17184427.464000] cy_convert_wan_idx buffer=, len=1

May 25 14:09:12 SPA122 kern.err [17184427.480000] voice_kmode_thread_send(0): sock_sendmsg() failed witherror -101

May 25 14:09:12 SPA122 kern.err [17184427.500000] voice_kmode_thread_send(0): sock_sendmsg() failed witherror -101

May 25 14:09:12 SPA122 kern.warning [17184427.516000] WAN idx = [ <3>voice_kmode_thread_send(0): sock_sendmsg() failed witherror -101

May 25 14:09:12 SPA122 kern.warning [17184427.524000] ]

May 25 14:09:12 SPA122 kern.err [17184427.540000] voice_kmode_thread_send(0): sock_sendmsg() failed witherror -101

May 25 14:09:12 SPA122 kern.err [17184427.556000] voice_kmode_thread_send(0): sock_sendmsg() failed witherror -101

May 25 14:09:12 SPA122 kern.err [17184427.576000] voice_kmode_thread_send(0): sock_sendmsg() failed witherror -101

May 25 14:09:12 SPA122 kern.err [17184427.596000] voice_kmode_thread_send(0): sock_sendmsg() failed witherror -101

May 25 14:09:12 SPA122 kern.err [17184427.620000] voice_kmode_thread_send(0): sock_sendmsg() failed witherror -101

May 25 14:09:12 SPA122 kern.err [17184427.636000] voice_kmode_thread_send(0): sock_sendmsg() failed witherror -101

May 25 14:09:12 SPA122 kern.err [17184427.656000] voice_kmode_thread_send(0): sock_sendmsg() failed witherror -101

May 25 14:09:12 SPA122 kern.warning [17184427.660000] cy_convert_wan1_idx buffer=, len=1

May 25 14:09:12 SPA122 kern.warning [17184427.660000] WAN1 idx = [ ]

May 25 14:09:12 SPA122 kern.err [17184427.676000] voice_kmode_thread_send(0): sock_sendmsg() failed witherror -101

May 25 14:09:12 SPA122 kern.err [17184427.696000] voice_kmode_thread_send(0): sock_sendmsg() failed witherror -101

May 25 14:09:12 SPA122 kern.warning [17184427.716000] ### conntrack_flush

May 25 14:09:12 SPA122 kern.err [17184427.716000] voice_kmode_thread_send(0): sock_sendmsg() failed witherror -101

May 25 14:09:12 SPA122 kern.warning [17184427.720000] ### conntrack_flush done

May 25 14:09:12 SPA122 kern.err [17184427.736000] voice_kmode_thread_send(0): sock_sendmsg() failed witherror -101

.........

May 25 14:09:23 SPA122 kern.debug [17184438.216000] sysevt_comm_sendto: (103, rc)=>

May 25 14:09:23 SPA122 kern.err [17184438.220000] voice_kmode_thread_send(0): sock_sendmsg() failed witherror -101

May 25 14:09:23 SPA122 daemon.notice system[1]: WAN br0 connected

May 25 14:09:23 SPA122 kern.err [17184438.240000] voice_kmode_thread_send(0): sock_sendmsg() failed witherror -101

May 25 14:09:23 SPA122 daemon.notice system[1]: Link name=br0

May 25 14:09:23 SPA122 kern.err [17184438.256000] voice_kmode_thread_send(0): sock_sendmsg() failed witherror -101

May 25 14:09:23 SPA122 daemon.notice system[1]: IP address=64.229.56.156

May 25 14:09:23 SPA122 daemon.notice system[1]: Netmask=255.255.255.248

May 25 14:09:23 SPA122 kern.err [17184438.280000] voice_kmode_thread_send(0): sock_sendmsg() failed witherror -101

May 25 14:09:23 SPA122 kern.err [17184438.296000] voice_kmode_thread_send(0): sock_sendmsg() failed witherror -101

May 25 14:09:23 SPA122 daemon.notice system[1]: Gateway=64.229.56.155

May 25 14:09:23 SPA122 kern.err [17184438.316000] voice_kmode_thread_send(0): sock_sendmsg() failed witherror -101

May 25 14:09:23 SPA122 daemon.notice system[1]: hostname=

May 25 14:09:23 SPA122 daemon.notice system[1]: domain=gateway.2wire.net

May 25 14:09:23 SPA122 kern.err [17184438.336000] voice_kmode_thread_send(0): sock_sendmsg() failed witherror -101

May 25 14:09:23 SPA122 kern.err [17184438.356000] voice_kmode_thread_send(0): sock_sendmsg() failed witherror -101

May 25 14:09:23 SPA122 kern.debug [17184438.360000] sysevt_comm_sendto: (16, rc)=>

May 25 14:09:23 SPA122 daemon.notice system[1]: dns_0=192.168.2.1

Someone have an idea?

Thanks

robbikay1 · ‎05-25-2013

Is your spa122 behind a 2wire residential gateway?

I had voice drops every 10 minutes due to the 2wire(in DMZplus mode) releasing and renewing the spa122's ip address.

I ended up going into the spa122's admin panel and in the Internet Settings, copying the DHCP info into the Static ip connection type.

The only thing to watch is if your isp frequently changes your ip address, this can can cause more hassle than it's worth. I was initially worried about this but it turned out that although my service says dynamic ip, my ip address was "sticky" and did not actually change for the whole year that I had that service.

fredk2500 · ‎05-25-2013

Yes, it is in the DMZplus too. It make sense. I will set it behind de router in NAT mode instead.

Thanks

syncrotechnology · ‎04-26-2013

This sounds like a familiar problem that in my case was traced (I believe) to a memory leak. How long do they stay up? I was seeing failures about every 4 days and the engineering release 1.3.2.(004) solved this issue - I have units up over a month at a time now. Are you doing anything else unusual? Also, I used SNMP to look at free memory and CPU load and was able to predict a failure coming - then used the rebooter from Mike (http://www.freymond.ca/SPA112/) to reboot remotely - you could also just do that every night without messing with SNMP. I know this is not a solution but sometimes a workaround is all you have! --Dale

Daniele Giordano · ‎04-30-2013

Interesting log from SPA122:

Apr 30 19:04:47 10.35.41.140 Apr 27 04:15:09 me [17195206.692000] __ratelimit: 282 messages suppressed

Apr 30 19:04:47 10.35.41.140 Apr 27 04:15:09 me [17195206.692000] ksoftirqd/0: page allocation failure. order:0, mode:0x20

Apr 30 19:04:47 10.35.41.140 Apr 27 04:15:09 me [17195206.692000] [] (dump_stack+0x0/0x14) from [] (__alloc_pages_internal+0x3bc/0x3d4)

Apr 30 19:04:47 10.35.41.140 Apr 27 04:15:09 me [17195206.692000] [] (__alloc_pages_internal+0x0/0x3d4) from [] (__alloc_pages+0x14/0x18)

Apr 30 19:04:47 10.35.41.140 Apr 27 04:15:09 me [17195206.692000] [] (__alloc_pages+0x0/0x18) from [] (cache_alloc_refill+0x2e8/0x5a8)

Apr 30 19:04:47 10.35.41.140 Apr 27 04:15:09 me [17195206.692000] [] (cache_alloc_refill+0x0/0x5a8) from [] (__kmalloc+0x94/0xd0)

Apr 30 19:04:47 10.35.41.140 Apr 27 04:15:09 me [17195206.692000] [] (__kmalloc+0x0/0xd0) from [] (__alloc_skb+0x54/0xf8)

Apr 30 19:04:47 10.35.41.140 Apr 27 04:15:09 me [17195206.692000] r7:00000620 r6:b5807d40 r5:00000614 r4:b47a6560

Apr 30 19:04:47 10.35.41.140 Apr 27 04:15:09 me [17195206.692000] [] (__alloc_skb+0x0/0xf8) from [] (dev_alloc_skb+0x20/0x44)

Apr 30 19:04:47 10.35.41.140 Apr 27 04:15:09 me [17195206.692000] r8:b5850000 r7:0000001e r6:b5850360 r5:b5850360 r4:b501a800

Apr 30 19:04:47 10.35.41.140 Apr 27 04:15:09 me [17195206.692000] [] (dev_alloc_skb+0x0/0x44) from [] (ip3912_alloc_new_rx_skb+0x28/0x120)

Apr 30 19:04:47 10.35.41.140 Apr 27 04:15:09 me [17195206.692000] [] (ip3912_alloc_new_rx_skb+0x0/0x120) from [] (ip3912_poll+0x230/0x4b4)

Apr 30 19:04:47 10.35.41.140 Apr 27 04:15:09 me [17195206.692000] [] (ip3912_poll+0x0/0x4b4) from [] (net_rx_action+0x74/0x1ac)

Apr 30 19:04:47 10.35.41.140 Apr 27 04:15:09 me [17195206.692000] [] (net_rx_action+0x0/0x1ac) from [] (__do_softirq+0x74/0xf4)

Apr 30 19:04:47 10.35.41.140 Apr 27 04:15:09 me [17195206.692000] [] (__do_softirq+0x0/0xf4) from [] (do_softirq+0x4c/0x58)

Apr 30 19:04:47 10.35.41.140 Apr 27 04:15:09 me [17195206.692000] r6:00000000 r5:00000000 r4:60000013

Apr 30 19:04:47 10.35.41.140 Apr 27 04:15:09 me [17195206.692000] [] (do_softirq+0x0/0x58) from [] (ksoftirqd+0x7c/0x150)

Apr 30 19:04:47 10.35.41.140 Apr 27 04:15:09 me [17195206.692000] r4:b581e000

Apr 30 19:04:47 10.35.41.140 Apr 27 04:15:09 me [17195206.692000] [] (ksoftirqd+0x0/0x150) from [] (kthread+0x5c/0x94)

Apr 30 19:04:47 10.35.41.140 Apr 27 04:15:09 me [17195206.692000] r6:b403dc5c r5:00000000 r4:b581e000

Apr 30 19:04:47 10.35.41.140 Apr 27 04:15:09 me [17195206.692000] [] (kthread+0x0/0x94) from [] (do_exit+0x0/0x694)

Apr 30 19:04:48 10.35.41.140 Apr 27 04:15:09 me [17195206.692000] r6:00000000 r5:00000000 r4:00000000

Apr 30 19:04:48 10.35.41.140 Apr 27 04:15:09 me [17195206.692000] Mem-info:

Apr 30 19:04:48 10.35.41.140 Apr 27 04:15:09 me [17195206.692000] DMA per-cpu:

Apr 30 19:04:48 10.35.41.140 Apr 27 04:15:09 me [17195206.692000] CPU 0: hi: 0, btch: 1 usd: 0

Apr 30 19:04:48 10.35.41.140 Apr 27 04:15:09 me [17195206.692000] Active:2754 inactive:1930 dirty:0 writeback:0 unstable:0

Apr 30 19:04:48 10.35.41.140 Apr 27 04:15:09 me [17195206.692000] free:67 slab:1762 mapped:863 pagetables:63 bounce:0

Apr 30 19:04:48 10.35.41.140 Apr 27 04:15:09 me [17195206.692000] DMA free:268kB min:708kB low:884kB high:1060kB active:11016kB inactive:7720kB present:31496kB pages_scanned:0 all_unreclaimable? no

Apr 30 19:04:48 10.35.41.140 Apr 27 04:15:09 me [17195206.692000] lowmem_reserve[]: 0 0 0

Apr 30 19:04:48 10.35.41.140 Apr 27 04:15:09 me [17195206.692000] DMA: 1*4kB 1*8kB 0*16kB 0*32kB 0*64kB 0*128kB 1*256kB 0*512kB 0*1024kB 0*2048kB 0*4096kB = 268kB

Apr 30 19:04:48 10.35.41.140 Apr 27 04:15:09 me [17195206.692000] 2996 total pagecache pages

Apr 30 19:04:48 10.35.41.140 Apr 27 04:15:09 me [17195206.692000] Swap cache: add 0, delete 0, find 0/0

Apr 30 19:04:48 10.35.41.140 Apr 27 04:15:09 me [17195206.692000] Free swap = 0kB

Apr 30 19:04:48 10.35.41.140 Apr 27 04:15:09 me [17195206.692000] Total swap = 0kB

Apr 30 19:04:48 10.35.41.140 Apr 27 04:15:09 me [17195206.692000] 7936 pages of RAM

Apr 30 19:04:48 10.35.41.140 Apr 27 04:15:09 me [17195206.692000] 256 free pages

Apr 30 19:04:48 10.35.41.140 Apr 27 04:15:09 me [17195206.692000] 839 reserved pages

Apr 30 19:04:48 10.35.41.140 Apr 27 04:15:09 me [17195206.692000] 1762 slab pages

Apr 30 19:04:48 10.35.41.140 Apr 27 04:15:09 me [17195206.692000] 3124 pages shared

Apr 30 19:04:48 10.35.41.140 Apr 27 04:15:09 me [17195206.692000] 0 pages swap cached

Apr 30 19:04:48 10.35.41.140 Apr 27 04:15:09 me [17195206.696000] ip3912: Allocating new skb in RX IRQ handler failed, losing one DMA descriptor for eth0!

Apr 30 19:04:48 10.35.41.140 Apr 27 04:15:09 me [17195206.700000] ksoftirqd/0: page allocation failure. order:0, mode:0x20

Apr 30 19:04:48 10.35.41.140 Apr 27 04:15:09 me [17195206.700000] [] (dump_stack+0x0/0x14) from [] (__alloc_pages_internal+0x3bc/0x3d4)

Apr 30 19:04:48 10.35.41.140 Apr 27 04:15:09 me [17195206.700000] [] (__alloc_pages_internal+0x0/0x3d4) from [] (__alloc_pages+0x14/0x18)

Apr 30 19:04:49 10.35.41.140 Apr 27 04:15:09 me [17195206.700000] [] (__alloc_pages+0x0/0x18) from [] (cache_alloc_refill+0x2e8/0x5a8)

Apr 30 19:04:49 10.35.41.140 Apr 27 04:15:09 me [17195206.700000] [] (cache_alloc_refill+0x0/0x5a8) from [] (__kmalloc+0x94/0xd0)

Apr 30 19:04:49 10.35.41.140 Apr 27 04:15:09 me [17195206.700000] [] (__kmalloc+0x0/0xd0) from [] (__alloc_skb+0x54/0xf8)

Apr 30 19:04:49 10.35.41.140 Apr 27 04:15:09 me [17195206.700000] r7:00000620 r6:b5807d40 r5:00000614 r4:b47a6560

Apr 30 19:04:49 10.35.41.140 Apr 27 04:15:09 me [17195206.700000] [] (__alloc_skb+0x0/0xf8) from [] (dev_alloc_skb+0x20/0x44)

Apr 30 19:04:49 10.35.41.140 Apr 27 04:15:09 me [17195206.700000] r8:b5850000 r7:0000001f r6:b5850360 r5:b5850360 r4:b501a800

Apr 30 19:04:49 10.35.41.140 Apr 27 04:15:09 me [17195206.700000] [] (dev_alloc_skb+0x0/0x44) from [] (ip3912_alloc_new_rx_skb+0x28/0x120)

Apr 30 19:04:49 10.35.41.140 Apr 27 04:15:09 me [17195206.700000] [] (ip3912_alloc_new_rx_skb+0x0/0x120) from [] (ip3912_poll+0x230/0x4b4)

Apr 30 19:04:49 10.35.41.140 Apr 27 04:15:09 me [17195206.700000] [] (ip3912_poll+0x0/0x4b4) from [] (net_rx_action+0x74/0x1ac)

Apr 30 19:04:49 10.35.41.140 Apr 27 04:15:09 me [17195206.700000] [] (net_rx_action+0x0/0x1ac) from [] (__do_softirq+0x74/0xf4)

Apr 30 19:04:49 10.35.41.140 Apr 27 04:15:09 me [17195206.700000] [] (__do_softirq+0x0/0xf4) from [] (do_softirq+0x4c/0x58)

Apr 30 19:04:49 10.35.41.140 Apr 27 04:15:09 me [17195206.700000] r6:00000000 r5:00000000 r4:60000013

Apr 30 19:04:49 10.35.41.140 Apr 27 04:15:09 me [17195206.700000] [] (do_softirq+0x0/0x58) from [] (ksoftirqd+0x7c/0x150)

Apr 30 19:04:49 10.35.41.140 Apr 27 04:15:09 me [17195206.700000] r4:b581e000

Apr 30 19:04:49 10.35.41.140 Apr 27 04:15:09 me [17195206.700000] [] (ksoftirqd+0x0/0x150) from [] (kthread+0x5c/0x94)

Apr 30 19:04:49 10.35.41.140 Apr 27 04:15:09 me [17195206.700000] r6:b403dc5c r5:00000000 r4:b581e000

Apr 30 19:04:49 10.35.41.140 Apr 27 04:15:09 me [17195206.700000] [] (kthread+0x0/0x94) from [] (do_exit+0x0/0x694)

Apr 30 19:04:49 10.35.41.140 Apr 27 04:15:09 me [17195206.700000] r6:00000000 r5:00000000 r4:00000000

Apr 30 19:04:49 10.35.41.140 Apr 27 04:15:09 me [17195206.700000] Mem-info:

Apr 30 19:04:49 10.35.41.140 Apr 27 04:15:09 me [17195206.700000] DMA per-cpu:

Apr 30 19:04:49 10.35.41.140 Apr 27 04:15:09 me [17195206.700000] CPU 0: hi: 0, btch: 1 usd: 0

Apr 30 19:04:49 10.35.41.140 Apr 27 04:15:09 me [17195206.700000] Active:2754 inactive:1930 dirty:0 writeback:0 unstable:0

Apr 30 19:04:49 10.35.41.140 Apr 27 04:15:09 me [17195206.700000] free:67 slab:1762 mapped:863 pagetables:63 bounce:0

Apr 30 19:04:50 10.35.41.140 Apr 27 04:15:09 me [17195206.700000] DMA free:268kB min:708kB low:884kB high:1060kB active:11016kB inactive:7720kB present:31496kB pages_scanned:0 all_unreclaimable? no

Apr 30 19:04:50 10.35.41.140 Apr 27 04:15:09 me [17195206.708000] [] (__alloc_skb+0x0/0xf8) from [] (dev_alloc_skb+0x20/0x44)

Apr 30 19:04:50 10.35.41.140 Apr 27 04:15:09 me [17195206.708000] 256 free pages

Apr 30 19:04:50 10.35.41.140 Apr 27 04:15:09 me [17195206.724000] [] (__alloc_skb+0x0/0xf8) from [] (dev_alloc_skb+0x20/0x44)