WAP 371 - Segfault

thetatwalter · ‎06-01-2015

I have a WAP371 in which hostapd has crashed as well as the swapper process. The antenna (1 of four we have) is very inconsistent in maintaining its connection. I sit about twenty feet away with only gyprock walls between us. What counters should I be inspecting to try and diagnose the problem?

From the log:

crashdumps:

crashdump1:
Software Version: 1.1.2.3
Crash Log:
Kernel Uptime of the AP: 3 Days, 3 Hours, 18 Minutes and 17 Seconds
Page Table Data:
   pgd = c0004000
   *pgd=00000000
Registers:
   pc : bf0c2dcc lr : bf2523ac psr: 60000113
   sp : c7843c88 ip : bf25223c fp : c52f0000
   r10: 00000004 r9 : c52f0000 r8 : 00000016
   r7 : 00000000 r6 : c70b0b40 r5 : c4f10000 r4 : c4454540
   r3 : 00000000 r2 : 40000008 r1 : c4454540 r0 : 00000000
   Flags: nZCv IRQs on FIQs on

do_page_fault() #2: sending SIGSEGV to process - swapper
pc = bf0c2dcc, lr = bf2523ac

Stack Trace:
[<bf0c2dcc>] (wlc_sdu_data+0x14/0x24 [wl]) from [<bf2523ac>] (wlc_ampdu_agg+0x170/0x69c [wl])

crashdump2:
Software Version: 1.1.2.3
Crash Log:
Kernel Uptime of the AP: 14 Days, 16 Hours, 14 Minutes and 37 Seconds
Page Table Data:
   pgd = c4094000
   *pgd=0408d031
   *pte=00000000 *ppte=00000000

Registers:
   pc : bf0c6950 lr : c6d88174 psr: 80000113
   sp : c7195610 ip : 00000010 fp : 00000000
   r10: 00000000 r9 : c73380e0 r8 : 00000000
   r7 : c7b8bea0 r6 : c3a52000 r5 : c73380e0 r4 : c4f0e000
   r3 : 00001ab4 r2 : c4b292a0 r1 : c4b29260 r0 : c7b8bea0
   Flags: Nzcv IRQs on FIQs on

do_page_fault() #2: sending SIGSEGV to process - hostapd
pc = bf0c6950, lr = c6d88174

Stack Trace:
[<bf0c6950>] (wlc_d11hdrs+0x16e4/0x3694 [wl]) from [<bf0ca550>] (wlc_prep_pdu+0x110/0x274 [wl])
[<bf0ca550>] (wlc_prep_pdu+0x110/0x274 [wl]) from [<bf24a6b8>] (wlc_send_q+0xa0/0x338 [wl])

Eric Moyers · ‎06-15-2015

My name Eric Moyers. I am an Engineer in the Small Business Support Center.

I am sorry to hear that you are experiencing this issue. There are many things that could be effecting your connection. Could you provide your complete diagnostic file? Also How many gyprock walls are between you and the WAP371? gyprock walls are typically used for blocking sound waves. Because of the material it is also good at blocking other things, like wireless.

Have you done a wireless scan like Wifi Analytics- free for download and use. (provides a lot of good info).

Eric Moyers
.:|:.:|:. CISCO | Cisco Presales Technical Support | Wireless Subject Matter Expert

Please rate helpful Posts and Let others know when your Question has been answered.

thetatwalter · ‎06-15-2015

Hi Eric,

I am sorry but I fail to see how the quality of the walls would cause a process to segfault. I have four of these antennas and after being away for the weekend, three of them had to be rebooted this morning. This has been occurring since they were purchased. I have attached a diagnostic from one of the WAPs.

Best regards,

Todd

Eric Moyers · ‎06-15-2015

I wanted the diagnostic to look for anything that may have caused the segfault. My comment about the gyprock was in reference to you mentioning about the referring to your inconsistent connections.

However since your diagnostic was only 2 minutes old, there was really nothing there to see. With the complexity of this issue, it may be quicker to call in and open a case with one of our engineers.

http://www.cisco.com/c/en/us/support/web/tsd-cisco-small-business-support-center-contacts.html

Eric Moyers
.:|:.:|:. CISCO | Cisco Presales Technical Support | Wireless Subject Matter Expert

Please rate helpful Posts and Let others know when your Question has been answered.

thetatwalter · ‎06-15-2015

I rebooted all the WAPs recently. Attached is the oldest at 6 hrs.

Eric Moyers · ‎06-16-2015

After looking at your diagnostic file, I am noticing a couple of things.

1) Under Network ports:

Network Ports:
eth0 RX packets:113212312 errors:1 dropped:60991572 overruns:1 frame:1

wlan0 TX packets:28989181 errors:83185853 dropped:0 overruns:0 carrier:0

wlan1 TX packets:35414475 errors:76614178 dropped:0 overruns:0 carrier:0

wlan1vap2 TX packets:174671 errors:63282 dropped:0 overruns:0 carrier:0

Also under wlan0 wme_counters and wlan1 wme_counter (which is for 5 Ghz and2.4 GHz QoS under Wireless tab) I am seeing massive errors under AC_BE: Which is for Best Effort.

wlan0 wme_counters:
AC_BE: tx frames: 28991971 bytes: -1330146327 failed frames: 199158 failed bytes: 28881951
rx frames: 680 bytes: 179725 failed frames: 2286 failed bytes: 63269 foward frames: 0 bytes: 0
tx frames time expired: 0

wlan1 wme_counters:
AC_BE: tx frames: 35593691 bytes: -587140034 failed frames: 112736745 failed bytes: 1583315651
rx frames: 47737 bytes: 6793359 failed frames: 17361 failed bytes: 470883 foward frames: 0 bytes: 0
tx frames time expired: 0

Is this the Diagnostic from the one giving the Segfault or one of the other ones?

Do the others have similar errors? How long have you had your WAP371s?

Eric Moyers
.:|:.:|:. CISCO | Cisco Presales Technical Support | Wireless Subject Matter Expert

Please rate helpful Posts and Let others know when your Question has been answered.

thetatwalter · ‎06-16-2015

This is the diagnostic from one of the antennas. They all experience segfaults. I have had the WAPs for about two months I would say.

Thanks,

Todd

Eric Moyers · ‎06-16-2015

I would call in and open a case with one of our Engineers. More troubleshooting than we can do here is needed. If an RMA is determined as necessary then you will already have the case opened.

http://www.cisco.com/c/en/us/support/web/tsd-cisco-small-business-support-center-contacts.html

Eric Moyers

thetatwalter · ‎10-07-2015

Caught one of the WAPs in the process of crashing. Perhaps this log file will be more revealing.

Eric Moyers · ‎10-07-2015

Hello SIr

THat you for the diagnostic file. I did seem some unusual things in this one. Before making any conclusion, may I ask for a network diagram of your network, showing the router and anything connected between it and the WAP371.

Specifically I am looking for how the WAP371 is connected back to the network. Is it directly hardwired into the network to the router, a switch, connected to something wirelessly, etc. The issue seems to be with that communication but I need to understand how everything is hooked up.

Thanks

Eric Moyers

thetatwalter · ‎10-07-2015

WAP371---Brocade 6430 (PoE+ switch)---(Brocade 6450)--10Gbps backbone.

GigabitEthernet1/1/5 is up, line protocol is up
Hardware is GigabitEthernet, address is cc4e.245e.a644 (bia cc4e.245e.a644)
Configured speed auto, actual 1Gbit, configured duplex fdx, actual fdx
Configured mdi mode AUTO, actual MDIX
Member of 2 L2 VLANs, port is dual mode in Vlan 300, port state is FORWARDING
BPDU guard is Disabled, ROOT protect is Disabled
Link Error Dampening is Disabled
STP configured to ON, priority is level0, mac-learning is enabled
Flow Control is config enabled, oper enabled, negotiation disabled
Mirror disabled, Monitor disabled
Not member of any active trunks
Not member of any configured trunks
No port name
Inter-Packet Gap (IPG) is 96 bit times
MTU 10240 bytes
300 second input rate: 89432 bits/sec, 16 packets/sec, 0.00% utilization
300 second output rate: 71224 bits/sec, 27 packets/sec, 0.00% utilization
19024818 packets input, 3818006237 bytes, 0 no buffer
Received 991946 broadcasts, 1935313 multicasts, 16097559 unicasts
0 input errors, 0 CRC, 0 frame, 0 ignored
0 runts, 0 giants
2280222962 packets output, 243018952189 bytes, 0 underruns
Transmitted 30908383 broadcasts, 2221592230 multicasts, 27722349 unicasts
0 output errors, 0 collisions
Relay Agent Information option: Disabled

Inline Power:

Port Admin Oper ---Power(mWatts)--- PD Type PD Class Pri Fault/
State State Consumed Allocated Error
--------------------------------------------------------------------------

1/1/5 On On 10610 17705 802.3at Class 4 3 n/a