C9115AXI Reboot issue

[JeongSeok_Kim]

Hi

4 out of 6 wireless APs connected to a specific switch have an unknown reboot problem.

WLC Model: C9800-L-F-K9

WLC OS: 17.6.3

Uplink Type: 1G UTP

AP Model: C9115AXI

AP OS: 17.6.3

Uplink Speed: 1G UTP

Power: AIR-PWRINJ6

Switch Model: C2960L-24TS

Switch OS: 15.2(6)E

The Max Retransmission or Heartbeat Timeout on the WLC identifies the reason for disconnection, and the reboot reason is Unknown on the wireless AP itself. This phenomenon occurs repeatedly for as short as 20 minutes and as long as 4 hours.

There is no clear cause analysis result in Cisco TAC, and the action method for me to solve it myself is as follows.

1. Downgrade WLC OS 17.6.4 -> 17.6.3 : Unresolved

2. Replacing all 4 problematic wireless APs : Only 1 wireless AP problem is resolved

3. Wireless AP power injector and switch port replacement : Unresolved

4. Syslog setting in wireless AP: Wireless AP reboots without sending any specific log to syslog

5. The demolished wireless AP, which had an existing problem, was connected to the computer room for testing purposes : the reboot problem was not reproduced.

Has anyone experienced and resolved these symptoms?

[Polestar]

AP is currently :

cisco C9115AXI-E ARMv8 Processor rev 0 (v8l) with 1971660/1185988K bytes of memory.
Processor board ID FCW2614YF7H
AP Running Image : 17.3.5.42
Primary Boot Image : 17.3.5.42
Backup Boot Image : 8.10.130.0

1 Multigigabit Ethernet interfaces
2 802.11 Radios
Radio Driver version : Broadcom BCA: 17.18 RC2.0
Radio FW version : 24203.122.r44027 44027
NSS FW version : NA

C9300L stack :

Version 16.12.4, RELEASE SOFTWARE (fc5)

ROM: IOS-XE ROMMON
BOOTLDR: System Bootstrap, Version 17.8.1r[FC2], RELEASE SOFTWARE (P)

[Rich R]

Seriously consider getting the WLC and switch software updated (WLC is oldish, switch is antique).

The problem with running old versions like that is that even if they identify a bug you're not going to get a fix unless you migrate to a new version anyway and moving to new version resolves hundreds of existing bugs - one of which might be causing you problems.  It's much easier for TAC to engage BU on problems if the software is up to date to start with and at least you know you've eliminated those known bugs which have already been fixed.

[JeongSeok_Kim]

Hi Polestar

TAC did not provide any cause analysis, and it is suspected to be an electrical problem when judged by itself.
If possible, it would be nice if you could open the TAC case separately and proceed with EFA for switches and wireless APs.

[Polestar]

Just a quick update on this story:
We upgraded our WLC to 17.9.3 which seems to have fixed the issue for many of our C9115-AXI-E APs.

There is still 1 ( only 1 ) that seems to reload at random times and i cannot see anything / reason cause in the console.

Power levels ok / no other upstream issues although i have to say the upstream switch  C9300L is running on 16.12.4

All other sites / remote locations are running fine. Any ideas why the console output could be suppressed ? Other than last reload " unkown"

[stuart.pannell]

Hi all, we have been banging our heads over this issue for some time, we have opened 3 TAC cases and upgraded to the latest software at least 3 times. We are currently running 17.9.4a code on our WLC9800-40, 17.9.4a code on our 9200L switches and latest APSP 17.9.4.201. Yet we still have APs on multiple sites randomly rebooting with the error MAX RETRIES. If we move the AP off the ceiling mount then they appear to be stable and not reboot. Could I please ask how you are insulating these APs and if that is working for you?

[JeongSeok_Kim]

Hi stuart.pannell

When using a PoE switch to pinpoint electrical problems:

1. Redundant the PoE switch power modules.

2. Unplug and reconnect the main power cord of the PoE switch power module and check whether the wireless AP goes down. It would be even better if you could connect the console to the wireless AP, and if it is an electrical problem, a reboot will occur without any logs.

Insulation methods vary depending on the installation method.

1. When the problem occurred when using the rail bracket, I solved it by embedding it in the ceiling (placing it on top of the texture).

2. When using brackets and pieces: 1. Wrap the screw pieces with heat shrink tubing. or 2. Wrap the screw piece with insulating tape.

I utilized number 2 and that method does not guarantee 100% insulation.

[stuart.pannell]

We have moved 20 APs so far from the ceiling rail to the top of the tiles, all 20 are now stable. We have about 150 out of 535 that are flapping across numerous sites. 1 site had 5 APs out of 10 dropping out, we moved the 5 APs to the tiles and they are now stable however another 1 of the APs has begun flapping as if it has taken over the earthing process. We will be going back to site and relocate this one.