2702i AP 802.11a radio admin up but operatively DOWN

[mqontt]

Hey Guys,

i noticed that one of my APs (2702i with 8.10.185) in a remote branch office decided not to serve any 5GHz clients.

Upon some investigation i found out that its 802.11a radio is admin UP but oper DOWN. AP is powered via PoE and has full power, so that is not the issue. Radio configuration is fine, 802.11a is enabled

When i checked the AP itself it seems like it has some troubles with the 802.11a radio

See details:

Last radio RC(21)
Radio resets - total:41750146 retries:0 failed:0
Reset Stats: Start Cnt: 944106, Recovery: Cnt 0, Last Ret: 1000, Fails: 0, Recvry Status: Stalled NO, In Prog NO
Normal:
Code Count State When
10 4 STOP *20:08:16.475 UTC Mon Oct 9 2023
21 3706 STOP *20:22:12.987 UTC Mon Oct 9 2023
27 1 STOP *20:08:05.007 UTC Mon Oct 9 2023
39 2 START *00:00:47.199 UTC Mon Mar 1 1993
56 1 STOP *10:08:49.055 UTC Wed Sep 27 2023
Prefetch: Trigger Active: 0 Recovery Disabled?: 0
Prefetch: Stats updated every 10 seconds. Time since trigger:0
Prefetch: Trigger cnt: 0
Prefetch Limits: Legacy - 500 11ac - 10000 300000

The thing here that stands out is "21 3706 STOP *20:22:12.987 UTC Mon Oct 9 2023" it says that it recoreded 3706 radio resets because of the code (21)

I tried to search cisco documentation about what that code means, but im not sure whether im looking at "expected radio" code or "unexpected" code (https://www.cisco.com/c/en/us/td/docs/wireless/access_point/tech-notes/ap-radio-reset-codes.html#_Toc49248884)

My theory is that the AP is bust, or perhaps just the 5GHz radio chain. but i am open to any suggestions

Thx!

[Leo Laohoo]

Post the complete output to the following commands: 

1.  WLC:  sh sysinfo;
2.  AP:  sh version

[mqontt]

Manufacturer's Name.............................. Cisco Systems Inc.
Product Name..................................... Cisco Controller
Product Version.................................. 8.10.185.0
RTOS Version..................................... 8.10.185.0
Bootloader Version............................... 8.1.102.0
Emergency Image Version.......................... 8.1.102.0

OUI File Last Update Time........................ Tue Feb 06 10:44:07 UTC 2018

Build Type....................................... DATA + WPS

System Name...................................... W01
System Location..................................
System Contact...................................
System ObjectID..................................
Redundancy Mode.................................. SSO
IP Address....................................... 10.x.x.x
IPv6 Address..................................... ::
System Up Time................................... 39 days 16 hrs 1 mins 39 secs
System Timezone Location......................... (GMT +1:00) Amsterdam, Berlin, Rome, Vienna
System Stats Realtime Interval................... 5
System Stats Normal Interval..................... 180

Configured Country............................... SK - Slovak Republic
Operating Environment............................ Commercial (10 to 35 C)
Internal Temp Alarm Limits....................... 10 to 38 C
Internal Temperature............................. +21 C
Fan Status....................................... OK

State of 802.11b Network......................... Enabled
State of 802.11a Network......................... Enabled
Number of WLANs.................................. 14
Number of Active Clients......................... 2224

OUI Classification Failure Count................. 627373

Memory Current Usage............................. 12
Memory Average Usage............................. 12
CPU Current Usage................................ 0
CPU Average Usage................................ 1

Flash Type....................................... Compact Flash Card
Flash Size....................................... 1073741824

Burned-in MAC Address............................
Power Supply 1................................... Present, OK
Power Supply 2................................... Present, OK
Maximum number of APs supported.................. 1500
System Nas-Id.................................... Cisco_ec:cb:00
WLC MIC Certificate Types........................ SHA1/SHA2
Licensing Type................................... RTU

----

cisco AIR-CAP2702I-E-K9 (PowerPC) processor (revision A0) with 376814K/134656K bytes of memory.
Processor board ID FCW2218N9SS
PowerPC CPU at 800Mhz, revision number 0x2151
Last reset from power-on
LWAPP image version 8.10.185.0
1 Gigabit Ethernet interface
2 802.11 Radios

[Leo Laohoo]

I want to see the complete output to the command "sh version".

[mqontt]

Cisco IOS Software, C2700 Software (AP3G2-K9W8-M), Version 15.3(3)JK9, RELEASE SOFTWARE (fc1)
Technical Support: http://www.cisco.com/techsupport
Copyright (c) 1986-2023 by Cisco Systems, Inc.
Compiled Fri 24-Mar-23 01:54 by mcpre

ROM: Bootstrap program is C2700 boot loader
BOOTLDR: C2700 Boot Loader (AP3G2-BOOT-M) LoaderVersion 15.2(4)JB5m, RELEASE SOFTWARE (fc2)

S9LC01-A01 uptime is 13 hours, 51 minutes
System returned to ROM by power-on
System image file is "flash:/ap3g2-k9w8-mx.153-3.JK9/ap3g2-k9w8-xx.153-3.JK9"
Last reload reason: Reset Request from Controller.

This product contains cryptographic features and is subject to United
States and local country laws governing import, export, transfer and
use. Delivery of Cisco cryptographic products does not imply
third-party authority to import, export, distribute or use encryption.
Importers, exporters, distributors and users are responsible for
compliance with U.S. and local country laws. By using this product you
agree to comply with applicable laws and regulations. If you are unable
to comply with U.S. and local laws, return this product immediately.

A summary of U.S. laws governing Cisco cryptographic products may be found at:
http://www.cisco.com/wwl/export/crypto/tool/stqrg.html

If you require further assistance please contact us by sending email to
export@cisco.com.

cisco AIR-CAP2702I-E-K9 (PowerPC) processor (revision A0) with 376814K/134656K bytes of memory.
Processor board ID #
PowerPC CPU at 800Mhz, revision number 0x2151
Last reset from power-on
LWAPP image version 8.10.185.0
1 Gigabit Ethernet interface
2 802.11 Radios

32K bytes of flash-simulated non-volatile configuration memory.
Base ethernet MAC Address: 70:69:5A:B7:XX:XX
Part Number : 73-15824-04
PCB Serial Number : #
Top Assembly Part Number : 068-100379-02
Top Assembly Serial Number : #
Top Revision Number : C0
Product/Model Number : AIR-CAP2702I-E-K9

[marce1000]

   - Reboot the particular AP and just after boot , connect to it and issue the command show logging ; watch for errors related to the radios if any. 
     According to the control model being used also look into : https://www.cisco.com/c/en/us/support/docs/wireless/wireless-lan-controller-software/200046-tac-recommended-aireos.html

 M.

[mqontt]

thx.

i tried to reboot and 5GHz radio came up, but i expect it to go down within few days as it has before. Will check the logs from the AP to see if there is anything useful.

With regards to the new AirOS version:

The AP is wave1 and it is not using DFS channels (using only UNII-1 channels), and if i read the patch notes correctly the fixed image is only to allow APs to detect the radar to be in line with ETSI and vacate the channel.

So on the current image they might not detect the radar signal even if its there and keep using the DFS channel which would be against regulations. However i think this would cause regulatory issues mostly for the outdoor deployment rather than indoor.

[marce1000]

                >....Will check the logs from the AP to see if there is anything useful.
 - Note  that you need to do this just after reboot in order to get clues about the original problem , it's kind of online checking of the logs just after the reboot of the AP

 M.

[Leo Laohoo]

Remote into the AP and post the complete output to the command "dir".

[mqontt]

Directory of flash:/

2 -rwx 282 Jan 1 1970 00:10:48 +00:00 info
3 -rwx 367 Oct 9 2023 20:36:19 +00:00 capwap-saved-config
4 -rwx 68871 Oct 9 2023 20:33:42 +00:00 event.log
5 -rwx 0 Mar 1 1993 00:00:35 +00:00 config.txt
38 drwx 576 Jan 1 1970 00:10:47 +00:00 ap3g2-rcvk9w8-mx
6 drwx 0 Mar 1 1993 00:01:03 +00:00 configs
10 drwx 2368 Jun 20 2023 22:12:22 +00:00 ap3g2-k9w8-mx.153-3.JK9
80 -rwx 64 Oct 9 2023 20:33:35 +00:00 sensord_CSPRNG0
81 -rwx 130211 Sep 27 2023 10:06:01 +00:00 event.r1
82 -rwx 367 Oct 9 2023 20:41:36 +00:00 capwap-saved-config-bak
83 -rwx 130035 May 25 2023 09:27:46 +00:00 event.r0
84 -rwx 64 Oct 9 2023 20:33:35 +00:00 sensord_CSPRNG1
8 -rwx 12312 Oct 9 2023 20:41:36 +00:00 private-multiple-fs
19 -rwx 417 Oct 10 2023 10:36:46 +00:00 env_vars

BTW the radio came up after reboot, but as far as i am aware it will go down eventually again within few days. At least thats what was reported from customer.

So far the 5GHz radio is UP and im continuously monitoring the AP, and checking when it comes down to see the logs on the AP if there are any clues 

EDIT:

i checked the event.r1 from 27.9.2023  and noticed some errors there (could be that this was the log from when the radio went down)

Sep 27 10:05:53.951: %DOT11-2-FAILURE_RADIO_RESET: Interface Dot11Radio1, reset due to Radio Interface Bus Down(34)
Sep 27 10:05:53.951: %EVT-5-NTC: Radio d1 RST 34 Flags 202010D BCN 1

[Leo Laohoo]

---

@mqontt wrote:

Sep 27 10:05:53.951: %DOT11-2-FAILURE_RADIO_RESET: Interface Dot11Radio1, reset due to Radio Interface Bus Down(34)
Sep 27 10:05:53.951: %EVT-5-NTC: Radio d1 RST 34 Flags 202010D BCN 1

---

RMA the AP.  

The 2700/3700 has a well-known track record for the 5.0 Ghz to completely fail. 

I have recently RMAed seven (7) nine (9) 3700 because their 5.0 Ghz has failed.  I am about to RMA another one where both the 2.4 Ghz and 5.0 Ghz are dead.