Re: AP1852 not joined to controller

TimurKaneev · ‎07-11-2021

Hello.

I have the problem with AP1852, i tried to joned the ap to WLC 2504 with 8.5.171 software version .

On AP logs i saw fallow:

[*07/11/2021 00:08:27.4705] ethernet_port wired0, ip 10.15.1.6, netmask 255.255.255.0, gw 10.15.1.1, mtu 1500, bcast 10.15.1.255, dns1 0.0.0.0, vid 0, static_ip_failover false, dhcp_vlan_failover false
[*07/11/2021 00:08:38.2671] DOT11_CFG[0] Radio Mode is changed from Local to Local
[*07/11/2021 00:08:38.2671] DOT11_CFG[1] Radio Mode is changed from Local to Local
[*07/11/2021 00:08:38.3171] AP IPv4 Address updated from 0.0.0.0 to 10.15.1.6
[*07/11/2021 00:08:38.3371] dtls_init: Use MIC certificate
[*07/11/2021 00:08:38.3371]
[*07/11/2021 00:08:38.3371] CAPWAP State: Init
[*07/11/2021 00:08:38.3371]
[*07/11/2021 00:08:38.3371] Config not found, PNP is required, Starting PNP
[*07/11/2021 00:08:38.3371]
[*07/11/2021 00:08:44.8251] PNP:Server not reachable, Start CAPWAP Discovery
[*07/11/2021 00:08:44.8251]
[*07/11/2021 00:08:44.8251] CAPWAP State: Discovery
[*07/11/2021 00:08:44.8351] Got WLC address 10.5.1.11 from DHCP.
[*07/11/2021 00:08:44.8351] Got WLC address 10.5.1.11 from DHCP.
[*07/11/2021 00:08:44.8351] Discovery Request sent to 10.5.1.11, discovery type DHCP(2)
[*07/11/2021 00:08:44.8351] Discovery Request sent to 255.255.255.255, discovery type UNKNOWN(0)
[*07/11/2021 00:08:44.8351]
[*07/11/2021 00:08:44.8351] CAPWAP State: Discovery
[*07/11/2021 00:08:44.8451] Discovery Response from 10.15.1.11
[*07/11/2021 00:19:18.0000] can't find the Ip from discoveryRequest array
[*07/11/2021 00:19:18.0000]
[*07/11/2021 00:19:18.0000]
[*07/11/2021 00:19:18.0000] CAPWAP State: DTLS Setup
[*07/11/2021 00:19:18.3799]
[*07/11/2021 00:19:18.3799] CAPWAP State: Join
[*07/11/2021 00:19:18.3799] Sending Join request to 10.15.1.11 through port 5248
[*07/11/2021 00:19:18.3799] Join Response from 10.15.1.11
[*07/11/2021 00:19:18.4698] nss_capwapmgr_enable_tunnel[1661]:ef30e800: tunnel 0 is already enabled
[*07/11/2021 00:19:18.4898]
[*07/11/2021 00:19:18.4898] CAPWAP State: Image Data
[*07/11/2021 00:19:18.5398] do NO_UPGRADE, part2 is active part
[*07/11/2021 00:19:18.5398]
[*07/11/2021 00:19:18.5398] CAPWAP State: Configure
[*07/11/2021 00:19:18.5398] DOT11_CFG[0] Radio Mode is changed from Local to Local
[*07/11/2021 00:19:18.5398] DOT11_CFG[1] Radio Mode is changed from Local to Local
[*07/11/2021 00:19:18.6198] DOT11_DRV[0]: Start Radio0
[*07/11/2021 00:19:18.6398] DOT11_DRV[0]: Stop Radio0
[*07/11/2021 00:19:19.0797] osif_vap_stop:Timeout for VAP-16 start response event
[*07/11/2021 00:19:19.0797] DOT11_DRV[0]: Start Radio0
[*07/11/2021 00:19:19.0797] _ieee80211_resmgr_vap_start line 151 Warning: VAP-16 Init in progress!
[*07/11/2021 00:19:19.0897] DOT11_DRV[1]: Start Radio1
[*07/11/2021 00:19:19.0997] DOT11_DRV[1]: Stop Radio1
[*07/11/2021 00:19:19.3196] DOT11_DRV[1]: Start Radio1
[*07/11/2021 00:19:19.3196] DOT11_DRV[1]: Stop Radio1
[*07/11/2021 00:19:19.3196] DOT11_DRV[1]: Start Radio1
[*07/11/2021 00:19:19.6295] Discarding msg CAPWAP_WTP_EVENT_REQUEST(type 9) in CAPWAP state: Configure(8).
[*07/11/2021 00:19:19.6295] Discarding msg CAPWAP_WTP_EVENT_REQUEST(type 9) in CAPWAP state: Configure(8).
[*07/11/2021 00:19:19.6295] Discarding msg CAPWAP_WTP_EVENT_REQUEST(type 9) in CAPWAP state: Configure(8).
[*07/11/2021 00:19:19.6295] Discarding msg CAPWAP_WTP_EVENT_REQUEST(type 9) in CAPWAP state: Configure(8).
[*07/11/2021 00:19:19.6295] Discarding msg CAPWAP_WTP_EVENT_REQUEST(type 9) in CAPWAP state: Configure(8).
[*07/11/2021 00:19:19.6295] Discarding msg CAPWAP_WTP_EVENT_REQUEST(type 9) in CAPWAP state: Configure(8).
[*07/11/2021 00:19:19.6495] DOT11_DRV[0]: Stop Radio0
[*07/11/2021 00:19:20.0893] osif_vap_stop:Timeout for VAP-16 start response event
[*07/11/2021 00:19:20.0893] DOT11_DRV[0]: Start Radio0
[*07/11/2021 00:19:20.0893] _ieee80211_resmgr_vap_start line 151 Warning: VAP-16 Init in progress!
[*07/11/2021 00:19:20.0993] DOT11_DRV[1]: Stop Radio1
[*07/11/2021 00:19:20.0993] DOT11_DRV[1]: Start Radio1
[*07/11/2021 00:19:20.0993] DOT11_DRV[1]: Stop Radio1
[*07/11/2021 00:19:20.1093] DOT11_DRV[1]: Start Radio1
[*07/11/2021 00:19:20.1693]
[*07/11/2021 00:19:20.1693] CAPWAP State: Run
[*07/11/2021 00:19:20.3193] AP has joined controller WLC2504
[*07/11/2021 00:19:20.3693] _ieee80211_resmgr_vap_start line 151 Warning: VAP-16 Init in progress!
[*07/11/2021 00:19:20.3693] DOT11_DRV[0]: Stop Radio0
[*07/11/2021 00:19:20.8091] osif_vap_stop:Timeout for VAP-16 start response event
[*07/11/2021 00:19:20.8091] DOT11_DRV[0]: Start Radio0
[*07/11/2021 00:19:20.8091] _ieee80211_resmgr_vap_start line 151 Warning: VAP-16 Init in progress!
[*07/11/2021 00:19:24.0681] osif_vap_open vap 1 failed, iv_node_count=1, bss_node_ref_count=2
[*07/11/2021 00:19:24.0681] DOT11_DRV[0]:
[*07/11/2021 00:19:24.0681]
[*07/11/2021 00:19:24.0681] ***** FAILED TO START VAP[1] apr0v1 1 try*****
[*07/11/2021 00:19:24.0681]
[*07/11/2021 00:19:24.0681] vap-1(apr0v1):osif_vap_open vap 1 starts
[*07/11/2021 00:19:24.0681] vap-1(apr0v1):start running, ifname=apr0v1, opmode=6
[*07/11/2021 00:19:33.0653] vap-1(apr0v1):OL vap_stop +
[*07/11/2021 00:19:33.0653] vap-1(apr0v1):ol_ath_vap_stopping: Warning:Stop during VAP Init !
[*07/11/2021 00:19:33.0653] vap-1(apr0v1):OL vap_stop -
[*07/11/2021 00:19:33.0653] vap-1(apr0v1):ieee80211_state_event: VAP state event 0, cur_state=0, vap_deleted_is_set=0
[*07/11/2021 00:19:33.0653] vap-1(apr0v1):ieee80211_state_event: VAP state event 11, cur_state=0, vap_deleted_is_set=0
[*07/11/2021 00:19:33.0653] vap-1(apr0v1):ieee80211_state_event: VAP EVENT_NODE_FREED node count 1
[*07/11/2021 00:19:33.0953] vap-1(apr0v1):ieee80211_state_event: VAP state event 11, cur_state=0, vap_deleted_is_set=0
[*07/11/2021 00:19:33.0953] vap-1(apr0v1):ieee80211_state_event: VAP EVENT_NODE_FREED node count 1
[*07/11/2021 00:19:33.0953] vap-1(apr0v1):OL vap_start +
[*07/11/2021 00:19:33.0953] _ieee80211_resmgr_vap_start line 151 Warning: VAP-1 Init in progress!
[*07/11/2021 00:19:33.0953] vap-1(apr0v1):osif_vap_init :mlme returned error 16
[*07/11/2021 00:19:36.1043] vap-1(apr0v1):osif_vap_init : timed out waitinfor AP to come up
[*07/11/2021 00:19:36.1043] osif_vap_open vap 1 failed, iv_node_count=1, bss_node_ref_count=2
[*07/11/2021 00:19:36.1043] vap-1(apr0v1):osif_vap_open vap 1 ends, result 16
[*07/11/2021 00:19:36.1043] DOT11_DRV[0]:
[*07/11/2021 00:19:36.1043]
[*07/11/2021 00:19:36.1043] ***** FAILED TO START VAP[1] apr0v1 2 try*****
[*07/11/2021 00:19:36.1043]
[*07/11/2021 00:19:36.1043] vap-1(apr0v1):osif_vap_open vap 1 starts
[*07/11/2021 00:19:36.1043] vap-1(apr0v1):start running, ifname=apr0v1, opmode=6
[*07/11/2021 00:19:45.1015] vap-1(apr0v1):OL vap_stop +
[*07/11/2021 00:19:45.1015] vap-1(apr0v1):ol_ath_vap_stopping: Warning:Stop during VAP Init !
[*07/11/2021 00:19:45.1015] vap-1(apr0v1):OL vap_stop -
[*07/11/2021 00:19:45.1015] vap-1(apr0v1):ieee80211_state_event: VAP state event 0, cur_state=0, vap_deleted_is_set=0
[*07/11/2021 00:19:45.1015] vap-1(apr0v1):ieee80211_state_event: VAP state event 11, cur_state=0, vap_deleted_is_set=0
[*07/11/2021 00:19:45.1015] vap-1(apr0v1):ieee80211_state_event: VAP EVENT_NODE_FREED node count 1
[*07/11/2021 00:19:45.1315] vap-1(apr0v1):ieee80211_state_event: VAP state event 11, cur_state=0, vap_deleted_is_set=0
[*07/11/2021 00:19:45.1315] vap-1(apr0v1):ieee80211_state_event: VAP EVENT_NODE_FREED node count 1
[*07/11/2021 00:19:45.1315] vap-1(apr0v1):OL vap_start +
[*07/11/2021 00:19:45.1315] _ieee80211_resmgr_vap_start line 151 Warning: VAP-1 Init in progress!
[*07/11/2021 00:19:45.1315] vap-1(apr0v1):osif_vap_init :mlme returned error 16
[*07/11/2021 00:19:48.1306] vap-1(apr0v1):osif_vap_init : timed out waitinfor AP to come up
[*07/11/2021 00:19:48.1306] osif_vap_open vap 1 failed, iv_node_count=1, bss_node_ref_count=2
[*07/11/2021 00:19:48.1306] vap-1(apr0v1):osif_vap_open vap 1 ends, result 16
[*07/11/2021 00:19:48.1306] DOT11_DRV[0]:
[*07/11/2021 00:19:48.1306]
[*07/11/2021 00:19:48.1306] ***** FAILED TO START VAP[1] apr0v1 3 try*****
[*07/11/2021 00:19:48.1306]
[*07/11/2021 00:19:48.1306] DOT11_DRV[0]: *** Triggered FW assert for radio failure (VAP start failure)
[*07/11/2021 00:19:48.1306]
[*07/11/2021 00:19:48.1306] DOT11_DRV[0]: Stop Radio0
[*07/11/2021 00:19:48.5704] osif_vap_stop:Timeout for VAP-16 start response event
[*07/11/2021 00:19:48.5704] DOT11_DRV[0]: *** Resetting Radio 0
[*07/11/2021 00:19:48.5904] DOT11_DRV[1]: Stop Radio1
[*07/11/2021 00:19:48.6104] DOT11_DRV[1]: Start Radio1
[*07/11/2021 00:19:48.8404] Re-Tx Count=1, Max Re-Tx Value=5, SendSeqNum=40, NumofPendingMsgs=30
[*07/11/2021 00:19:48.8404]
[*07/11/2021 00:19:51.6695] Re-Tx Count=2, Max Re-Tx Value=5, SendSeqNum=45, NumofPendingMsgs=35
[*07/11/2021 00:19:51.6695]
[*07/11/2021 00:19:54.5186] Re-Tx Count=3, Max Re-Tx Value=5, SendSeqNum=45, NumofPendingMsgs=35
[*07/11/2021 00:19:54.5186]
[*07/11/2021 00:19:57.3677] Re-Tx Count=4, Max Re-Tx Value=5, SendSeqNum=45, NumofPendingMsgs=35
[*07/11/2021 00:19:57.3677]
[*07/11/2021 00:19:58.7773]
[*07/11/2021 00:19:58.7773] phy_value=0: org="0x1800" phy_reg="0x1800"
[*07/11/2021 00:20:00.2168] Re-Tx Count=5, Max Re-Tx Value=5, SendSeqNum=46, NumofPendingMsgs=36
[*07/11/2021 00:20:00.2168]
[*07/11/2021 00:20:03.0659] Re-Tx Count=6, Max Re-Tx Value=5, SendSeqNum=46, NumofPendingMsgs=36
[*07/11/2021 00:20:03.0659]
[*07/11/2021 00:20:03.0659] Max retransmission count exceeded, going back to DISCOVER mode.
[*07/11/2021 00:20:03.0659] GOING BACK TO DISCOVER MODE
[*07/11/2021 00:20:03.0659]
[*07/11/2021 00:20:03.0659] CAPWAP State: DTLS Teardown
[*07/11/2021 00:20:07.8144] DOT11_DRV[1]: Stop Radio1
[*07/11/2021 00:20:07.9444]
[*07/11/2021 00:20:07.9444] CAPWAP State: Discovery
[*07/11/2021 00:20:07.9544] Got WLC address 10.5.1.11 from DHCP.
[*07/11/2021 00:20:07.9544] Got WLC address 10.5.1.11 from DHCP.
[*07/11/2021 00:20:07.9544] Discovery Request sent to 10.15.1.11, discovery type STATIC_CONFIG(1)
[*07/11/2021 00:20:07.9544] Discovery Request sent to 10.5.1.11, discovery type DHCP(2)
[*07/11/2021 00:20:07.9544] Discovery Request sent to 255.255.255.255, discovery type UNKNOWN(0)
[*07/11/2021 00:20:07.9544] Discovery Response from 10.15.1.11
[*07/11/2021 00:20:07.9744] Discovery Response from 10.15.1.11
[*07/11/2021 00:20:08.5942] DOT11_DRV[0]:
[*07/11/2021 00:20:08.5942]
[*07/11/2021 00:20:08.5942] *** Warning: FW dump after assert didn't finish in time***
[*07/11/2021 00:20:08.5942]
[*07/11/2021 00:20:08.5942] Received reboot request event 'radio failure(radio recovery failed)'

I update the AP from TFTP and try to join again and saw that AP joined to WLC , then download software, then updated , and after update doens't join to WLC

On WLC i have fallow:

(Cisco Controller) >show sysinfo

Manufacturer's Name.............................. Cisco Systems Inc.
Product Name..................................... Cisco Controller
Product Version.................................. 8.5.171.0
Bootloader Version............................... 1.0.20
Field Recovery Image Version..................... 7.6.101.1
Firmware Version................................. PIC 16.0

OUI File Last Update Time........................ Sun Sep 07 10:44:07 IST 2014

Build Type....................................... DATA + WPS

System Name...................................... WLC2504
System Location..................................
System Contact...................................
System ObjectID.................................. 1.3.6.1.4.1.9.1.1279
IP Address....................................... 10.15.1.11
IPv6 Address..................................... ::
Last Reset....................................... Software reset
System Up Time................................... 0 days 0 hrs 22 mins 32 secs

--More-- or (q)uit
System Timezone Location.........................
System Stats Realtime Interval................... 5
System Stats Normal Interval..................... 180

Configured Country............................... RU - Russian Federation
Operating Environment............................ Commercial (0 to 40 C)
Internal Temp Alarm Limits....................... 0 to 65 C
Internal Temperature............................. +35 C
External Temperature............................. +38 C
Fan Status....................................... 4300 rpm

State of 802.11b Network......................... Enabled
State of 802.11a Network......................... Enabled
Number of WLANs.................................. 2
Number of Active Clients......................... 0

OUI Classification Failure Count................. 0

Memory Current Usage............................. 38
Memory Average Usage............................. 38
CPU Current Usage................................ 0
CPU Average Usage................................ 0

--More-- or (q)uit
Flash Type....................................... Compact Flash Card
Flash Size....................................... 1073741824

Burned-in MAC Address............................ 3C:CE:73:D9:69:60
Maximum number of APs supported.................. 75
System Nas-Id....................................
WLC MIC Certificate Types........................ SHA1

(Cisco Controller) >

What can in

Rps-Cheers · ‎07-11-2021

From the perspective of the phenomenon, the AP can be registered, but then restarted due to radio failure.

It seems this is a bug.

Cisco 18xx APs unexpectedly reload due to 'radio failure(radio recovery failed)'

CSCvm90337

Description

Symptom:
Beacon stuck for 120 seconds followed by AP reset

Crashfile shows "radio failure(radio recovery failed)" as reboot reason

Conditions:
AP in local mode

Workaround:
At this time, there is no workaround

Further Problem Description:

And Release 8.5.171 is not in fixed release list.

Known Fixed Releases:

(15)

ap-16.12.1

ap-16.12.0.2

ap-16.11.2.15

ap-16.11.1.130

ap-16.10.1.45

Gibraltar-16.12.1

8.10(105.0)

8.10(1.37)

8.9(111.0)

8.9(104.41)

8.8(120.0)

8.8(114.78)

8.8(111.2)

8.5(140.0)

8.5(137.109)

You can check the crashinfo on the AP again to see if it is due to APs unexpectedly reload due to'radio failure.

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Rps-Cheers | If it solves your problem, please mark as answer. Thanks !

Rich R · ‎07-12-2021

CSCvm90337 was fixed in 8.5.140.0 so 8.5.171.0 would have inherited the fix from that!

Reset the AP to factory default (mode button) and try again.

Check you have a reliable network connection: "Max retransmission count exceeded, going back to DISCOVER mode" suggests there could be packet loss between AP and WLC.

Otherwise TAC case/RMA - replace AP.

------------------------------
Please click Helpful if this post helped you and Select as Solution (drop down menu at top right of this reply) if this answered your query.
------------------------------
TAC recommended codes for AireOS WLC's and TAC recommended codes for 9800 WLC's
Best Practices for AireOS WLC's, Best Practices for 9800 WLC's and Cisco Wireless compatibility matrix
Check your 9800 WLC config with Wireless Config Analyzer using "show tech wireless" output or "config paging disable" then "show run-config" output on AireOS and use Wireless Debug Analyzer to analyze your WLC client debugs
Field Notice: FN63942 APs and WLCs Fail to Create CAPWAP Connections Due to Certificate Expiration
Field Notice: FN72424 Later Versions of WiFi 6 APs Fail to Join WLC - Software Upgrade Required
Field Notice: FN72524 IOS APs stuck in downloading state after 4 Dec 2022 due to Certificate Expired
- Fixed in 8.10.196.0, latest 9800 releases, 8.5.182.12 (8.5.182.13 for 3504) and 8.5.182.109 (IRCM, 8.5.182.111 for 3504)
Field Notice: FN70479 AP Fails to Join or Joins with 1 Radio due to Country Mismatch, RMA needed
How to avoid boot loop due to corrupted image on Wave 2 and Catalyst 11ax Access Points (CSCvx32806)
Field Notice: FN74035 - Wave2 APs DFS May Not Detect Radar After Channel Availability Check Time
Leo's list of bugs affecting 2800/3800/4800/1560 APs

Rps-Cheers · ‎07-12-2021

Thanks for the discussion.

I thought about the problem you mentioned, but some fixed releases don't seem to be effective. For similar problems, it is indeed possible that it is a hardware problem of a single AP.
Resetting the AP seems to be the only way before trying other methods. No matter what, further testing is needed, including the link quality between AP and WLC, and the problems of the AP itself.

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Rps-Cheers | If it solves your problem, please mark as answer. Thanks !

TimurKaneev · ‎07-13-2021

Customer have 3 types of AP: 1852, 3802, 1560. Only AP1852 have the problem.

Now customer have 16 AP that doesn't join to WLC.

Rps-Cheers · ‎07-13-2021

Seeing that you plan to upgrade and waiting for your good news, thank you.

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Rps-Cheers | If it solves your problem, please mark as answer. Thanks !

TimurKaneev · ‎07-13-2021

Hi, i tried to make factory reset ( use reset button) and tftp update the AP.

One time the AP joined to wlc, but SSID doen't broadcast from AP, after reload AP doesn't joined to WLC .

.Jaidev Hattiangadi. · ‎07-12-2021

Seems like CSCvg12122 from the errors on the console log. Fixed on 8.8 and 8.10.

TimurKaneev · ‎07-13-2021

Today i planning update the WLC5520 to 8.10.142 version and i hope the problem will be fix.

TimurKaneev · ‎07-13-2021

Updated WLC to 8.10.142, but have the same problem - access point does't join to wlc.

Will open the case