LWAP APs do not join anymore

[derobbacher]

Hello,

we have a problem with LWAP APs that do not join anymore after a power outage at the remote side.

For example a AIR-AP1142N-E-K9 in HREAP-mode is trying to reach the WLC in vain.

The AP join statistics say:

Last AP Connection Failure
Last AP Disconnect Reason
Last Error occurred

Number of message retransmission to the AP has reached maximum

Message retransmission to the AP has reached maximum

AP got or has been disconnecte

The WLC is a 4402 with enough licenses running on SW-level 7.0.230

2 other APs of same model on same switch and identical portconfig where able to join

We did already a shutdown on the switchport and can see that the WLAN AP is receiving an IP-address.

What is wrong ?

Here is additnional info catched from adebug on the 4400 WLC:

*spamReceiveTask: Sep 12 01:44:16.299: a4:56:30:a2:96:b0 No AP entry exist in temporary database for 163.157.75.199:23796
*spamReceiveTask: Sep 12 01:47:26.656: a4:56:30:a2:96:b0 length = 4, packet received from a4:56:30:a2:96:b0

*spamReceiveTask: Sep 12 01:47:26.656: a4:56:30:a2:96:b0 Discovery Request from 163.157.75.199:23796

*spamReceiveTask: Sep 12 01:47:26.656: a4:56:30:a2:96:b0 Discovery request: Total msgEleLen = 93

*spamReceiveTask: Sep 12 01:47:26.656: a4:56:30:a2:96:b0 Discovery Type = Static Config

*spamReceiveTask: Sep 12 01:47:26.656: a4:56:30:a2:96:b0 Discovery request: Vendor payload type = 207, length = 10

*spamReceiveTask: Sep 12 01:47:26.656: a4:56:30:a2:96:b0 Discovery request: Vendor payload type = 5, length = 16

*spamReceiveTask: Sep 12 01:47:26.656: a4:56:30:a2:96:b0 Join Priority Processing status = 0, Incoming Ap's Priority 1, MaxLrads = 25, joined Aps =20
*spamReceiveTask: Sep 12 01:47:26.656: a4:56:30:a2:96:b0 msgLength = 36

*spamReceiveTask: Sep 12 01:47:26.656: a4:56:30:a2:96:b0 encodeLen = 121 len = 16

*spamReceiveTask: Sep 12 01:47:26.656: a4:56:30:a2:96:b0 Discovery Response sent to 163.157.75.199:23796

*spamReceiveTask: Sep 12 01:47:36.656: a4:56:30:a2:96:b0 DTLS Connection not found for 163.157.75.199:23796

*spamReceiveTask: Sep 12 01:47:36.656: a4:56:30:a2:96:b0 DTLS connection not found, creating new connection for 163:157:75:199 (23796) 163:157:231:7 (5246)

*spamReceiveTask: Sep 12 01:47:36.733: a4:56:30:a2:96:b0 record_consumed or sucess
*spamReceiveTask: Sep 12 01:47:36.753: a4:56:30:a2:96:b0 record_consumed or sucess
*spamReceiveTask: Sep 12 01:48:36.699: a4:56:30:a2:96:b0 DTLS connection was closed
*spamReceiveTask: Sep 12 01:48:36.699: a4:56:30:a2:96:b0 DTLS connection closed event receivedserver (163:157:231:7/5246) client (163:157:75:199/23796)
*spamReceiveTask: Sep 12 01:48:36.699: a4:56:30:a2:96:b0 DTLS Connection not found for 163.157.75.199:23796

*spamReceiveTask: Sep 12 01:48:36.699: a4:56:30:a2:96:b0 No entry exists for AP (163:157:75:199/23796)

[Leo Laohoo]

Console into the AP and reboot the AP.  

Post the entire boot-up process.

[derobbacher]

Hello Leo, thank You for Your answer.

It is not possible to do that. The APs are mounted in remote locations of customer without local IT support. Does it make sense to dismount the failing AP and investigate at my office ? Or will this influence how to replay the error situation ?

I can say that we are doing Cisco WLANs for years and I haven't noticed this behaviour in the past.

It looks like it is appearing after SW-upgrades. The APs worked for months and years already.

The error-message is not appearing by the way on our central Jump-in WLC, which is reached via DNS-lookup "Cisco-CAPWAP-CONTROLLER". Here I can see: " Received Discovery request and sent response"

This is a 5508 running 7.4.122.23-code while the normal WLC in the remote location is a 4402 running on 7.0.230.0-SW-release.

It looks like the answer from the elder 4402-WLC is always faster and the AP is never trying the newer 5508.

This problem seems to hit several customers meanwhile, because I can see identical  discussions around this subject.

Would it help to change the country code or disabling WLAN AP management interface for a while on the 4402 to try to block the join  and force the WLAN AP to talk andtry to join to the remote 5508 instead ?

To me it looks like a homemade problem of Cisco with Flexconnect-APs. Not a single AP in Local-Mode has shown this behaviour. We are really stuck and afraid of doing SW-upgrades, because we have to fear that we will loose more APs by doing so.

 Who has a good idea how to get those god-forsaken WLAN APs of Cisco reconnect to thei home-WLC ?