cancel
Showing results for 
Search instead for 
Did you mean: 
cancel
10032
Views
0
Helpful
11
Replies

After the AP runs for a few hours, it disconnects from the WLC and cannot register again !

At present, I encountered a problem, some AP1600 showed abnormal symptoms. They can register to the WLC normally, but after a few hours of operation, they will disconnect from the WLC and cannot register again.

 

I extracted some key information:

  • Msglog information on WLC:

*spamApTask5: Mar 01 11:59:55.452: %CAPWAP-3-DTLS_CLOSED_ERR: capwap_ac_sm.c:6985 0c:68:03:dd:6b:b0: DTLS connection closed forAP 10:191:2:5 (18943), Controller: 10:1:2:253 (5246) Echo Timer Expiry <<<<<<<<
*spamApTask4: Mar 01 12:00:38.393: %CAPWAP-3-DTLS_CLOSED_ERR: capwap_ac_sm.c:6985 ac:7e:8a:68:77:80: DTLS connection closed forAP 10:191:2:4 (23864), Controller: 10:1:2:253 (5246) Echo Timer Expiry
.....
*osapiBsnTimer: Mar 01 12:03:40.785: %DTLS-3-HANDSHAKE_FAILURE: openssl_dtls.c:3191 Failed to complete DTLS handshake with peer 10.1.2.4
*osapiBsnTimer: Mar 01 12:03:01.689: %DTLS-3-HANDSHAKE_FAILURE: openssl_dtls.c:3191 Failed to complete DTLS handshake with peer 10.1.2.5

.....
A lot of the same information:
*bcastReceiveTask: Mar 01 12:03:01.481: %APF-3-INVALID_INET: apf_net.c:4752 Invalid Inet address source: (fe80::1ede:a7ff:fe06:85e0) destination: (::). Discarding packet.
*apfOrphanSocketTask: Mar 01 12:03:01.136: %LOG-3-Q_IND: apf_net.c:4752 Invalid Inet address source: (fe80::1ede:a7ff:fe06:85e0) destination: (::). Discarding packet.[...It occurred 22 times.!]

 

(Cisco Controller) >show ap join stats summary all

Sync phase statistics
- Time at sync request received............................ Not applicable
- Time at sync completed................................... Not applicable

Discovery phase statistics
- Discovery requests received.............................. 64
- Successful discovery responses sent...................... 0
- Unsuccessful discovery request processing................ 64
- Reason for last unsuccessful discovery attempt........... Discovery request decoding with subnet broadcast and wrong AP IP address
- Time at last successful discovery attempt................ Not applicable
- Time at last unsuccessful discovery attempt.............. Mar 01 12:15:40.844

Join phase statistics
- Join requests received................................... 0
- Successful join responses sent........................... 0
- Unsuccessful join request processing..................... 0
- Reason for last unsuccessful join attempt................ Not applicable
- Time at last successful join attempt..................... Not applicable
- Time at last unsuccessful join attempt................... Not applicable

Configuration phase statistics

--More-- or (q)uit
- Configuration requests received.......................... 0
- Successful configuration responses sent.................. 0
- Unsuccessful configuration request processing............ 0
- Reason for last unsuccessful configuration attempt....... Not applicable
- Time at last successful configuration attempt............ Not applicable
- Time at last unsuccessful configuration attempt.......... Not applicable

Last AP message decryption failure details
- Reason for last message decryption failure............... Not applicable

Last AP disconnect details
- Reason for last AP connection failure.................... Not applicable
- Last AP disconnect reason................................ Not applicable

Last join error summary
- Type of error that occurred last......................... Lwapp discovery request rejected
- Reason for error that occurred last...................... Discovery request decoding with subnet broadcast and wrong AP IP address
- Time at which the last join error occurred............... Mar 01 12:15:40.844

AP disconnect details
- Reason for last AP connection failure.................... Not applicable
Ethernet Mac : 00:00:00:00:00:00 Ip Address : 10.1.3.32

 

  • logging infomation on AP:

*Mar 1 12:00:05.000: %CAPWAP-5-DTLSREQSEND: DTLS connection request sent peer_ip: 10.1.2.253 peer_port: 5246
*Mar 1 12:01:04.999: %DTLS-5-SEND_ALERT: Send FATAL : Close notify Alert to 10.1.2.253:5246
*Mar 1 12:01:05.000: %CAPWAP-5-DTLSREQSEND: DTLS connection request sent peer_ip: 10.1.2.253 peer_port: 5246
*Mar 1 12:01:10.999: %DTLS-5-SEND_ALERT: Send FATAL : Unexpected message Alert to 10.1.2.253:5246
*Mar 1 12:01:10.999: %DTLS-5-SEND_ALERT: Send FATAL : Close notify Alert to 10.1.2.253:5246
*Mar 1 12:02:38.000: %CAPWAP-5-DTLSREQSEND: DTLS connection request sent peer_ip: 10.1.2.253 peer_port: 5246
*Mar 1 12:02:51.999: %DTLS-5-SEND_ALERT: Send FATAL : Unexpected message Alert to 10.1.2.253:5246

 

Based on this information, I found some Cisco documentation.

Problem 5: Controller receives AP discovery message on wrong VLAN (you see the discovery message debug, but not response)
You see this message in the debug capwap events enable command output:

Received a Discovery Request with subnet broadcast with wrong AP IP address (A.B.C.D)!
This message means that the controller received a discovery request via a broadcast IP address that has a source IP address which is not in any configured subnets on the controller. This also means the controller is dropping the packet.

The problem is that the AP is not sending the discovery request to the management IP address. The controller is reporting a broadcast discovery request from a VLAN that is not configured on the controller. This typically occurs when the customer trunks allowed VLANs instead of restricting them to wireless VLANs.

Complete these steps in order to resolve this problem:

1.If the controller is on another subnet, the APs must be primed for the controller IP address, or the APs must receive the controllers IP address using any one of the discovery methods.

2.The switch is configured to allow some VLANs that are not on the controller. Restrict the allowed VLANs on the trunks.

 

https://www.cisco.com/c/en/us/support/docs/wireless/5500-series-wireless-controllers/119286-lap-notjoin-wlc-tshoot.html#anc17

 

The switch interfaces connecting the WLC and AP are in trunk mode.So I suspect this problem caused by the switch interface configuration.I would like to ask everyone to help me analyze whether it is the same problem as "Problem 5" or what other reasons?

 

Thanks

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Rps-Cheers | If it solves your problem, please mark as answer. Thanks !
11 Replies 11

Sandeep Choudhary
VIP Alumni
VIP Alumni

AP is in local mode or flexconnect ?

Are AP and WLC in same subnet or ?

 

Paste the complete output of these commands:

show sysinfo from WLC

show version from AP

Bootup process from AP console

show run inter <WLC connected>

show run inter <AP connected>

show time form WLC

 

Regards

Dont forget to rate helpful posts

 

I am sorry for late reply
All APs went offline this morning and cannot register to the WLC. It's weird. Basically, APs will be disconnected after a few hours. Very unstable. We need to use the "shu / no shu" command to restart the switch interfaces connected to APs, and restart these APs to register with the WLC again.

 

Attached is the information I collected. But there is no information about the AP bootup process, but I attached the show logging information of the AP.

 

Please help to see if there is any problem.

 

Thanks

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Rps-Cheers | If it solves your problem, please mark as answer. Thanks !

Hi

 

    You're hitting a well-known bug (CSCvn08995), upgrade the controller to a stabler version.

 

Regards,

Cristian Matei.

Thanks for you response! I did not find CSCvn08995, may I ask if there are any bug details or links

BR
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Rps-Cheers | If it solves your problem, please mark as answer. Thanks !

i have found this bug(Bug CSCvn08995 is a duplicate of CSCvk53499),Thanks .
I will confirm the debug info again.
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Rps-Cheers | If it solves your problem, please mark as answer. Thanks !

Doesn't seem to be this bug:
I do not see debugging similar to the following in the debug information:
* spamApTask1: Failed to create DTLS connection for: 54455
* spamApTask1: System reached max concurrent DTLS Handshakes

* spamApTask0: Ignoring peer connection because concurrent handshake threshold reached.

 

In addition, the msglog on the WLC looks like this:
* osapiBsnTimer: Mar 06 09: 07: 20.977:% DTLS-3-HANDSHAKE_FAILURE: openssl_dtls.c: 3191 Failed to complete DTLS handshake with peer 10.1.2.4
* spamApTask7: Mar 06 08: 08: 43.969:% CAPWAP-3-MAX_RETRANSMISSIONS_REACHED: capwap_ac_sm.c: 7533 Max retransmissions reached on AP (00: 2a: 10: xx: xx: xx), message (CAPWAP_CONFIGURATION_UPDATE_REQUEST
), number of pending messages (37)
* spamApTask0: Mar 06 08: 08: 35.417:% CAPWAP-3-DTLS_CLOSED_ERR: capwap_ac_sm.c: 6985 f0: 29: 29: 27: 03: 00: DTLS connection closed forAP 10: 1: 2: 4 (24630), Controller: 10: 1: 2: 253 (5246) Echo Timer Expiry

 

There are also a large number of logs on WLC:
* spamApTask4: Mar 06 08: 05: 05.790:% LOG-3-Q_IND: apf_net.c: 4752 Invalid Inet address source: (fe80 :: c6e7: d5ff: ffe4: 6877) destination: (: :). Discarding packet. [... It occurred 26 times.!]
* bcastReceiveTask: Mar 06 08: 05: 02.172:% APF-3-INVALID_INET: apf_net.c: 4752 Invalid Inet address source: (fe80 :: c6e7: d5ff: ffe4: 6877) destination: (: :). Discarding packet.
* spamApTask3: Mar 06 08: 04: 41.317:% LOG-3-Q_IND: apf_net.c: 4752 Invalid Inet address source: (fe80 :: c6e7: d5ff: ffe4: 6877) destination: (: :). Discarding packet. [... It occurred 26 times.!]
* bcastReceiveTask: Mar 06 08: 04: 40.404:% APF-3-INVALID_INET: apf_net.c: 4752 Invalid Inet address source: (fe80 :: c6e7: d5ff: ffe4: 6877) destination: (: :). Discarding packet.

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Rps-Cheers | If it solves your problem, please mark as answer. Thanks !

Hi,

  

     The Bug Search Toll is a bit sneaky here, to say the least. go to the bug search tool, look for the ONLY relevant error from your attached file (%DTLS-5-SEND_ALERT: Send FATAL : Close notify Alert to) and you'll see the bug; if you click it, it will redirect you to the other bug, ever the errors are different, but outcome and behaviour is the same. See the attached pic for the correct bug output.

HI Cristian,

Thank you for sharing, I also plan to upgrade to 8.5.161 and try it out
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Rps-Cheers | If it solves your problem, please mark as answer. Thanks !

What's the problem? It's going crazy. At present, the WLC has been upgraded to 8.5.161, but the AP will still be offline after a few hours of operation.
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Rps-Cheers | If it solves your problem, please mark as answer. Thanks !

Hi,

   

   You've seen the bug, currently opened. Open a TAC case urgently and see how it goes, you may need to do some RMA.

 

Regards,

Cristian Matei.

Thanks for you response! I will try the final test. If it does n’t work, I think I have to open TAC CASE.

Thx
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Rps-Cheers | If it solves your problem, please mark as answer. Thanks !
Getting Started

Find answers to your questions by entering keywords or phrases in the Search bar above. New here? Use these resources to familiarize yourself with the community:

Review Cisco Networking products for a $25 gift card