Hello everyone -
We have a 5508 controller that manages AP's at approximately 20 branches - each branch has their own subnet. We have a single branch (subnet) with new 2602 AP's that will not register with the controller. All communications to this subnet appear normal and there are no ACL's in place between the AP's and the controller. The AP's are able to resolve the controller IP address via DNS and begin the registration process but then timeout. We have successfully installed 2602 AP's at other branch locations and they register with no problems - this is only a problem at a single branch. I've attached some debug messages below for a single AP (this is a production environment so I parsed un-necessary info) and also included the console messages from a different AP (the console messages on the AP's are the same). There are currently 9 AP's at this location and none of them will register. Any ideas??
Debug Capwap Events:
*spamApTask0: Dec 11 14:39:32.904: 44:2b:03:9a:d1:10 Discovery Request from 10.29.9.190:44306
*spamApTask0: Dec 11 14:39:32.904: 44:2b:03:9a:d1:10 Join Priority Processing status = 0, Incoming Ap's Priority 1, MaxLrads = 300, joined Aps =272
*spamApTask0: Dec 11 14:39:32.904: 44:2b:03:9a:d1:10 Discovery Response sent to 10.29.9.190:44306
*spamApTask0: Dec 11 14:39:32.904: 44:2b:03:9a:d1:10 Discovery Response sent to 10.29.9.190:44306
*spamApTask0: Dec 11 14:39:32.992: 44:2b:03:9a:d1:10 Received LWAPP DISCOVERY REQUEST to 68:ef:bd:8e:48:6f on port '13'
*spamApTask0: Dec 11 14:39:32.992: 44:2b:03:9a:d1:10 Discarding discovery request in LWAPP from AP supporting CAPWAP
*spamApTask0: Dec 11 14:39:42.903: 44:2b:03:9a:d1:10 DTLS connection not found, creating new connection for 10:29:9:190 (44306) 10:5:13:4 (5246)
*spamApTask0: Dec 11 14:57:52.301: e8:ba:70:dc:d1:c0 DTLS connection closed event receivedserver (10:5:13:4/5246) client (10:29:9:190/44306)
*spamApTask0: Dec 11 14:57:52.301: e8:ba:70:dc:d1:c0 No entry exists for AP (10:29:9:190/44306)
*spamApTask0: Dec 11 14:57:52.301: e8:ba:70:dc:d1:c0 No AP entry exist in temporary database for 10.29.9.190:44306
*spamApTask0: Dec 11 14:57:53.828: 44:2b:03:9a:d1:10 Discovery Request from 10.29.9.190:44306
*spamApTask0: Dec 11 14:57:53.828: 44:2b:03:9a:d1:10 Join Priority Processing status = 0, Incoming Ap's Priority 1, MaxLrads = 300, joined Aps =272
*spamApTask0: Dec 11 14:57:53.828: 44:2b:03:9a:d1:10 Discovery Response sent to 10.29.9.190:44306
*spamApTask0: Dec 11 14:57:53.828: 44:2b:03:9a:d1:10 Discovery Response sent to 10.29.9.190:44306
*spamApTask0: Dec 11 14:57:53.916: 44:2b:03:9a:d1:10 Received LWAPP DISCOVERY REQUEST to 68:ef:bd:8e:48:6f on port '13'
*spamApTask0: Dec 11 14:57:53.916: 44:2b:03:9a:d1:10 Discarding discovery request in LWAPP from AP supporting CAPWAP
*spamApTask0: Dec 11 14:58:03.824: 44:2b:03:9a:d1:10 DTLS connection not found, creating new connection for 10:29:9:190 (44306) 10:5:13:4 (5246)
Debug Capwap Errors:
*spamApTask0: Dec 11 15:17:33.715: e8:ba:70:dc:d1:c0 Deleting AP 10.29.9.190 which has not been plumbed
*spamApTask0: Dec 11 15:17:33.716: e8:ba:70:dc:d1:c0 DTLS connection was closed
Debug Capwap Details:
*spamApTask0: Dec 11 15:24:29.419: 44:2b:03:9a:d1:10 CAPWAP Control Msg Received from 10.29.9.190:44306
*spamApTask0: Dec 11 15:24:35.542: 44:2b:03:9a:d1:10 CAPWAP Control Msg Received from 10.29.9.190:44306
*spamApTask0: Dec 11 15:24:41.555: 44:2b:03:9a:d1:10 CAPWAP Control Msg Received from 10.29.9.190:44306
*spamApTask0: Dec 11 15:24:49.555: 44:2b:03:9a:d1:10 CAPWAP Control Msg Received from 10.29.9.190:44306
*spamApTask0: Dec 11 15:25:29.420: 44:2b:03:9a:d1:10 CAPWAP Control Msg Received from 10.29.9.190:44306
*spamApTask0: Dec 11 15:25:29.420: 44:2b:03:9a:d1:10 DTLS connection 0x1a1703c8 closed by controller
*spamApTask0: Dec 11 15:25:29.421: CAPWAP DTLS connection closed msg
AP Console log:
Translating "CISCO-CAPWAP-CONTROLLER.ad.pps.k12.va.us"...domain server (10.29.8.3)
*Mar 1 00:00:57.511: %CAPWAP-3-ERRORLOG: Did not get log server settings from DHCP. [OK]
*Mar 1 00:01:10.511: %CAPWAP-3-ERRORLOG: Go join a capwap controller
*Dec 11 16:05:56.000: %CAPWAP-5-DTLSREQSEND: DTLS connection request sent peer_ip: 10.5.13.4 peer_port: 5246
*Dec 11 16:06:17.495: %CDP_PD-2-POWER_LOW: All radios disabled - NEGOTIATED WS-C3560E-24PD (68bc.0c03.8015)
*Dec 11 16:06:28.231: DTLS_CLIENT_ERROR: ../capwap/base_capwap/dtls/base_capwap_dtls_connection_db.c:2051 Max retransmission count reached!
logging facility kern
^
% Invalid input detected at '^' marker.
logging trap emergencies
^
% Invalid input detected at '^' marker.
logging facility kern
^
% Invalid input detected at '^' marker.
logging trap emergencies
^
% Invalid input detected at '^' marker.
*Dec 11 16:06:55.999: %DTLS-5-SEND_ALERT: Send FATAL : Close notify Alert to 10.5.13.4:5246
*Dec 3 16:06:55.999: %LWAPP-3-CLIENTERRORLOG: LWAPP LED Init: incorrect led state 255
*Dec 11 16:07:06.367: %CAPWAP-3-ERRORLOG: Go join a capwap controller
*Dec 11 16:07:07.000: %CAPWAP-5-DTLSREQSEND: DTLS connection request sent peer_ip: 10.5.13.4 peer_port: 5246
*Dec 11 16:07:39.151: DTLS_CLIENT_ERROR: ../capwap/base_capwap/dtls/base_capwap_dtls_connection_db.c:2051 Max retransmission count reached!
logging facility kern
^
% Invalid input detected at '^' marker.
logging trap emergencies
^
% Invalid input detected at '^' marker.
logging facility kern
^
% Invalid input detected at '^' marker.
logging trap emergencies
^
% Invalid input detected at '^' marker.
*Dec 11 16:08:06.999: %DTLS-5-SEND_ALERT: Send FATAL : Close notify Alert to 10.5.13.4:5246
*Dec 11 16:08:06.999: %LWAPP-3-CLIENTERRORLOG: LWAPP LED Init: incorrect led state 255
*Dec 11 16:08:17.367: %CAPWAP-3-ERRORLOG: Go join a capwap controller
*Dec 11 16:08:17.000: %CAPWAP-5-DTLSREQSEND: DTLS connection request sent peer_ip: 10.5.13.4 peer_port: 5246
