Re: AP 2802 not registering with Cisco 3650 WLC

serenjek · ‎10-12-2019

We have Cisco 3650 switch which is acting as wireless controller. The IOS version of 3650 is 16.3.9. Cisco 2802 is version on firmware version 8.5.131.0. We configured switch as wireless mobility controller but none of the APs are registering to switch. AP is connected to same switch.

In switch we are getting following error

Oct 12 14:27:52.813: *%DTLS-3-PKI_ERROR:Switch 1 R0/0: wcm: PKI initialization error : Certificate initialization failed
Oct 12 14:27:52.813: *%CAPWAP-3-DTLS_DB_ERR:Switch 1 R0/0: wcm: 0000.0000.0000: Failed to create DTLS connection for AP 192:168:10:21 (5264).

In 2802 AP we are getting following error

Oct 12 14:21:46 kernel: [*10/12/2019 14:21:46.0000] CAPWAP State: DTLS Setup
Oct 12 14:21:46 kernel: [*10/12/2019 14:21:46.0005] dtls_connectionDB_add_connection: Added Connection 0x1179c00 Server [192.168.10.1]:5246 Client [192.168.10.21]:5264
Oct 12 14:21:46 kernel: [*10/12/2019 14:21:46.0005]
Oct 12 14:21:46 kernel: [*10/12/2019 14:21:46.0005] create_dtls_connection: Creating DTLS Ctrl Connection 0x1179c00
Oct 12 14:21:46 kernel: [*10/12/2019 14:21:46.0005] DTLS connection created sucessfully local_ip: 192.168.10.21 local_port: 5264 peer_ip: 192.168.10.1 peer_port : 5246
Oct 12 14:22:07 FIPS[31259]: *** shell: FIPS Mode = disabled ***
Oct 12 14:22:43 kernel: [*10/12/2019 14:22:43.0161] Wait DTLS timer has expired
Oct 12 14:22:43 kernel: [*10/12/2019 14:22:43.0161] Dtls session establishment failed
Oct 12 14:22:43 kernel: [*10/12/2019 14:22:43.0162] local_in_addr_comp: Client and server addresses/port/version of 2 nodes are [192.168.10.21]:5264(0)--[192.16 8.10.1]:5246(0) [192.168.10.21]:5264--[192.168.10.1]:5246

Oct 12 14:22:43 kernel: [*10/12/2019 14:22:43.0162] dtls_connection_find_using_l ink_info: Searching connection [192.168.10.21]:5264--[192.168.10.1]:5246, result 0x1179c00
Oct 12 14:22:43 kernel: [*10/12/2019 14:22:43.0162] wtpCloseAllDtlsConnections: Closing DTLS-CTRL connection 0x1179c00.
Oct 12 14:22:43 kernel: [*10/12/2019 14:22:43.0162] dtls_disconnect: ERROR shutt ing down dtls connection ...
Oct 12 14:22:43 kernel: [*10/12/2019 14:22:43.0162]
Oct 12 14:22:43 kernel: [*10/12/2019 14:22:43.0163] local_in_addr_comp: Client a nd server addresses/port/version of 2 nodes are [192.168.10.21]:5264(0)--[192.16 8.10.1]:5246(0) [192.168.10.21]:5264--[192.168.10.1]:5246
Oct 12 14:22:43 kernel: [*10/12/2019 14:22:43.0163] wtpDtlsCallback: DTLS-Ctrl C onnection 0x1179c00 closed
Oct 12 14:22:43 kernel: [*10/12/2019 14:22:43.0163] dtls_free_connection: Free d one... for connection 0x1179c00
Oct 12 14:22:43 kernel: [*10/12/2019 14:22:43.0172] dtls_connectionDB_del_connec tion: Deleted Connection 0x1179c00, Server [192.168.10.1]:5246, Client [192.168. 10.21]:5264, Count 0, rc_return 2
Oct 12 14:22:43 kernel: [*10/12/2019 14:22:43.0230] [DP] Deleting capwap datapat h
Oct 12 14:22:43 kernel: [*10/12/2019 14:22:43.0230] CAPWAP data tunnel delete fr om forwarding succeeded

Please advice how we can make AP 2802 working with Cisco 3650 controller.

marce1000 · ‎10-12-2019

- Make sure that the country code on the AP and the controller match.

M.

-- ' 'Good body every evening' ' this sentence was once spotted on a logo at the entrance of a Weight Watchers Club !

Leo Laohoo · ‎10-12-2019

Post the complete output to the following commands:
1. WLC: sh sysinfo;
2. WLC: sh time;
3. AP: sh version; and
4. AP: sh capwap ap client rcb

serenjek · ‎10-13-2019

@Leo Laohoo : below are the outputs from the AP side:

APBC26.C792.5EFA#show capwap client rcb
AdminState : ADMIN_ENABLED
OperationState : DTLS SETUP
Name : APBC26.C792.5EFA
SwVer : 8.5.131.0
HwVer : 1.0.0.0
MwarApMgrIp : 192.168.10.1
MwarName : GW_3650
MwarHwVer : 0.0.0.0
Location : default location
ApMode : Local
ApSubMode : Not Configured
CAPWAP Path MTU : 576
CAPWAP UDP-Lite : Enabled
IP Prefer-mode : IPv4
AP Link DTLS Encryption : OFF
AP TCP MSS Adjust : Enabled
AP TCP MSS size : 1250
LinkAuditing : disabled
AP Group Name : default-group
Cisco Trustsec Config
AP Inline Tagging Mode : Disabled
AP Sgacl Enforcement : Disabled
AP Override Status : Disabled
APBC26.C792.5EFA#

Cisco AP Software, (ap3g3), C2802, RELEASE SOFTWARE
Technical Support: http://www.cisco.com/techsupport
Copyright (c) 1986-2015 by Cisco Systems, Inc.
Compiled Tue Jun 5 07:27:33 PDT 2018

ROM: Bootstrap program is U-Boot boot loader
BOOTLDR: U-Boot boot loader Version 2013.01-gad8bd06 (Sep 28 2017 - 17:03:04)

APBC26.C792.5EFA uptime is 6 days, 14 hours, 39 minutes
Last reload time : Sat Sep 14 11:25:16 UTC 2019
Last reload reason : unknown

cisco AIR-AP2802I-E-K9 ARMv7 Processor rev 1 (v7l) with 1028584/591148K bytes of memory.
Processor board ID FGL2236A2C2
AP Running Image : 8.5.131.0
Primary Boot Image : 8.5.131.0
Backup Boot Image : 0.0.0.0
AP Image type : MOBILITY EXPRESS IMAGE
AP Configuration : NOT MOBILITY EXPRESS CAPABLE
2 Gigabit Ethernet interfaces
2 802.11 Radios
Radio Driver version : 9.0.5.5-W8964
Radio FW version : 9.1.8.1
NSS FW version : 2.4.24

Base ethernet MAC Address : BC:26:C7:92:5E:FA
Part Number : 73-100821-05
PCA Assembly Number : 000-00000-00
PCA Revision Number :
PCB Serial Number : FDO223413Q1
Top Assembly Part Number : 068-100534-02
Top Assembly Serial Number : FGL2236A2C2
Top Revision Number : A0
Product/Model Number : AIR-AP2802I-E-K9

APBC26.C792.5EFA#

below is the output from the Controller side;

GW_3650#show sys
GW_3650#show system ?
mtu Show the global Maximum Transmission Unit (MTU)

GW_3650#show tim
GW_3650#show time ?
WORD Name of entry to show
ipc Show statistics of time-range ipc messages
| Output modifiers
<cr>

GW_3650#show time
GW_3650#show tim
GW_3650#show time-range ?
WORD Name of entry to show
ipc Show statistics of time-range ipc messages
| Output modifiers
<cr>

GW_3650#show time-range
GW_3650#show time ?
WORD Name of entry to show
ipc Show statistics of time-range ipc messages
| Output modifiers
<cr>

GW_3650#show time ipc?
WORD ipc

GW_3650#show time ipc
GW_3650#sh
GW_3650#show sy
GW_3650#show system ?
mtu Show the global Maximum Transmission Unit (MTU)

GW_3650#show system
% Incomplete command.

GW_3650#show system

Leo Laohoo · ‎10-14-2019

@serenjek wrote:
AP Image type : MOBILITY EXPRESS IMAGE

AP is loaded with Mobility Express is the reason.

Converting an AP from Mobility Express to CAPWAP Type

serenjek · ‎10-14-2019

hi @Leo Laohoo see below output after trying to convert the AP to capwap

APBC26.C792.5EFA#ap-type capwap
APBC26.C792.5EFA#Oct 14 15:07:15 kernel: [*10/14/2019 15:07:15.3905]
Oct 14 15:07:15 kernel: [*10/14/2019 15:07:15.3905] .....No change in AP Type Configuration......
Oct 14 15:07:15 kernel: [*10/14/2019 15:07:15.3905]

APBC26.C792.5EFA#
APBC26.C792.5EFA#
APBC26.C792.5EFA#
APBC26.C792.5EFA#Oct 14 15:08:06 kernel: [*10/14/2019 15:08:06.0163] dtls_disconnect: ERROR shutting down dtls connection ...
Oct 14 15:08:06 kernel: [*10/14/2019 15:08:06.0163]
Oct 14 15:08:06 kernel: [*10/14/2019 15:08:06.0164]
Oct 14 15:08:06 kernel: [*10/14/2019 15:08:06.0164] CAPWAP State: DTLS Teardown
Oct 14 15:08:10 kernel: [*10/14/2019 15:08:10.7679] No more AP manager addresses remain..
Oct 14 15:08:10 kernel: [*10/14/2019 15:08:10.7680] No valid AP manager found for controller 'GW_3650' (ip: 192.168.10.1)
Oct 14 15:08:10 kernel: [*10/14/2019 15:08:10.7680] Failed to join controller GW_3650.
Oct 14 15:08:10 kernel: [*10/14/2019 15:08:10.7680] Failed to join controller.
Oct 14 15:08:10 kernel: [*10/14/2019 15:08:10.8727]
Oct 14 15:08:10 kernel: [*10/14/2019 15:08:10.8727] CAPWAP State: Discovery
Oct 14 15:08:10 kernel: [*10/14/2019 15:08:10.8749] Discovery Request sent to 192.168.10.1, discovery type STATIC_CONFIG(1)
Oct 14 15:08:10 kernel: [*10/14/2019 15:08:10.8765] Discovery Request sent to 192.168.10.1, discovery type STATIC_CONFIG(1)
Oct 14 15:08:10 kernel: [*10/14/2019 15:08:10.8786] Discovery Request sent to 255.255.255.255, discovery type UNKNOWN(0)
Oct 14 15:08:10 kernel: [*10/14/2019 15:08:10.8787] Discovery Response from 192.168.10.1
Oct 14 15:08:10 kernel: [*10/14/2019 15:08:10.8864] Discovery Response from 192.168.10.1
Oct 14 15:08:10 kernel: [*10/14/2019 15:08:10.8958] Discovery Response from 192.168.10.1
Oct 14 15:08:20 kernel: [*10/14/2019 15:08:20.0000]
Oct 14 15:08:20 kernel: [*10/14/2019 15:08:20.0000] CAPWAP State: DTLS Setup

APBC26.C792.5EFA#Oct 14 15:09:17 kernel: [*10/14/2019 15:09:17.0162] dtls_disconnect: ERROR shutting down dtls connection ...
Oct 14 15:09:17 kernel: [*10/14/2019 15:09:17.0162]
Oct 14 15:09:17 kernel: [*10/14/2019 15:09:17.0164]
Oct 14 15:09:17 kernel: [*10/14/2019 15:09:17.0164] CAPWAP State: DTLS Teardown
Oct 14 15:09:21 kernel: [*10/14/2019 15:09:21.7679] No more AP manager addresses remain..
Oct 14 15:09:21 kernel: [*10/14/2019 15:09:21.7680] No valid AP manager found for controller 'GW_3650' (ip: 192.168.10.1)
Oct 14 15:09:21 kernel: [*10/14/2019 15:09:21.7680] Failed to join controller GW_3650.
Oct 14 15:09:21 kernel: [*10/14/2019 15:09:21.7680] Failed to join controller.
Oct 14 15:09:21 kernel: [*10/14/2019 15:09:21.9770]
Oct 14 15:09:21 kernel: [*10/14/2019 15:09:21.9770] CAPWAP State: Discovery
Oct 14 15:09:21 kernel: [*10/14/2019 15:09:21.9790] Discovery Request sent to 192.168.10.1, discovery type STATIC_CONFIG(1)
Oct 14 15:09:21 kernel: [*10/14/2019 15:09:21.9808] Discovery Request sent to 192.168.10.1, discovery type STATIC_CONFIG(1)
Oct 14 15:09:21 kernel: [*10/14/2019 15:09:21.9826] Discovery Request sent to 255.255.255.255, discovery type UNKNOWN(0)
Oct 14 15:09:21 kernel: [*10/14/2019 15:09:21.9828] Discovery Response from 192.168.10.1
Oct 14 15:09:21 kernel: [*10/14/2019 15:09:21.9907] Discovery Response from 192.168.10.1
Oct 14 15:09:22 kernel: [*10/14/2019 15:09:21.9982] Discovery Response from 192.168.10.1
Oct 14 15:09:31 kernel: [*10/14/2019 15:09:31.0000]
Oct 14 15:09:31 kernel: [*10/14/2019 15:09:31.0000] CAPWAP State: DTLS Setup
Oct 14 15:10:28 kernel: [*10/14/2019 15:10:28.0162] dtls_disconnect: ERROR shutting down dtls connection ...
Oct 14 15:10:28 kernel: [*10/14/2019 15:10:28.0162]
Oct 14 15:10:28 kernel: [*10/14/2019 15:10:28.0164]
Oct 14 15:10:28 kernel: [*10/14/2019 15:10:28.0164] CAPWAP State: DTLS Teardown
Oct 14 15:10:32 kernel: [*10/14/2019 15:10:32.7680] No more AP manager addresses remain..
Oct 14 15:10:32 kernel: [*10/14/2019 15:10:32.7681] No valid AP manager found for controller 'GW_3650' (ip: 192.168.10.1)
Oct 14 15:10:32 kernel: [*10/14/2019 15:10:32.7681] Failed to join controller GW_3650.
Oct 14 15:10:32 kernel: [*10/14/2019 15:10:32.7681] Failed to join controller.
Oct 14 15:10:32 kernel: [*10/14/2019 15:10:32.9771]
Oct 14 15:10:32 kernel: [*10/14/2019 15:10:32.9771] CAPWAP State: Discovery
Oct 14 15:10:32 kernel: [*10/14/2019 15:10:32.9797] Discovery Request sent to 192.168.10.1, discovery type STATIC_CONFIG(1)
Oct 14 15:10:32 kernel: [*10/14/2019 15:10:32.9813] Discovery Request sent to 192.168.10.1, discovery type STATIC_CONFIG(1)
Oct 14 15:10:32 kernel: [*10/14/2019 15:10:32.9827] Discovery Request sent to 255.255.255.255, discovery type UNKNOWN(0)
Oct 14 15:10:32 kernel: [*10/14/2019 15:10:32.9829] Discovery Response from 192.168.10.1
Oct 14 15:10:32 kernel: [*10/14/2019 15:10:32.9910] Discovery Response from 192.168.10.1
Oct 14 15:10:33 kernel: [*10/14/2019 15:10:32.9985] Discovery Response from 192.168.10.1

Leo Laohoo · ‎10-14-2019

@serenjek wrote: Oct 14 15:08:10 kernel: [*10/14/2019 15:08:10.7680] No valid AP manager found for controller 'GW_3650' (ip: 192.168.10.1)
Oct 14 15:08:10 kernel: [*10/14/2019 15:08:10.7680] Failed to join controller GW_3650.
Oct 14 15:08:10 kernel: [*10/14/2019 15:08:10.7680] Failed to join controller.

Problem is now with the switch/controller.

serenjek · ‎10-21-2019

@Leo Laohoo pliz make me understand how is the switch / controller a problem? have gone through the configs again just to make sure tht wat @Rasika Nayanajith and @marce1000 advised if am missing sumthing but i seem not to c anything??

am really stack!!!!

Rasika Nayanajith · ‎10-21-2019

would you be able to attach your switch config with sensitive information removed (like passwords,etc) ?

Rasika

serenjek · ‎10-21-2019

! @Rasika Nayanajith below are the current config from the switch side
no ip domain lookup
ip domain name GSA.local
ip dhcp excluded-address 192.168.10.1 192.168.10.20
!
ip dhcp pool Wireless
network 192.168.10.0 255.255.255.0
default-router 192.168.10.1
dns-server 10.6.0.89 10.7.0.8

interface GigabitEthernet1/0/3
description Wireless Test AP
switchport access vlan 10
switchport mode access
power inline port priority high
spanning-tree portfast

interface Vlan10
description WIireless-VLAN
ip address 192.168.10.1 255.255.255.0

wireless mobility controller
wireless management interface Vlan10
wlan id 1 Wireless-Test
client vlan Wireless-VLAN
ip dhcp server 192.168.10.1
radio dot11a
wmm require
no shutdown
ap dot1x username XXX password 0XXXXXX
ap link-encryption
ap country ZA
ap dot11 airtime-fairness policy-name Default 0
description "OFFICE WIRELESS NETWORK"
hyperlocation
ap group default-group
ap hyperlocation ble-beacon 0
ap hyperlocation ble-beacon 1
ap hyperlocation ble-beacon 2
ap hyperlocation ble-beacon 3
ap hyperlocation ble-beacon 4