topic Re: APs won't join vWLC 8.3 in Wireless

APs won't join vWLC 8.3

Austin Godbey — Mon, 05 Jul 2021 17:25:42 GMT

Have a customer with all APs already associated to their main vWLC running 8.3. Created a new vWLC to act as their secondary controller. Statically assigned WLCs in HA tab for all APs. None of the APs will join the controller. APs are 3602i, 3702i, and 3802i. Same code on both vWLCs.

I see the following logs on the WLC:

*spamApTask7: May 16 18:11:37.611: %CAPWAP-3-DECODE_ERR: capwap_ac_sm.c:2732 Error decoding discovery request from AP 00:00:00:00:00:00
*spamApTask7: May 16 18:11:37.611: %CAPWAP-3-INVALID_PAYLOAD3: capwap_ac_decode.c:629 The system detects an invalid vendor type 12846 in WTP descriptor message element

On the AP Join status page the APs keep rising the discovery counter, but never transition to the join phase.

Capwap debugs don't show any errors, just keep repeating the discovery phase. Packet captures show the same.

Re: APs won't join vWLC 8.3

Sathiyanarayanan Ravindran — Fri, 17 May 2019 19:02:47 GMT

Check the below things on the controller.

* Time and date (NTP configuration)

* Appropriate Country Code is enabled on the WLC.

* Licensing on the Controller.

If all are fine but still the AP didn’t joined enable . Please connect console of one AP and share the Error logs on that while trying to join secondary WLC.

Re: APs won't join vWLC 8.3

Austin Godbey — Fri, 17 May 2019 20:29:40 GMT

First 3 things I checked. Times are up to date, new licenses and eval licenses won't work, and country code matches both WLCs.

All the APs are currently joined to 1 controller, but when I try to fail them over to the new controller that is configured exactly the same, they don't join.

Re: APs won't join vWLC 8.3

balaji.bandi — Sat, 18 May 2019 06:56:02 GMT

How is your second vWLC Setup, is the same version

Couple of the things need to check, Is the AP able to reach secondary controller IP address ?

is the DHCP configured Option 43 setup for the secondary controller ?

Re: APs won't join vWLC 8.3

Jurgens L — Sun, 19 May 2019 13:49:49 GMT

@Austin Godbey wrote:
First 3 things I checked. Times are up to date, new licenses and eval licenses won't work, and country code matches both WLCs.

When you say new licenses and eval licenses won't work, does the vWLC show how many AP's it can support?

You can confirm by doing a "show license all" in the CLI

<<< Please help the community by marking useful posts helpful, or accept as a solution if it resolved your issue >>>

Re: APs won't join vWLC 8.3

Austin Godbey — Mon, 20 May 2019 15:29:35 GMT

(Cisco Controller) >show license all

Feature name: ap_count
License type: Evaluation
License Eula: Accepted
Evaluation total period: 12 weeks 6 days
Evaluation period left: 89 days
License state: Inactive, Not-In-Use
License Nodelocked: Yes
RTU License Count: 200

Feature name: ap_count (adder)
License type: Permanent
License state: Active, Not-In-Use
License Nodelocked: No
RTU License Count: 93

==================================
Total available count : 93
Total inuse count : 0

Re: APs won't join vWLC 8.3

Austin Godbey — Mon, 20 May 2019 19:37:13 GMT

I see the following from 'debug capwap errors' on the AP (removed names and IPs):

May 20 17:28:53 kernel: [*05/20/2019 17:28:53.0000] CAPWAP State: DTLS Setup
May 20 17:28:53 kernel: [*05/20/2019 17:28:53.0002] dtls_new_connection: Connection 0x24f0a00 is already there for this server port 5246, Deleting it. Number of connections: 388
May 20 17:28:53 kernel: [*05/20/2019 17:28:53.0002]
May 20 17:28:53 kernel: [*05/20/2019 17:28:53.0004] dtls_connectionDB_add_connection: Number of DTLS connections exceeded two
May 20 17:28:53 kernel: [*05/20/2019 17:28:53.0369] dtls_load_ca_certs: LSC Root Certificate not present
May 20 17:28:53 kernel: [*05/20/2019 17:28:53.0369]
May 20 17:28:53 kernel: [*05/20/2019 17:28:53.0375] dtls_verify_con_cert: Controller certificate verification error
May 20 17:28:53 kernel: [*05/20/2019 17:28:53.0375] dtls_process_packet: controller cert verification failed
May 20 17:28:53 kernel: [*05/20/2019 17:28:53.0397] DTLS: Received packet 0x256c000 caused DTLS to close connection
May 20 17:28:53 kernel: [*05/20/2019 17:28:53.0398] sendPacketToDtls: DTLS: Closing connection 0x24f0a00.
May 20 17:28:53 kernel: [*05/20/2019 17:28:53.0398]
May 20 17:28:53 kernel: [*05/20/2019 17:28:53.0398] Lost connection to the controller, going to restart CAPWAP...
May 20 17:28:53 kernel: [*05/20/2019 17:28:53.0398]
May 20 17:28:53 kernel: [*05/20/2019 17:28:53.0401] DTLS: Error while processing DTLS packet 0x250a000.
May 20 17:28:53 kernel: [*05/20/2019 17:28:53.0401] Restarting CAPWAP State Machine.
May 20 17:28:53 kernel: [*05/20/2019 17:28:57.7510] No more AP manager addresses remain..
May 20 17:28:53 kernel: [*05/20/2019 17:28:57.7510] No valid AP manager found for controller '***-****-*' (ip: *.*.*.*)
May 20 17:28:53 kernel: [*05/20/2019 17:28:57.7510] Failed to join controller ***-****-*.
May 20 17:28:53 kernel: [*05/20/2019 17:28:57.7510] Failed to join controller.
May 20 17:28:53 kernel: [*05/20/2019 17:28:53.0000]

Re: APs won't join vWLC 8.3

Austin Godbey — Mon, 20 May 2019 19:38:08 GMT

Also I just updated from 8.3.112 to 8.3.143 in case it was a bug with that release. APs still won't join controller.

Re: APs won't join vWLC 8.3

Jurgens L — Mon, 20 May 2019 21:16:44 GMT

Have you tried to clear the CAPWAP settings on one of the AP's?

Re: APs won't join vWLC 8.3

Austin Godbey — Mon, 20 May 2019 21:49:27 GMT

Yes if we're talking about Wireless>AP>General "Clear all config" or "Clear config Except Static IP"

Re: APs won't join vWLC 8.3

RaffyLindogan — Tue, 21 May 2019 02:08:10 GMT

Hi mate,

It seems on primary wlc, you are running Local Significant Cert.

You may have not configured it on the other WLC.

Basically AP is not joining since it is not trusting the 2nd WLC.

Can you run this command on both WLC "show certificate lsc summary"

Cheers,

Raffy

Re: APs won't join vWLC 8.3

Austin Godbey — Tue, 21 May 2019 14:59:57 GMT

PRIMARY CONTROLLER

(Cisco Controller) >show certificate lsc summary

LSC Enabled...................................... No
LSC CA-Server.................................... None

LSC AP-Provisioning.............................. No

LSC Params:
Country......................................
State........................................
City.........................................
Orgn.........................................
Dept.........................................
Email........................................
KeySize...................................... 2048

LSC Certs:
CA Cert...................................... Not Configured
RA Cert...................................... Not Configured
DEV Cert..................................... Not Configured

SECONDARY CONTROLLER

(Cisco Controller) >show certificate lsc sum

LSC Enabled...................................... No
LSC CA-Server.................................... None

LSC AP-Provisioning.............................. No

LSC Certs:
CA Cert...................................... Not Configured
RA Cert...................................... Not Configured
DEV Cert..................................... Not Configured

Re: APs won't join vWLC 8.3

Sathiyanarayanan Ravindran — Tue, 21 May 2019 18:14:47 GMT

Issue these commands on Secondary WLC and check the status.

(Cisco Controller) >config ap cert-expiry-ignore mic enable
(Cisco Controller) >config ap cert-expiry-ignore ssc enable

Re: APs won't join vWLC 8.3

Austin Godbey — Tue, 21 May 2019 19:39:23 GMT

Made this change and it did not do anything. AP still won't join.

Re: APs won't join vWLC 8.3

Sathiyanarayanan Ravindran — Tue, 21 May 2019 19:56:54 GMT

Please run the below debug commands and share the output.

(Cisco Controller) >debug capwap events enable
(Cisco Controller) >debug pm pki enable
(Cisco Controller) >debug capwap packet enable
(Cisco Controller) >debug mac addr <ap-mac-address>

Re: APs won't join vWLC 8.3

Austin Godbey — Tue, 21 May 2019 20:41:03 GMT

(Cisco Controller) >debug capwap packet enable

Warning! Enabling this debug with large number of AP's connected, May flood console with message logs !!!.
Are you sure you want to continue? (y/N)y

(Cisco Controller) >
(Cisco Controller) >*spamApTask7: May 21 14:37:07.957: <<<< Start of CAPWAP Packet >>>>
*spamApTask7: May 21 14:37:07.957: CAPWAP Control mesg Recd from 172.19.38.26, Port 5264
*spamApTask7: May 21 14:37:07.957: HLEN 4, Radio ID 0, WBID 1
*spamApTask7: May 21 14:37:07.957: Msg Type : CAPWAP_PRIMARY_DISCOVERY_REQUEST
*spamApTask7: May 21 14:37:07.957: Msg Length : 187
*spamApTask7: May 21 14:37:07.957: Msg SeqNum : 0
*spamApTask7: May 21 14:37:07.957:
*spamApTask7: May 21 14:37:07.957: Type : CAPWAP_MSGELE_DISCOVERY_TYPE, Length 1
*spamApTask7: May 21 14:37:07.957: Discovery Type : CAPWAP_DISCOVERY_TYPE_STATIC_CONFIG
*spamApTask7: May 21 14:37:07.957:
*spamApTask7: May 21 14:37:07.957: Type : CAPWAP_MSGELE_WTP_BOARD_DATA, Length 73
*spamApTask7: May 21 14:37:07.957: Vendor Identifier : 0x00409600
*spamApTask7: May 21 14:37:07.957: WTP_SERIAL_NUMBER : AIR-AP3802I-B-K9
*spamApTask7: May 21 14:37:07.957:
*spamApTask7: May 21 14:37:07.957: Type : CAPWAP_MSGELE_WTP_DESCRIPTOR, Length 40
*spamApTask7: May 21 14:37:07.957: Maximum Radios Supported : 2
*spamApTask7: May 21 14:37:07.957: Radios in Use : 2
*spamApTask7: May 21 14:37:07.957: Encryption Capabilities : 0x00 0x01
*spamApTask7: May 21 14:37:07.957:
*spamApTask7: May 21 14:37:07.957: Type : CAPWAP_MSGELE_WTP_FRAME_TUNNEL, Length 1
*spamApTask7: May 21 14:37:07.957: WTP Frame Tunnel Mode : NATIVE_FRAME_TUNNEL_MODE
*spamApTask7: May 21 14:37:07.957:
*spamApTask7: May 21 14:37:07.957: Type : CAPWAP_MSGELE_WTP_MAC_TYPE, Length 1
*spamApTask7: May 21 14:37:07.957: WTP Mac Type : SPLIT_MAC
*spamApTask7: May 21 14:37:07.957:
*spamApTask7: May 21 14:37:07.957: Type : CAPWAP_DOT11_MSGELE_WTP_RADIO_INFORMATION, Length 5
*spamApTask7: May 21 14:37:07.957: Radio ID : 0
*spamApTask7: May 21 14:37:07.957: Radio Type : RADIO_80211A
*spamApTask7: May 21 14:37:07.957: Radio Type : RADIO_80211B
*spamApTask7: May 21 14:37:07.957:
*spamApTask7: May 21 14:37:07.957: Type : CAPWAP_DOT11_MSGELE_WTP_RADIO_INFORMATION, Length 5
*spamApTask7: May 21 14:37:07.957: Radio ID : 1
*spamApTask7: May 21 14:37:07.957: Radio Type : RADIO_80211B
*spamApTask7: May 21 14:37:07.957:
*spamApTask7: May 21 14:37:07.957: Type : CAPWAP_MSGELE_VENDOR_SPECIFIC_PAYLOAD, Length 10
*spamApTask7: May 21 14:37:07.957: Vendor Identifier : 0x00409600
*spamApTask7: May 21 14:37:07.957:
IE : UNKNOWN IE 207

*spamApTask7: May 21 14:37:07.957: IE Length : 4

*spamApTask7: May 21 14:37:07.957: Decode routine not available, Printing Hex Dump

*spamApTask7: May 21 14:37:07.957: 00000000: 01 00 00 01 ....

*spamApTask7: May 21 14:37:07.957:
*spamApTask7: May 21 14:37:07.957: Type : CAPWAP_MSGELE_VENDOR_SPECIFIC_PAYLOAD, Length 15
*spamApTask7: May 21 14:37:07.957: Vendor Identifier : 0x00409600
*spamApTask7: May 21 14:37:07.957:
IE : RAD_NAME_PAYLOAD

*spamApTask7: May 21 14:37:07.957: IE Length : 9

*spamApTask7: May 21 14:37:07.957: Rad Name :
*spamApTask7: May 21 14:37:07.957: AP-BFP-31

*spamApTask7: May 21 14:37:07.957: <<<< End of CAPWAP Packet >>>>

*spamApTask7: May 21 14:37:13.659: <<<< Start of CAPWAP Packet >>>>
*spamApTask7: May 21 14:37:13.659: CAPWAP Control mesg Recd from 172.19.38.26, Port 5264
*spamApTask7: May 21 14:37:13.659: HLEN 4, Radio ID 0, WBID 1
*spamApTask7: May 21 14:37:13.659: Msg Type : CAPWAP_DISCOVERY_REQUEST
*spamApTask7: May 21 14:37:13.659: Msg Length : 187
*spamApTask7: May 21 14:37:13.659: Msg SeqNum : 0
*spamApTask7: May 21 14:37:13.659:
*spamApTask7: May 21 14:37:13.659: Type : CAPWAP_MSGELE_DISCOVERY_TYPE, Length 1
*spamApTask7: May 21 14:37:13.659: Discovery Type : CAPWAP_DISCOVERY_TYPE_STATIC_CONFIG
*spamApTask7: May 21 14:37:13.659:
*spamApTask7: May 21 14:37:13.659: Type : CAPWAP_MSGELE_WTP_BOARD_DATA, Length 73
*spamApTask7: May 21 14:37:13.659: Vendor Identifier : 0x00409600
*spamApTask7: May 21 14:37:13.659: WTP_SERIAL_NUMBER : AIR-AP3802I-B-K9
*spamApTask7: May 21 14:37:13.659:
*spamApTask7: May 21 14:37:13.659: Type : CAPWAP_MSGELE_WTP_DESCRIPTOR, Length 40
*spamApTask7: May 21 14:37:13.659: Maximum Radios Supported : 2
*spamApTask7: May 21 14:37:13.659: Radios in Use : 2
*spamApTask7: May 21 14:37:13.659: Encryption Capabilities : 0x00 0x01
*spamApTask7: May 21 14:37:13.659:
*spamApTask7: May 21 14:37:13.659: Type : CAPWAP_MSGELE_WTP_FRAME_TUNNEL, Length 1
*spamApTask7: May 21 14:37:13.659: WTP Frame Tunnel Mode : NATIVE_FRAME_TUNNEL_MODE
*spamApTask7: May 21 14:37:13.659:
*spamApTask7: May 21 14:37:13.659: Type : CAPWAP_MSGELE_WTP_MAC_TYPE, Length 1
*spamApTask7: May 21 14:37:13.659: WTP Mac Type : SPLIT_MAC
*spamApTask7: May 21 14:37:13.659:
*spamApTask7: May 21 14:37:13.659: Type : CAPWAP_DOT11_MSGELE_WTP_RADIO_INFORMATION, Length 5
*spamApTask7: May 21 14:37:13.659: Radio ID : 0
*spamApTask7: May 21 14:37:13.659: Radio Type : RADIO_80211A
*spamApTask7: May 21 14:37:13.659: Radio Type : RADIO_80211B
*spamApTask7: May 21 14:37:13.659:
*spamApTask7: May 21 14:37:13.659: Type : CAPWAP_DOT11_MSGELE_WTP_RADIO_INFORMATION, Length 5
*spamApTask7: May 21 14:37:13.659: Radio ID : 1
*spamApTask7: May 21 14:37:13.659: Radio Type : RADIO_80211B
*spamApTask7: May 21 14:37:13.659:
*spamApTask7: May 21 14:37:13.659: Type : CAPWAP_MSGELE_VENDOR_SPECIFIC_PAYLOAD, Length 10
*spamApTask7: May 21 14:37:13.659: Vendor Identifier : 0x00409600
*spamApTask7: May 21 14:37:13.659:
IE : UNKNOWN IE 207

*spamApTask7: May 21 14:37:13.659: IE Length : 4

*spamApTask7: May 21 14:37:13.659: Decode routine not available, Printing Hex Dump

*spamApTask7: May 21 14:37:13.659: 00000000: 01 00 00 01 ....

*spamApTask7: May 21 14:37:13.659:
*spamApTask7: May 21 14:37:13.659: Type : CAPWAP_MSGELE_VENDOR_SPECIFIC_PAYLOAD, Length 15
*spamApTask7: May 21 14:37:13.659: Vendor Identifier : 0x00409600
*spamApTask7: May 21 14:37:13.659:
IE : RAD_NAME_PAYLOAD

*spamApTask7: May 21 14:37:13.659: IE Length : 9

*spamApTask7: May 21 14:37:13.659: Rad Name :
*spamApTask7: May 21 14:37:13.659: AP-BFP-31

*spamApTask7: May 21 14:37:13.659: <<<< End of CAPWAP Packet >>>>

*spamApTask7: May 21 14:37:13.660: <<<< Start of CAPWAP Packet >>>>
*spamApTask7: May 21 14:37:13.660: CAPWAP Control mesg Recd from 172.19.38.26, Port 5264
*spamApTask7: May 21 14:37:13.660: HLEN 4, Radio ID 0, WBID 1
*spamApTask7: May 21 14:37:13.660: Msg Type : CAPWAP_DISCOVERY_REQUEST
*spamApTask7: May 21 14:37:13.660: Msg Length : 187
*spamApTask7: May 21 14:37:13.660: Msg SeqNum : 0
*spamApTask7: May 21 14:37:13.660:
*spamApTask7: May 21 14:37:13.660: Type : CAPWAP_MSGELE_DISCOVERY_TYPE, Length 1
*spamApTask7: May 21 14:37:13.660: Discovery Type : CAPWAP_DISCOVERY_TYPE_STATIC_CONFIG
*spamApTask7: May 21 14:37:13.660:
*spamApTask7: May 21 14:37:13.660: Type : CAPWAP_MSGELE_WTP_BOARD_DATA, Length 73
*spamApTask7: May 21 14:37:13.660: Vendor Identifier : 0x00409600
*spamApTask7: May 21 14:37:13.660: WTP_SERIAL_NUMBER : AIR-AP3802I-B-K9
*spamApTask7: May 21 14:37:13.660:
*spamApTask7: May 21 14:37:13.660: Type : CAPWAP_MSGELE_WTP_DESCRIPTOR, Length 40
*spamApTask7: May 21 14:37:13.660: Maximum Radios Supported : 2
*spamApTask7: May 21 14:37:13.660: Radios in Use : 2
*spamApTask7: May 21 14:37:13.660: Encryption Capabilities : 0x00 0x01
*spamApTask7: May 21 14:37:13.660:
*spamApTask7: May 21 14:37:13.660: Type : CAPWAP_MSGELE_WTP_FRAME_TUNNEL, Length 1
*spamApTask7: May 21 14:37:13.660: WTP Frame Tunnel Mode : NATIVE_FRAME_TUNNEL_MODE
*spamApTask7: May 21 14:37:13.660:
*spamApTask7: May 21 14:37:13.660: Type : CAPWAP_MSGELE_WTP_MAC_TYPE, Length 1
*spamApTask7: May 21 14:37:13.660: WTP Mac Type : SPLIT_MAC
*spamApTask7: May 21 14:37:13.660:
*spamApTask7: May 21 14:37:13.660: Type : CAPWAP_DOT11_MSGELE_WTP_RADIO_INFORMATION, Length 5
*spamApTask7: May 21 14:37:13.660: Radio ID : 0
*spamApTask7: May 21 14:37:13.660: Radio Type : RADIO_80211A
*spamApTask7: May 21 14:37:13.660: Radio Type : RADIO_80211B
*spamApTask7: May 21 14:37:13.660:
*spamApTask7: May 21 14:37:13.660: Type : CAPWAP_DOT11_MSGELE_WTP_RADIO_INFORMATION, Length 5
*spamApTask7: May 21 14:37:13.660: Radio ID : 1
*spamApTask7: May 21 14:37:13.660: Radio Type : RADIO_80211B
*spamApTask7: May 21 14:37:13.660:
*spamApTask7: May 21 14:37:13.660: Type : CAPWAP_MSGELE_VENDOR_SPECIFIC_PAYLOAD, Length 10
*spamApTask7: May 21 14:37:13.660: Vendor Identifier : 0x00409600
*spamApTask7: May 21 14:37:13.660:
IE : UNKNOWN IE 207

*spamApTask7: May 21 14:37:13.660: IE Length : 4

*spamApTask7: May 21 14:37:13.660: Decode routine not available, Printing Hex Dump

*spamApTask7: May 21 14:37:13.660: 00000000: 01 00 00 01 ....

*spamApTask7: May 21 14:37:13.660:
*spamApTask7: May 21 14:37:13.660: Type : CAPWAP_MSGELE_VENDOR_SPECIFIC_PAYLOAD, Length 15
*spamApTask7: May 21 14:37:13.660: Vendor Identifier : 0x00409600
*spamApTask7: May 21 14:37:13.660:
IE : RAD_NAME_PAYLOAD

*spamApTask7: May 21 14:37:13.660: IE Length : 9

*spamApTask7: May 21 14:37:13.660: Rad Name :
*spamApTask7: May 21 14:37:13.660: AP-BFP-31

*spamApTask7: May 21 14:37:13.660: <<<< End of CAPWAP Packet >>>>

*spamApTask7: May 21 14:37:13.666: <<<< Start of CAPWAP Packet >>>>
*spamApTask7: May 21 14:37:13.666: CAPWAP Control mesg Recd from 172.19.38.26, Port 5264
*spamApTask7: May 21 14:37:13.666: HLEN 4, Radio ID 0, WBID 1
*spamApTask7: May 21 14:37:13.666: Msg Type : CAPWAP_DISCOVERY_REQUEST
*spamApTask7: May 21 14:37:13.666: Msg Length : 187
*spamApTask7: May 21 14:37:13.666: Msg SeqNum : 0
*spamApTask7: May 21 14:37:13.666:
*spamApTask7: May 21 14:37:13.666: Type : CAPWAP_MSGELE_DISCOVERY_TYPE, Length 1
*spamApTask7: May 21 14:37:13.666: Discovery Type : CAPWAP_DISCOVERY_TYPE_STATIC_CONFIG
*spamApTask7: May 21 14:37:13.666:
*spamApTask7: May 21 14:37:13.666: Type : CAPWAP_MSGELE_WTP_BOARD_DATA, Length 73
*spamApTask7: May 21 14:37:13.666: Vendor Identifier : 0x00409600
*spamApTask7: May 21 14:37:13.666: WTP_SERIAL_NUMBER : AIR-AP3802I-B-K9
*spamApTask7: May 21 14:37:13.666:
*spamApTask7: May 21 14:37:13.666: Type : CAPWAP_MSGELE_WTP_DESCRIPTOR, Length 40
*spamApTask7: May 21 14:37:13.666: Maximum Radios Supported : 2
*spamApTask7: May 21 14:37:13.666: Radios in Use : 2
*spamApTask7: May 21 14:37:13.666: Encryption Capabilities : 0x00 0x01
*spamApTask7: May 21 14:37:13.666:
*spamApTask7: May 21 14:37:13.666: Type : CAPWAP_MSGELE_WTP_FRAME_TUNNEL, Length 1
*spamApTask7: May 21 14:37:13.666: WTP Frame Tunnel Mode : NATIVE_FRAME_TUNNEL_MODE
*spamApTask7: May 21 14:37:13.666:
*spamApTask7: May 21 14:37:13.666: Type : CAPWAP_MSGELE_WTP_MAC_TYPE, Length 1
*spamApTask7: May 21 14:37:13.666: WTP Mac Type : SPLIT_MAC
*spamApTask7: May 21 14:37:13.666:
*spamApTask7: May 21 14:37:13.666: Type : CAPWAP_DOT11_MSGELE_WTP_RADIO_INFORMATION, Length 5
*spamApTask7: May 21 14:37:13.666: Radio ID : 0
*spamApTask7: May 21 14:37:13.666: Radio Type : RADIO_80211A
*spamApTask7: May 21 14:37:13.666: Radio Type : RADIO_80211B
*spamApTask7: May 21 14:37:13.666:
*spamApTask7: May 21 14:37:13.666: Type : CAPWAP_DOT11_MSGELE_WTP_RADIO_INFORMATION, Length 5
*spamApTask7: May 21 14:37:13.666: Radio ID : 1
*spamApTask7: May 21 14:37:13.666: Radio Type : RADIO_80211B
*spamApTask7: May 21 14:37:13.666:
*spamApTask7: May 21 14:37:13.666: Type : CAPWAP_MSGELE_VENDOR_SPECIFIC_PAYLOAD, Length 10
*spamApTask7: May 21 14:37:13.666: Vendor Identifier : 0x00409600
*spamApTask7: May 21 14:37:13.666:
IE : UNKNOWN IE 207

*spamApTask7: May 21 14:37:13.666: IE Length : 4

*spamApTask7: May 21 14:37:13.666: Decode routine not available, Printing Hex Dump

*spamApTask7: May 21 14:37:13.666: 00000000: 01 00 00 01 ....

*spamApTask7: May 21 14:37:13.666:
*spamApTask7: May 21 14:37:13.666: Type : CAPWAP_MSGELE_VENDOR_SPECIFIC_PAYLOAD, Length 15
*spamApTask7: May 21 14:37:13.666: Vendor Identifier : 0x00409600
*spamApTask7: May 21 14:37:13.666:
IE : RAD_NAME_PAYLOAD

*spamApTask7: May 21 14:37:13.666: IE Length : 9

*spamApTask7: May 21 14:37:13.666: Rad Name :
*spamApTask7: May 21 14:37:13.666: AP-BFP-31

*spamApTask7: May 21 14:37:13.666: <<<< End of CAPWAP Packet >>>>

Re: APs won't join vWLC 8.3

Austin Godbey — Tue, 21 May 2019 20:41:34 GMT

(Cisco Controller) >debug capwap events enable

(Cisco Controller) >*spamApTask7: May 21 14:30:17.438: sshpmFreePublicKeyHandle: freeing public key

*spamApTask7: May 21 14:34:44.466: 6c:8b:d3:ee:a0:60 Primary Discovery Request from 172.19.38.26:5264

*spamApTask7: May 21 14:34:44.466: 6c:8b:d3:ee:a0:60 Join Priority Processing status = 0, Incoming Ap's Priority 1, MaxLrads = 200, MaxLicense=93 joined Aps =0
*spamApTask7: May 21 14:34:44.466: 6c:8b:d3:ee:a0:60 apType = 52 apModel: AIR-AP3802I-B-K9

*spamApTask7: May 21 14:34:44.466: 6c:8b:d3:ee:a0:60 apType: Ox34 bundleApImageVer: 8.3.150.0
*spamApTask7: May 21 14:34:44.466: 6c:8b:d3:ee:a0:60 version:8 release:3 maint:150 build:0
*spamApTask7: May 21 14:34:44.466: 6c:8b:d3:ee:a0:60 Primary Discovery Response sent to 172.19.38.26:5264

*spamApTask7: May 21 14:34:50.165: 6c:8b:d3:ee:a0:60 Discovery Request from 172.19.38.26:5264

*spamApTask7: May 21 14:34:50.165: 6c:8b:d3:ee:a0:60 Join Priority Processing status = 0, Incoming Ap's Priority 1, MaxLrads = 200, MaxLicense=93 joined Aps =0
*spamApTask7: May 21 14:34:50.165: 6c:8b:d3:ee:a0:60 apType = 52 apModel: AIR-AP3802I-B-K9

*spamApTask7: May 21 14:34:50.165: 6c:8b:d3:ee:a0:60 apType: Ox34 bundleApImageVer: 8.3.150.0
*spamApTask7: May 21 14:34:50.165: 6c:8b:d3:ee:a0:60 version:8 release:3 maint:150 build:0
*spamApTask7: May 21 14:34:50.165: 6c:8b:d3:ee:a0:60 Discovery Response sent to 172.19.38.26 port 5264

*spamApTask7: May 21 14:34:50.165: 6c:8b:d3:ee:a0:60 Discovery Response sent to 172.19.38.26:5264

*spamApTask7: May 21 14:34:50.166: 6c:8b:d3:ee:a0:60 Discovery Request from 172.19.38.26:5264

*spamApTask7: May 21 14:34:50.166: 6c:8b:d3:ee:a0:60 Join Priority Processing status = 0, Incoming Ap's Priority 1, MaxLrads = 200, MaxLicense=93 joined Aps =0
*spamApTask7: May 21 14:34:50.166: 6c:8b:d3:ee:a0:60 apType = 52 apModel: AIR-AP3802I-B-K9

*spamApTask7: May 21 14:34:50.166: 6c:8b:d3:ee:a0:60 apType: Ox34 bundleApImageVer: 8.3.150.0
*spamApTask7: May 21 14:34:50.166: 6c:8b:d3:ee:a0:60 version:8 release:3 maint:150 build:0
*spamApTask7: May 21 14:34:50.166: 6c:8b:d3:ee:a0:60 Discovery Response sent to 172.19.38.26 port 5264

*spamApTask7: May 21 14:34:50.166: 6c:8b:d3:ee:a0:60 Discovery Response sent to 172.19.38.26:5264

*spamApTask7: May 21 14:34:50.170: 6c:8b:d3:ee:a0:60 Discovery Request from 172.19.38.26:5264

*spamApTask7: May 21 14:34:50.170: 6c:8b:d3:ee:a0:60 Join Priority Processing status = 0, Incoming Ap's Priority 1, MaxLrads = 200, MaxLicense=93 joined Aps =0
*spamApTask7: May 21 14:34:50.170: 6c:8b:d3:ee:a0:60 apType = 52 apModel: AIR-AP3802I-B-K9

*spamApTask7: May 21 14:34:50.170: 6c:8b:d3:ee:a0:60 apType: Ox34 bundleApImageVer: 8.3.150.0
*spamApTask7: May 21 14:34:50.170: 6c:8b:d3:ee:a0:60 version:8 release:3 maint:150 build:0
*spamApTask7: May 21 14:34:50.170: 6c:8b:d3:ee:a0:60 Discovery Response sent to 172.19.38.26 port 5264

*spamApTask7: May 21 14:34:50.170: 6c:8b:d3:ee:a0:60 Discovery Response sent to 172.19.38.26:5264

*spamApTask7: May 21 14:34:59.666: 00:08:e3:ff:fd:90 DTLS connection not found, creating new connection for 172.19.38.26 (5264) 10.2.0.98 (5246)

*spamApTask7: May 21 14:34:59.698: 00:08:e3:ff:fd:90 DTLS Session established server (10.2.0.98:5246), client (172.19.38.26:5264)
*spamApTask7: May 21 14:34:59.699: 00:08:e3:ff:fd:90 Starting wait join timer for AP: 172.19.38.26:5264

*spamApTask7: May 21 14:34:59.709: 00:08:e3:ff:fd:90 DTLS connection closed event receivedserver (10.2.0.98/5246) client (172.19.38.26/5264)
*spamApTask7: May 21 14:34:59.710: 00:08:e3:ff:fd:90 No entry exists for AP (172.19.38.26/5264)
*spamApTask7: May 21 14:34:59.710: 00:08:e3:ff:fd:90 Deleting AP entry 172.19.38.26:5264 from temporary database.
*spamApTask7: May 21 14:35:04.421: 00:08:e3:ff:fd:90 DTLS connection not found, creating new connection for 172.19.38.26 (5264) 10.2.0.98 (5246)

*spamApTask7: May 21 14:35:04.453: 00:08:e3:ff:fd:90 DTLS Session established server (10.2.0.98:5246), client (172.19.38.26:5264)
*spamApTask7: May 21 14:35:04.453: 00:08:e3:ff:fd:90 Starting wait join timer for AP: 172.19.38.26:5264

*spamApTask7: May 21 14:35:04.464: 00:08:e3:ff:fd:90 DTLS connection closed event receivedserver (10.2.0.98/5246) client (172.19.38.26/5264)
*spamApTask7: May 21 14:35:04.464: 00:08:e3:ff:fd:90 No entry exists for AP (172.19.38.26/5264)
*spamApTask7: May 21 14:35:04.464: 00:08:e3:ff:fd:90 Deleting AP entry 172.19.38.26:5264 from temporary database.
*spamApTask7: May 21 14:35:09.169: 00:08:e3:ff:fd:90 DTLS connection not found, creating new connection for 172.19.38.26 (5264) 10.2.0.98 (5246)

*spamApTask7: May 21 14:35:09.204: 00:08:e3:ff:fd:90 DTLS Session established server (10.2.0.98:5246), client (172.19.38.26:5264)
*spamApTask7: May 21 14:35:09.204: 00:08:e3:ff:fd:90 Starting wait join timer for AP: 172.19.38.26:5264

*spamApTask7: May 21 14:35:09.215: 00:08:e3:ff:fd:90 DTLS connection closed event receivedserver (10.2.0.98/5246) client (172.19.38.26/5264)
*spamApTask7: May 21 14:35:09.215: 00:08:e3:ff:fd:90 No entry exists for AP (172.19.38.26/5264)
*spamApTask7: May 21 14:35:09.215: 00:08:e3:ff:fd:90 Deleting AP entry 172.19.38.26:5264 from temporary database.

(Cisco Controller) >

Re: APs won't join vWLC 8.3

Austin Godbey — Tue, 21 May 2019 20:42:43 GMT

(Cisco Controller) >debug pm pki enable

(Cisco Controller) >*spamApTask7: May 21 14:42:10.147: sshpmGetCID: called to evaluate <cscoSha2IdCert>

*spamApTask7: May 21 14:42:10.147: sshpmGetCID: failed to find matching cert name cscoSha2IdCert

*spamApTask7: May 21 14:42:10.147: GetIDCert: Using SHA2 Id cert on WLC

*spamApTask7: May 21 14:42:10.147: sshpmGetCID: called to evaluate <cscoDefaultIdCert>

*spamApTask7: May 21 14:42:10.147: sshpmGetCID: Found matching ID cert cscoDefaultIdCert in row 2
*spamApTask7: May 21 14:42:10.147: Get Cert from CID: For CID 148fdb8c certType 1
*spamApTask7: May 21 14:42:10.147: Get Cert from CID: Found match of ID Cert in row 2
*spamApTask7: May 21 14:42:10.147: sshpmGetCID: called to evaluate <cscoSha2IdCert>

*spamApTask7: May 21 14:42:10.147: sshpmGetCID: failed to find matching cert name cscoSha2IdCert

*spamApTask7: May 21 14:42:10.147: GetDERIDKey: Using SHA2 Id cert Private Keys on WLC

*spamApTask7: May 21 14:42:10.147: sshpmGetCID: called to evaluate <cscoDefaultIdCert>

*spamApTask7: May 21 14:42:10.147: sshpmGetCID: Found matching ID cert cscoDefaultIdCert in row 2
*spamApTask7: May 21 14:42:10.147: GetPrivateKey: called to get key for CID 148fdb8c

*spamApTask7: May 21 14:42:10.147: Private Key found row 2 KeyBufLen 2048 Keylen 1190 PrivateKeyPtr 0x7f31ad87a9b0

*spamApTask7: May 21 14:42:10.157: OpenSSL Get Issuer Handles: locking ca cert table

*spamApTask7: May 21 14:42:10.157: OpenSSL Get Issuer Handles: x509 subject_name /C=US/ST=California/L=San Jose/O=Cisco Systems/CN=AP3G3-6C8BD313E052/emailAddress=support@cisco.com

*spamApTask7: May 21 14:42:10.157: OpenSSL Get Issuer Handles: issuer_name /O=Cisco/CN=Cisco Manufacturing CA SHA2

*spamApTask7: May 21 14:42:10.157: OpenSSL Get Issuer Handles: CN AP3G3-6C8BD313E052

*spamApTask7: May 21 14:42:10.157: OpenSSL Get Issuer Handles: issuerCertCN Cisco Manufacturing CA SHA2

*spamApTask7: May 21 14:42:10.157: GetMac: MAC: 6c8b.d313.e052

*spamApTask7: May 21 14:42:10.157: OpenSSL Get Issuer Handles: openssl Mac Address in subject is 6c:8b:d3:13:e0:52

*spamApTask7: May 21 14:42:10.157: OpenSSL Get Issuer Handles: Cert Name in subject is AP3G3-6C8BD313E052

*spamApTask7: May 21 14:42:10.157: OpenSSL Get Issuer Handles: Extracted cert issuer from subject name.

*spamApTask7: May 21 14:42:10.157: NMSP:: Algo name matched SHA256

*spamApTask7: May 21 14:42:10.157: OpenSSL Get Issuer Handles: Cert is issued by Cisco Systems.

*spamApTask7: May 21 14:42:10.157: Retrieving x509 cert for CertName cscoMfgSha2CaCert

*spamApTask7: May 21 14:42:10.157: sshpmGetCID: called to evaluate <cscoMfgSha2CaCert>

*spamApTask7: May 21 14:42:10.157: sshpmGetCID: Found matching CA cert cscoMfgSha2CaCert in row 7
*spamApTask7: May 21 14:42:10.157: Found CID 227056ab for certname cscoMfgSha2CaCert

*spamApTask7: May 21 14:42:10.157: CACertTable: Found matching CID cscoMfgSha2CaCert in row 7 x509 0x7f31b05cb218
*spamApTask7: May 21 14:42:10.157: Retrieving x509 cert for CertName cscoRootSha2CaCert

*spamApTask7: May 21 14:42:10.157: sshpmGetCID: called to evaluate <cscoRootSha2CaCert>

*spamApTask7: May 21 14:42:10.157: sshpmGetCID: Found matching CA cert cscoRootSha2CaCert in row 6
*spamApTask7: May 21 14:42:10.157: Found CID 2883937e for certname cscoRootSha2CaCert

*spamApTask7: May 21 14:42:10.157: CACertTable: Found matching CID cscoRootSha2CaCert in row 6 x509 0x7f31b05cb4e8
*spamApTask7: May 21 14:42:10.157: Verify User Certificate: X509 Cert Verification return code: 1
*spamApTask7: May 21 14:42:10.157: Verify User Certificate: X509 Cert Verification result text: ok
*spamApTask7: May 21 14:42:10.157: sshpmGetCID: called to evaluate <cscoMfgSha2CaCert>

*spamApTask7: May 21 14:42:10.157: sshpmGetCID: Found matching CA cert cscoMfgSha2CaCert in row 7
*spamApTask7: May 21 14:42:10.157: Verify User Certificate: OPENSSL X509_Verify: AP Cert Verfied Using >cscoMfgSha2CaCert<

*spamApTask7: May 21 14:42:10.157: OpenSSL Get Issuer Handles: Check cert validity times (allow expired YES)
*spamApTask7: May 21 14:42:10.157: sshpmGetCID: called to evaluate <cscoDefaultIdCert>

*spamApTask7: May 21 14:42:10.157: sshpmGetCID: Found matching ID cert cscoDefaultIdCert in row 2
*spamApTask7: May 21 14:42:10.157: sshpmFreePublicKeyHandle: called with 0x7f31ad64d368

*spamApTask7: May 21 14:42:10.157: sshpmFreePublicKeyHandle: freeing public key

*spamApTask7: May 21 14:42:14.898: sshpmGetCID: called to evaluate <cscoSha2IdCert>

*spamApTask7: May 21 14:42:14.898: sshpmGetCID: failed to find matching cert name cscoSha2IdCert

*spamApTask7: May 21 14:42:14.898: GetIDCert: Using SHA2 Id cert on WLC

*spamApTask7: May 21 14:42:14.899: sshpmGetCID: called to evaluate <cscoDefaultIdCert>

*spamApTask7: May 21 14:42:14.899: sshpmGetCID: Found matching ID cert cscoDefaultIdCert in row 2
*spamApTask7: May 21 14:42:14.899: Get Cert from CID: For CID 148fdb8c certType 1
*spamApTask7: May 21 14:42:14.899: Get Cert from CID: Found match of ID Cert in row 2
*spamApTask7: May 21 14:42:14.899: sshpmGetCID: called to evaluate <cscoSha2IdCert>

*spamApTask7: May 21 14:42:14.899: sshpmGetCID: failed to find matching cert name cscoSha2IdCert

*spamApTask7: May 21 14:42:14.899: GetDERIDKey: Using SHA2 Id cert Private Keys on WLC

*spamApTask7: May 21 14:42:14.899: sshpmGetCID: called to evaluate <cscoDefaultIdCert>

*spamApTask7: May 21 14:42:14.899: sshpmGetCID: Found matching ID cert cscoDefaultIdCert in row 2
*spamApTask7: May 21 14:42:14.899: GetPrivateKey: called to get key for CID 148fdb8c

*spamApTask7: May 21 14:42:14.899: Private Key found row 2 KeyBufLen 2048 Keylen 1190 PrivateKeyPtr 0x7f31ad87a9b0

*spamApTask7: May 21 14:42:14.920: OpenSSL Get Issuer Handles: locking ca cert table

*spamApTask7: May 21 14:42:14.920: OpenSSL Get Issuer Handles: x509 subject_name /C=US/ST=California/L=San Jose/O=Cisco Systems/CN=AP3G3-6C8BD313E052/emailAddress=support@cisco.com

*spamApTask7: May 21 14:42:14.920: OpenSSL Get Issuer Handles: issuer_name /O=Cisco/CN=Cisco Manufacturing CA SHA2

*spamApTask7: May 21 14:42:14.920: OpenSSL Get Issuer Handles: CN AP3G3-6C8BD313E052

*spamApTask7: May 21 14:42:14.920: OpenSSL Get Issuer Handles: issuerCertCN Cisco Manufacturing CA SHA2

*spamApTask7: May 21 14:42:14.920: GetMac: MAC: 6c8b.d313.e052

*spamApTask7: May 21 14:42:14.920: OpenSSL Get Issuer Handles: openssl Mac Address in subject is 6c:8b:d3:13:e0:52

*spamApTask7: May 21 14:42:14.920: OpenSSL Get Issuer Handles: Cert Name in subject is AP3G3-6C8BD313E052

*spamApTask7: May 21 14:42:14.920: OpenSSL Get Issuer Handles: Extracted cert issuer from subject name.

*spamApTask7: May 21 14:42:14.920: NMSP:: Algo name matched SHA256

*spamApTask7: May 21 14:42:14.920: OpenSSL Get Issuer Handles: Cert is issued by Cisco Systems.

*spamApTask7: May 21 14:42:14.920: Retrieving x509 cert for CertName cscoMfgSha2CaCert

*spamApTask7: May 21 14:42:14.920: sshpmGetCID: called to evaluate <cscoMfgSha2CaCert>

*spamApTask7: May 21 14:42:14.920: sshpmGetCID: Found matching CA cert cscoMfgSha2CaCert in row 7
*spamApTask7: May 21 14:42:14.920: Found CID 227056ab for certname cscoMfgSha2CaCert

*spamApTask7: May 21 14:42:14.920: CACertTable: Found matching CID cscoMfgSha2CaCert in row 7 x509 0x7f31b05cb218
*spamApTask7: May 21 14:42:14.920: Retrieving x509 cert for CertName cscoRootSha2CaCert

*spamApTask7: May 21 14:42:14.920: sshpmGetCID: called to evaluate <cscoRootSha2CaCert>

*spamApTask7: May 21 14:42:14.920: sshpmGetCID: Found matching CA cert cscoRootSha2CaCert in row 6
*spamApTask7: May 21 14:42:14.920: Found CID 2883937e for certname cscoRootSha2CaCert

*spamApTask7: May 21 14:42:14.920: CACertTable: Found matching CID cscoRootSha2CaCert in row 6 x509 0x7f31b05cb4e8
*spamApTask7: May 21 14:42:14.920: Verify User Certificate: X509 Cert Verification return code: 1
*spamApTask7: May 21 14:42:14.920: Verify User Certificate: X509 Cert Verification result text: ok
*spamApTask7: May 21 14:42:14.920: sshpmGetCID: called to evaluate <cscoMfgSha2CaCert>

*spamApTask7: May 21 14:42:14.920: sshpmGetCID: Found matching CA cert cscoMfgSha2CaCert in row 7
*spamApTask7: May 21 14:42:14.920: Verify User Certificate: OPENSSL X509_Verify: AP Cert Verfied Using >cscoMfgSha2CaCert<

*spamApTask7: May 21 14:42:14.920: OpenSSL Get Issuer Handles: Check cert validity times (allow expired YES)
*spamApTask7: May 21 14:42:14.920: sshpmGetCID: called to evaluate <cscoDefaultIdCert>

*spamApTask7: May 21 14:42:14.920: sshpmGetCID: Found matching ID cert cscoDefaultIdCert in row 2
*spamApTask7: May 21 14:42:14.921: sshpmFreePublicKeyHandle: called with 0x7f31ad64d800

*spamApTask7: May 21 14:42:14.921: sshpmFreePublicKeyHandle: freeing public key

(Cisco Controller) >debug pm pki disable

Re: APs won't join vWLC 8.3

kai schoene — Wed, 28 Aug 2019 09:56:32 GMT

I have the same issue. Did you figure it out?

Re: APs won't join vWLC 8.3

Austin Godbey — Wed, 28 Aug 2019 14:46:50 GMT

Yes. We rebuilt the new secondary controller and got the AP to join, but with other errors. Found the bug

https://bst.cloudapps.cisco.com/bugsearch/bug/CSCva69352 that was specific to two virtual WLCs used this way.

Under "Controller > Mobility Management > Mobility Groups" we added each of the controllers to each other and used their Hash Key. Once they were added the APs would be able to join either and fail back to either.

Good luck!