topic 2702i not joining 9800-CL in Wireless

2702i not joining 9800-CL

Mikolaj Moryto — Mon, 05 Jul 2021 17:57:55 GMT

Hi,

Because of some vWLC limitations I decided to test 9800-CL (16.11.1c) and after initial configuration I am stuck with AP not joining the controller. I followed all guides I could find and all discussions here I could find but nothing worked so far. Also tried to factory reset the AP with MODE button and clearing AP with following command:

delete flash:private-multiple-fs
clear capwap private-multiple-fs

As I want to migrate AP from vWLC to 9800 before factory resetting the AP I also tried to disable SSC and set authentication token on AP (through vWLC) and also on the 9800 side. That didn't work either.

Initial configuration completed:

- access (passwords, SSH, etc.)

- interfaces

- AP country

- wireless management interface

- wireless config vwlc-ssc key-size 2048 signature-algo sha256 password 0 <pwd>

AP (DHCP - 192.168.1.8):

*Sep  8 21:31:35.003: %LWAPP-3-CLIENTERRORLOG: Config load from flash failed. Initialising Cfg

*Sep  8 21:31:35.015: %LWAPP-3-CLIENTERRORLOG: Config load from flash failed. Initialising Cfg

*Sep  8 21:31:45.015: AP has SHA2 MIC certificate - Using SHA2 MIC certificate for DTLS.

*Sep  8 21:31:45.000: %CAPWAP-5-DTLSREQSEND: DTLS connection request sent peer_ip: 192.168.1.151 peer_port: 5246
*Sep  8 21:31:45.000: DTLS_CLIENT_EVENT: dtls_connectionDB_add_connection: Added Connection 0x57FC158 Server 192.168.1.151:147E, Client 192.168.1.8:2531

*Sep  8 21:31:45.000: DTLS_CLIENT_EVENT: dtls_process_HelloVerifyRequest: Processing...
*Sep  8 21:31:45.000: DTLS_CLIENT_EVENT: dtls_process_HelloVerifyRequest:
 dtls_process_HelloVerifyRequest: cookie_len = 20

*Sep  8 21:31:45.003: DTLS_CLIENT_EVENT: dtls_process_ServerHello: Processing...
*Sep  8 21:31:45.003: DTLS_CLIENT_EVENT: dtls_connection_set_cipher: Setting cipher to TLS_RSA_WITH_AES_128_CBC_SHA
*Sep  8 21:31:45.003: DTLS_CLIENT_EVENT: dtls_connection_is_resumed_session:
DTLS connection database is initialized

*Sep  8 21:31:45.003: DTLS_CLIENT_EVENT: dtls_connection_is_resumed_session:
DTLS connection database is initialized

*Sep  8 21:31:45.003: DTLS_CLIENT_EVENT: dtls_process_Certificate: Processing...
*Sep  8 21:31:45.023: DTLS_CLIENT_EVENT: wtpDtlsCallback: Certificate verified ok!
*Sep  8 21:31:45.023: DTLS_CLIENT_EVENT: dtls_process_CertificateRequest: Processingt...
*Sep  8 21:31:45.023: DTLS_CLIENT_EVENT: dtls_process_ServerHelloDone: Processing...
*Sep  8 21:31:45.023: DTLS_CLIENT_EVENT: wtpDtlsCallback: Certificate installed for PKI based authentication.
*Sep  8 21:31:45.023: DTLS_CLIENT_EVENT: dtls_send_Certificate: Sending 1 certificates
*Sep  8 21:31:45.023: DTLS_CLIENT_EVENT: dtls_handshake_fragment_and_send: Re-aligning the last fragmenti by 1
*Sep  8 21:31:45.219: DTLS_CLIENT_EVENT: dtls_connection_is_resumed_session:
DTLS connection database is initialized

*Sep  8 21:31:45.219: DTLS_CLIENT_EVENT: dtls_prf: Called...
*Sep  8 21:31:45.219: DTLS_CLIENT_EVENT: dtls_prf: Called...
*Sep  8 21:31:45.219: DTLS_CLIENT_EVENT: dtls_prf: Called...
*Sep  8 21:31:45.219: DTLS_CLIENT_EVENT: dtls_connection_is_resumed_session:
DTLS connection database is initialized

*Sep  8 21:31:45.231: DTLS_CLIENT_EVENT: dtls_connection_is_resumed_session:
DTLS connection database is initialized

*Sep  8 21:31:45.231: DTLS_CLIENT_EVENT: dtls_process_Finished: Processing...
*Sep  8 21:31:45.231: DTLS_CLIENT_EVENT: dtls_prf: Called...
*Sep  8 21:31:45.231: DTLS_CLIENT_EVENT: dtls_connection_is_resumed_session:
DTLS connection database is initialized

*Sep  8 21:31:45.231: DTLS_CLIENT_EVENT: dtls_connection_is_resumed_session:
DTLS connection database is initialized

*Sep  8 21:31:45.231: DTLS_CLIENT_EVENT: dtls_connection_established: Connection established!
*Sep  8 21:31:45.231: %CAPWAP-5-DTLSREQSUCC: DTLS connection created sucessfully peer_ip: 192.168.1.151 peer_port: 5246
*Sep  8 21:31:45.231: %CAPWAP-5-SENDJOIN: sending Join Request to 192.168.1.151
*Sep  8 21:31:45.231: DTLS_CLIENT_EVENT: wtpDtlsCallback: DTLS-Ctrl Connection 0x57FC158 established
*Sep  8 21:31:50.231: %CAPWAP-5-SENDJOIN: sending Join Request to 192.168.1.151

9800-CL (static - 192.168.1.151):

Sep  8 21:31:45.592: %CAPWAPAC_SMGR_TRACE_MESSAGE-3-EWLC_GEN_ERR: Chassis 1 R0/0: wncd: Error in Session-IP:192.168.1.8[9521] CAPWAP DTLS session closed for AP, cause: DTLS handshake error

2019/09/08 21:05:55.067 {wncd_x_R0-0}{1}: [capwapac-smgr-srvr] [23645]: (ERR): Unknown message type from load balancer, dropping
2019/09/08 21:05:55.067 {wncd_x_R0-0}{1}: [capwapac-smgr-srvr] [23645]: (ERR): Unknown message type from load balancer, dropping
2019/09/08 21:05:55.067 {wncd_x_R0-0}{1}: [capwapac-smgr-srvr] [23645]: (ERR): Unknown message type from load balancer, dropping
2019/09/08 21:05:55.084 {wncmgrd_R0-0}{1}: [capwapac-srvr] [23098]: (info): MAC: 80e0.1d92.530c  IP:192.168.1.8[9521], Discovery Request received
2019/09/08 21:05:55.097 {wncmgrd_R0-0}{1}: [capwapac-srvr] [23098]: (info): MAC: 80e0.1d92.530c  IP:192.168.1.8[9521], Discovery Response sent
2019/09/08 21:05:55.097 {wncmgrd_R0-0}{1}: [ewlc-infra-evq] [23098]: (debug): instance :0 port:11585MAC: 4b39.0001.000b  
2019/09/08 21:05:55.097 {wncmgrd_R0-0}{1}: [capwapac-srvr] [23098]: (info): MAC: 80e0.1d92.530c  Source IP:192.168.1.8[9521], Discovery handling completed for AP.
2019/09/08 21:05:55.097 {wncmgrd_R0-0}{1}: [capwapac-srvr] [23098]: (info): MAC: 80e0.1d92.530c  IP:192.168.1.8[9521], Discovery Request received
2019/09/08 21:05:55.097 {wncmgrd_R0-0}{1}: [capwapac-srvr] [23098]: (info): MAC: 80e0.1d92.530c  IP:192.168.1.8[9521], Discovery Response sent
2019/09/08 21:05:55.097 {wncmgrd_R0-0}{1}: [ewlc-infra-evq] [23098]: (debug): instance :0 port:11585MAC: 4b39.0001.000b  
2019/09/08 21:05:55.097 {wncmgrd_R0-0}{1}: [capwapac-srvr] [23098]: (info): MAC: 80e0.1d92.530c  Source IP:192.168.1.8[9521], Discovery handling completed for AP.
2019/09/08 21:05:55.097 {wncmgrd_R0-0}{1}: [capwapac-srvr] [23098]: (info): MAC: 80e0.1d92.530c  IP:192.168.1.8[9521], Discovery Request received
2019/09/08 21:05:55.097 {wncmgrd_R0-0}{1}: [capwapac-srvr] [23098]: (info): MAC: 80e0.1d92.530c  IP:192.168.1.8[9521], Discovery Response sent
2019/09/08 21:05:55.097 {wncmgrd_R0-0}{1}: [ewlc-infra-evq] [23098]: (debug): instance :0 port:11585MAC: 4b39.0001.000b  
2019/09/08 21:05:55.097 {wncmgrd_R0-0}{1}: [capwapac-srvr] [23098]: (info): MAC: 80e0.1d92.530c  Source IP:192.168.1.8[9521], Discovery handling completed for AP.
2019/09/08 21:05:55.098 {wncmgrd_R0-0}{1}: [capwapac-srvr] [23098]: (info): MAC: 80e0.1d92.530c  IP:192.168.1.8[9521], Discovery Request received
2019/09/08 21:05:55.098 {wncmgrd_R0-0}{1}: [capwapac-srvr] [23098]: (info): MAC: 80e0.1d92.530c  IP:192.168.1.8[9521], Discovery Response sent
2019/09/08 21:05:55.098 {wncmgrd_R0-0}{1}: [ewlc-infra-evq] [23098]: (debug): instance :0 port:11585MAC: 4b39.0001.000b  
2019/09/08 21:05:55.098 {wncmgrd_R0-0}{1}: [capwapac-srvr] [23098]: (info): MAC: 80e0.1d92.530c  Source IP:192.168.1.8[9521], Discovery handling completed for AP.
2019/09/08 21:05:55.098 {wncmgrd_R0-0}{1}: [capwapac-srvr] [23098]: (info): MAC: 80e0.1d92.530c  IP:192.168.1.8[9521], Discovery Request received
2019/09/08 21:05:55.098 {wncmgrd_R0-0}{1}: [capwapac-srvr] [23098]: (info): MAC: 80e0.1d92.530c  IP:192.168.1.8[9521], Discovery Response sent
2019/09/08 21:05:55.098 {wncmgrd_R0-0}{1}: [ewlc-infra-evq] [23098]: (debug): instance :0 port:11585MAC: 4b39.0001.000b  
2019/09/08 21:05:55.098 {wncmgrd_R0-0}{1}: [capwapac-srvr] [23098]: (info): MAC: 80e0.1d92.530c  Source IP:192.168.1.8[9521], Discovery handling completed for AP.
2019/09/08 21:05:55.098 {wncmgrd_R0-0}{1}: [capwapac-srvr] [23098]: (info): MAC: 80e0.1d92.530c  IP:192.168.1.8[9521], Discovery Request received
2019/09/08 21:05:55.098 {wncmgrd_R0-0}{1}: [capwapac-srvr] [23098]: (info): MAC: 80e0.1d92.530c  IP:192.168.1.8[9521], Discovery Response sent
2019/09/08 21:05:55.098 {wncmgrd_R0-0}{1}: [ewlc-infra-evq] [23098]: (debug): instance :0 port:11585MAC: 4b39.0001.000b  
2019/09/08 21:05:55.098 {wncmgrd_R0-0}{1}: [capwapac-srvr] [23098]: (info): MAC: 80e0.1d92.530c  Source IP:192.168.1.8[9521], Discovery handling completed for AP.
2019/09/08 21:06:05.082 {wncd_x_R0-0}{1}: [ewlc-infra-evq] [23645]: (info): DTLS record type: 22, handshake
2019/09/08 21:06:05.082 {wncd_x_R0-0}{1}: [ewlc-infra-evq] [23645]: (info): DTLS client hello
2019/09/08 21:06:05.083 {wncd_x_R0-0}{1}: [ewlc-infra-evq] [23645]: (info): DTLS record type: 22, handshake
2019/09/08 21:06:05.083 {wncd_x_R0-0}{1}: [ewlc-infra-evq] [23645]: (info): DTLS client hello
2019/09/08 21:06:05.083 {wncd_x_R0-0}{1}: [ewlc-infra-evq] [23645]: (info): DTLS record type: 22, handshake
2019/09/08 21:06:05.083 {wncd_x_R0-0}{1}: [ewlc-infra-evq] [23645]: (info): DTLS client hello
2019/09/08 21:06:05.084 {wncd_x_R0-0}{1}: [ewlc-infra-evq] [23645]: (info): DTLS record type: 22, handshake
2019/09/08 21:06:05.084 {wncd_x_R0-0}{1}: [ewlc-infra-evq] [23645]: (info): DTLS client hello
2019/09/08 21:06:05.085 {wncd_x_R0-0}{1}: [capwapac-smgr-srvr] [23645]: (ERR): IPv4: 192.168.1.8Failed to Process DTLS Hello message from loadbalancer server
2019/09/08 21:06:05.085 {wncd_x_R0-0}{1}: [capwapac-smgr-srvr] [23645]: (ERR): IPv4: 192.168.1.8Failed to Process DTLS Hello message from loadbalancer server
2019/09/08 21:06:05.303 {wncd_x_R0-0}{1}: [ewlc-infra-evq] [23645]: (info): DTLS record type: 22, handshake
2019/09/08 21:06:05.303 {wncd_x_R0-0}{1}: [ewlc-infra-evq] [23645]: (info): DTLS record type: 22, handshake
2019/09/08 21:06:05.303 {wncd_x_R0-0}{1}: [ewlc-infra-evq] [23645]: (info): DTLS record type: 22, handshake
2019/09/08 21:06:05.304 {wncd_x_R0-0}{1}: [ewlc-infra-evq] [23645]: (info): DTLS record type: 22, handshake
2019/09/08 21:06:05.304 {wncd_x_R0-0}{1}: [ewlc-infra-evq] [23645]: (info): DTLS record type: 22, handshake
2019/09/08 21:06:05.304 {wncd_x_R0-0}{1}: [ewlc-infra-evq] [23645]: (info): DTLS record type: 22, handshake
2019/09/08 21:06:05.304 {wncd_x_R0-0}{1}: [ewlc-infra-evq] [23645]: (info): DTLS record type: 22, handshake
2019/09/08 21:06:05.304 {wncd_x_R0-0}{1}: [ewlc-infra-evq] [23645]: (info): DTLS record type: 22, handshake
2019/09/08 21:06:05.304 {wncd_x_R0-0}{1}: [ewlc-infra-evq] [23645]: (info): DTLS record type: 0, unknown type
2019/09/08 21:06:05.304 {wncd_x_R0-0}{1}: [ewlc-infra-evq] [23645]: (info): DTLS record type: 22, handshake
2019/09/08 21:06:05.304 {wncd_x_R0-0}{1}: [ewlc-infra-evq] [23645]: (info): DTLS record type: 22, handshake
2019/09/08 21:06:05.304 {wncd_x_R0-0}{1}: [ewlc-infra-evq] [23645]: (info): DTLS record type: 22, handshake
2019/09/08 21:06:05.304 {wncd_x_R0-0}{1}: [ewlc-infra-evq] [23645]: (info): DTLS record type: 22, handshake
2019/09/08 21:06:05.304 {wncd_x_R0-0}{1}: [ewlc-infra-evq] [23645]: (info): DTLS record type: 22, handshake
2019/09/08 21:06:05.304 {wncd_x_R0-0}{1}: [ewlc-infra-evq] [23645]: (info): DTLS record type: 22, handshake
2019/09/08 21:06:05.304 {wncd_x_R0-0}{1}: [ewlc-infra-evq] [23645]: (info): DTLS record type: 0, unknown type
2019/09/08 21:06:05.304 {wncd_x_R0-0}{1}: [ewlc-infra-evq] [23645]: (info): DTLS record type: 22, handshake
2019/09/08 21:06:05.305 {wncd_x_R0-0}{1}: [ewlc-infra-evq] [23645]: (info): DTLS record type: 22, handshake
2019/09/08 21:06:05.305 {wncd_x_R0-0}{1}: [ewlc-infra-evq] [23645]: (info): DTLS record type: 22, handshake
2019/09/08 21:06:05.305 {wncd_x_R0-0}{1}: [ewlc-infra-evq] [23645]: (info): DTLS record type: 0, unknown type
2019/09/08 21:06:05.305 {wncd_x_R0-0}{1}: [ewlc-infra-evq] [23645]: (info): DTLS record type: 22, handshake
2019/09/08 21:06:05.308 {wncd_x_R0-0}{1}: [ewlc-dtls-sessmgr] [23645]: (info): Remote Host: 192.168.1.8[9521] Completed cert verification, status:CERT_VALIDATE_SUCCESS 
2019/09/08 21:06:05.314 {wncd_x_R0-0}{1}: [capwapac-smgr-sess-fsm] [23645]: (info): Session-IP: 192.168.1.8[9521] Mac:  Close CAPWAP DTLS session.
2019/09/08 21:06:05.314 {wncd_x_R0-0}{1}: [capwapac-smgr-sess-fsm] [23645]: (note): Session-IP: 192.168.1.8[9521] Mac:  Last Control Packet information not available.AP terminated in DTLS phase.
2019/09/08 21:06:05.314 {wncd_x_R0-0}{1}: [capwapac-smgr-sess-fsm] [23645]: (note): Session-IP: 192.168.1.8[9521] Mac:  Last Data Keep Alive Packet information not available.Data session was not established
2019/09/08 21:06:05.314 {wncd_x_R0-0}{1}: [apmgr-db] [23645]: (ERR): 80e0.1d92.530c Mismatch in session handles.Record already deleted and recreated
2019/09/08 21:06:10.312 {wncd_x_R0-0}{1}: [capwapac-smgr-srvr] [23645]: (ERR): Unknown message type from load balancer, dropping
2019/09/08 21:06:10.312 {wncd_x_R0-0}{1}: [capwapac-smgr-srvr] [23645]: (ERR): Unknown message type from load balancer, dropping
2019/09/08 21:06:10.312 {wncd_x_R0-0}{1}: [capwapac-smgr-srvr] [23645]: (ERR): Unknown message type from load balancer, dropping

Thanks,

Mikolaj

Re: 2702i not joining 9800-CL

Sandeep Choudhary — Mon, 09 Sep 2019 06:20:13 GMT

please try this command on AP:

clear capwap ap all-config

Regards

Dont forget to rate helpful posts

Re: 2702i not joining 9800-CL

Mikolaj Moryto — Mon, 09 Sep 2019 06:28:56 GMT

Hi Sandeep,

Does the AP need to be in specific mode for that? I have it currently registered to vWLC and I have this:

When I was trying previously with recovery mode (MODE button) I had "clear capwap ap" command available but "all-config" was not there. There were only "clear capwap ap controller", "clear capwap ap ip" but not "clear capwap ap all-config".

Re: 2702i not joining 9800-CL

Sandeep Choudhary — Mon, 09 Sep 2019 07:17:23 GMT

first run this command:

debug capwap console cli

Regards

Dont forget to rate helpful posts

Re: 2702i not joining 9800-CL

Mikolaj Moryto — Mon, 09 Sep 2019 12:43:54 GMT

These two commands are not available now when the AP is registered to vWLC. Does it need to be factory reset? When I did the factory reset with MODE button for the last time I did not see "clear capwap ap all-config" there either.

Re: 2702i not joining 9800-CL

Sandeep Choudhary — Mon, 09 Sep 2019 13:29:36 GMT

you need to copy and paste the command on AP console.(Its a hidden command)

debug capwap console cli

without this command you cant go in config mode.

Regards

Dont forget to rate helpful posts

Re: 2702i not joining 9800-CL

Mikolaj Moryto — Tue, 10 Sep 2019 01:44:54 GMT

That command worked after using "debug capwap console cli" but it did not solve the issue. I am still having the same "DTLS handshake" error on the controller side.

Re: 2702i not joining 9800-CL

Daniel Ordóñez Flores — Tue, 10 Sep 2019 02:12:06 GMT

Hi @Mikolaj Moryto

Try whit "erase /all nvram: " and reload the AP

Re: 2702i not joining 9800-CL

Mikolaj Moryto — Tue, 10 Sep 2019 15:40:44 GMT

So that wasn't really an AP or WLC configuration issue. Below commands were doing what they were supposed to.

debug capwap console cli
clear capwap ap all-config
erase /all nvram:

The reason for getting "DTLS handshake" error was because I misconfigured the interfaces and put all of them into single port group on ESXi side. Once changing it AP instantly registered to the WLC.

Thanks everyone for your assistance.

Re: 2702i not joining 9800-CL

Nickolus Looper — Wed, 25 Dec 2019 17:55:54 GMT

I'm noticing the same issue with a 9800-CL on ESXi...but I've verified that my interfaces are in different port groups.

Also, I've shut Gig1 on my 9800-CL. Only using Gig2, and have it configured as a trunk.

Thanks

Re: 2702i not joining 9800-CL

Nickolus Looper — Wed, 25 Dec 2019 17:58:51 GMT

I replied too soon - the command:

clear capwap ap all-config

Fixed the issue for me.

Thanks!