topic Re: 1602 APs fail to join 7500 controller after update in Wireless

1602 APs fail to join 7500 controller after update

clybumat1 — Mon, 05 Jul 2021 17:21:01 GMT

I have two 1602 APs that are doing the same thing. They initially join the 7500 controller, download the needed software, but after they reboot, they fail to re-join. Here is the log info from one of the APs:

*Apr 22 23:36:17.067: AP has SHA2 MIC certificate - Using SHA2 MIC certificate for DTLS.

*Apr 22 23:36:17.000: %CAPWAP-5-DTLSREQSEND: DTLS connection request sent peer_ip: 10.1.6.60 peer_port: 5246
*Apr 22 23:36:19.183: DTLS_CLIENT_ERROR: ../capwap/base_capwap/dtls/base_capwap_dtls_handshake.c:929 Unexpected message received while expecting HelloVerifyRequest
*Apr 22 23:36:19.183: %DTLS-5-SEND_ALERT: Send FATAL : Unexpected message Alert to "WLC IP ADDRESS REMOVED FOR PRIVACY REASONS"
*Apr 22 23:36:19.183: %DTLS-5-SEND_ALERT: Send FATAL : Close notify Alert to "WLC IP ADDRESS REMOVED FOR PRIVACY REASONS"
*Apr 22 23:37:22.067: AP has SHA2 MIC certificate - Using SHA2 MIC certificate for DTLS.

*Apr 22 23:37:22.000: %CAPWAP-5-DTLSREQSEND: DTLS connection request sent peer_ip: 10.1.6.60 peer_port: 5246
*Apr 22 23:37:28.571: DTLS_CLIENT_ERROR: ../capwap/base_capwap/dtls/base_capwap_dtls_record.c:394 BD is not of DTLS Change Cipher Spec type
*Apr 22 23:37:28.571: %DTLS-5-SEND_ALERT: Send FATAL : Internal error Alert to "WLC IP ADDRESS REMOVED FOR PRIVACY REASONS"
*Apr 22 23:37:28.571: %DTLS-5-SEND_ALERT: Send FATAL : Close notify Alert to "WLC IP ADDRESS REMOVED FOR PRIVACY REASONS"

Re: 1602 APs fail to join 7500 controller after update

Jurgens L — Wed, 08 May 2019 12:56:55 GMT

What version of code are you running on the 7500 and can you also provide a debug output on the wlc:
debug capwap errors
debug capwap events

Re: 1602 APs fail to join 7500 controller after update

clybumat1 — Wed, 08 May 2019 13:13:11 GMT

Here is the code: 8.3.143.4

I have enabled those two debug commands on the WLC. How can I view the output? I assume it will need to be filtered by the APs in question (there are over 1,000 APs on this controller)

Re: 1602 APs fail to join 7500 controller after update

Sandeep Choudhary — Wed, 08 May 2019 13:38:49 GMT

paste the output of these commands:

sh version from AP

sh sysinfo (or country code configured on WLC)

Boot-up process from AP console.

Also check the date and time on cisco WLC.

Regards

Dont forget to rate helpful posts

Re: 1602 APs fail to join 7500 controller after update

Leo Laohoo — Wed, 08 May 2019 13:53:12 GMT

Look at the time and date of the AP.
See if the WLC has NTP configured.

Re: 1602 APs fail to join 7500 controller after update

clybumat1 — Wed, 08 May 2019 14:07:31 GMT

Date and time on WLC

show time

Time............................................. Wed May 8 10:04:56 2019

Below are the country codes on the WLC. The two APs are physically located in Brazil (BR):

AR, BO, BR, CA, CL, CO, CR, DO, EC, JM, MX, PA, PE, PR, PY, US, UY, VE

I will have to arrange console to run the sh version, and sysinfo. However, the software loaded is: ap1g2-k9w8-tar.153-3.JF9.tar

Thanks.

Re: 1602 APs fail to join 7500 controller after update

clybumat1 — Wed, 08 May 2019 14:14:40 GMT

We do have NTP configured, although it says "AUTH DISABLED".

So basically the time and date on the AP need to match that of the WLC? I wonder why I haven't had to do that before. We have hundreds of 1600s that work fine.

Re: 1602 APs fail to join 7500 controller after update

Sathiyanarayanan Ravindran — Wed, 08 May 2019 14:34:59 GMT

Are you able to reach the WLC IP from this AP ?

Try to ping the WLC IP and See, whether its reachable. Also see on the Monitor --> Statistics --> AP Join status of the WLC, are you getting hits for the AP.

Am suspecting this could be a reach-ability issue.

Re: 1602 APs fail to join 7500 controller after update

Leo Laohoo — Wed, 08 May 2019 14:42:59 GMT

If NTP is configured, post the complete output to the following commands:
1. WLC: sh sysinfo;
2. WLC: sh time; and
3. AP: sh version

Re: 1602 APs fail to join 7500 controller after update

clybumat1 — Wed, 08 May 2019 14:46:10 GMT

Thanks. I will try to ping the WLC next time I have console access. However, it was able to join the controller fine the first time to download the software. It does have a hit in the AP join stats - Status not joined.

Re: 1602 APs fail to join 7500 controller after update

Sathiyanarayanan Ravindran — Wed, 08 May 2019 14:52:04 GMT

No, You can see the logs of AP, which fails to join the controller. If the reach-ability of the AP is there towards the WLC at-least you will get a hit on your WLC.

Check the reach-ability and logs on the controller.

Re: 1602 APs fail to join 7500 controller after update

clybumat1 — Wed, 08 May 2019 15:52:14 GMT

Sorry double post.

Re: 1602 APs fail to join 7500 controller after update

clybumat1 — Wed, 08 May 2019 15:02:18 GMT

Ok I have console access now. I can ping the WLC fine.

Re: 1602 APs fail to join 7500 controller after update

Sathiyanarayanan Ravindran — Wed, 08 May 2019 15:13:55 GMT

Try using the command config ap primary-base <wlcname> <ip address>

if its not joining post the output of the AP console logs. Lets check the error logs.

Re: 1602 APs fail to join 7500 controller after update

clybumat1 — Wed, 08 May 2019 15:38:19 GMT

Here are the AP logs. It repeats this over and over:

*May 7 23:35:23.000: %CAPWAP-5-DTLSREQSEND: DTLS connection request sent peer_ip: "WLC IP HIDDEN FOR PRIVACY REASONS" peer_port: 5246
*May 7 23:35:29.579: DTLS_CLIENT_ERROR: ../capwap/base_capwap/dtls/base_capwap_dtls_record.c:394 BD is not of DTLS Change Cipher Spec type
*May 7 23:35:29.579: %DTLS-5-SEND_ALERT: Send FATAL : Internal error Alert to "WLC IP HIDDEN FOR PRIVACY REASONS"
*May 7 23:35:29.579: %DTLS-5-SEND_ALERT: Send FATAL : Close notify Alert to "WLC IP HIDDEN FOR PRIVACY REASONS"
*May 7 23:36:28.067: AP has SHA2 MIC certificate - Using SHA2 MIC certificate for DTLS.

Re: 1602 APs fail to join 7500 controller after update

Sathiyanarayanan Ravindran — Wed, 08 May 2019 15:40:39 GMT

capwap ap controller ip address <wlc ip>

Re: 1602 APs fail to join 7500 controller after update

clybumat1 — Wed, 08 May 2019 15:41:46 GMT

show sysinfo

Manufacturer's Name.............................. Cisco Systems Inc.

Product Name..................................... Cisco Controller

Product Version.................................. 8.3.143.4

RTOS Version..................................... 8.3.143.4

Bootloader Version............................... 8.1.102.0

Emergency Image Version.......................... 8.1.102.0

OUI File Update Time............................. Sun Sep 07 10:44:07 IST 2014

Build Type....................................... DATA + WPS

System Name...................................... hidden

System Location.................................. hidden

System Contact................................... hidden

System ObjectID.................................. hidden

Redundancy Mode.................................. SSO

IP Address....................................... hidden

IPv6 Address..................................... ::

System Up Time................................... 153 days 3 hrs 1 mins 35 secs

System Timezone Location......................... (GMT -5:00) Eastern Time (US and Canada)

System Stats Realtime Interval................... 5

--More or (q)uit current module or <ctrl-z> to abort

System Stats Normal Interval..................... 180

Configured Country............................... Multiple Countries : AR,BO,BR,CA,CL,CO,CR,DO,EC,JM,

............................................... MX,PA,PE,PR,PY,US,UY,VE

Operating Environment............................ Commercial (10 to 35 C)

Internal Temp Alarm Limits....................... 10 to 38 C

Internal Temperature............................. +24 C

Fan Status....................................... OK

RAID Volume Status

Drive 0.......................................... Good

Drive 1.......................................... Good

State of 802.11b Network......................... Enabled

State of 802.11a Network......................... Enabled

Number of WLANs.................................. 9

Number of Active Clients......................... 3584

OUI Classification Failure Count................. 117045861

Burned-in MAC Address............................ 64:9E:F3:65:74:60

Power Supply 1................................... Present, OK

--More or (q)uit current module or <ctrl-z> to abort

Power Supply 2................................... Present, OK

Maximum number of APs supported.................. 6000

System Nas-Id.................................... hidden

WLC MIC Certificate Types........................ SHA1

Licensing Type................................... RTU

show time

Time............................................. Wed May 8 10:51:00 2019

Timezone delta................................... 0:0

Timezone location................................ (GMT -5:00) Eastern Time (US and Canada)

NTP Servers

NTP Polling Interval......................... 3600

Index NTP Key Index NTP Server Status NTP Msg Auth Status

------- ----------------------------------------------------------------------------------------------

1 0 hidden In Progress AUTH DISABLED

sh ver

Cisco IOS Software, C1600 Software (AP1G2-K9W8-M), Experimental Version 15.3(201 80904:192853) [vipendya 130]

Compiled Tue 04-Sep-18 12:30 by vipendya

ROM: Bootstrap program is C1600 boot loader

BOOTLDR: C1600 Boot Loader (AP1G2-BOOT-M) LoaderVersion 15.2(2)JAX, RELEASE SOFT WARE (fc1)

"NAME HIDDEN" uptime is 17 hours, 56 minutes

System returned to ROM by power-on

System image file is "flash:/ap1g2-k9w8-mx.ap_smr4_esc.201809041149/ap1g2-k9w8-x x.ap_"

Last reload reason:

This product contains cryptographic features and is subject to United

States and local country laws governing import, export, transfer and

use. Delivery of Cisco cryptographic products does not imply

third-party authority to import, export, distribute or use encryption.

Importers, exporters, distributors and users are responsible for

compliance with U.S. and local country laws. By using this product you

agree to comply with applicable laws and regulations. If you are unable

to comply with U.S. and local laws, return this product immediately.

A summary of U.S. laws governing Cisco cryptographic products may be found at:

http://www.cisco.com/wwl/export/crypto/tool/stqrg.html

If you require further assistance please contact us by sending email to

export@cisco.com.

cisco AIR-CAP1602I-T-K9 (PowerPC) processor (revision B0) with 187386K/74672K bytes of memory.

Processor board ID TSP1912AABU

PowerPC CPU at 533Mhz, revision number 0x2151

Last reset from power-on

LWAPP image version 8.3.143.4

1 Gigabit Ethernet interface

2 802.11 Radios

32K bytes of flash-simulated non-volatile configuration memory.

Base ethernet MAC Address: 64:F6:9D:C6:B4:67

Part Number : 73-14671-04

PCB Serial Number : TSP19080181

Top Assembly Part Number : 800-38552-03

Top Assembly Serial Number : TSP1912AABU

Top Revision Number : A0

Product/Model Number : AIR-SAP1602I-T-K9

Configuration register is 0xF

Re: 1602 APs fail to join 7500 controller after update

clybumat1 — Wed, 08 May 2019 15:42:46 GMT

Yes I tried that command, thanks. It still does not join with the same log info in my previous post.

Re: 1602 APs fail to join 7500 controller after update

Sathiyanarayanan Ravindran — Wed, 08 May 2019 16:16:21 GMT

There are so many Country codes enabled on the WLC . So are you running multiple domain APs on this single WLC?

Re: 1602 APs fail to join 7500 controller after update

clybumat1 — Wed, 08 May 2019 17:08:45 GMT

I'm not sure about multiple Domains, but the WLC holds over 1,000 APs, many from countries all over the world.