Lightweight AP 3500 fail to join capwap/lwapp

patoberli — Mon, 05 Jul 2021 13:24:41 GMT

Hi All

I just took an old, never used 2012 manufactured AP 3502i-E out of the box (it had image 7.0.114.40 (or something like this) preinstalled) and was unable to join it to any of my controllers. It's unable to build the DTLS connection to the controller.

What I tried so far:

manually upgrade the image in recovery on the AP to ap3g1-k9w8-mx.153-3.JD (from the 8.3.x release)

changed the clock on an old WiSM running 7.0.252.0 to the year 2012

set this command on the old controller "config ap lifetime-check mic enable "

also tested with the newer command on a WLC 5520 running 8.2.141.0 (but didn't change the clock on the 5520).

So far nothing of this helped.

Here's the boot output from the ap. Controller with the IP 172.16.102.24 is the WiSM with the old date.

r WRDTR,CLKTR: 0x8200083f 0x40000000 
r RQDC ,RFDC : 0x80000033 0x00000212
using  eeprom values
WRDTR,CLKTR: 0x8200083f 0x40000000 
RQDC ,RFDC : 0x80000033 0x00000212
using MCNG ddr static values from serial eeprom
ddr init done
Running Normal Memtest...
Passed.
IOS Bootloader - Starting system.
FLASH CHIP:  Numonyx P33
Checking for Over Erased blocks
......................................................................................................................................................................................................................................................
Xmodem file system is available.
DDR values used from system serial eeprom.
WRDTR,CLKTR: 0x8200083f, 0x40000000
RQDC, RFDC : 0x80000033, 0x00000212
PCIE0: link is up.
PCIE0: VC0 is active
PCIE1: link is up.
PCIE1: VC0 is active
64bit PCIE devices
PCIEx: initialization done
flashfs[0]: 42 files, 8 directories
flashfs[0]: 0 orphaned files, 0 orphaned directories
flashfs[0]: Total bytes: 31739904
flashfs[0]: Bytes used: 10029568
flashfs[0]: Bytes available: 21710336
flashfs[0]: flashfs fsck took 9 seconds.
Reading cookie from system serial eeprom...Done
Base Ethernet MAC address: a4:93:4c:f3:1d:8b
Ethernet speed is 1000 Mb - FULL duplex
Loading "flash:/ap3g1-k9w8-mx.153-3.JD/ap3g1-k9w8-mx.153-3.JD"...###############
File "flash:/ap3g1-k9w8-mx.153-3.JD/ap3g1-k9w8-mx.153-3.JD" uncompressed and installed, entry point: 0x4000
executing...
enet halted
IOS Secondary Bootloader - Starting system.
FLASH CHIP:  Numonyx P33
Checking for Over Erased blocks
......................................................................................................................................................................................................................................................
Xmodem file system is available.
DDR values used from system serial eeprom.
WRDTR,CLKTR: 0x8200083f, 0x40000000
RQDC, RFDC : 0x80000033, 0x00000212
PCIE0: link is up.
PCIE0: VC0 is active
PCIE1: link is up.
PCIE1: VC0 is active
Radio 0 : Vendor 0x11AB, Device 0x8350
64bit PCIE devices
Radio 1 : Vendor 0x11AB, Device 0x8324
PCIEx: initialization done
flashfs[0]: 42 files, 8 directories
flashfs[0]: 0 orphaned files, 0 orphaned directories
flashfs[0]: Total bytes: 31739904
flashfs[0]: Bytes used: 10029568
flashfs[0]: Bytes available: 21710336
flashfs[0]: flashfs fsck took 10 seconds.
Reading cookie from system serial eeprom...Done
Base Ethernet MAC address: a4:93:4c:f3:1d:8b
Creating Test Kernel diagnostic commands
Radio 0 : Vendor 0x11AB, Device 0x8324
Radio 1 : Vendor 0x11AB, Device 0x8350
Radio 2 : Vendor 0x8909, Device 0x40
Radio 3 : Vendor 0x1204, Device 0x841
******** AUTOMATIC DDR CALIBRATION UPGRADE LOGIC *********
=== 1. Is original FCS bootloader in BS:?  If not, skip upgrade ===
    ---> original FCS bootloader not detected -- skip upgrade
Boot CMD: 'boot  flash:/ap3g1-k9w8-mx.153-3.JD/ap3g1-k9w8-xx.153-3.JD;flash:/ap3g1-k9w8-mx.153-3.JD/ap3g1-k9w8-xx.153-3.JD'
Loading "flash:/ap3g1-k9w8-mx.153-3.JD/ap3g1-k9w8-xx.153-3.JD"...####################################
File "flash:/ap3g1-k9w8-mx.153-3.JD/ap3g1-k9w8-xx.153-3.JD" uncompressed and installed, entry point: 0x100000
executing...
              Restricted Rights Legend
Use, duplication, or disclosure by the Government is
subject to restrictions as set forth in subparagraph
(c) of the Commercial Computer Software - Restricted
Rights clause at FAR sec. 52.227-19 and subparagraph
(c) (1) (ii) of the Rights in Technical Data and Computer
Software clause at DFARS sec. 252.227-7013.
           cisco Systems, Inc.
           170 West Tasman Drive
           San Jose, California 95134-1706
Cisco IOS Software, C3500 Software (AP3G1-K9W8-M), Version 15.3(3)JD, RELEASE SOFTWARE (fc3)
Technical Support: http://www.cisco.com/techsupport
Copyright (c) 1986-2016 by Cisco Systems, Inc.
Compiled Fri 29-Jul-16 03:37 by prod_rel_team
Initializing flashfs...
FLASH CHIP:  Numonyx P33
Checking for Over Erased blocks
......................................................................................................................................................................................................................................................
flashfs[2]: 42 files, 8 directories
flashfs[2]: 0 orphaned files, 0 orphaned directories
flashfs[2]: Total bytes: 31481856
flashfs[2]: Bytes used: 10029568
flashfs[2]: Bytes available: 21452288
flashfs[2]: flashfs fsck took 9 seconds.
flashfs[2]: Initialization complete.
flashfs[4]: 0 files, 1 directories
flashfs[4]: 0 orphaned files, 0 orphaned directories
flashfs[4]: Total bytes: 11999232
flashfs[4]: Bytes used: 1024
flashfs[4]: Bytes available: 11998208
flashfs[4]: flashfs fsck took 1 seconds.
flashfs[4]: Initialization complete.
Copying radio files from flash: to ram:
Copy in progress...CCC
Copy in progress...CCC
Copy in progress...CC
Uncompressing radio files...
...done Initializing flashfs.
Ethernet speed is 1000 Mb - FULL duplex
Radio0  present 8364B 8000 B8020000 0 B8030000 10
Rate table has 300 entries (16 legacy/64 11n/220 11ac)
POWER TABLE FILENAME = ram:/Z2.bin
Radio1  present 8364B 8000 B0020000 0 B0030000 C
POWER TABLE FILENAME = ram:/Z5.bin
This product contains cryptographic features and is subject to United
States and local country laws governing import, export, transfer and
use. Delivery of Cisco cryptographic products does not imply
third-party authority to import, export, distribute or use encryption.
Importers, exporters, distributors and users are responsible for
compliance with U.S. and local country laws. By using this product you
agree to comply with applicable laws and regulations. If you are unable
to comply with U.S. and local laws, return this product immediately.
A summary of U.S. laws governing Cisco cryptographic products may be found at:
http://www.cisco.com/wwl/export/crypto/tool/stqrg.html
If you require further assistance please contact us by sending email to
export@cisco.com.
cisco AIR-CAP3502I-E-K9 (PowerPC460exr) processor (revision A0) with 98294K/32768K bytes of memory.
Processor board ID FCZ1626Z02N
PowerPC460exr CPU at 666Mhz, revision number 0x18A8
Last reset from reload
LWAPP image version 8.3.102.0
1 Gigabit Ethernet interface
2 802.11 Radios
32K bytes of flash-simulated non-volatile configuration memory.
Base ethernet MAC Address: A4:93:4C:F3:1D:8B
Part Number                          : 73-14857-01
PCB Serial Number                    : FOC16232K2Z
Top Assembly Part Number             : 800-32891-02
Top Assembly Serial Number           : FCZ1626Z02N
Top Revision Number                  : B0
Product/Model Number                 : AIR-CAP3502I-E-K9   
% Please define a domain-name first.

Press RETURN to get started!

*Mar  1 00:00:12.894: %SOAP_FIPS-2-SELF_TEST_IOS_SUCCESS: IOS crypto FIPS self test passed (15)
*Mar  1 00:00:12.897: *** CRASH_LOG = YES
*Mar  1 00:00:12.897: 64bit PCIE devices
*Mar  1 00:00:14.004: %SOAP_FIPS-2-SELF_TEST_HW_SUCCESS: HW crypto FIPS self test passed (1-6)
*Mar  1 00:00:14.004: Security Core found.
*Mar  1 00:00:14.017: Registering HW DTLS
Base Ethernet MAC address: A4:93:4C:F3:1D:8B
*Mar  1 00:00:16.244: %LINK-6-UPDOWN: Interface GigabitEthernet0, changed state to up
*Mar  1 00:00:17.575: %SOAP_FIPS-2-SELF_TEST_RAD_SUCCESS: RADIO crypto FIPS self test passed on interface Dot11Radio 0 (4)
*Mar  1 00:00:17.581: %LINEPROTO-5-UPDOWN: Line protocol on Interface GigabitEthernet0, changed state to up
*Mar  1 00:00:17.701: loading Power Tables from ram:/Z2.bin. Class = E
*Mar  1 00:00:17.701:  record size of 2ss: 404 read_ptr: 2868DF8
*Mar  1 00:00:20.884: %SOAP_FIPS-2-SELF_TEST_RAD_SUCCESS: RADIO crypto FIPS self test passed on interface Dot11Radio 1 (4)
*Mar  1 00:00:20.934: loading Power Tables from ram:/Z5.bin. Class = E
*Mar  1 00:00:20.934:  record size of 2ss: 404 read_ptr: 2868DF8
*Jan 20 12:21:15.088: %DOT11-5-EXPECTED_RADIO_RESET: Restarting Radio interface Dot11Radio0 due to hostname change
*Jan 20 12:21:15.088: %DOT11-5-EXPECTED_RADIO_RESET: Restarting Radio interface Dot11Radio1 due to hostname change
*Jan 20 12:21:15.106: %SYS-5-RESTART: System restarted --
Cisco IOS Software, C3500 Software (AP3G1-K9W8-M), Version 15.3(3)JD, RELEASE SOFTWARE (fc3)
Technical Support: http://www.cisco.com/techsupport
Copyright (c) 1986-2016 by Cisco Systems, Inc.
Compiled Fri 29-Jul-16 03:37 by prod_rel_team
*Jan 20 12:21:15.106: %SNMP-5-COLDSTART: SNMP agent on host APa493.4cf3.1d8b is undergoing a cold start
*Jan 20 12:21:15.235: %DOT11-5-EXPECTED_RADIO_RESET: Restarting Radio interface Dot11Radio0 due to interface reset
*Jan 20 12:21:15.239: %LINK-6-UPDOWN: Interface Dot11Radio0, changed state to up
*Jan 20 12:21:15.242: %DOT11-5-EXPECTED_RADIO_RESET: Restarting Radio interface Dot11Radio1 due to interface reset
*Jan 20 12:21:15.402: %SSH-5-ENABLED: SSH 2.0 has been enabled
*Jan 20 12:21:15.402: %LINK-6-UPDOWN: Interface Dot11Radio1, changed state to uplwapp_crypto_init: MIC Present and Parsed Successfully
*Jan 20 12:21:16.163: %LINEPROTO-5-UPDOWN: Line protocol on Interface BVI1, changed state to up
*Jan 20 12:21:32.314: %DHCP-6-ADDRESS_ASSIGN: Interface BVI1 assigned DHCP address 172.16.102.235, mask 255.255.255.0, hostname APa493.4cf3.1d8b
*Jan 20 12:21:32.776: Currently running a Release Image
validate_sha2_block: Failed to get certificate chain
*Jan 20 12:21:32.798: Using SHA-1 signed certificate for image signing validation.%Default route without gateway, if not a point-to-point interface, may impact performance
*Jan 20 12:21:38.341: AP image integrity check PASSED
*Jan 20 12:21:38.350: Non-recovery image. PNP Not required.
*Jan 20 12:21:38.410:  validate_sha2_block:No SHA2 Block present on this AP.
*Jan 20 12:21:38.441: %LINK-5-CHANGED: Interface Dot11Radio0, changed state to reset
*Jan 20 12:21:38.441: %LINK-5-CHANGED: Interface Dot11Radio1, changed state to reset
*Jan 20 12:21:48.473: %SYS-6-LOGGINGHOST_STARTSTOP: Logging to host 255.255.255.255 port 0 CLI Request Triggered
Translating "CISCO-CAPWAP-CONTROLLER.[removed]"...domain server (152.96.20.10) [OK]
*Jan 20 12:22:03.091: %DOT11-5-EXPECTED_RADIO_RESET: Restarting Radio interface Dot11Radio0 due to interface reset
*Jan 20 12:22:03.091: %DOT11-5-EXPECTED_RADIO_RESET: Restarting Radio interface Dot11Radio1 due to interface reset
*Jan 20 12:22:03.091: %CDP_PD-4-POWER_OK: Full power - NEGOTIATED inline power source
*Jan 20 12:22:04.189: %LINK-6-UPDOWN: Interface Dot11Radio0, changed state to up
*Jan 20 12:22:05.190: %LINEPROTO-5-UPDOWN: Line protocol on Interface Dot11Radio0, changed state to up
*Jan 20 12:22:05.281: %LINK-6-UPDOWN: Interface Dot11Radio1, changed state to up
*Jan 20 12:22:06.281: %LINEPROTO-5-UPDOWN: Line protocol on Interface Dot11Radio1, changed state to up
*Jan 20 12:24:38.000: %CAPWAP-5-DTLSREQSEND: DTLS connection request sent peer_ip: 172.16.102.24 peer_port: 5246
*Jan 20 12:25:08.088: DTLS_CLIENT_ERROR: ../capwap/base_capwap/dtls/base_capwap_dtls_connection_db.c:2214 Max retransmission count reached for Connection 0x543B5F0!
*Jan 20 12:25:38.001: %DTLS-5-SEND_ALERT: Send FATAL : Close notify Alert to 172.16.102.24:5246
*Jan 20 12:25:38.000: %CAPWAP-5-DTLSREQSEND: DTLS connection request sent peer_ip: 172.16.102.24 peer_port: 5246
*Jan 20 12:26:08.085: DTLS_CLIENT_ERROR: ../capwap/base_capwap/dtls/base_capwap_dtls_connection_db.c:2214 Max retransmission count reached for Connection 0x543B5F0!
*Jan 20 12:26:09.372: %CDP_PD-4-POWER_OK: Full power - NEGOTIATED inline power source
*Jan 20 12:26:38.001: %DTLS-5-SEND_ALERT: Send FATAL : Close notify Alert to 172.16.102.24:5246
*Jan 20 12:26:45.000: %CAPWAP-5-DTLSREQSEND: DTLS connection request sent peer_ip: 172.16.102.11 peer_port: 5246
*Jan 20 12:26:51.017: DTLS_CLIENT_ERROR: ../capwap/base_capwap/dtls/base_capwap_dtls_record.c:394 BD is not of DTLS Change Cipher Spec type
*Jan 20 12:26:51.017: %DTLS-5-SEND_ALERT: Send FATAL : Internal error Alert to 172.16.102.11:5246
*Jan 20 12:26:51.017: %DTLS-5-SEND_ALERT: Send FATAL : Close notify Alert to 172.16.102.11:5246

I'm open for more ideas, besides doing an RMA.

Thanks

Take a look at the following

Emerson Rodrigues — Fri, 20 Jan 2017 17:01:18 GMT

Take a look at the following link: https://supportforums.cisco.com/document/12453081/lightweight-ap-fail-create-capwaplwapp-connection-due-certificate-expiration

Also disable your NTP server.

Forgot to add that I disabled

patoberli — Mon, 23 Jan 2017 07:53:07 GMT

Forgot to add that I disabled NTP before I changed the date to 2012.

I also thought it's that issue. But my AP was produced (based on the serial) in 2012, so it shouldn't be affected. This is what is confusing me.

This is the DTLS debug, if

patoberli — Tue, 07 Feb 2017 08:02:52 GMT

This is the DTLS debug, if anybody is curious.

Hi,

Sandeep Choudhary — Tue, 07 Feb 2017 12:20:16 GMT

Hi,

Frankly speaking cant find the exact reason of failure but just go the below bug which has the same issue...

*Jan 20 12:26:51.017: DTLS_CLIENT_ERROR: ../capwap/base_capwap/dtls/base_capwap_dtls_record.c:394 BD is not of DTLS Change Cipher Spec type

Check these two bugs has slimier errors:

https://bst.cloudapps.cisco.com/bugsearch/bug/CSCuy15766/?referring_site=bugquickviewredir

https://bst.cloudapps.cisco.com/bugsearch/bug/CSCut21564/?referring_site=bugquickviewclick

Workaround: I think reload to WLC will resolve your problem.

Regards

Dont forget to rate helpful post

I tested with three different

patoberli — Tue, 07 Feb 2017 12:24:08 GMT

I tested with three different WLCs (one on 7.0.252.0, one on 8.0.131.40 and one on 8.2.141.0), it doesn't connect to any of those. The first two WLCs don't have a single AP connected (those are my old ones, replaced by the 8.2 based one), so it's probably not CPU load (which is at 0%).

Then best is to

Sandeep Choudhary — Tue, 07 Feb 2017 12:29:02 GMT

Then best is to

Step1:raise a TAC case with cisco if you have a vlaid service contract

or Step2: RMAed it.

Regards

Dont forget to rate helpful posts

I feared that. The RMA costs

patoberli — Tue, 07 Feb 2017 12:30:18 GMT

I feared that. The RMA costs are sadly nearly as high as a new one and it's not anymore really worth for this old model. Thanks for your help.

topic Take a look at the following in Wireless

Lightweight AP 3500 fail to join capwap/lwapp

Take a look at the following

Forgot to add that I disabled

This is the DTLS debug, if

Hi,

I tested with three different

Then best is to

I feared that. The RMA costs