Configuring Virtual WLC - RVS 4000 router - No VLAN

[sardarjion]

I am using AIR_CTVM-K9_8_2_170_0.ova as my virtual WLC running in VMware Workstation on Windows 10. This is home lab setup.

The planned setup is as follows:
1. RVS 4000 router connected to internet and hands outs dynamic IPs in range of 192.168.1.100 - 192.168.1.199. There is no VLAN configured on this router. Router IP is 192.168.1.1.
2. Windows 10 host running VMware Workstation is connected to one of the LAN ports of RVS 4000. Windows 10 host has static IP 192.168.1.2.
3. Virtual WLC is running on Windows 10 and two virtual interfaces are configured as:
3.1. Service Port Interface - VMware NAT Configuration (changed it from Bridge Mode to NAT in Virtual Machine settings) - Dynamic IP assigned by VMware - 192.168.233.130.
3.2. Management Interface - Bridge Mode - Configured statically as 192.168.1.200 in WLC during setup.

I have two 2700i 1142 APs configured in lightweight mode and want them to connect to the WLC. The two APs are connected to the two LAN ports of router.
I configured two AP's static IPs in router based on the MAC addressed as 192.168.1.11 and 192.168.1.12. I see them getting these IPs in the router DHCP list.

I kept the management interface IP on subnet 192.168.1.x because I want the APs to communicate with WLC. However, I cannot ping the WLC's management interface or router from WLC.
The APs cannot find the WLC.

Could someone please help me understand what I am missing and how to get this setup rolling. Thank you..

Below are few commands I ran on the WLC configured interfaces

(Cisco Controller) >show interface summary
Number of Interfaces.......................... 3

Interface Name Port Vlan Id IP Address Type Ap Mgr Guest
-------------------------------- ---- -------- --------------- ------- ------ -----
management 1 untagged 192.168.1.200 Static Yes N/A
service-port N/A N/A 192.168.233.130 DHCP No N/A
virtual N/A N/A 2.2.2.2 Static No N/A

(Cisco Controller) >show interface detailed management
Interface Name................................... management
MAC Address...................................... xx:xx:xx:xx:xx:xx
IP Address....................................... 192.168.1.200
IP Netmask....................................... 255.255.255.0
IP Gateway....................................... 192.168.1.1
External NAT IP State............................ Disabled
External NAT IP Address.......................... 0.0.0.0
Link Local IPv6 Address.......................... fe80::20c:29ff:fe84:6c47/64
STATE ........................................... REACHABLE
Primary IPv6 Address............................. ::/128
STATE ........................................... NONE
Primary IPv6 Gateway............................. ::
Primary IPv6 Gateway Mac Address................. 00:00:00:00:00:00
STATE ........................................... INCOMPLETE
VLAN............................................. untagged
Quarantine-vlan.................................. 0
Physical Port.................................... 1
DHCP Proxy Mode.................................. Global
Primary DHCP Server.............................. 192.168.1.1
Secondary DHCP Server............................ Unconfigured
DHCP Option 82................................... Disabled

(Cisco Controller) >show interface detailed service-port
Interface Name................................... service-port
MAC Address...................................... xx:xx:xx:xx:xx:xx
IP Address....................................... 192.168.233.130
IP Netmask....................................... 255.255.255.0
Link Local IPv6 Address.......................... fe80::20c:29ff:fe84:6c3d/64
STATE ........................................... NONE
IPv6 Address..................................... ::/128
STATE ........................................... NONE
SLAAC............................................ Disabled
DHCP Protocol.................................... Enabled
AP Manager....................................... No
Guest Interface.................................. N/A
Speed ........................................... 1Gbps
Duplex .......................................... Full
Auto Negotiation ................................ Enabled
Link Status...................................... Up
Port specific Information:
inet addr:192.168.233.130 Bcast:192.168.233.255 Mask:255.255.255.0
inet6 addr: fe80::20c:29ff:fe84:6c3d/64 Scope:Link
UP BROADCAST RUNNING MULTICAST MTU:1430 Metric:1
RX packets:775 errors:0 dropped:0 overruns:0 frame:0

(Cisco Controller) >show route summary
Number of Routes................................. 0
Destination Network Netmask Gateway
------------------- ------------------- -------------------

[Rich R]

- 8.2.170.0 is *very* old and likely to have numerous bugs!
- what version of software is on the APs?
- what do the join stats on the WLC show?
- Post the full console log from the AP from power on - that should give us some clues?
- Have you followed https://www.cisco.com/c/en/us/td/docs/wireless/technology/mesh/8-2/b_Virtual_Wireless_LAN_Controller_Deployment_Guide_8-2.html#id_10327? (in particular promiscuous mode)

[sardarjion]

Hi Richard,

Thank you for reply. The reason I am on this old vWLC version is because I have two 1142 APs and I read in the documentation that if I use anything higher than 8.3.x it will not let these AP register. 
Right now, I am testing the setup with 1142 AP and it has c1140-k9w8-tar.153-3.JBB4.tar.

I tried the promiscuous mode on Vmware Workstation but it didn't make a difference. I have enabled it just to be safe.
I am not sure what happened but I am able to ping the vWLC control from the machines on the same subnet i.e. 192.168.1.xx but the ping from vWLC to any client isn't working. 
The good news is that I see that AP(192.168.1.14) is trying to talk to vWLC (192.168.1.200). But it fails the authentication at the certificate validation.

I did run these commands on the WLC to bypass certificate validation but it didnt help.

config ap lifetime-check mic enable
config ap lifetime-check ssc enable
config ap cert-expiry-ignore mic enable
config ap cert-expiry-ignore ssc enable
save config

Earlier, I had the AP with version c1140-k9w8-tar.124-23c.JA4.tar and was getting this error on vWLC
*spamApTask1: Oct 21 21:36:59.244: %DTLS-3-HANDSHAKE_FAILURE: openssl_dtls.c:872 Failed to complete DTLS handshake with peer 192.168.1.14
*spamApTask6: Oct 21 21:35:15.035: %DTLS-3-HANDSHAKE_FAILURE: openssl_dtls.c:872 Failed to complete DTLS handshake with peer 192.168.1.14
*emWeb: Oct 21 21:34:13.575: %AAA-3-ACCTREQ_SEND_FAILED: aaa.c:3820 Unable to send Accounting Request for User admin. No accounting server is configured.
*spamApTask1: Oct 21 21:33:58.036: %DTLS-3-HANDSHAKE_FAILURE: openssl_dtls.c:872 Failed to complete DTLS handshake with peer 192.168.1.14
*emWeb: Oct 21 21:33:07.167: %AAA-3-ACCTREQ_SEND_FAILED: aaa.c:3820 Unable to send Accounting Request for User admin. No accounting server is configured.
*fp_main_task: Oct 21 21:29:35.719: %BONJOUR-3-CONFIG_ERR: bonjour_cfg.c:570 Failed to do initial configuration: Failed to add service from xml to profile DB
*ethoipSocketTask: Oct 21 21:29:35.020: %ETHOIP-2-SOCKET_OPEN_ERROR: ethoip.c:205 Unable to open Ethernet-over-IP socket

This error went away when I moved to newer AP c1140-k9w8-tar.153-3.JBB4.tar but started seeing the below error.
I tried even setting time back to 1/1/2013, 1/1/2016, 1/1/2019 and disable NTP but it didnt help as mentioned in this page: https://www.cisco.com/c/en/us/support/docs/field-notices/639/fn63942.html

-----------------------------------------------------------------------------------------------------------------------------------------------------------

vWLC log

*spamApTask6: Oct 22 19:37:22.380: %DTLS-6-RECORD_IGNORED: openssl_dtls.c:2597 Record ignored - expired sequence number.
*spamApTask6: Oct 22 19:37:18.376: %DTLS-6-RECORD_IGNORED: openssl_dtls.c:2597 Record ignored - expired sequence number.
*spamApTask6: Oct 22 19:37:16.376: %DTLS-6-DUPLICATE_RECORD: openssl_dtls.c:2618 Duplicate record received - ignored.
*spamApTask6: Oct 22 19:37:16.376: %LOG-6-Q_IND: openssl_dtls.c:2597 Record ignored - expired sequence number.
*spamApTask6: Oct 22 19:36:12.453: %DTLS-6-RECORD_IGNORED: openssl_dtls.c:2597 Record ignored - expired sequence number.
*SISF BT Process: Oct 22 19:35:18.777: %SISF-6-ENTRY_DELETED: sisf_shim_utils.c:482 Entry deleted A=fe80::6600:f1ff:fec5:e0e8 V=0 I=wired:1 P=0000 M=
*DHCP Server: Oct 22 19:35:18.610: %DHCP-6-SCOPE_NOT_FOUND: dhcpd.c:308 Dropping packet from 192.168.1.200 (unable to match to a dhcp scope)
*DHCP Server: Oct 22 19:35:17.609: %DHCP-6-SCOPE_NOT_FOUND: dhcpd.c:308 Dropping packet from 192.168.1.200 (unable to match to a dhcp scope)
*sisfSwitcherTask: Oct 22 19:35:16.349: %SISF-6-ENTRY_CREATED: sisf_shim_utils.c:485 Entry created A=fe80::6600:f1ff:fec5:e0e8 V=0 I=wired:1 P=0000 M=
*osapiBsnTimer: Oct 22 19:35:10.145: %DTLS-3-HANDSHAKE_FAILURE: openssl_dtls.c:3083 Failed to complete DTLS handshake with peer 192.168.1.14
*emWeb: Oct 22 19:34:26.670: %AAA-3-ACCTREQ_SEND_FAILED: aaa.c:3820 Unable to send Accounting Request for User admin. No accounting server is configured.
*emWeb: Oct 22 19:34:26.670: %AAA-5-AAA_AUTH_ADMIN_USER: aaa.c:2739 Authentication succeeded for admin user 'admin' on 192.168.233.1
*spamApTask1: Oct 22 19:33:56.523: %DTLS-6-RECORD_IGNORED: openssl_dtls.c:2597 Record ignored - expired sequence number.
*spamApTask1: Oct 22 19:33:54.524: %DTLS-6-DUPLICATE_RECORD: openssl_dtls.c:2618 Duplicate record received - ignored.
*osapiReaper: Oct 22 19:32:53.842: %OSAPI-5-CLEAN_TASK: osapi_task.c:3510 Reaper cleaning up exited task 'autoInstallTask' (0x9119c00)
*osapiReaper: Oct 22 19:32:23.818: %OSAPI-5-CLEAN_TASK: osapi_task.c:3510 Reaper cleaning up exited task 'ethoipSocketTask' (0x910d000)
*osapiReaper: Oct 22 19:32:23.813: %OSAPI-5-CLEAN_TASK: osapi_task.c:3510 Reaper cleaning up exited task 'cliWebTask' (0x9118f40)
*osapiReaper: Oct 22 19:32:23.813: %LOG-6-Q_IND: openssl_dtls.c:2597 Record ignored - expired sequence number.
*spamApTask1: Oct 22 19:32:17.124: %DTLS-6-RECORD_IGNORED: openssl_dtls.c:2597 Record ignored - expired sequence number.
*gccp_t: Oct 22 19:32:13.768: %DOT1D-7-PORT_FIND_FAIL: gid.c:580 Port number 1 is not found for GARP Information Declaration (GID)
*fp_main_task: Oct 22 19:32:13.738: %CLIWEB-6-CLIWEB_NGINX_CFG_INFO: cli_web_api.c:5296 NGINX config: This IP is already configured -1062731320
*fp_main_task: Oct 22 19:32:13.736: %CLIWEB-6-CLIWEB_NGINX_CFG_INFO: cli_web_api.c:5268 NGINX config: This port is already configured as listen port 16000
*Bonjour_Socket_Task: Oct 22 19:32:13.733: %SOCKET_TASK-6-STARTING: socket_task.c:58 Starting socket task for protocol 21 (callback 0x121be78)
*fp_main_task: Oct 22 19:32:13.728: %BONJOUR-3-CONFIG_ERR: bonjour_cfg.c:570 Failed to do initial configuration: Failed to add service from xml to profile DB
*ipv6SocketTask: Oct 22 19:32:13.217: %SOCKET_TASK-6-STARTING: socket_task.c:58 Starting socket task for protocol 18 (callback 0x1321960)
*ethoipSocketTask: Oct 22 19:32:13.197: %ETHOIP-2-SOCKET_OPEN_ERROR: ethoip.c:205 Unable to open Ethernet-over-IP socket
*DHCP Server: Oct 22 19:32:13.185: %DHCP-6-DISP_SERV_ADDR: dhcpd.c:108 dhcp server: binding to 0.0.0.0
*DHCP Server: Oct 22 19:32:13.185: %LOG-6-Q_IND: sig_v1.c:410 Sig Interval set to default[...It occurred 6 times.!]
*fp_main_task: Oct 22 19:32:13.178: %WPS-6-IDS_TOKEN_INTERVAL_DEF: sig_v1.c:410 Sig Interval set to default
*fp_main_task: Oct 22 19:32:13.176: %DHCP-6-CHADDR_FILTER_STATE: dhcp_config.c:639 NPU/Driver DHCP CHADDR Filter is disabled
*cdpSocketTask: Oct 22 19:32:13.165: %SOCKET_TASK-6-STARTING: socket_task.c:58 Starting socket task for protocol 13 (callback 0x6cfe60)
*apfOpenDtlSocket: Oct 22 19:31:48.076: %SOCKET_TASK-6-STARTING: socket_task.c:58 Starting socket task for protocol 2 (callback 0x45edd0)
*apfProbeThread: Oct 22 19:31:48.074: %SOCKET_TASK-6-STARTING: socket_task.c:58 Starting socket task for protocol 6 (callback 0x45edd0)
*apfOrphanSocketTask: Oct 22 19:31:48.073: %SOCKET_TASK-6-STARTING: socket_task.c:58 Starting socket task for protocol 17 (callback 0x45eea0)
*fp_main_task: Oct 22 19:31:48.042: %APF-4-DOT1P_TAGS: apf_api.c:292 Dot1P Tags ENABLED for all APs connected to this switch
*fp_main_task: Oct 22 19:31:48.029: %APF-6-SUP_MOBILE_CLIENTS: apf.c:3544 Support 6000 mobile clients!
*fp_main_task: Oct 22 19:31:47.763: %APF-6-AIRSPC_WARP_KCID: apf_warp_utils.c:409 WARP KCID: b9:ae:3d:89:9e:4b:49:a5
*fp_main_task: Oct 22 19:31:47.763: %APF-6-WARP_ENABLE: apf_warp_utils.c:460 Enabling WARP...
*fp_main_task: Oct 22 19:31:47.656: %MM-6-INET_MEMBER_ADD_FAILED: mm_dir.c:1439 Could not add Mobility Member. Reason: Zero IP Addr member add request, not allowed, Member-Count:1,MAC: 00:00:00:00:00:00, IP: 0.0.0.0
*fp_main_task: Oct 22 19:31:47.547: %AAA-6-FUNC_RUNNING: rfc3576.c:171 Running initRfc3576...
*fp_main_task: Oct 22 19:31:47.547: %AAA-6-DB_ADD_USER: file_db.c:1130 Adding user 'admin' to AAA database.
*fp_main_task: Oct 22 19:31:47.547: %AAA-6-CREATE_AVL_TREE: file_db.c:311 Creating an AVL tree with 2048 entries
*iappSocketTask: Oct 22 19:31:47.541: %SOCKET_TASK-6-STARTING: socket_task.c:58 Starting socket task for protocol 7 (callback 0x1275ea0)
*dot1xSocketTask: Oct 22 19:31:47.541: %SOCKET_TASK-6-STARTING: socket_task.c:58 Starting socket task for protocol 4 (callback 0x7074c0)
*fp_main_task: Oct 22 19:31:47.538: %CCX-6-MSGTAG014: ccx_rm_task.c:62 Created CCX RM Task
*fp_main_task: Oct 22 19:31:47.537: %CCX-6-MSGTAG006: ccx_rm.c:615 Creating AVL Tree with 500 entries for CCX RM ClientDatabase
*fp_main_task: Oct 22 19:31:47.535: %CCX-6-MSGTAG004: ccx_rm.c:566 Creating AVL Tree with 100 entries for CCX RM ClientDatabase
*capwapSocketTask: Oct 22 19:31:47.533: %SOCKET_TASK-6-STARTING: socket_task.c:58 Starting socket task for protocol 10 (callback 0x9a90c0)
*rmgrPing: Oct 22 19:31:47.013: %SOCKET_TASK-6-STARTING: socket_task.c:58 Starting socket task for protocol 24 (callback 0xf6ad00)
*fp_main_task: Oct 22 19:31:42.965: %SSHPM-6-MANUF_CERT_INFO: sshpmcert.c:4565 Found Manufacturing-installed device certificates
*fp_main_task: Oct 22 19:31:42.960: %OSAPI-6-FILE_DOES_NOT_EXIST: osapi_file.c:424 File : /mnt/application/.usb_dev does not exist.(errno 2)
-Traceback: 0xf142bd 0x40aca1 0xca585d 0xc8453f 0xf31af9 0x4069ec 0xf21613 0x2013f8f 0x20abf49

--------------------------------------------------------------------------------------------------------------------------
ap1142-ax202#reload
Proceed with reload? [confirm]
Writing out the event log to flash:/event.log ...

*Oct 23 02:33:49.007: %SYS-5-RELOAD: Reload requested by cisco on console. Reload Reason: Reload Command.
*Oct 23 02:33:49.010: %LWAPP-5-CHANGED: CAPWAP changed state to DOWN
Write of event.log done

*Oct 23 02:33:54.000: %CAPWAP-5-DTLSREQSEND: DTLS connection request sent peer_ip: 192.168.1.200 peer_port: 5246
using eeprom values

WRDTR,CLKTR: 0x86000800 0x40000000
RQDC ,RFDC : 0x80000036 0x0000020c

ddr init done

Running Normal Memtest...
Passed.
IOS Bootloader - Starting system.
FLASH CHIP: Numonyx P33
Checking for Over Erased blocks
......................................................................................................................................................................................................................................................
Xmodem file system is available.

DDR values used from system serial eeprom.
WRDTR,CLKTR: 0x86000800, 0x40000000
RQDC, RFDC : 0x80000036, 0x0000020c

PCIE0: link is up.
PCIE0: VC0 is active
PCIE1: link is up.
PCIE1: VC0 is active
PCIEx: initialization done
flashfs[0]: 36 files, 9 directories
flashfs[0]: 0 orphaned files, 0 orphaned directories
flashfs[0]: Total bytes: 32385024
flashfs[0]: Bytes used: 15514624
flashfs[0]: Bytes available: 16870400
flashfs[0]: flashfs fsck took 27 seconds.
Reading cookie from system serial eeprom...Done
Base Ethernet MAC address: xx:xx:xx:xx:xx:xx
Ethernet speed is 1000 Mb - FULL duplex
Loading "flash:/c1140-k9w8-mx.153-3.JBB4/c1140-k9w8-mx.153-3.JBB4"...############

File "flash:/c1140-k9w8-mx.153-3.JBB4/c1140-k9w8-mx.153-3.JBB4" uncompressed and installed, entry point: 0x4000
executing...
enet halted

Secondary Bootloader - Starting system.
FLASH CHIP: Numonyx P33
Checking for Over Erased blocks
......................................................................................................................................................................................................................................................
Xmodem file system is available.
flashfs[0]: 36 files, 9 directories
flashfs[0]: 0 orphaned files, 0 orphaned directories
flashfs[0]: Total bytes: 32385024
flashfs[0]: Bytes used: 15514624
flashfs[0]: Bytes available: 16870400
flashfs[0]: flashfs fsck took 7 seconds.
Reading cookie from system serial eeprom...Done
Base Ethernet MAC address: xx:xx:xx:xx:xx:xx

Secondary bootloader Ethernet not enabled, skip ether_init
Boot CMD: 'boot flash:/c1140-k9w8-mx.153-3.JBB4/c1140-k9w8-xx.153-3.JBB4;flash:/c1140-k9w8-mx.153-3.JBB4/c1140-k9w8-xx.153-3.JBB4'
Loading "flash:/c1140-k9w8-mx.153-3.JBB4/c1140-k9w8-xx.153-3.JBB4"...####################################
File "flash:/c1140-k9w8-mx.153-3.JBB4/c1140-k9w8-xx.153-3.JBB4" uncompressed and installed, entry point: 0x4000
executing...

Restricted Rights Legend

Use, duplication, or disclosure by the Government is
subject to restrictions as set forth in subparagraph
(c) of the Commercial Computer Software - Restricted
Rights clause at FAR sec. 52.227-19 and subparagraph
(c) (1) (ii) of the Rights in Technical Data and Computer
Software clause at DFARS sec. 252.227-7013.

cisco Systems, Inc.
170 West Tasman Drive
San Jose, California 95134-1706

Cisco IOS Software, C1140 Software (C1140-K9W8-M), Version 15.3(3)JBB4, RELEASE SOFTWARE (fc1)
Technical Support: http://www.cisco.com/techsupport
Copyright (c) 1986-2015 by Cisco Systems, Inc.
Compiled Thu 10-Sep-15 03:57 by prod_rel_team

Initializing flashfs...
FLASH CHIP: Numonyx P33
Checking for Over Erased blocks
......................................................................................................................................................................................................................................................

flashfs[2]: 36 files, 9 directories
flashfs[2]: 0 orphaned files, 0 orphaned directories
flashfs[2]: Total bytes: 32126976
flashfs[2]: Bytes used: 15514624
flashfs[2]: Bytes available: 16612352
flashfs[2]: flashfs fsck took 7 seconds.
flashfs[2]: Initialization complete.
flashfs[4]: 0 files, 1 directories
flashfs[4]: 0 orphaned files, 0 orphaned directories
flashfs[4]: Total bytes: 11999232
flashfs[4]: Bytes used: 1024
flashfs[4]: Bytes available: 11998208
flashfs[4]: flashfs fsck took 0 seconds.
flashfs[4]: Initialization complete....done Initializing flashfs.

Ethernet speed is 1000 Mb - FULL duplex

Radio0 present 8363 8000 90020000 0 90030000 B
Rate table has 300 entries (16 legacy/64 11n/220 11ac)

POWER TABLE FILENAME = flash:/c1140-k9w8-mx.153-3.JBB4/T2.bin

Radio1 present 8363 8000 98020000 0 98030000 0
POWER TABLE FILENAME = flash:/c1140-k9w8-mx.153-3.JBB4/T5.bin

This product contains cryptographic features and is subject to United
States and local country laws governing import, export, transfer and
use. Delivery of Cisco cryptographic products does not imply
third-party authority to import, export, distribute or use encryption.
Importers, exporters, distributors and users are responsible for
compliance with U.S. and local country laws. By using this product you
agree to comply with applicable laws and regulations. If you are unable
to comply with U.S. and local laws, return this product immediately.

A summary of U.S. laws governing Cisco cryptographic products may be found at:
http://www.cisco.com/wwl/export/crypto/tool/stqrg.html

If you require further assistance please contact us by sending email to
export@cisco.com.

cisco AIR-LAP1142N-A-K9 (PowerPC405ex) processor (revision A0) with 98294K/32768K bytes of memory.
Processor board ID XXXXXXXXXX
PowerPC405ex CPU at 586Mhz, revision number 0x147E
Last reset from reload
LWAPP image version 8.1.121.0
1 Gigabit Ethernet interface
2 802.11 Radios

32K bytes of flash-simulated non-volatile configuration memory.
Base ethernet MAC Address: xx:xx:xx:xx:xx:xx
Part Number : 73-12836-01
PCA Assembly Number : 800-33767-01
PCA Revision Number : A0
PCB Serial Number : XXXXXXXXXX
Top Assembly Part Number : 800-33775-01
Top Assembly Serial Number : XXXXXXXXXX
Top Revision Number : A0
Product/Model Number : AIR-LAP1142N-A-K9
% Please define a domain-name first.

Press RETURN to get started!

*Mar 1 00:00:11.245: %SOAP_FIPS-2-SELF_TEST_IOS_SUCCESS: IOS crypto FIPS self test passed (15)
*Mar 1 00:00:11.247: *** CRASH_LOG = YES

*Mar 1 00:00:12.355: %SOAP_FIPS-2-SELF_TEST_HW_SUCCESS: HW crypto FIPS self test passed (1-6)
*Mar 1 00:00:12.356: Security Core found.

*Mar 1 00:00:12.368: Registering HW DTLS
Base Ethernet MAC address: xx:xx:xx:xx:xx:xx

*Mar 1 00:00:14.298: %SOAP_FIPS-2-SELF_TEST_RAD_SUCCESS: RADIO crypto FIPS self test passed on interface Dot11Radio 0 (4)
*Mar 1 00:00:14.475: Loading Power Tables from flash:/c1140-k9w8-mx.153-3.JBB4/T2.bin. Class = A
*Mar 1 00:00:14.475: record size of 2ss: 404 read_ptr: 2605330

*Mar 1 00:00:15.332: %SOAP_FIPS-2-SELF_TEST_RAD_SUCCESS: RADIO crypto FIPS self test passed on interface Dot11Radio 1 (4)
*Mar 1 00:00:15.417: Loading Power Tables from flash:/c1140-k9w8-mx.153-3.JBB4/T5.bin. Class = A
*Mar 1 00:00:15.417: record size of 2ss: 404 read_ptr: 2605330
capwap_read_version_info: Info file flash:/c1140-k9w8-mx.124-21a.JHB1/info not find
*Mar 1 00:00:16.792: %LINK-6-UPDOWN: Interface GigabitEthernet0, changed state to up
*Mar 1 00:00:19.798: %SYS-5-RESTART: System restarted --
Cisco IOS Software, C1140 Software (C1140-K9W8-M), Version 15.3(3)JBB4, RELEASE SOFTWARE (fc1)
Technical Support: http://www.cisco.com/techsupport
Copyright (c) 1986-2015 by Cisco Systems, Inc.
Compiled Thu 10-Sep-15 03:57 by prod_rel_team
*Mar 1 00:00:19.798: %SNMP-5-COLDSTART: SNMP agent on host ap1142-ax202 is undergoing a cold start
*Mar 1 00:00:19.819: %LINEPROTO-5-UPDOWN: Line protocol on Interface GigabitEthernet0, changed state to up
*Oct 23 02:33:48.060: %CDP_PD-4-POWER_OK: Full power - AC_ADAPTOR inline power source
*Oct 23 02:33:48.638: %LINK-6-UPDOWN: Interface Dot11Radio0, changed state to up
*Oct 23 02:33:49.040: %LINEPROTO-5-UPDOWN: Line protocol on Interface BVI1, changed state to uplwapp_crypto_init: MIC Present and Parsed Successfully

*Oct 23 02:33:49.590: %SSH-5-ENABLED: SSH 2.0 has been enabled
*Oct 23 02:33:49.590: %LINK-6-UPDOWN: Interface Dot11Radio1, changed state to up
*Oct 23 02:33:57.139: %DHCP-6-ADDRESS_ASSIGN: Interface BVI1 assigned DHCP address 192.168.1.14, mask 255.255.0.0, hostname ap1142-ax202

*Oct 23 02:34:05.038: Currently running a Release Image
validate_sha2_block: Failed to get certificate chain
*Oct 23 02:34:05.055: Using SHA-1 signed certificate for image signing validation.%Default route without gateway, if not a point-to-point interface, may impact performance
*Oct 23 02:34:09.703: AP image integrity check PASSED

*Oct 23 02:34:09.782: validate_sha2_block:No SHA2 Block present on this AP.

*Oct 23 02:34:09.815: %LINK-5-CHANGED: Interface Dot11Radio0, changed state to reset
*Oct 23 02:34:10.838: %LINK-6-UPDOWN: Interface Dot11Radio0, changed state to up
*Oct 23 02:34:10.844: %LINK-5-CHANGED: Interface Dot11Radio1, changed state to reset
*Oct 23 02:34:11.839: %LINEPROTO-5-UPDOWN: Line protocol on Interface Dot11Radio0, changed state to up
*Oct 23 02:34:11.869: %LINK-6-UPDOWN: Interface Dot11Radio1, changed state to up
*Oct 23 02:34:12.869: %LINEPROTO-5-UPDOWN: Line protocol on Interface Dot11Radio1, changed state to up
%Error opening flash:/capwap-saved-config (No such file or directory)
*Oct 23 02:34:19.842: %SYS-6-LOGGINGHOST_STARTSTOP: Logging to host 255.255.255.255 port 0 CLI Request Triggered
Translating "CISCO-CAPWAP-CONTROLLER"...domain server (1.1.1.2)

*Oct 23 02:36:11.000: %CAPWAP-5-DTLSREQSEND: DTLS connection request sent peer_ip: 192.168.1.200 peer_port: 5246
*Oct 23 02:36:12.083: %PKI-3-CERTIFICATE_INVALID_NOT_YET_VALID: Certificate chain validation has failed. The certificate (SN: 1000) is not yet valid Validity period starts on 12:18:34 UTC Oct 19 2022Peer certificate verification failed 001A

*Oct 23 02:36:12.084: DTLS_CLIENT_ERROR: ../capwap/base_capwap/capwap/base_capwap_wtp_dtls.c:497 Certificate verified failed!
*Oct 23 02:36:12.085: %DTLS-5-SEND_ALERT: Send FATAL : Bad certificate Alert to 192.168.1.200:5246
*Oct 23 02:36:12.085: %DTLS-5-SEND_ALERT: Send FATAL : Close notify Alert to 192.168.1.200:5246
*Oct 23 02:37:16.000: %CAPWAP-5-DTLSREQSEND: DTLS connection request sent peer_ip: 192.168.1.200 peer_port: 5246
*Oct 23 02:37:22.002: DTLS_CLIENT_ERROR: ../capwap/base_capwap/dtls/base_capwap_dtls_handshake.c:929 Unexpected message received while expecting HelloVerifyRequest
*Oct 23 02:37:22.003: %DTLS-5-SEND_ALERT: Send FATAL : Unexpected message Alert to 192.168.1.200:5246
*Oct 23 02:37:22.003: %DTLS-5-SEND_ALERT: Send FATAL : Close notify Alert to 192.168.1.200:5246

--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

AP Join Stats Detail >
General

Base MAC Address
1c:aa:07:23:2b:20
AP Name
ap1142-ax202
Ethernet MAC Address
00:00:00:00:00:00
IP Address(Ipv4/Ipv6)
192.168.1.14
Status
Not joined
Last AP Join

Timestamp Message
Oct 22 20:06:50.300
Received Discovery request and sent response
Discovery Phase Statistics

Requests Received
26
Responses Sent
26
Unsuccessful Request Processed
0
Reason For Last Unsuccessful Attempt
-
Last Successful Attempt Time
Oct 22 20:06:50.300

--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

[sardarjion · Accepted solution]

Finally got the AP registered on the WLC. Pings from WLC to any client on 192.168.1.x still is not going through but will debug it later.

1. Run the commands on wlc to disable expired WLC/AP certificate checks.
2. Reboot WLC
3. Using console cable connected to the AP, load c1140-k9w8-tar.153-3.JBB4.tar version. I don't know what is special about this image but only this image worked. I tried c1140-k9w8-tar.153-3.JBB1.tar, c1140-rcvk9w8-tar.124-21a.JA2.tar and few others.
4. Connect the AP to network. 
5. At this point the AP will join the WLC and show in the Wireless and image download will start.
6. AP automatically reboots and then joins the WLC. 

[Rich R]

There was no need to reboot the WLC (apart from when upgrading the code) - the config change does not require a reboot.  If you followed the instructions in the field notice:

1. Upgrade to latest version which supports your APs and WLC - probably 8.3.150.0
   https://www.cisco.com/c/en/us/support/docs/wireless/wireless-lan-controller-software/200046-tac-recommended-aireos.html#anc10
   The corresponding AP code image is c1140-k9w8-tar.153-3.JD17.tar if you want to upgrade manually but otherwise it will download from the controller after joining.
   2. Apply the config workaround on the WLC
   3. Disable NTP and set time manually to before your certs expired
   4. Allow all the APs to join, download new code, pick up the config workaround
   5 Re-enable NTP

[sardarjion]

In my case, I didn't have to move the time back manually before the certs expire. NTP was disabled because I selected no option during the setup. The time on the VM was set to PC's time which is current. I had tried moving time in my earlier post, but it didn't help. 

Thank you for the help and links in your signature are very helpful for a newbie like me.