Solved: cisco virtual WLC 8.0.121.0 AP 1131AG

p3tter123 · ‎01-23-2017

Hi,

I have a Cisco virtual WLC 8.0.121.0 and 2x Cisco 1131AG access points.
I having trouble register the AP to the controller. all i could see from the debug output from the AP is:

Jan 23 21:32:08.000: %CAPWAP-5-DTLSREQSEND: DTLS connection request sent peer_ip: 192.168.9.10 peer_port: 5246
*Jan 23 21:32:08.001: %CAPWAP-5-CHANGED: CAPWAP changed state to
*Jan 23 21:32:08.031: %LWAPP-3-CLIENTERRORLOG: Peer certificate verification failed
*Jan 23 21:32:08.031: %CAPWAP-3-ERRORLOG: Certificate verification failed!
*Jan 23 21:32:08.031: DTLS_CLIENT_ERROR: ../capwap/capwap_wtp_dtls.c:352 Certificate verified failed!
*Jan 23 21:32:08.031: %DTLS-4-BAD_CERT: Certificate verification failed. Peer IP: 192.168.9.10
*Jan 23 21:32:08.031: %DTLS-5-SEND_ALERT: Send FATAL : Bad certificate Alert to 192.168.9.10:5246
*Jan 23 21:32:08.032: %DTLS-3-BAD_RECORD: Erroneous record received from 192.168.9.10: Malformed Certificate
*Jan 23 21:32:08.032: %DTLS-5-SEND_ALERT: Send FATAL : Close notify Alert to 192.168.9.10:5246
*Jan 23 21:32:08.033: %CAPWAP-3-ERRORLOG: Invalid event 38 & state 3 combination.

they are on same local vlan, i have also tried with WLC version 8.0.140.0 with same result.

I have searched through forums and found some tips etc. with the time and date with no luck.

anyone have some ideas?

Thanks.

Leo Laohoo · ‎01-23-2017

Post the output to the following commands:

1. WLC: sh sysinfo;

2. AP: sh version; and

3. AP: sh inventory

I have a suspicion the AP's certificate has expired and this can be determined by looking at the serial number of the AP.

Read Field Notice 63942.

View solution in original post

Leo Laohoo · ‎01-23-2017

Post the output to the following commands:

1. WLC: sh sysinfo;

2. AP: sh version; and

3. AP: sh inventory

I have a suspicion the AP's certificate has expired and this can be determined by looking at the serial number of the AP.

Read Field Notice 63942.

Rasika Nayanajith · ‎01-23-2017

*Jan 23 21:32:08.031: %LWAPP-3-CLIENTERRORLOG: Peer certificate verification failed
*Jan 23 21:32:08.031: %CAPWAP-3-ERRORLOG: Certificate verification failed!
*Jan 23 21:32:08.031: DTLS_CLIENT_ERROR: ../capwap/capwap_wtp_dtls.c:352 Certificate verified failed!
*Jan 23 21:32:08.031: %DTLS-4-BAD_CERT: Certificate verification failed. Peer IP: 192.168.9.10
*Jan 23 21:32:08.031: %DTLS-5-SEND_ALERT: Send FATAL : Bad certificate Alert to 192.168.9.10:5246
*Jan 23 21:32:08.032: %DTLS-3-BAD_RECORD: Erroneous record received from 192.168.9.10: Malformed Certificate

It's look like certificate issue causing this problem. Check this post & apply the command to ignore certificate expiry of AP

https://supportforums.cisco.com/document/12453081/lightweight-ap-fail-create-capwaplwapp-connection-due-certificate-expiration

HTH

Rasika

*** Pls rate all useful responses ***

p3tter123 · ‎01-23-2017

Hi, its still the same result.

Leo Laohoo · ‎01-23-2017

If the problem still persist, kindly post the output to the following commands:

1. WLC: sh time; and

2. AP: sh ip interface brief

Plug a console cable to the AP and reboot the AP. Post the entire boot-up process.

p3tter123 · ‎01-23-2017

WLC:

>show time

Time............................................. Tue Jan 24 00:51:14 2017

Timezone delta................................... 0:0
Timezone location................................ (GMT +1:00) Amsterdam, Berlin, Rome, Vienna

NTP Servers
NTP Polling Interval......................... 86400

Index NTP Key Index NTP Server NTP Msg Auth Status
------- ----------------------------------------------------------------------------------

AP0024.1445.a2de#sh ip interface brief
Interface IP-Address OK? Method Status Protocol
Dot11Radio0 unassigned NO unset up up
Dot11Radio1 unassigned NO unset up up
FastEthernet0 192.168.9.25 YES DHCP up up
AP0024.1445.a2de#

AP0024.1445.a2de#reload
Proceed with reload? [confirm]
Writing out the event log to nvram...

*Jan 23 23:50:28.700: %SYS-5-RELOAD: Reload requested by Cisco on console. Reload Reason: Reload Command.
*Jan 23 23:50:28.704: %LWAPP-5-CHANGED: CAPWAP changed state to DOWNXmodem file system is available.
flashfs[0]: 26 files, 8 directories
flashfs[0]: 0 orphaned files, 0 orphaned directories
flashfs[0]: Total bytes: 15998976
flashfs[0]: Bytes used: 5160960
flashfs[0]: Bytes available: 10838016
flashfs[0]: flashfs fsck took 28 seconds.
Base ethernet MAC Address: 00:24:14:45:a2:de
Initializing ethernet port 0...
Reset ethernet port 0...
Reset done!
ethernet link up, 100 mbps, full-duplex
Ethernet port 0 initialized: link is up
Loading "flash:/c1130-k9w8-mx.124-23c.JA9/c1130-k9w8-mx.124-23c.JA9"...#################################################################################################################################################################################################################################################################################################################################################################################################################################################################

File "flash:/c1130-k9w8-mx.124-23c.JA9/c1130-k9w8-mx.124-23c.JA9" uncompressed and installed, entry point: 0x3000
executing...

Restricted Rights Legend

Use, duplication, or disclosure by the Government is
subject to restrictions as set forth in subparagraph
(c) of the Commercial Computer Software - Restricted
Rights clause at FAR sec. 52.227-19 and subparagraph
(c) (1) (ii) of the Rights in Technical Data and Computer
Software clause at DFARS sec. 252.227-7013.

cisco Systems, Inc.
170 West Tasman Drive
San Jose, California 95134-1706

Cisco IOS Software, C1130 Software (C1130-K9W8-M), Version 12.4(23c)JA9, RELEASE SOFTWARE (fc1)
Technical Support: http://www.cisco.com/techsupport
Copyright (c) 1986-2014 by Cisco Systems, Inc.
Compiled Wed 03-Dec-14 12:25 by prod_rel_team

Proceeding with system init

Proceeding to unmask interrupts
Initializing flashfs...

flashfs[1]: 26 files, 8 directories
flashfs[1]: 0 orphaned files, 0 orphaned directories
flashfs[1]: Total bytes: 15740928
flashfs[1]: Bytes used: 5160960
flashfs[1]: Bytes available: 10579968
flashfs[1]: flashfs fsck took 4 seconds.
flashfs[1]: Initialization complete....done Initializing flashfs.

Radio0 present A506 7100 E8000000 A0000000 80000000 3
Radio1 present A506 6700 E8000100 A0040000 80010000 2
This product contains cryptographic features and is subject to United
States and local country laws governing import, export, transfer and
use. Delivery of Cisco cryptographic products does not imply
third-party authority to import, export, distribute or use encryption.
Importers, exporters, distributors and users are responsible for
compliance with U.S. and local country laws. By using this product you
agree to comply with applicable laws and regulations. If you are unable
to comply with U.S. and local laws, return this product immediately.

A summary of U.S. laws governing Cisco cryptographic products may be found at:
http://www.cisco.com/wwl/export/crypto/tool/stqrg.html

If you require further assistance please contact us by sending email to
export@cisco.com.

%Error opening flash:/c1130-k9w8-mx.124-23c.JA/info (No such file or directory)
%Error opening flash:/c1130-k9w8-mx.124-23c.JA/info (No such file or directory)cisco AIR-LAP1131AG-A-K9 (PowerPCElvis) processor (revision B0) with 27638K/5120K bytes of memory.
Processor board ID FTX1326T00P
PowerPCElvis CPU at 262Mhz, revision number 0x0950
Last reset from power-on
LWAPP image version 7.0.251.2
1 FastEthernet interface
2 802.11 Radio(s)

32K bytes of flash-simulated non-volatile configuration memory.
Base ethernet MAC Address: 00:24:14:45:A2:DE
Part Number : 73-8962-14
PCA Assembly Number : 800-24818-13
PCA Revision Number : A0
PCB Serial Number : FOC13233R8V
Top Assembly Part Number : 800-29144-03
Top Assembly Serial Number : FTX1326T00P
Top Revision Number : A0
Product/Model Number : AIR-LAP1131AG-A-K9
% Please define a domain-name first.

Press RETURN to get started!

Translating "CISCO-LWAPP-CONTROLLER"...domain server (255.255.255.255)
*Mar 1 00:00:06.084: %SOAP_FIPS-2-SELF_TEST_IOS_SUCCESS: IOS crypto FIPS self test passed
*Mar 1 00:00:07.495: %SOAP_FIPS-2-SELF_TEST_RAD_SUCCESS: RADIO crypto FIPS self test passed on interface Dot11Radio 0
*Mar 1 00:00:08.901: %SOAP_FIPS-2-SELF_TEST_RAD_SUCCESS: RADIO crypto FIPS self test passed on interface Dot11Radio 1
*Mar 1 00:00:09.000: %LWAPP-3-CLIENTEVENTLOG: Read and initialized AP event log (contains, 1024 messages)

*Mar 1 00:00:09.031: status of voice_diag_test from WLC is falsecapwap_read_version_info: Info file flash:/c1130-k9w8-mx.124-23c.JA/info not find
*Mar 1 00:00:11.339: %LINK-3-UPDOWN: Interface FastEthernet0, changed state to up
*Mar 1 00:00:11.398: %SYS-5-RESTART: System restarted --
Cisco IOS Software, C1130 Software (C1130-K9W8-M), Version 12.4(23c)JA9, RELEASE SOFTWARE (fc1)
Technical Support: http://www.cisco.com/techsupport
Copyright (c) 1986-2014 by Cisco Systems, Inc.
Compiled Wed 03-Dec-14 12:25 by prod_rel_team
*Mar 1 00:00:11.398: %SNMP-5-COLDSTART: SNMP agent on host AP0024.1445.a2de is undergoing a cold start
*Mar 1 00:00:11.500: %LINK-5-CHANGED: Interface Dot11Radio1, changed state to reset
*Mar 1 00:00:11.500: %LINK-5-CHANGED: Interface Dot11Radio0, changed state to reset
*Mar 1 00:00:11.501: %CDP_PD-4-POWER_OK: Full power - AC_ADAPTOR inline power source
*Mar 1 00:00:11.567: %DOT11-6-FREQ_SCAN: Interface Dot11Radio0, Scanning frequencies for 8 seconds
*Mar 1 00:00:12.339: %LINEPROTO-5-UPDOWN: Line protocol on Interface FastEthernet0, changed state to up
*Mar 1 00:00:12.500: %LINEPROTO-5-UPDOWN: Line protocol on Interface Dot11Radio1, changed state to down
*Mar 1 00:00:12.500: %LINEPROTO-5-UPDOWN: Line protocol on Interface Dot11Radio0, changed state to down
*Mar 1 00:00:19.574: %DOT11-6-FREQ_USED: Interface Dot11Radio0, frequency 2447 selected

User Access Verification

Username:
*Mar 1 00:00:19.575: %LINK-3-UPDOWN: Interface Dot11Radio0, changed state to up
*Mar 1 00:00:20.575: %LINEPROTO-5-UPDOWN: Line protocol on Interface Dot11Radio0, changed state to up

Translating "CISCO-CAPWAP-CONTROLLER"...domain server (255.255.255.255)
Username:
Username:

*Mar 1 00:00:32.498: %CAPWAP-5-CHANGED: CAPWAP changed state to DISCOVERY
*Mar 1 00:00:32.782: %SSH-5-ENABLED: SSH 2.0 has been enabled
*Mar 1 00:00:32.782: %LINK-3-UPDOWN: Interface Dot11Radio1, changed state to up
*Mar 1 00:00:33.033: %LINK-5-CHANGED: Interface Dot11Radio1, changed state to reset
*Mar 1 00:00:33.178: status of voice_diag_test from WLC is false
*Mar 1 00:00:33.266: Logging LWAPP message to 255.255.255.255.

*Mar 1 00:00:33.549: %SYS-6-LOGGINGHOST_STARTSTOP: Logging to host 255.255.255.255 started - CLI initiated
*Mar 1 00:00:33.719: %LINK-3-UPDOWN: Interface Dot11Radio1, changed state to up
*Mar 1 00:00:33.784: %LINK-3-UPDOWN: Interface Dot11Radio0, changed state to down
*Mar 1 00:00:33.784: %LINEPROTO-5-UPDOWN: Line protocol on Interface Dot11Radio0, changed state to down
*Mar 1 00:00:33.809: %LINK-5-CHANGED: Interface Dot11Radio0, changed state to reset
*Mar 1 00:00:33.995: %LINK-3-UPDOWN: Interface Dot11Radio0, changed state to up
*Mar 1 00:00:34.719: %LINEPROTO-5-UPDOWN: Line protocol on Interface Dot11Radio1, changed state to up
*Mar 1 00:00:34.995: %LINEPROTO-5-UPDOWN: Line protocol on Interface Dot11Radio0, changed state to up
*Jan 23 23:54:55.000: %CAPWAP-5-DTLSREQSEND: DTLS connection request sent peer_ip: 192.168.9.10 peer_port: 5246
*Jan 23 23:54:55.001: %CAPWAP-5-CHANGED: CAPWAP changed state to
*Jan 23 23:54:55.030: %LWAPP-3-CLIENTERRORLOG: Peer certificate verification failed
*Jan 23 23:54:55.030: %CAPWAP-3-ERRORLOG: Certificate verification failed!
*Jan 23 23:54:55.030: DTLS_CLIENT_ERROR: ../capwap/capwap_wtp_dtls.c:352 Certificate verified failed!
*Jan 23 23:54:55.030: %DTLS-4-BAD_CERT: Certificate verification failed. Peer IP: 192.168.9.10
*Jan 23 23:54:55.030: %DTLS-5-SEND_ALERT: Send FATAL : Bad certificate Alert to 192.168.9.10:5246
*Jan 23 23:54:55.031: %DTLS-3-BAD_RECORD: Erroneous record received from 192.168.9.10: Malformed Certificate
*Jan 23 23:54:55.031: %DTLS-5-SEND_ALERT: Send FATAL : Close notify Alert to 192.168.9.10:5246
*Jan 23 23:54:55.032: %CAPWAP-3-ERRORLOG: Invalid event 38 & state 3 combination.

I also tried to boot the recovery image. still no luck.

here is the debug output from the WLC with debug capwap ap errors:

*spamApTask3: Jan 24 00:57:15.719: #DTLS-3-HANDSHAKE_FAILURE: openssl_dtls.c:824 Failed to complete DTLS handshake with peer 192.168.9.20
*spamApTask3: Jan 24 00:56:10.697: #DTLS-3-HANDSHAKE_FAILURE: openssl_dtls.c:824 Failed to complete DTLS handshake with peer 192.168.9.20
*spamApTask3: Jan 24 00:55:05.670: #DTLS-3-HANDSHAKE_FAILURE: openssl_dtls.c:824 Failed to complete DTLS handshake with peer 192.168.9.20
*ipv6SocketTask: Jan 24 00:54:55.871: #LOG-3-Q_IND: spam_lrad.c:1689 Ignoring discovery request received on a wrong VLAN (48) on interface (1) in L3 LWAPP mode from AP 00:25:84:95:b3:10
*spamApTask0: Jan 24 00:54:55.641: #LWAPP-3-DISC_INTF_ERR2: spam_lrad.c:1689 Ignoring discovery request received on a wrong VLAN (48) on interface (1) in L3 LWAPP mode from AP 00:25:84:95:b3:10
*spamApTask7: Jan 24 00:52:01.789: #DTLS-3-HANDSHAKE_FAILURE: openssl_dtls.c:824 Failed to complete DTLS handshake with peer 192.168.9.25
*spamApTask7: Jan 24 00:50:56.763: #DTLS-3-HANDSHAKE_FAILURE: openssl_dtls.c:824 Failed to complete DTLS handshake with peer 192.168.9.25
*spamApTask7: Jan 24 00:49:51.743: #DTLS-3-HANDSHAKE_FAILURE: openssl_dtls.c:824 Failed to complete DTLS handshake with peer 192.168.9.25

Thanks

Leo Laohoo · ‎01-23-2017

Console into the AP and see if the AP can ping the Management Interface IP address of the controller.

If it can, then enter the command "capwap ap controller ip address <Management Interface IP address>".

p3tter123 · ‎01-23-2017

(Cisco Controller) >show sys

Incorrect usage. Use the '?' or <TAB> key to list commands.

(Cisco Controller) >show sysinfo

Manufacturer's Name.............................. Cisco Systems Inc.
Product Name..................................... Cisco Controller
Product Version.................................. 8.0.121.0
RTOS Version..................................... 8.0.121.0
Bootloader Version............................... 8.0.121.0
Emergency Image Version.......................... 8.0.121.0

Build Type....................................... DATA + WPS

System Name...................................... Cisco_6a:41:c7
System Location..................................
System Contact...................................
System ObjectID.................................. 1.3.6.1.4.1.9.1.1631
IP Address....................................... 192.168.9.10
IPv6 Address..................................... ::
System Up Time................................... 0 days 1 hrs 18 mins 21 secs
System Timezone Location......................... (GMT +1:00) Amsterdam, Berlin, Rome, Vienna
System Stats Realtime Interval................... 5
System Stats Normal Interval..................... 180

Configured Country............................... US - United States

--More-- or (q)uit

State of 802.11b Network......................... Enabled
State of 802.11a Network......................... Enabled
Number of WLANs.................................. 1
Number of Active Clients......................... 0

Burned-in MAC Address............................ 00:0C:29:6A:41:C7
Maximum number of APs supported.................. 200
System Nas-Id.................................... Cisco_6a:41:c7
WLC MIC Certificate Types........................ SHA1

AP0024.1445.a2de#sh version
Cisco IOS Software, C1130 Software (C1130-K9W8-M), Version 12.4(23c)JA9, RELEASE SOFTWARE (fc1)
Technical Support: http://www.cisco.com/techsupport
Copyright (c) 1986-2014 by Cisco Systems, Inc.
Compiled Wed 03-Dec-14 12:25 by prod_rel_team

ROM: Bootstrap program is C1130 boot loader
BOOTLDR: C1130 Boot Loader (C1130-BOOT-M) Version 12.3(8)JEA, RELEASE SOFTWARE (fc2)

AP0024.1445.a2de uptime is 5 minutes
System returned to ROM by power-on
System image file is "flash:/c1130-k9w8-mx.124-23c.JA9/c1130-k9w8-mx.124-23c.JA9"

This product contains cryptographic features and is subject to United
States and local country laws governing import, export, transfer and
use. Delivery of Cisco cryptographic products does not imply
third-party authority to import, export, distribute or use encryption.
Importers, exporters, distributors and users are responsible for
compliance with U.S. and local country laws. By using this product you
agree to comply with applicable laws and regulations. If you are unable
to comply with U.S. and local laws, return this product immediately.

A summary of U.S. laws governing Cisco cryptographic products may be found at:
http://www.cisco.com/wwl/export/crypto/tool/stqrg.html

If you require further assistance please contact us by sending email to
export@cisco.com.

cisco AIR-LAP1131AG-A-K9 (PowerPCElvis) processor (revision B0) with 27638K/5120K bytes of memory.
Processor board ID FTX1326T00P
PowerPCElvis CPU at 262Mhz, revision number 0x0950
Last reset from power-on
LWAPP image version 7.0.251.2
1 FastEthernet interface
2 802.11 Radio(s)

32K bytes of flash-simulated non-volatile configuration memory.
Base ethernet MAC Address: 00:24:14:45:A2:DE
Part Number : 73-8962-14
PCA Assembly Number : 800-24818-13
PCA Revision Number : A0
PCB Serial Number : FOC13233R8V
Top Assembly Part Number : 800-29144-03
Top Assembly Serial Number : FTX1326T00P
Top Revision Number : A0
Product/Model Number : AIR-LAP1131AG-A-K9

Configuration register is 0xF

#sh inventory
NAME: "AP1130", DESCR: "Cisco Aironet 1130 Series (IEEE 802.11a/g) Access Point"
PID: AIR-LAP1131AG-A-K9, VID: V05, SN: FTX1326T00P