topic Re: Cisco 3502I cannot connect to vWLC in Wireless

Cisco 3502I cannot connect to vWLC

mohfarid20 — Mon, 05 Jul 2021 18:54:49 GMT

Hello

I am new to wireless and I got Cisco AIR-CAP3502I-A-K9 wireless access point and installed vWLC (version: 8.5.161.0) on ESXi.
vWLC: management interface is bridged to outside and it is reached by switch and AP.
AP: is POE from the switch, getting IP from DHCP pool on Switch (also tried to statically assign CAPWAP IP/DG/controller IP).

the problem is, AP cannot join the WLC, and i ran Wireshark and could not see any CAPWAP traffic from the AP to WLC. see attached screenshot (ip in the screen shot is maybe changed from what is in the running config).

this log message keeps on appearing on the AP console:
(Translating "CISCO-CAPWAP-CONTROLLER.MOH.LOCAL"...domain server (172.16.1.1))

and sometimes the AP keeps on renewing its IP from DHCP: below logs:
*Apr 4 18:00:33.459: %DHCP-6-ADDRESS_ASSIGN: Interface BVI1 assigned DHCP address 172.16.1.58, mask 255.255.255.0, hostname AP4055.398e.0872
Translating "CISCO-CAPWAP-CONTROLLER.MOH.LOCAL"...domain server (172.16.1.1)
*Apr 4 18:00:39.423: %CAPWAP-5-DHCP_OPTION_43: Controller address 172.16.1.101 obtained through DHCP

attached show commands from AP, WLC and switch. and AP debug output and boot process.

Thanks in advance for you support

Re: Cisco 3502I cannot connect to vWLC

Scott Fella — Sat, 04 Apr 2020 16:11:31 GMT

Clear the AP’s nvram following this thread:

https://community.cisco.com/t5/other-wireless-mobility-subjects/how-to-restore-the-factory-settings/td-p/1991301

Did you also make sure you followed the vNIC settings in the guide. One thing is you need promiscuous mode enabled.

Re: Cisco 3502I cannot connect to vWLC

mohfarid20 — Sat, 04 Apr 2020 20:00:06 GMT

Thank you @Scott Fella ,

I have already seen your reply in this post, and did the steps to erase but still no luck.

for vNIC, i enabled promiscuse mode (it was disabled) but nothing changed.

Its strange while using wireshark with SPAN port, i don't see any traffic (what so ever) sourced from the AP IP address to any IP (except when doing Ping), isn't the AP should send CAPWAP packets to the WLC IP address? i dont see that in the packet capture. it seems that the CAPWAP packets never leave the AP.

Re: Cisco 3502I cannot connect to vWLC

mohfarid20 — Sat, 04 Apr 2020 20:03:45 GMT

the current status of the AP is:

1- got an IP from DHCP with all options.

%CAPWAP-5-DHCP_OPTION_43: Controller address 172.16.1.101 obtained through DHCP

%CAPWAP-3-DHCP_RENEW: Could not discover WLC. Either IP address is not assigned or assigned IP is wrong. Renewing DHCP IP.

2- AP assigns it self a new IP

then repeat the above process.

is this normal??

Re: Cisco 3502I cannot connect to vWLC

Scott Fella — Sat, 04 Apr 2020 21:02:31 GMT

That is not normal. When the ap can’t discover the wlc, the ap will reboot and mark the up as bad and will want to get a different IP address from the dhcp server. You would see bad address if using a Windows dhcp server.
So back to your situation, you can try to download the recovery image and tftp the image to the ap and delete the image you currently have in flash. Other than that, I don’t know what else you can do. If the ap is on the same subnet as the controller management, that is the simplest way for the ap to discover the controller.
Things I would do:

* factory rest the ap
* erase nvram
* upload recovery image to flash
* delete all other image files in flash
* try a different cable and switchport
* place the ap on the same subnet as the management
* use option 43 on the dhcp to see if that helps

Re: Cisco 3502I cannot connect to vWLC

mohfarid20 — Sat, 04 Apr 2020 21:53:17 GMT

I will check out the recovery image,

this is the output of #dir flash:

AP4055.398e.0872#dir
Directory of flash:/

2 -rwx 214 Apr 4 2020 18:01:57 +00:00------------------------- env_vars
3 -rwx 61676 Apr 4 2020 18:02:05 +00:00 ---------------------- event.log
6 -rwx 6168 Apr 4 2020 18:03:03 +00:00 ----------------------- private-multiple-fs
5 -rwx 296 Apr 4 2020 18:02:34 +00:00 ------------------------ capwap-saved-config
43 drwx 0 Mar 1 2002 00:14:37 +00:00 ------------------------- configs
4 -rwx 128305 Sep 16 2014 02:18:27 +00:00 ------------------- event.r0
45 -rwx 64 Apr 4 2020 18:01:57 +00:00 ------------------------ sensord_CSPRNG1
46 -rwx 64 Apr 4 2020 18:01:57 +00:00 ------------------------ sensord_CSPRNG0
52 drwx 1152 Mar 1 1993 00:24:39 +00:00 ---------------------- ap3g1-k9w8-mx.153-3.JF12
11 -rwx 85 Mar 1 1993 00:40:19 +00:00 ------------------------ mesh_port_cfg.txt
8 -rwx 127211 May 2 2015 11:38:33 +00:00 -------------------- event.r1
7 -rwx 0 Mar 1 1993 00:40:37 +00:00 -------------------------- config.txt
9 -rwx 296 Mar 1 1993 01:18:11 +00:00 ------------------------ capwap-saved-config-bak

31481856 bytes total (20784640 bytes free)

even after typing #erase command in your previous reply, i didnt see any of those files get deleted.

what is not needed so i can manually delete.

Re: Cisco 3502I cannot connect to vWLC

Scott Fella — Sat, 04 Apr 2020 22:48:31 GMT

Erase nvram to another way to factory reset. You need to delete the image file but not until you have the rcv (recovery image) uploaded to the AP. Here is a thread that also goes over best ways to upload an image to an ap:

https://community.cisco.com/t5/wireless-and-mobility/recovery-image-topology-3602i-thru-2504-controller-from-tftp/td-p/2499395

Re: Cisco 3502I cannot connect to vWLC

Leo Laohoo — Sun, 05 Apr 2020 00:26:00 GMT

@mohfarid20 wrote:

11 -rwx 85 Mar 1 1993 00:40:19 +00:00 ------------------------ mesh_port_cfg.txt

Console into the AP and post the complete output to the command "sh capwap client rcb".

Re: Cisco 3502I cannot connect to vWLC

mohfarid20 — Sun, 05 Apr 2020 05:22:51 GMT

thanks Mr. Leo

i've attached the output.

Re: Cisco 3502I cannot connect to vWLC

Leo Laohoo — Sun, 05 Apr 2020 05:51:59 GMT

Ok, nothing there.
NTP is not configured. Is the time/date of the WLC accurate?

Re: Cisco 3502I cannot connect to vWLC

mohfarid20 — Sun, 05 Apr 2020 06:16:52 GMT

I have configure NTP server on CSR and using it as NTP server for the lab

here is the time on WLC

(Cisco Controller) >show time

Time............................................. Sun Apr 5 09:12:21 2020

Timezone delta................................... 3:0
Timezone location................................

NTP Servers
NTP Version.................................. 3
NTP Polling Interval......................... 600

Index NTP Key Index NTP Server Status NTP Msg Auth Status
------- ---------------------------------------------------------------------
1----- 0------- 172.16.1.7 ------In Sync ------- AUTH DISABLED

(Cisco Controller) >

Re: Cisco 3502I cannot connect to vWLC

Leo Laohoo — Sun, 05 Apr 2020 06:31:31 GMT

Please provide the output to the command "show ap join stats detailed <AP MAC ADDRESS>".

Re: Cisco 3502I cannot connect to vWLC

mohfarid20 — Sun, 05 Apr 2020 08:14:06 GMT

(Cisco Controller) >show ap join stats detailed 40:55:39:8E:08:72
No join information found for AP: 40:55:39:8e:08:72


(Cisco Controller) >

it seems the packets are never made it to WLC, or even never made it out of the AP itself.

Re: Cisco 3502I cannot connect to vWLC

mohfarid20 — Sun, 05 Apr 2020 08:15:30 GMT

i have downloaded the rcv image, but this method requires an external power, i only have POE source. 😞

Re: Cisco 3502I cannot connect to vWLC

Leo Laohoo — Sun, 05 Apr 2020 09:08:19 GMT

@mohfarid20 wrote:
i have downloaded the rcv image, but this method requires an external power, i only have POE source. 😞

No, you don't. Console into the AP and issue the following command:

archive download-sw tftp://<TFTP_IP_ADDRESS>/filename.tar

And then reboot.

@mohfarid20 wrote:
it seems the packets are never made it to WLC

Can the AP ping the WLC?
Check firewall.

Re: Cisco 3502I cannot connect to vWLC

mohfarid20 — Sun, 05 Apr 2020 10:18:42 GMT

ping was successful from the beginning, but sure it was the firewall block CAPWAP port.

i disabled the firewall and started to see good things happening :D,

but still not yet registered with the WLC

CAPWAP messages are exchanged between AP and WLC (see packet captures)

and also see "#debug capwap packets" on both the AP and WLC.

thanks for your support and for following up with the post.

Re: Cisco 3502I cannot connect to vWLC

Leo Laohoo — Sun, 05 Apr 2020 12:01:50 GMT

(Cisco Controller) >show ap join stats detailed 40:55:39:8E:08:72

What is the output to the above command?

Re: Cisco 3502I cannot connect to vWLC

mohfarid20 — Sun, 05 Apr 2020 12:46:27 GMT

(Cisco Controller) >show ap join stats detailed 40:55:39:8E:08:72
No join information found for AP: 40:55:39:8e:08:72


(Cisco Controller) >

nothing,

could u check "debug capwap events" output (attached)

i also read somewhere, i should add the AP mac to:

Security > AP Policies > (enabled) Authorize MIC APs against auth-list or AAA

and added AP MAC to > AP Authorization List

Re: Cisco 3502I cannot connect to vWLC

mohfarid20 — Sun, 05 Apr 2020 14:14:38 GMT

when i issue "sh ver" on AP, i got this;

Base ethernet MAC Address: 40:55:39:8E:08:72

but when i issued "sh ap join stats summary all" on WLC, this is what i got

it seems the MAC is different

(Cisco Controller) >show ap join stats summary all

Number of APs.............................................. 1 

Base Mac             AP EthernetMac       AP Name                 IP Address         Status
e8:40:40:df:64:60    N A                  AP4055.398e.0872        172.16.1.97        Not Joined

so here is the output of "show ap join stats detailed e8:40:40:df:64:60"

(Cisco Controller) >show ap join stats detailed e8:40:40:df:64:60

Sync phase statistics
- Time at sync request received............................ Not applicable
- Time at sync completed................................... Not applicable

Discovery phase statistics
- Discovery requests received.............................. 80
- Successful discovery responses sent...................... 80
- Unsuccessful discovery request processing................ 0
- Reason for last unsuccessful discovery attempt........... Not applicable
- Time at last successful discovery attempt................ Apr 05 17:08:53.094
- Time at last unsuccessful discovery attempt.............. Not applicable

Join phase statistics
- Join requests received................................... 140
- Successful join responses sent........................... 0
- Unsuccessful join request processing..................... 140
- Reason for last unsuccessful join attempt................ Failed to add database entry
- Time at last successful join attempt..................... Not applicable
- Time at last unsuccessful join attempt................... Apr 05 17:09:08.269

Configuration phase statistics

--More-- or (q)uit
- Configuration requests received.......................... 0
- Successful configuration responses sent.................. 0
- Unsuccessful configuration request processing............ 0
- Reason for last unsuccessful configuration attempt....... Not applicable
- Time at last successful configuration attempt............ Not applicable
- Time at last unsuccessful configuration attempt.......... Not applicable

Last AP message decryption failure details
- Reason for last message decryption failure............... Not applicable

Last AP disconnect details
- Reason for last AP connection failure.................... Not applicable
- Last AP disconnect reason................................ Not applicable

Last join error summary
- Type of error that occurred last......................... Unsuccessful Lwapp join response sent
- Reason for error that occurred last...................... Failed to add database entry
- Time at which the last join error occurred............... Apr 05 17:09:08.269

AP disconnect details
- Reason for last AP connection failure.................... Not applicable
Ethernet Mac : 00:00:00:00:00:00 Ip Address : 172.16.1.97

if you got time please have a look at the wireshark capture it maybe helpful.

Re: Cisco 3502I cannot connect to vWLC

Scott Fella — Sun, 05 Apr 2020 14:35:31 GMT

Look at the output from the console of the ap now since you have disabled the FW.