AP Not Joining WLC 8.10.183 - No valid AP manager found for controller

[datacenter]

First of all I'd like to say that I found other topics here in this forum with same error message, but unfortunately none of the answers helped me.

I'm raising a new vWLC 8.10.183 with a bunch of 28xx, 27xx, 9130 and 9120, but I'm getting continuosly this:

[*09/08/2023 10:55:01.1690] Discarding msg CAPWAP_WTP_EVENT_REQUEST(type 9) in CAPWAP state: DTLS Teardown(4).
[*09/08/2023 10:55:01.1690] Discarding msg CAPWAP_WTP_EVENT_REQUEST(type 9) in CAPWAP state: DTLS Teardown(4).
[*09/08/2023 10:55:05.7790] No more AP manager addresses remain..
[*09/08/2023 10:55:05.7790] No valid AP manager found for controller 'WLCXXXXX' (ip: 10.x.x.x)
[*09/08/2023 10:55:05.7790] Failed to join controller WLCXXXXX.
[*09/08/2023 10:55:05.7790] Failed to join controller.

Controller gets the join request, and happened TWICE that one of the APs was able to register, and get basic configuration (credentials, etc), but after changing it rebooted and now is not registering anymore (awkward, looks intermittent).

What did I checked already:

AP and WLC are both at same VLAN and have good conectivity (stable pinging, no firewall in the middle)

APs are all -E models and country code is proper

Licenses are RTU set to X licenses that is enough to cover

Tried with and without NTP (with proper time and date)

APs are getting WLC by DHCP and it looks OK, they are all reaching proper WLC

Tried hardreset, capwap erase all, etc, rebooting controller, etc, no luck at all

Tried a 9120 AXI and 9120 AXE both are running already (out of the box) version 8.10.183 too.

Management Port is checked for ENABLE DYNAMIC AP MANAGEMENT

Service Port is disabled and in a different SUBNET, same fov VIRTUAL interface.

I've deployed over an OVA (AIR_CTVM-K9_8_10_183_0.ova) and did a manual configuration as I always do, for any reason this time I'm facing this issue and dont know why. We have same AP and WLC models running in lots of countries without any issue.

I'm running out of options, tried debugging on both WLC and AP but got no specific messages, just the ones above.

[balaji.bandi]

Ok lets clarify some information - is the issue while you setting up new system or is this Failed working one ?

is this HA /SSO ?

Check is the Certs are expired ?

Most case - No more AP manager addresses remain.. ( my observation is Duplicate IP address ?)

(check on the switch arp tables see any Duplicate address )  - Shutdown the vWLC see are you still able to ping the IP ? (that confirms duplicate IP  in the network ?)

[datacenter]

Thank you for the response!

Single vWLC deploy, brand new one, will be used to migrate APs from an old HW controller.

IP is not duplicated, forgot to mention that I've tested it shutting down the VM for 10 minutes and loosing all pings.

It has only SSC certificate and it not expired (I've just brought the WLC up): 

SSC Device Certificate details:

Subject Name :

C=US, ST=California, L=San Jose, O=Cisco Virtual Wireless LAN Controller, 

CN=DEVICE-vWLC-AIR-CTVM-K9-00#######3F8###, emailAddress=support@vwlc.com

Validity :

Start : Sep  8 07:47:32 2023 GMT

End   : Jul 17 07:47:32 2033 GMT

There are no LSC or CSR

Tested it also resetting a 2802 and symptoms are the same of the 9120.

[marce1000]

  - Check available  AP licenses/licensed  on the intended controller , make sure you have sufficient AP-licenses , 

 M.

[datacenter]

But yes, licenses were good, 200 available and active.

[datacenter]

Guys, thanks for the help, but I found the solution and it has nothing to do with the WLC itself.

We've activated PROMISCUOUS MODE in the vSwitch from VMWARE, tested right after and all three APs I have in my desk registered.