WLC 5520 licensing problem

[Scott Paananen]

Hello everyone, 

so this mornring when i came in,  about half of my APs were not connected to my WLC. I went into the logs, and this is what is there:

*spamApTask4: Dec 03 14:48:38.267: %CAPWAP-3-AP_DB_ALLOC: [SA]capwap_ac_db.c:151 The system is unable to alloc AP entry in database for 192.168.68.140:5256
*spamApTask4: Dec 03 14:48:38.267: %LWAPP-3-LICENSE_ACQUIRE_ERR: [SA]spam_lrad.c:398 Failed to acquire license from the licensing module, maximum APs Joined 15/0.
*spamApTask4: Dec 03 14:48:38.163: %LWAPP-3-LICENSE_ACQUIRE_ERR: [SA]spam_lrad.c:398 Failed to acquire license from the licensing module, maximum APs Joined 15/0.
*spamApTask6: Dec 03 14:48:35.222: %LWAPP-3-LIST_ERR8: [SA]spam_list.c:1074 The sytem is unable to delete AP entry, entry not allocated
*spamApTask7: Dec 03 14:48:30.955: %CAPWAP-3-AP_DB_ALLOC: [SA]capwap_ac_db.c:151 The system is unable to alloc AP entry in database for 192.168.68.143:5264
*spamApTask7: Dec 03 14:48:30.955: %LWAPP-3-LICENSE_ACQUIRE_ERR: [SA]spam_lrad.c:398 Failed to acquire license from the licensing module, maximum APs Joined 15/0.
*spamApTask3: Dec 03 14:48:27.944: %LWAPP-3-LIST_ERR8: [SA]spam_list.c:1074 The sytem is unable to delete AP entry, entry not allocated
*spamApTask2: Dec 03 14:48:21.959: %LWAPP-3-LIST_ERR8: [SA]spam_list.c:1074 The sytem is unable to delete AP entry, entry not allocated
*spamApTask1: Dec 03 14:48:21.713: %LWAPP-3-LIST_ERR8: [SA]spam_list.c:1074 The sytem is unable to delete AP entry, entry not allocated
*spamApTask5: Dec 03 14:48:17.999: %LWAPP-3-LIST_ERR8: [SA]spam_list.c:1074 The sytem is unable to delete AP entry, entry not allocated
*spamApTask4: Dec 03 14:48:17.742: %LWAPP-3-LIST_ERR8: [SA]spam_list.c:1074 The sytem is unable to delete AP entry, entry not allocated
*spamApTask4: Dec 03 14:48:17.638: %LWAPP-3-LIST_ERR8: [SA]spam_list.c:1074 The sytem is unable to delete AP entry, entry not allocated
*spamApTask7: Dec 03 14:48:10.428: %LWAPP-3-LIST_ERR8: [SA]spam_list.c:1074 The sytem is unable to delete AP entry, entry not allocated
*spamApTask5: Dec 03 14:47:57.418: %CAPWAP-3-AP_DB_ALLOC: [SA]capwap_ac_db.c:151 The system is unable to alloc AP entry in database for 192.168.68.149:5256
*spamApTask5: Dec 03 14:47:57.418: %LWAPP-3-LICENSE_ACQUIRE_ERR: [SA]spam_lrad.c:398 Failed to acquire license from the licensing module, maximum APs Joined 15/0.
*spamApTask0: Dec 03 14:47:53.492: %CAPWAP-3-AP_DB_ALLOC: [SA]capwap_ac_db.c:151 The system is unable to alloc AP entry in database for 192.168.68.136:5248
*spamApTask0: Dec 03 14:47:53.492: %LWAPP-3-LICENSE_ACQUIRE_ERR: [SA]spam_lrad.c:398 Failed to acquire license from the licensing module, maximum APs Joined 15/0.
*spamApTask6: Dec 03 14:47:51.005: %CAPWAP-3-AP_DB_ALLOC: [SA]capwap_ac_db.c:151 The system is unable to alloc AP entry in database for 192.168.68.150:5272
*spamApTask6: Dec 03 14:47:51.005: %LWAPP-3-LICENSE_ACQUIRE_ERR: [SA]spam_lrad.c:398 Failed to acquire license from the licensing module, maximum APs Joined 15/0.
*spamApTask3: Dec 03 14:47:50.694: %LWAPP-3-LICENSE_ACQUIRE_ERR: [SA]spam_lrad.c:398 Failed to acquire license from the licensing module, maximum APs Joined 15/0.
*spamApTask6: Dec 03 14:47:44.477: %CAPWAP-3-AP_DB_ALLOC: [SA]capwap_ac_db.c:151 The system is unable to alloc AP entry in database for 192.168.68.142:5256
*spamApTask6: Dec 03 14:47:44.477: %LWAPP-3-LICENSE_ACQUIRE_ERR: [SA]spam_lrad.c:398 Failed to acquire license from the licensing module, maximum APs Joined 15/0.
*spamApTask3: Dec 03 14:47:37.199: %CAPWAP-3-AP_DB_ALLOC: [SA]capwap_ac_db.c:151 The system is unable to alloc AP entry in database for 192.168.68.147:5264
*spamApTask3: Dec 03 14:47:37.199: %LWAPP-3-LICENSE_ACQUIRE_ERR: [SA]spam_lrad.c:398 Failed to acquire license from the licensing module, maximum APs Joined 15/0.
*spamApTask5: Dec 03 14:47:36.893: %LWAPP-3-LIST_ERR8: [SA]spam_list.c:1074 The sytem is unable to delete AP entry, entry not allocated
*spamApTask0: Dec 03 14:47:32.971: %LWAPP-3-LIST_ERR8: [SA]spam_list.c:1074 The sytem is unable to delete AP entry, entry not allocated

At first glance, it looks like it is saying that i dont have licensing for it. but i have 50 RTU licenses i purchased with the device. 

I can verify this by going to Management -> Software Activation - Licenses  where it shows,  the name (Ap count) the type, which is Permanent adn the RTU count, which is 50.  When i go to Licensing usage it is showing a max count of 0,  current count 15 and remaining count is also 15. 

so what is happening here? everything was fine till this morning. Not sure why the licensing would have disappeared. 

Please help. 

[marce1000 · Accepted solution]

  - Reboot the controller , check if that can help.

 M.

[Scott Paananen]

Thanks, that seemed to do it. Not sure why my primary one went down like that, but the reboot  resolved it. 

thank you 

Scott

[marce1000]

 - From the first message-only (CAPWAP-3-AP_DB_ALLOC....) ; I presume the controller suffered from resource exhaustion due to long uptimes. In fact you executed a 'cleanup'-reboot. If that would happen often look at the software version being used and compare with https://www.cisco.com/c/en/us/support/docs/wireless/wireless-lan-controller-software/200046-tac-recommended-aireos.html . Of course when upgrading you need to check if all access points remain compatible.

 M.

[Scott Paananen]

Great thanks for the into, yea, i resource exhaustion makes a lot of since, I'm gonna have to make a note to check on the servers once ever few weeks to avoid it happening again.  this is the first time we've had this happen, and we have had the system for a few years now.