2960X 15.02(EX5) %AAA-3-ACCT_LOW_MEM_UID_FAIL:AAA unable to create UID for incoming calls due to insufficient processor memory.

[NeverOutofTune]

Deployed four 2960X switches in a stack.  All okay for about one month then tried to web browse for the first time via firefox which partially displayed the page.  I assumed this was a browser error.  So tried Chrome then IE which both failed.  Chrome was a bad display and IE fails to connect.

After this, I could not telnet or ssh.  Plugged into the console and immediately started receiving:

%AAA-3-ACCT_LOW_MEM_UID_FAIL: AAA unable to create UID for incoming calls due to insufficient processor memory
%% Low on memory; try again later

I am unable to log in.  I have a TAC case logged but the first step to try is a reboot which will be difficult until I can get a maintenance window.  When I do get a maintenance window, I would also like to deploy a fix such as a different version of code or a work-around cofig command.  I don't mind disabling HTTP.

Any suggestions?

[Leo Laohoo]

%% Low on memory; try again later

First "rule" when you troubleshoot IOS issues:  If you see an error message like this, I can guarantee a part of my human anatomy this is an IOS bug.  I strongly recommend you downgrade to 15.0(2)EX3 as this is the most stable IOS for the 2960X.

[NeverOutofTune]

I am currently working with TAC

The switches failed about 18 hours later and had to be rebooted to get back up.  Now that I have console/telnet access, I can see the memory being depleted mostly by the Auth Manager process at about the same rate as free memory is dropping.

SW13#sho proc mem sort | i Auth Manager
 191   0  177721332   95004616   34757416          0          0 Auth Manager
SW13#sho proc mem sort | i Auth Manager
 191   0  177754888   95025696   34759780          0          0 Auth Manager
SW13#sho proc mem sort | i Auth Manager
 191   0  177774316   95037928   34761056          0          0 Auth Manager
SW13#sho proc mem sort | i Auth Manager
 191   0  177799720   95053940   34762888          0          0 Auth Manager
SW13#sho proc mem sort | i Auth Manager
 191   0  177824976   95069732   34764696          0          0 Auth Manager
SW13#
SW13#
SW13#sho proc mem sort | i Processor
Processor Pool Total:  442796836 Used:  103448576 Free:  339348260
SW13#sho proc mem sort | i Processor
Processor Pool Total:  442796836 Used:  103454416 Free:  339342420
SW13#sho proc mem sort | i Processor
Processor Pool Total:  442796836 Used:  103455860 Free:  339340976
SW13#sho proc mem sort | i Processor
Processor Pool Total:  442796836 Used:  103459236 Free:  339337600
SW13#sho proc mem sort | i Processor
Processor Pool Total:  442796836 Used:  103461040 Free:  339335796