Cisco ACS Problem

[josephqiu]

I recently have some problem with the Cisco ACS server, which is the login authentication server for my switches and routers. Everyday at a certain period of time, I just can't login to the gears; or I can login, but after dozens of attempts. A "debug tacacs" shows the following error messages:

-ERROR 1-

Sep 25 08:58:09.638 EST: TAC+: 192.168.100.100 (1005873793) AUTHEN/CONT -- TIMED OUT

Sep 25 08:58:09.638 EST: TAC+: (1005873793) AUTHEN/CONT processed

Sep 25 08:58:09.638 EST: TAC+: Error sending continue packet.

Sep 25 08:58:09.638 EST: TAC+: Closing TCP/IP 0x1D1A608 connection to 192.168.100.100/49

-ERROR 2-

Sep 25 09:09:49.397 EST: TAC+: 192.168.100.100 (1396526313) AUTHEN/CONT -- TIMED OUT

Sep 25 09:09:49.397 EST: TAC+: (1396526313) AUTHEN/CONT processed

Sep 25 09:09:49.397 EST: TAC+: received bad AUTHEN packet: type = 0, expected 1

Sep 25 09:09:49.397 EST: TAC+: received corrupt data from server.

Sep 25 09:09:49.397 EST: TAC+: Closing TCP/IP 0x77D128 connection to 192.168.100.100/49

-ERROR 3-

Sep 25 09:10:15.148 EST: TAC+: send AUTHEN/CONT packet id=3826363357

Sep 25 09:10:15.148 EST: TAC+: 192.168.100.100 (3826363357) AUTHEN/CONT queued

Sep 25 09:10:15.247 EST: TAC+: (3826363357) AUTHEN/CONT processed

Sep 25 09:10:15.247 EST: TAC+: received bad AUTHEN packet: session id = 13965263

13, expected 3826363357

Sep 25 09:10:15.250 EST: TAC+: received corrupt data from server.

Sep 25 09:10:15.250 EST: TAC+: Closing TCP/IP 0x76EC68 connection to 192.168.100.100/49

Apparently, I don't always get the same error when I failed to login. I checked the activity reports on the ACS server, and found that, for all those failed attempts, the server actually has passed my authentication and replied to the gear. No password errors or other failure records on the server.

Is there anyone has similar experience? Or could anyone explain the possible reason for those errors in the debug output?

Thanks a lot!

[psmith]

I am also seeing the exact same error after the ACS has been up and running fine for about 6 weeks. I haven't found any resolution, but count me as someone having a "similar experience"...

[p-dolbow]

You mentioned that this only happens at "certain period of time". Check your backup/database replication schedule(s) and see if they coincide. The ACS system can become temporarily unavailable during the times that it is performing these procedures. If that is the cause, you might consider changing your backup/replication schedules.

-=Phil=-

[josephqiu]

Thanks Phil. I'm glad to see my question got a reply after 4 months. :) Also, I'm not alone...

Actually, I was also thinking it's a problem just happens when database replication is undergoing. However, I checked all my ACS servers, none of them has replication scheduled at the time the problem normally happens. In other words, for my case, database replication should not be the cause.

Anyway, thanks a lot for your input.

[psmith]

Well, I opened a TAC case for our problem and it turns out this is related to timeout issues with logging and the remote agent.

We're using the ACS appliance (not the software) and had configured remote logging on the agent. When remote logging is disabled there are no more timeouts and TACACS authentication works correctly.

Our authentication problems were not intermittent, they occurred all the time, so this may not be the same as your issue. But this may be a bug related to the remote agent - if you have remote logging enabled try disabling it.

Hope that helps,

Paul

[josephqiu]

I have ACS installed on dedicated server, but not Cisco appliance. I don't have remote logging enabled. My problem is intermittent - probably is caused by network performance. I will further investigate.

Anyway, thank you for sharing the information! Merry X'mas!

[cammaher]

Hi,

It looks like the forums have come to my help again.

I have been having this exact issue. TACACS authentication works fine, but as soon as Remote Logging is turned on, TACACS authentication does not work.

Does anyone know if this issues has been resolved.

I am using Cisco ACS Solution Engine v4.0.1.42 and the Remote Agent is running on a Windows 2000 server.

Thanks,

Cam

[darpotter]

For an alternative to remote logging, take a look at www.extraxi.com/utils.htm

We have a ACS specific utility to collect CSV logs over HTTP(S) called csvsync.

It can be scheduled, works with ALL versions and types of ACS, collects from ANY number of ACSs and can be scheduled.