topic Re: Problem in ACS v 4.1 in Network Access Control

Problem in ACS v 4.1

ranjit123 — Mon, 11 Mar 2019 00:30:01 GMT

Dear All,

We had faced a problem in our ACS Server 4.1, it refused all the user connections for 15 mins and we were not able to authenticate through our TACACS username and password during this perdiod. After 15 mins things became normal

Below were the logs generated by the server during this period.

==========================================================================================

Fri Oct 15 17:02:24 2010): Info: GetApplNICConfig GetIfTable size = 11192
(Fri Oct 15 17:02:24 2010): Info: GetApplNICConfig, adpt Idx = 16777220, en adpt Idx = 16777219
(Fri Oct 15 17:02:24 2010): Info: GetApplNICConfig, adpt Idx = 16777219, en adpt Idx = 16777219
(Fri Oct 15 17:02:24 2010): Info: GetApplNICConfig ip < ip address>, mask 255.255.255.240, gateway < ip address>,
(Fri Oct 15 17:09:00 2010): Trying to get current administrator name...
(Fri Oct 15 17:09:01 2010): checking Administrator: admin...
(Fri Oct 15 17:09:01 2010): Administrator admin found
(Fri Oct 15 17:09:01 2010): Trying to get current administrator name...
(Fri Oct 15 17:09:01 2010): checking Administrator: admin...
(Fri Oct 15 17:09:01 2010): Administrator admin found
(Fri Oct 15 17:19:54 2010): Trying to get current administrator name...
(Fri Oct 15 17:19:54 2010): checking Administrator: admin...
(Fri Oct 15 17:19:54 2010): Administrator admin found
(Fri Oct 15 17:19:54 2010): Trying to get current administrator name...
(Fri Oct 15 17:19:54 2010): checking Administrator: admin...
(Fri Oct 15 17:19:54 2010): Administrator admin found

===========================================================================================

Regards,

Ranjit

Re: Problem in ACS v 4.1

Tiago Antunes — Fri, 15 Oct 2010 12:29:47 GMT

Do you have replication configured?

If yes, can you check if this 15 mins were during the replication process? If yes, it is expected.

Can you share with us the csmon.log file from the C:\Program Files\CiscoSecure ACS v4.2\CSMon\Logs directory?

Cheers,
Tiago

If this answers your question please mark the question as "answered" and rate it, so other users can easily find it.

Re: Problem in ACS v 4.1

ranjit123 — Fri, 15 Oct 2010 12:39:44 GMT

Hi,

Thanks for your reply,replications is configured but its duration is 120 mins

please find the csmon.log file attached along with this mail.

ACS was implemented in 2008 and this problem occured the first time since then.

Regards,

Ranjit

Re: Problem in ACS v 4.1

Tiago Antunes — Fri, 15 Oct 2010 12:47:20 GMT

Hi,

Thanks but this is not the file I was asking for.

Can you share with us the "csmon.log" text file from the C:\Program Files\CiscoSecure ACS v4.2\CSMon\Logs directory?

Are you sure no one else configure replication?

Thanks,

Tiago

Re: Problem in ACS v 4.1

ranjit123 — Sat, 16 Oct 2010 05:45:00 GMT

Hi Tiago,

It is a Appliance and LINUX based.

below is the snap shot of the Diagnostic logs avaiable on the box.

Please update me which logs do you want for reference.

Regards,

Ranjit

Re: Problem in ACS v 4.1

Tiago Antunes — Sat, 16 Oct 2010 06:36:27 GMT

Hi Ranjit,

Yes, indeed it is an appliance, however please be aware that it is Windows based even though you don't have access to the OS level.

Ok, so you can collect the package.cab file that you can obtain when you go to System Configuration -> Support -> Collect log file, and collect log files from previous x days making sure you catch the time of the outage.

Thanks,

Tiago

Re: Problem in ACS v 4.1

ranjit123 — Sat, 16 Oct 2010 07:14:00 GMT

Hi!,

Please find the package.cab file attached.

Regards,

Ranjit

Re: Problem in ACS v 4.1

Tiago Antunes — Sat, 16 Oct 2010 07:50:59 GMT

Hi Ranjit,

I see that the timestamp on your initial post isa bit deslocated in relation to the time on the ACS.

On the ACS i see that the authentications stopped between 10/15/2010 16:49:46 and 17:08:09:

...

CSMon 10/15/2010 16:49:46 A 0523 15836 CSTacacs: Failed to authenticate on test account.

CSMon 10/15/2010 16:49:56 I 0718 15836 Auth Failure Retry 1 (Successful auths this cycle 0)

CSMon 10/15/2010 16:50:06 I 0718 15836 Auth Failure Retry 2 (Successful auths this cycle 0)

CSMon 10/15/2010 16:50:16 I 0718 15836 Auth Failure Retry 3 (Successful auths this cycle 0)

CSMon 10/15/2010 16:50:26 I 0718 15836 Auth Failure Retry 4 (Successful auths this cycle 0)

CSMon 10/15/2010 16:50:46 I 0747 15836 Confirmed alert on CSTacacs
CSMon 10/15/2010 16:50:46 E 0748 15836 CSTacacs: Failed to authenticate on test account.

CSMon 10/15/2010 16:50:46 A 0641 43980 CSTacacs: State 6 0 Event Detected Level:4 Message:CSTacacs: Failed to authenticate on test account.

CSMon 10/15/2010 17:06:36 A 0152 43980 Services were all restarted. Attempt 1.

CSMon 10/15/2010 17:08:09 I 0530 15836 CSTacacs: Authenticated
CSMon 10/15/2010 17:08:09 I 0653 43980 CSTacacs: State 0 6 No Problems

...

This tell us that something happened with the tacacs+ service that made the ACS restart the services to resume normal operations.

Unfortunately the TCS logs of the package.cab you sent do not include any logs prior to Oct 16th... Have you collected the package.cab for how many previous days? Please try to collect for previous 3 days, to make sure we get the logs of the 15th Oct.

Thanks,
Tiago

If this helps you and/or answers your question please mark the question as "answered" and rate it, so other users can easily find it.

Re: Problem in ACS v 4.1

ranjit123 — Sat, 16 Oct 2010 10:04:00 GMT

Hi!,

Please check the same

Regards,

Ranjit

Re: Problem in ACS v 4.1

Tiago Antunes — Sat, 16 Oct 2010 10:42:31 GMT

Hi Ranjit,

Unfortunately, there is nothing there again...if you open the package.cab yourself, you will see that the file TCS.log contains no logs for the 15th Oct...

Sorry but without them there is no way to know why the tacacs+ service was failing...

Cheers,

Tiago

Re: Problem in ACS v 4.1

ranjit123 — Mon, 18 Oct 2010 05:01:20 GMT

Hi!,

If replication happening i guess it will refuse all connections.

Regards,

Ranjit

Re: Problem in ACS v 4.1

Tiago Antunes — Mon, 18 Oct 2010 08:09:19 GMT

Hi,

Correct, with replication all services would stop, and not only TACACS+.

This was for sure something specific with TACACS+.

I would leave the LogLevel to FULL and monitor the ACS so that if it happens again, you can collect the pacage.cab imediately after the problem occurs and the needed logs will be there.

HTH,
Tiago

If this helps you and/or answers your question please mark the question as "answered" and/or rate it, so other users can easily find it.

Re: Problem in ACS v 4.1

ranjit123 — Mon, 18 Oct 2010 08:18:09 GMT

HI!,

What do you suggest me to do next, so that we can capture the logs if the problem reoccurs again.

Regards,

Ranjit

Re: Problem in ACS v 4.1

Tiago Antunes — Mon, 18 Oct 2010 12:22:17 GMT

As i wrote before:

I would leave the LogLevel to FULL and monitor the ACS so that if it happens again, you can collect the package.cab imediately after the problem occurs and the needed logs will be there.

HTH,
Tiago

If this helps you and/or answers your question please mark the question as "answered" and/or rate it, so other users can easily find it.

Re: Problem in ACS v 4.1

samir ahmed — Fri, 30 Nov 2012 10:44:44 GMT

We are having a same issue. here it is log.

find attached log for you reference.

(Thu Oct 25 10:29:23 2012): Trying to get current administrator name...

(Thu Oct 25 10:29:23 2012): checking Administrator: admin...

(Thu Oct 25 10:29:23 2012): Administrator admin found

(Fri Nov 02 09:08:30 2012): Trying to get current administrator name...

(Fri Nov 02 09:08:30 2012): checking Administrator: admin...

(Fri Nov 02 09:08:30 2012): Administrator admin found

(Fri Nov 02 09:08:30 2012): Trying to get current administrator name...

(Fri Nov 02 09:08:30 2012): checking Administrator: admin...

(Fri Nov 02 09:08:30 2012): Administrator admin found

(Fri Nov 02 09:10:19 2012): Trying to get current administrator name...

(Fri Nov 02 09:10:19 2012): checking Administrator: admin...

(Fri Nov 02 09:10:19 2012): Administrator admin found

(Fri Nov 02 09:10:19 2012): Trying to get current administrator name...

(Fri Nov 02 09:10:19 2012): checking Administrator: admin...

(Fri Nov 02 09:10:19 2012): Administrator admin found

(Fri Nov 02 09:12:24 2012): Trying to get current administrator name...

(Fri Nov 02 09:12:24 2012): checking Administrator: admin...

(Fri Nov 02 09:12:24 2012): Administrator admin found

(Fri Nov 02 09:12:24 2012): Trying to get current administrator name...

(Fri Nov 02 09:12:24 2012): checking Administrator: admin...

(Fri Nov 02 09:12:24 2012): Administrator admin found

(Fri Nov 02 09:13:07 2012): Trying to get current administrator name...

(Fri Nov 02 09:13:07 2012): checking Administrator: admin...

(Fri Nov 02 09:13:07 2012): Administrator admin found

(Fri Nov 02 09:13:07 2012): Trying to get current administrator name...

(Fri Nov 02 09:13:07 2012): checking Administrator: admin...

(Fri Nov 02 09:13:07 2012): Administrator admin found

(Fri Nov 02 09:15:11 2012): Trying to get current administrator name...

(Fri Nov 02 09:15:11 2012): checking Administrator: admin...

(Fri Nov 02 09:15:11 2012): Administrator admin found

(Fri Nov 02 09:15:11 2012): Trying to get current administrator name...

(Fri Nov 02 09:15:11 2012): checking Administrator: admin...

(Fri Nov 02 09:15:11 2012): Administrator admin found

(Fri Nov 02 09:28:01 2012): Trying to get current administrator name...

(Fri Nov 02 09:28:01 2012): checking Administrator: admin...

(Fri Nov 02 09:28:01 2012): Administrator admin found

(Fri Nov 02 09:28:01 2012): Trying to get current administrator name...

(Fri Nov 02 09:28:01 2012): checking Administrator: admin...

(Fri Nov 02 09:28:01 2012): Administrator admin found

(Wed Nov 07 20:49:33 2012): Trying to get current administrator name...

(Wed Nov 07 20:49:33 2012): checking Administrator: admin...

(Wed Nov 07 20:49:33 2012): Administrator admin found

(Wed Nov 07 20:49:33 2012): Trying to get current administrator name...

(Wed Nov 07 20:49:33 2012): checking Administrator: admin...

(Wed Nov 07 20:49:33 2012): Administrator admin found

(Wed Nov 07 20:50:21 2012): Trying to get current administrator name...

(Wed Nov 07 20:50:21 2012): checking Administrator: admin...

(Wed Nov 07 20:50:21 2012): Administrator admin found

(Wed Nov 07 20:50:21 2012): Trying to get current administrator name...

(Wed Nov 07 20:50:21 2012): checking Administrator: admin...

(Wed Nov 07 20:50:21 2012): Administrator admin found

(Mon Nov 12 15:48:06 2012): Trying to get current administrator name...

(Mon Nov 12 15:48:06 2012): checking Administrator: admin...

(Mon Nov 12 15:48:06 2012): Administrator admin found

(Mon Nov 12 15:48:06 2012): Trying to get current administrator name...

(Mon Nov 12 15:48:06 2012): checking Administrator: admin...

(Mon Nov 12 15:48:06 2012): Administrator admin found

(Mon Nov 12 15:51:36 2012): Trying to get current administrator name...

(Mon Nov 12 15:51:36 2012): checking Administrator: admin...

(Mon Nov 12 15:51:36 2012): Administrator admin found

(Mon Nov 12 15:51:36 2012): Trying to get current administrator name...

(Mon Nov 12 15:51:36 2012): checking Administrator: admin...

(Mon Nov 12 15:51:36 2012): Administrator admin found

(Tue Nov 20 14:17:34 2012): Trying to get current administrator name...

(Tue Nov 20 14:17:34 2012): checking Administrator: admin...

(Tue Nov 20 14:17:34 2012): Administrator admin found

(Tue Nov 20 14:17:34 2012): Trying to get current administrator name...

(Tue Nov 20 14:17:34 2012): checking Administrator: admin...

(Tue Nov 20 14:17:34 2012): Administrator admin found

(Wed Nov 21 15:29:00 2012): Trying to get current administrator name...

(Wed Nov 21 15:29:00 2012): checking Administrator: admin...

(Wed Nov 21 15:29:00 2012): Administrator admin found

(Wed Nov 21 15:29:00 2012): Trying to get current administrator name...

(Wed Nov 21 15:29:00 2012): checking Administrator: admin...

(Wed Nov 21 15:29:00 2012): Administrator admin found

(Wed Nov 21 18:08:26 2012): Trying to get current administrator name...

(Wed Nov 21 18:08:26 2012): checking Administrator: admin...

(Wed Nov 21 18:08:26 2012): Administrator admin found

(Wed Nov 21 18:08:26 2012): Trying to get current administrator name...

(Wed Nov 21 18:08:26 2012): checking Administrator: admin...

(Wed Nov 21 18:08:26 2012): Administrator admin found

(Fri Nov 23 12:48:34 2012): Trying to get current administrator name...

(Fri Nov 23 12:48:34 2012): checking Administrator: admin...

(Fri Nov 23 12:48:34 2012): Administrator admin found

(Fri Nov 23 12:48:34 2012): Trying to get current administrator name...

(Fri Nov 23 12:48:34 2012): checking Administrator: admin...

(Fri Nov 23 12:48:34 2012): Administrator admin found

(Fri Nov 23 12:51:35 2012): Trying to get current administrator name...

(Fri Nov 23 12:51:35 2012): checking Administrator: admin...

(Fri Nov 23 12:51:35 2012): Administrator admin found

(Fri Nov 23 12:51:35 2012): Trying to get current administrator name...

(Fri Nov 23 12:51:35 2012): checking Administrator: admin...

(Fri Nov 23 12:51:35 2012): Administrator admin found

(Fri Nov 23 12:52:01 2012): Trying to get current administrator name...

(Fri Nov 23 12:52:01 2012): checking Administrator: admin...

(Fri Nov 23 12:52:01 2012): Administrator admin found

(Fri Nov 23 12:52:01 2012): Trying to get current administrator name...

(Fri Nov 23 12:52:01 2012): checking Administrator: admin...

(Fri Nov 23 12:52:01 2012): Administrator admin found

(Fri Nov 23 14:15:11 2012): Trying to get current administrator name...

(Fri Nov 23 14:15:11 2012): checking Administrator: admin...

(Fri Nov 23 14:15:11 2012): Administrator admin found

(Fri Nov 23 14:15:11 2012): Trying to get current administrator name...

(Fri Nov 23 14:15:11 2012): checking Administrator: admin...

(Fri Nov 23 14:15:11 2012): Administrator admin found

(Fri Nov 23 14:15:29 2012): Info: GetApplNICConfig GetIfTable size = 11192

(Fri Nov 23 14:15:29 2012): Info: GetApplNICConfig, adpt Idx = 65540, en adpt Idx = 65539

(Fri Nov 23 14:15:29 2012): Info: GetApplNICConfig, adpt Idx = 65539, en adpt Idx = 65539

(Fri Nov 23 14:15:29 2012): Info: GetApplNICConfig, adpt Idx = 65540, en adpt Idx = 65540

(Fri Nov 23 14:15:29 2012): Info: GetApplNICConfig ip 10.212.15.1, mask 255.255.240.0, gateway 10.212.0.1

(Sun Nov 25 19:58:41 2012): Trying to get current administrator name...

(Sun Nov 25 19:58:41 2012): checking Administrator: admin...

(Sun Nov 25 19:58:41 2012): Administrator admin found

(Sun Nov 25 19:58:41 2012): Trying to get current administrator name...

(Sun Nov 25 19:58:41 2012): checking Administrator: admin...

(Sun Nov 25 19:58:41 2012): Administrator admin found

(Sun Nov 25 20:01:34 2012): Trying to get current administrator name...

(Sun Nov 25 20:01:34 2012): checking Administrator: admin...

(Sun Nov 25 20:01:34 2012): Administrator admin found

(Sun Nov 25 20:01:34 2012): Trying to get current administrator name...

(Sun Nov 25 20:01:34 2012): checking Administrator: admin...

(Sun Nov 25 20:01:34 2012): Administrator admin found

(Sun Nov 25 20:10:31 2012): Trying to get current administrator name...

(Sun Nov 25 20:10:31 2012): checking Administrator: admin...

(Sun Nov 25 20:10:31 2012): Administrator admin found

(Sun Nov 25 20:10:31 2012): Trying to get current administrator name...

(Sun Nov 25 20:10:31 2012): checking Administrator: admin...

(Sun Nov 25 20:10:31 2012): Administrator admin found

(Fri Nov 30 10:44:28 2012): Info: GetApplNICConfig GetIfTable size = 11192

(Fri Nov 30 10:44:28 2012): Info: GetApplNICConfig, adpt Idx = 65540, en adpt Idx = 65539

(Fri Nov 30 10:44:28 2012): Info: GetApplNICConfig, adpt Idx = 65539, en adpt Idx = 65539

(Fri Nov 30 10:44:28 2012): Info: GetApplNICConfig, adpt Idx = 65540, en adpt Idx = 65540

(Fri Nov 30 10:44:28 2012): Info: GetApplNICConfig ip 10.212.15.1, mask 255.255.240.0, gateway 10.212.0.1

(Fri Nov 30 10:46:16 2012): ApplGetSnmpConfig: service SNMP is running 1