Investigating ACS user keeps being disabled

[tal_shaul]

Hi everyone,

My ACS 5.5 keeps disabling a TACACS+ user that we use in a post-auth expect script with freeradius to configure something on the switches with a message that the user had too many authentication failures. The user is entitled to use all commands and is marked with "never expire password" policy. Of course the script uses the right password for this user. When I re-enable the user it being disabled again after couple of seconds.

I can't find any information about the issue in ACS troubleshoot authentication (empty) or reports (which doesn't work) screens. How can I find the reason for that and prevent this user from being disabled?

Thanks,

Tal

[pieterh]

you mention both ACS and freeradius. I'm not clear what account-database is used for the user authentication?

"entitled to use all commands" is authorization, authentication failure is before authorization occurs

[InternetDefenders430]

We use freeradius with mab for dynamic vlan functionality, but the admin logins to the switches using ACS with TACACS+.

The freeradius server is configured with post-auth script that configures the access vlan on the interface that had been authenticated using ssh with a TACACS+ admin user to the switch, to ensure that even in a DataCenter failure when the freeradius server is down/unreachable, the users' workstations will be configured in the right vlan.

The TACACS+ admin user that the post-auth script is using is being disabled automatically again and again and we can't figure why. One of the thoughts we had about why it could get disabled is an authorization vaiolation, anyway this is not the case either. 

How can we find the reason that this user is being disabled?

Tal

[pieterh]

to ensure that even in a DataCenter failure when the freeradius server is down/unreachable, the users' workstations will be configured in the right vlan

the situation described may be resolved by using the interface command

    authentication event server dead action authorize <VLAN_number>

apart from this the tacacs+ admin user is a only known at the  (ACS)?

or does the ACS authenticate this user at another user database like AD?

[InternetDefenders430]

authentication event server dead action authorize <VLAN_number> 

VLAN_number can't be pre-known. All the functionality of the freeradius server is to allow users to move their workstations to other rooms in the same site (or to install new workstations) without any manual change in network configuration. They have a web portal which updates the freeradius mysql database with mac to vlan mappings.

the tacacs+ admin user is only known via ACS internal users database. ACS does not authenticate the user against any external source..

[pieterh]

Then I'm not sure what your question is?

if it's for the situation where the DC/freeradius server is unreachable then you cannot dynamically assign vlans?

if you mean you have different vlans per room you need to configure the ports in that room individually with the command I suggested AND the correct vlan for that room. 

this vlan is only activated if the radiusserver is declared "dead" (multiple timeouts)

[pieterh]

as for reports to work you need to enable active-x/flash, which most browsers currently block

you can try the CLI  commands to search for a reason that the account gets locked

- show acs-logs 

- show acs-logs filename <log-filename> 

[InternetDefenders430]

freeradius works well. but the post authentication script it executes using ssh to switches fails because the user is being disabled, and this is what i'm trying to resolve. will check flash and those logs.. tnx

[InternetDefenders430]

I found in acsLocalStore.log:
Failed-Attempt: Authentication failed,....,FailureReason=22040

Failed-Attempt: Authentication failed,....,FailureReason=22040

Failed-Attempt: Authentication failed,....,FailureReason=22075

acs version 5.8.0.32-B.442.x86_64

and at the same time in catalina.log:

org.hinernate.event.def.AbstractFlushingEventListener performExecutions

SEVERE: Could not synchronize database state with session

org.hibernate.StaleObjectStateExepction: Row was updated or deleted by another transaction (or unsaved-value mapping was incorrect): [com.cisco.nm.acs.im.identity.ProtocolUser#648]

at org.hibernate.persister.entity.AbstractEntityPersister.check(AbstractEntityPersister.java:1635)

at org.hibernate.persister.entity.AbstractEntityPersister.update(AbstractEntityPersister.java:2208)

at org.hibernate.persister.entity.AbstractEntityPersister.updateOrInsert(AbstractEntityPersister.java:2118)

at org.hibernate.persister.entity.AbstractEntityPersister.update(AbstractEntityPersister.java:2374)

at org.hibernate.action.EntityUpdateAction.execute(EntityUpdateAction.java:91)

at org.hibernate.engine.ActionQueue.execute(ActionQueue.java:248)

at org.hibernate.engine.ActionQueue.executeActions(ActionQueue.java:232)

at org.hibernate.engine.ActionQueue.executeActions(ActionQueue.java:140)

at org.hibernate.event.def.AbstractFlushingEventListener.performExecutions(AbstractFlushingEventListener.java:297)

at org.hibernate.event.def.DefaultFlushEventListener.onFlush(DefaultFlushEventListener.java:27)

at org.hibernate.impl.SessionImpl.flush(SessionImpl.java:985)

at org.hibernate.impl.SessionImpl.managedFlush(SessionImpl.java:333)

at org.hibernate.transaction.JDBCTransaction.commit(JDBCTransaction.java:106)

at com.cisco.nm.acs.mgmt.db.Db.endTx(Db.java:309)

at com.cisco.nm.acs.mgmt.bl.framework.TransactionManager.commit(TransactionManager.java:78)

at com.cisco.nm.acs.mgmt.bl.framework.BaseManagementSession.updateImpl(BaseManagementSession.java:2028)

at com.cisco.nm.acs.mgmt.bl.framework.BaseManagementSession.update(BaseManagementSession.java:459)

at com.cisco.nm.acs.mgmt.bl.framework.BaseManagementSession.update(BaseManagementSession.java:449)

at sun.reflect.GeneratedMethodAccessor1325.invoke(Unknown Source)

at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)

at java.lang.reflect.Method.invoke(Method.java:606)

at com.cisco.nm.acs.mgmt.performancemonitoring.PerformanceProxy.invoke(PerformanceProxy.java:51)

at com.sun.proxy.$Proxy0.update(Unknown Source)

at com.cisco.nm.acs.mgmt.replication.RemoteReplicationManagmentImpl.remoteUpdateUserFailedAttempt(RemoteReplicationManagementImpl.java:707)

.....

Iv'e added a log to the script the freeradius uses to ssh to switches and authentication using TACACS against ACS to see if there's any error there in the time of locking... would love for more insights meanwhile. 

Thanks!

[pieterh]

Do you have a secundary ACS ?

     SEVERE: Could not synchronize database state with session

suggests they may not be in sync? (or take too long to sync)

[tal_shaul]

There is a secondary ACS server. How do I check if they are in sync? Can a long time to sync disable my user?Thanks,Tal