Hey everyone,
A couple of weeks ago one of our clients had a problem with an appliance ACS. Everyday at 5 AM an automatic compression of the View DB is programmed but it seems that, according to the logs, that procedure is having problems.
The thing is they only realized when one daythey came to the office and the admins couldnt access the Switches through TACACS+ policies. Other than that, normal users were given access through RADIUS. Before we got there, they rebooted the appliance and everything went back to normal.
From the logs we get following recurring errors at 5:00 am :
Sep 7 05:00:01 cod0309 [ACS-View-Decap-Clean]: Started to check the decap clean operation
Sep 7 05:00:21 cod0309 [ACS-View-compress-DB]: Checking ACS db actual and physical size started
Sep 7 05:00:21 cod0309 [ACS-View-compress-DB]: Physical ACS DB Size (GBs) :0.04
Sep 7 05:00:21 cod0309 [ACS-View-compress-DB]: Actual ACS DB Size (GBs) :0
Sep 7 05:00:21 cod0309 [ACS-View-compress-DB]: Checking ACS db actual and physical size completed
Sep 7 05:00:21 cod0309 [ACS-View-compress-DB]: Compress View DB was called to reduce the view db file size
Sep 7 05:00:21 cod0309 [ACS-View-compress-DB]: ACS View database compress option status from GUI FALSE
Sep 7 05:00:21 cod0309 [ACS-View-compress-DB]: upper limit value : 134.43physical DB size :1.57
Sep 7 05:00:24 cod0309 logger: warning:[ACSView Database Compress] Compressing ACS View Database failed
Sep 7 05:01:05 cod0309 adclient[22630]: INFO <fd:20 rt_daemon(6539)> Failed to send message: Socket closed, MsgTy
pe: 909.
To me, I think this is the error that caused the problem. It seems the ACS encountered and runtime error and the services restarted.
Sep 8 06:45:01 cod0309 [ACS-View-Decap-Clean]: Started to check the decap clean operation
Sep 8 06:54:58 cod0309 ACS runtime ERROR: terminate called after throwing an instance of 'std::invalid_argument'
Sep 8 06:54:58 cod0309 ACS runtime ERROR: what(): Invalid socket
Sep 8 06:55:46 cod0309 monit[10297]: 'runtime' process is not running
Sep 8 06:55:46 cod0309 monit[10297]: 'runtime' trying to restart
Sep 8 06:55:46 cod0309 monit[10297]: 'runtime' start: /opt/CSCOacs/bin/exec_wrapper.sh
Sep 8 06:55:46 cod0309 ACS runtime INFO: executing /opt/CSCOacs/runtime/bin/run.sh start
Sep 8 06:55:46 cod0309 ACS runtime INFO: Checking core files. Total size 0 M
Sep 8 06:55:46 cod0309 ACS runtime INFO: MaxCoreSize limit is 2000 M
Has anyone had a similar problem? I've checked any possible bugs but I could only find bugs for prior versions and not directly related to a compression of the View Database that could restart the services and mess up authorization policies.
Ed.
