Sourcefire DC 3500 very slow performance

[khendrick512]

Hi,

I am running two 3500 DC appliances in a high availability pair on DC version 5.4.1.2 and find that most tasks on the GUI, including just logging in, opening dashboards, editing policies, and especially applying policies, seems painfully slow.  Depending on what I'm pulling up, it can take 45+ seconds to open any page, and sometimes 3+ minutes to open a policy.  Could be just me being picky, but is this the kind of performance that others see?  I find myself not wanting to open pages because of the unresponsiveness.

I recently tried trimming down some of my database data limits to see if that would help and it did not.  I ran the DBCheck.pl script from the CLI, and this resulted in a good number of db warnings, but no errors.

Any ideas of things I can look at to optimize performance?

Your thoughts are appreciated.

[khendrick512]

I failed to mention above - my CPUs on the DCs seem to run consistently at ~70%, and memory at about 40%, so there doesn't seem to be a hardware performance overload.

[Aastha Bhardwaj]

Hi,

Can you run DBCheck.pl on the cli of the defense center and see if there are any corruptions in the database . You can run the command by ssh on the defense center  and then escalate the privilege to root by command : sudo su

Regards,

Aastha Bhardwaj

Rate if it helps!!!

[khendrick512]

Yes - I did run the DBCheck.pl script while investigating the issue.  It didn't come up with any errors, buy it report around 290 "warnings" similar to "[WARNING][missing referential data]column [current_user_ip_map.id]" and some other info.  I'm not sure if that constitutes a need for a repair or if warnings in the db are normal.

[Aastha Bhardwaj]

Hi,

Try to run the below:

>Ssh on the defense center.
>Escalate the privilege to root:
Sudo su
>Enter password

repair_users.pl –r
repair_users.pl –e

Then check DBCheck.pl and see if that makes a difference. Now generally we ignore warnings but 290 is huge , we might need to do slow  repair on the DB.

Regards,

Aastha Bhardwaj

Rate if that helps!!!

[khendrick512]

Aastha, thanks for your quick response.  

I ran the repair_users.pl script, this ran successfully but did not identify any needed fixes.

I ran the DBCheck.pl script a second time, and it came out with the same set of warnings - I was off, there are  about 390, not 290.  

Also, if it's useful, here's the full list of types of warnings it's listing (I cleared out generic identifier text):

• [WARNING][missing referential data]column [rna_client_app.clnt_app_fp_id], value [2000000742], ref column [appIdInfo.appId]
• [WARNING][missing referential data]column [current_user_ip_map.id], value [NUMBER], ref column [user_identities.id]
• [WARNING][missing referential data]column [cloud_version.uuid], value [IDENTIFIER], ref column [EM_peers.uuid]
• [WARNING][missing referential data]column [noncompliant_rna_client_app.[host_id,clnt_app_fp_id,version,clnt_app_type_id]], value [VALUES], ref column [rna_client_app.[host_id,clnt_app_fp_id,version,app_proto_id]]
• [WARNING][missing referential data]column [rna_sensor.peer], value [VALUE] ref column [EM_peers.uuid]
• [WARNING][missing referential data]column [user_last_login.user_id], value [VALUE] ref column [users.id]

If I do need to run a repair on the db, will this take down my Defense Center while it's running?

[Aastha Bhardwaj]

Hi,

I would suggest you to open up a TAC case. When we do a slow repair it will repair the complete database. you should still be able to access the DC.

 But slow repair takes time depending on the database size it might even take 2-3 hrs.

Regards,

Aastha Bhardwaj

Rate if that helps!!!

[khendrick512]

Are you referring to doing the slow repair with the /var/sf/bin/repair_table.pl script?  That script only allows me to repair a single table at a time, and plugging in what I see as tables from my error messages doesn't work - so, running "repair_table.pl -s rna_client_app" or "repair_table.pl -s current_user_ip_map" returns an error that says that InnoDB is not supported with this repair script.

[Aastha Bhardwaj · Accepted solution]

Hi,

No we have a for loop script which repairs all the tables.And apart from that there is also a script for optimization of tables . Moreover , you can also check the number of connection events on your defense center and check in data sheet the maximum number of conns supported.

I would definitely suggest you to open up a TAC case so that they can provide you with scripts accordingly.

Regards,

Aastha Bhardwaj

Rate if that helps!!!