Cisco Secure Endpoint Service Stopped

[mandrews]

Good day all!

From time to time, I find that there are several of our machines that have their service stopped with Secure Endpoint. I haven't found what has been stopping it, but has anyone seen this and know what has been causing this? And is there a way to detect machines whose service has been stopped from the console?

Thank you,

Maurice

[JennieZhang]

hello, 

do you mean from time to time you find 'Cisco Secure Endpoint' service was in a 'stopped' status? 

have you ticked the checkbox of 'enable connector protection'? you can find this option under 'advanced settings' -> 'administrative features' of your policy.

This feature can prevent  malware, application or user from disabling secure endpoint service.

[mandrews]

Hey Jennie! Thanks for responding. Yes, that's exactly what I mean on the services being stopped. I've found a few machines in the environment where it was stopped and it wasn't in positioned to be scanned or anything. In regards to your question about the connector protection, yes, we do have that turned on as well. I'm not sure if there's anything that's stopping it outside of that or if anyone has experienced this happening consistently. Outside of having people open CSE on their computers, I'm not sure if there's a way to check from the console if the services have stopped. 

[newberntac]

Is there any update on this?  We have the same issue, to the tune of 25% of our systems at a time.  And having to connect to EACH one just to restart the service is a time killer.

[mandrews]

Hey! Unfortunately, the closest solution that has been recommended was the connector protection. Unfortunately, that doesn't keep the connector services from stopping. I'm still not sure what is happening to cause the service to stop, but it's something I'd like to get to the bottom of to make sure that our environment is secured.

[DaphneG]

@mandrews, I suggest checking the Secure Endpoint directory for crash dumps then opening a TAC case. If you have an open SR, feel free to PM me the number so I can review/follow up with the TAC engineer. 

[mandrews]

Hey Daphne!

Do you know the file path or the file name for the file that would have the crash dump? I'm assuming that it would be a temp file. 

[newberntac]

I found this on setting the client up to collect debug info.  We're going to try and gather debugs too.

https://www.cisco.com/c/en/us/support/docs/security/amp-endpoints/216035-collect-debug-logs-file-in-amp-for-endpo.html

[mandrews]

I normally use debugs when CSE is using a lot of CPU utilization from scanning. I don't know if it'll reveal what is kicking off the connector, unless it scanning something too much is what stops the service.

[DaphneG]

With the debugging enabled, it provides us more context and enables us to correlate things with the dump. 

Btw, if after following the recommendations below you're still not seeing any .dmp files, please open a TAC case and use this discussion as reference.

[DaphneG]

You should see it under Program Files > Cisco > AMP. However, I forgot to mention earlier that there's a setting that'll determine if the dump will be written and saved locally or sent to the cloud. This setting, called "Automatic Crash Dump Uploads", can be found under Policy > Advanced Settings > Administrative Features. The "Connector Log Level" will also be in the same page so I recommend setting the log level to Debug then disabling the Automated Crash Dump Upload and see if there's a .dmp file created under the AMP directory once the issue reoccurs.

[mandrews]

That feature makes sense now. Assuming that we have crash dump logs sent to the cloud, do our organization have access to that? Also, is there a way to detect a stopped service with CSE from the console?

[Vibhor Amrodia]

Hello Maurice,

I wanted to know if you were able to file a TAC Case for this issue so that we can investigate this issue further.

To answer your queries above:

1) No, unfortunately, these logs(Crash Dumps) are not to the customers 

2) You should be able to detect stopped services on the endpoint using Orbital as a probe. On the portal, the "Last Seen" Date would be the only indicator to help you detect stopped service on the Endpoints.

Thanks,

Vibhor

[mandrews]

Hey Vibhor!

No, I haven't opened a TAC case for this yet. I haven't spotted any lately, but maybe your suggestion using Orbital may help. What query can I use to detect the stopped CSE service? If that returns any results, then I'll use that evidence and submit a TAC case with it.

[UMontero]

To further add on Vibhor's reply, you can also use Powershell to get a list of Services and their status

Get-Service -Name Cisco* | ft -auto

If you want the results in a txt file, please use Get-Service -Name Cisco* | ft -auto | Out-File "C:\Users\insertusernamehere\Desktop\cisco.txt"