cancel
Showing results for 
Search instead for 
Did you mean: 
cancel
26668
Views
185
Helpful
47
Replies

Resolving AMP for Endpoints TETRA Definition Issues

aledipas
Cisco Employee
Cisco Employee

 

Hi Everyone,

 

Recently we were made aware of a TETRA AV definition update which caused the Windows AMP for

Endpoints service to crash. 

 

Note:  Customers who do NOT have TETRA enabled are not affected by this issue. 

 

While we have already removed the problematic definition set, which was available for ~30 minutes (see further notes below), affected systems will need to be fixed manually by uninstalling/re-installing the Connector (instructions below). Once the connector has been re-installed, a non-affected definition set will be downloaded and resolve the issue.

 

How to determine if you are impacted:

The issue causes the AMP for Endpoints service to crash or hang. The best way to determine if you have an affected system is to determine if any Connectors have been offline since the bad definition set was published.

 

To get the Last Seen Timestamp from the AMP Console, go to the Management tab and select Computers. From here you can download a CSV file using the "Export to CSV" option. The CSV will contain the Last Seen Timestamp. You can sort and filter on Connectors that have not been seen since 16:00 UTC February 06 2018 – these are likely Connectors that have been affected by this issue.

 

Resolution:

We urge all customers who are affected by this issue to open a TAC case immediately.

 

Resolving this issue does involve uninstalling and reinstalling the Connector.

 

Uninstall via Add/Remove Programs:

a) Uninstall the connector (choose "No" when asked if you plan to install the Connector again)
b) Re-install connector

 

Uninstall via Command Line:

<installer> /R /S /stopservicecoe 1 /remove 1

 

Uninstall via Command Line with Connector Protection Enabled:

<installer> /R /S /stopservicecoe 1 /remove 1 /uninstallpassword <INSERT YOUR PASSWORD>

 

Affected Software Versions:

All Windows Connector versions with TETRA enabled are affected on both 32bit and 64bit versions of Windows 7/8/10, Windows Server 2008R2 and Server 2012

 

Notes:

 

TETRA Definition Sets:

Faulty TETRA definition revision (16:20 UTC)

32bit = 101032, 64bit = 70876

 

Updated TETRA definition revision (16:50 UTC)

32bit = 101034, 64bit = 70878

 

A Root Cause Analysis (RCA) document will be prepared and shared with affected customers. 


 

 

47 Replies 47

Your could change the resolution script like this:

 

    # remediation:
   
    $sfc = @(Get-ChildItem -Path "$env:ProgramFiles\Cisco\AMP" -Recurse -Include "sfc.exe")
    $sfc | foreach { .$($_.fullname) -k <YOUR PASSWORD> }
   
    $svc = Get-Service | Where-Object {$_.name -match "ciscoamp"}
    if ($svc.status -eq "running") {
        Stop-Service $svc -Force
    }

    Remove-Item "$env:ProgramFiles\Cisco\AMP\tetra\Plugins\*" -Force
    Remove-Item "$env:ProgramFiles\Cisco\AMP\update\Plugins\*" -Force
    Start-Service $svc

szielonka1
Level 1
Level 1

Same Problem

6261711391345530340.jpg

akton112233
Level 1
Level 1

Trying to modify the provided script in the OP using the documentation here ( https://www.cisco.com/c/en/us/support/docs/security/sourcefire-fireamp-endpoints/118588-technote-fireamp-00.html ) since we have connector protection enabled. Currently not working (access denied) but still trying.

  1. Try this addition to the script to get it to disable the connector protection.
  2. :: START cmd.exe /c "cd C:\Program Files\Cisco\AMP\<version you're running > & sfc.exe -k <connector protection password>" 

    START cmd.exe /c "cd C:\Program Files\Cisco\AMP\5.1.13 & sfc.exe -k Cisco123456"
    timeout 10 > nul
    del "C:\Program Files\Cisco\AMP\tetra\Plugins\*" /q
    del "C:\Program Files\Cisco\AMP\update\Plugins\*" /q
    timeout 5 > nul
    wmic service where "name like 'CiscoAMP%%'" call startservice > nul

tnhp
Level 1
Level 1

That one solved the problem - thanks.

 

Uninstall.

Choose no - (all files including quarantined are deleted).

Reboot.

Download and install connector.

damelton
Cisco Employee
Cisco Employee

Fixed my problem and the Window's security definition error, and the very slow response my PC was having.  All is well now thankfully.

UnUser
Level 1
Level 1

It is not possible to delete definition files after stopping the Service.

Looks like the Kerneldriver is preventing changes in AMP-Directory

 

 

thanks Cisco for this great Product

 

I haven't encountered duplicates in AMP by using the 'say no'-solution.

 

Advise: Try a few machines and check AMP before continuing.

 

you are right

changed my reply

Jens Nygaard
Level 1
Level 1

Are there any "Mass deployment" solution to this mess? this is really bad for a corporate environment to Uninstall the connector, Force a reboot, Install the connector and force another reboot..

Jeffsoh
Level 1
Level 1

Need a way to manage definitions through the console. Also need status information in the console on whether a connector is connected or not. Not really an enterprise solution with so little control over the endpoints. 

CE_Tristan
Level 1
Level 1
The delete_defs.bat script does not work for mass deployment purposes as it relies on manual elevation of the command prompt to delete the problematic files.

Alternatively, I'm unaware of a CLI parameter that can be set with the execution of the uninstall.exe executable in the local Cisco AMP version folder that would allow for the scripting of the AMP's uninstallation that includes removal of the definitions as well.

Cisco AMP customers need a better solution than has been provided to accommodate the large scope of machines that have been impacted. A manual machine-by-machine resolution is not optimal for companies managing hundreds or thousands of computers.

s22crocker
Level 1
Level 1

Can we get more details on this issue?  Are all versions of connector afffected?  What is the TETRA definition version number that is the issue?  Does the connector stop working completely until the fix is applied or does a reboot temporarily get it going again with the possibility of another crash? 
That information would greatly help us determine our exposure and assist in our remediation planning.

ESherlock
Level 1
Level 1

I would really like to get some explanation of the 0016 timestamp. Are you saying anything that has checked in from 0017 on is okay? Is it anything in a certain window?

Todd Everett
Cisco Employee
Cisco Employee

Rather than manually touching each endpoint, please remember you can push uninstalls and reinstalls via SCCM or any other deployment  tool using Command Line Switches. These are listed in the latest Deployment Strategy Guide on page 26.

 https://docs.amp.cisco.com/en/A4E/AMP%20for%20Endpoints%20Deployment%20Strategy.pdf

 

/remove 1       Uninstalls the Connector and removes all associated file

So you can use these commands:

<installer> /r /S /stopservicecoe 1 /remove 1

 <installer> /r /S /stopservicecoe 1 /remove 1 /uninstallpassword <INSERT YOUR PASSWORD>

 to remove the application. The <installer> should be the original installation file for the AMP for Endpoints Connector.

 

Todd

 

Getting Started

Find answers to your questions by entering keywords or phrases in the Search bar above. New here? Use these resources to familiarize yourself with the community: