cancel
Showing results for 
Search instead for 
Did you mean: 
cancel
1556
Views
5
Helpful
2
Replies

ETF The storage limit exceeded

Steflstefan
Beginner
Beginner

Hello

sometimes I recieve a warning from ESA

The Warning message is:

THREAT_FEEDS: The storage limit of 1250000 observables exceeded for the observable type: URL

 

I have two ETF souces configured; Haila_phishtank and OTX Alienvault.

Is there a option to increase the storage for ETF observable URL?
Maybe, the same URLs stored two times, one from Haila and the other from Alienvault? Could that be?

Can I have a look into ETF URL store or is there an option to download the stored data for further investigation?

 

All in all, we are satisfied with the possibility to discover URLs using the ETF sources.

Regards
Stefan 

 

2 Replies 2

SriramV
Cisco Employee
Cisco Employee

ETF Feature was brought to mitigate with very recent Threat or to use Internal Threat source. Mostly within a few hours/ days, the threat is updated in AV, antispam or other engines in ESA.

Even though we has provided 356 days of feed data to poll, it's not recommended configure it unless its needed. 

 

Answering your questions.

For a same source, ESA does not save duplicate URL's from feeds.

ESA stores an URL in 3 formats, <http and https>://<url> and only URL without protocol. 3 entries will be added in DB table for a URL. its done to increase search performance.

Currently we don't have any option increase DB limits. Can request for enhancement.

 

Hello Siram,

Thank you for your feedback.

 

The following is currently configured:
Age of Threat Feeds: 10
Time Span of Poll Segment: 10
Should this be reduced?

 

About the duplicates:
If I understand it correctly, the URL http://bad.hacker.com, for example, is stored twice if it is in Haila's feed and also in Alienvault's feed.
The URLs are not correlated across all ETF sources?

Regards Stefan

Getting Started

Find answers to your questions by entering keywords or phrases in the Search bar above. New here? Use these resources to familiarize yourself with the community:

Recognize Your Peers