01-29-2002 10:42 AM - edited 03-08-2019 09:42 PM
Hi. I'm trying to use STATE.HTTP to detect the User-Agent header of all my inbound Web traffic and parse the value of it to detect crawlers, robots, etc. Any suggestions?
01-29-2002 11:37 AM
I assume you know what will appear in the 'User-Agent' Field. If you don't that would be your first step by using tcpdump over a sustained period of time. With that information you could craft a simple HeaderRegex that would match on what you are looking for. The following example would fire upon detecting a Mozilla browser.....
Tune Signature Parameters : CSIDS Signature Wizard
___________________________________________________________________________
Current Signature: Engine STATE.HTTP SIGID 20000
SigName: Mozilla User Agent (example)
___________________________________________________________________________
0 - Edit ALL Parameters
1 - AlarmInterval =
2 - AlarmThrottle = FireOnce
3 - ArgNameRegex =
4 - ArgValueRegex =
5 - ChokeThreshold =
6 - DeObfuscate = True
7 - Direction = ToService
8 - FlipAddr =
9 - HeaderRegex = User[-]Agent[:][ ]Mozilla
10 - LimitSummary =
11 - MaxArgFieldLength =
12 - MaxHeaderFieldLength =
13 - MaxInspectLength =
14 - MaxRequestFieldLength =
15 - MaxUriFieldLength =
16 - MinHits = 1
17 - RequestRegex =
18 - ResetAfterIdle = 15
19 - ServicePorts = 80,3128,8000,8010,8080,8888,24326
20 - SigComment =
21 - SigName = Mozilla User Agent (example)
22 - SigStringInfo = Mozilla User Agent detected
23 - ThrottleInterval = 15
24 - UriRegex =
25 - WantFrag =
d - Delete a value
u - UNDO and continue
x - SAVE and continue
___________________________________________________________________________
Selection>
Hope that helps!
01-29-2002 12:33 PM
It almost did. My previous attempts did not take in all the rules for Regular Expressions. Following your example I have configured the sensor, however packetd now refuses to start. I get "W WARNING suppressed while parsing global parameter" messages in the error log.
01-29-2002 12:38 PM
Could you please cut and paste the SigWizMenu screen that describes your signature for us. This will help us to see what might be wrong. If you can't do that (it might prove to be fairly difficult) could you please forward your SigUser.conf file to anthall@cisco.com.
KLW
01-29-2002 02:30 PM
I think I found you problem. My example had a space in the brackets before Mozilla:
User[-]Agent[:][ ]Mozilla
Your SigUser.conf line did not:
User[-]Agent[:][]lwp
If you use SigWizMenu and edit that sig it should work.
Discover and save your favorite ideas. Come back to expert answers, step-by-step guides, recent topics, and more.
New here? Get started with these tips. How to use Community New member guide