topic Re: Web crawler/scraper signatures in Network Security https://community.cisco.com/t5/network-security/web-crawler-scraper-signatures/m-p/919401#M84698 <HTML><HEAD></HEAD><BODY><P>thanks, good link. I know about the custom signature configuration and will advise client to provide information from their web logs to asssit in an attempt to create a custom signature. It does not appear there are any pre-configured sigs. for "web crawler type traffic". I looked at what is available in version 6.0(3)E1 S315.</P></BODY></HTML> Tue, 12 Feb 2008 23:32:01 GMT 5creedus 2008-02-12T23:32:01Z Web crawler/scraper signatures https://community.cisco.com/t5/network-security/web-crawler-scraper-signatures/m-p/919397#M84694 <P>Running version 6.0(3)E1 S297. Is there a specific signature that will recognize web crawler/scraper traffic? If not one already defined what in particular would I need to create a customer signature.</P> Sun, 10 Mar 2019 10:59:02 GMT https://community.cisco.com/t5/network-security/web-crawler-scraper-signatures/m-p/919397#M84694 5creedus 2019-03-10T10:59:02Z Re: Web crawler/scraper signatures https://community.cisco.com/t5/network-security/web-crawler-scraper-signatures/m-p/919398#M84695 <HTML><HEAD></HEAD><BODY><P>what kindt of traffic is this ? some p2p application ?</P></BODY></HTML> Tue, 12 Feb 2008 21:15:29 GMT https://community.cisco.com/t5/network-security/web-crawler-scraper-signatures/m-p/919398#M84695 abinjola 2008-02-12T21:15:29Z Re: Web crawler/scraper signatures https://community.cisco.com/t5/network-security/web-crawler-scraper-signatures/m-p/919399#M84696 <HTML><HEAD></HEAD><BODY><P>it would be http or https type of traffic</P></BODY></HTML> Tue, 12 Feb 2008 22:57:06 GMT https://community.cisco.com/t5/network-security/web-crawler-scraper-signatures/m-p/919399#M84696 5creedus 2008-02-12T22:57:06Z Re: Web crawler/scraper signatures https://community.cisco.com/t5/network-security/web-crawler-scraper-signatures/m-p/919400#M84697 <HTML><HEAD></HEAD><BODY><P>check this :-</P><P></P><P><A class="jive-link-custom" href="http://www.cisco.com/en/US/docs/security/ips/5.0/configuration/guide/idm/dmsigwiz.html" target="_blank">http://www.cisco.com/en/US/docs/security/ips/5.0/configuration/guide/idm/dmsigwiz.html</A></P></BODY></HTML> Tue, 12 Feb 2008 23:03:16 GMT https://community.cisco.com/t5/network-security/web-crawler-scraper-signatures/m-p/919400#M84697 abinjola 2008-02-12T23:03:16Z Re: Web crawler/scraper signatures https://community.cisco.com/t5/network-security/web-crawler-scraper-signatures/m-p/919401#M84698 <HTML><HEAD></HEAD><BODY><P>thanks, good link. I know about the custom signature configuration and will advise client to provide information from their web logs to asssit in an attempt to create a custom signature. It does not appear there are any pre-configured sigs. for "web crawler type traffic". I looked at what is available in version 6.0(3)E1 S315.</P></BODY></HTML> Tue, 12 Feb 2008 23:32:01 GMT https://community.cisco.com/t5/network-security/web-crawler-scraper-signatures/m-p/919401#M84698 5creedus 2008-02-12T23:32:01Z Re: Web crawler/scraper signatures https://community.cisco.com/t5/network-security/web-crawler-scraper-signatures/m-p/919402#M84699 <HTML><HEAD></HEAD><BODY><P>I have heard of people embedding hidden URL's in their HTML and then firing when something requests the resource. You could also create something based on requests per second.</P></BODY></HTML> Thu, 14 Feb 2008 15:23:08 GMT https://community.cisco.com/t5/network-security/web-crawler-scraper-signatures/m-p/919402#M84699 mhellman 2008-02-14T15:23:08Z Re: Web crawler/scraper signatures https://community.cisco.com/t5/network-security/web-crawler-scraper-signatures/m-p/919403#M84706 <HTML><HEAD></HEAD><BODY><P>hmm, sounds good. Would creating a sig based upon # of requests per second be in the custom signature documentation?</P></BODY></HTML> Thu, 14 Feb 2008 22:06:24 GMT https://community.cisco.com/t5/network-security/web-crawler-scraper-signatures/m-p/919403#M84706 5creedus 2008-02-14T22:06:24Z