06-10-2009 03:18 AM - edited 03-08-2019 05:58 PM
One of the ASA features is url filtering. It can be used to block or allow users from going to certain urls/websites. This article aims to educate the user on how to use this feature. After reading it carefully someone should be able to take full advantage of url filtering and use it for his needs.
In this article we will either block or allow domains in urls and words in the uri. Of course the ASA can match on other things too. They can be found in the ASA configuration guides. From now and onwards we will allow or block the cisco.com domain. In other words, any user browsing to any page that is behind cisco.com will be subject to url filtering. Such pages would be www.cisco.com/index.html or cisco.com/exampledir/page.html. Also, we will allow or block "/test/" in the uri. In other words, any page path that contains "/test/" will be url filtered. Examples are www.examplesite.com/exampledir/test/page.html or www.anyurl.com/test/examplepage.jsp or www.anything.com/onedir/seconddir/test
The mechanism used to apply url filtering is Modular Policy Framework (MPF). We will create regular expressions (regex) that will be matched in class-maps of type http. These class-maps will be used in policy-maps to define the drop action. Then the policy-maps will be applied with an http inspection in another policy-map that will be applied to an interface. In that way the http inspection action will be applied to the traffic that hits an interface.
NOTE: Though, we need to highlight that for Enterprise URL Filtering, customers should be steered toward using WebSense or N2H2 integration with the ASA. Such web filtering engines can provide much more robust filtering based on classes of sites. URL filtering directly on the ASA using regex, should be used only sparsely when broad classifications can be applied, with limited regex patterns. The ASA will not scale being used in an enterprise with large regex matches and large volumes of HTTP traffic.
Lets assume that we want to block some specific websites. For example we want to block specific anything under cisco.com and uris that contain "/test/". We will create the regexes and match them in a class-map. Note that if ANY regex is matched then the class-map will actually be met. Then in the policy-map block-url-policy whatever meets the class-map (cisco.com OR uri containing "/test/") is reset. The rest are allowed (not cisco.com and not uri containing "/test/"). The policy-map block-url-policy is used for http inspection in another policy-map (global_policy) and applied with a service-policy.
regex blockex1 "/test/"
regex blockex2 "cisco\.com"
class-map type inspect http match-any block-url-class
match request uri regex blockex1
match request header host regex blockex2
policy-map type inspect http block-url-policy
parameters
class block-url-class
drop-connection log
policy-map global_policy
class inspection_default
inspect http block-url-policy
service-policy global_policy global
Finally lets say that we have a few hosts (administrators, i.e. ip address 192.168.1.2) that need access to any website and the rest of the users need to be blocked from specific websites (Note that you need to understand the example above to be able to follow the process of this example). We will use approximately the same configuration as in the example above but this time we will need an extra access-list, a class-map and a policy-map. This access-list (user-acl) will match all the users with the exception of the ones that need unrestricted access. It will be matched in a new class-map (block-user-class) which in turn will be used in a separate policy-map (block-user-url-policy). That policy-map will do the http inspection for the allowed websites policy-map block-url-policy (that part is the same as above). Thus the block-user-url-policy will be applied to an interface with a service-policy. What this policy-map is actually doing is to match on all the users except the unrestricted ones (class block-user-class) and block them from going to the specified websites (inspect http block-url-policy. The rest of the users (not matching/denied in the access-list) will be able to go anywhere.
regex blockex1 "/test/"
regex blockex2 "cisco\.com"
access-list user-acl extended deny tcp host 192.168.1.2 any eq www
access-list user-acl extended permit tcp any any eq www
class-map type inspect http match-any block-url-class
match request uri regex blockex1
match request header host regex blockex2
class-map block-user-class
match access-list user-acl
policy-map type inspect http block-url-policy
parameters
class block-url-class
drop-connection
policy-map block-user-url-policy
class block-user-class
inspect http block-url-policy
service-policy block-user-url-policy interface inside
Lets assume that we want to allow only cisco.com. We create the regex and match them (match statement) in a class-map. Note that if the match statements is matched the class-map is met. The match statement is met the regex is NOT matched by the url. Then in the policy-map allow-url-policy the connection is reset. The rest is allowed (cisco.com). In other words, if the page you browse is not cisco.com it will be reset. The policy map is used for http inspection in another policy-map (global_policy) and applied with a service-policy.
regex allowex2 "cisco\.com"
class-map type inspect http match-all allow-url-class
match not request header host regex allowex2
policy-map type inspect http allow-url-policy
parameters
class allow-url-class
drop-connection log
policy-map global_policy
class inspection_default
inspect http allow-url-policy
service-policy global_policy global
Lets assume that we want to allow only specific websites. For example we want to allow only cisco.com pages and only uris that contain "/test/". We create the regexes and match them in a class-map. Note that if both match statements are matched the class-map is met. Both the match statements are matched if and only if NEITHER regex is matched by the url browsed by the user. Then in the policy-map allow-url-policy the connection is reset. The rest is allowed (cisco.com or uri containing "/test/"). In other words, if the page you browse is neither cisco.com nor contains "/test/" in the url, it will be reset. The policy map is used for http inspection in another policy-map (global_policy) and applied with a service-policy.
regex allowex1 "/test/"
regex allowex2 "cisco\.com"
class-map type inspect http match-all allow-url-class
match not request uri regex allowex1
match not request header host regex allowex2
policy-map type inspect http allow-url-policy
parameters
class allow-url-class
drop-connection log
policy-map global_policy
class inspection_default
inspect http allow-url-policy
service-policy global_policy global
Now lets say that we have a few hosts (administrators, i.e. ip address 192.168.1.2) that need access to any website and the rest of the users need to be able to go only to specific websites (Note that you need to understand the example above to be able to follow the process of this example). We will use approximately the same configuration as in the example above but this time we will need an extra access-list, a class-map and a policy-map. This access-list (user-acl) will match all the users with the exception of the ones that need unrestricted access. It will be matched in a new class-map (allow-user-class) which in turn will be used in a separate policy-map (allow-user-url-policy). That policy-map will do the http inspection for the allowed websites policy-map allow-url-policy (that part is the same as above). Thus the allow-user-url-policy will be applied to an interface with a service-policy. What this policy-map is actually doing is to match on all the users except the unrestricted ones (class allow-user-class) and allow them only to go to the specified websites (inspect http allow-url-policy. The rest of the users (not matching/denied in the access-list) will be able to go anywhere.
regex allowex1 "/test/"
regex allowex2 "cisco\.com"
access-list user-acl extended deny tcp host 192.168.1.2 any eq www
access-list user-acl extended permit tcp any any eq www
class-map type inspect http match-all allow-url-class
match not request uri regex allowex1
match not request header host regex allowex2
class-map allow-user-class
match access-list user-acl
policy-map type inspect http allow-url-policy
parameters
class allow-url-class
drop-connection
policy-map allow-user-url-policy
class allow-user-class
inspect http allow-url-policy
service-policy allow-user-url-policy interface inside
The requirements are as follows:
1. deny facebook.com
2. allow developer.facebook.com which gets redirected automatically to developers.facebook.com
3. and all other domains like yahoo.com and google.com
regex block-fb ".*facebook\.com"
regex allow-fb "developer[s]*\.facebook\.com"
!
class-map type regex match-any block-fb
match regex block-fb
class-map type regex match-any allow-regex-class
match regex allow-fb
!
policy-map type inspect http http-inspect-pol
parameters
match not request header host regex class allow-regex-class
match request header host regex class block-fb
reset log
class-map http-class
match port tcp eq www
policy-map http-traffic
class http-class
inspect http http-inspect-pol
!
service-policy http-traffic interface inside
Alternatively, one could also tie the http policy-map to the default inspection under the global_policy.
Bear in mind, every time you add remove or change regex, you need to
remove the service-policy applied to the interface and add it again.
http://www.cisco.com/en/US/docs/security/asa/asa82/configuration/guide/inspect_basic.html#wp1514315
If you need to change a match command for HTTP inspection after configuring the inspection, you must
remove the attached service policy by using the no service policy command and then reconfigure the
service policy. Changing the class map by removing a match command causes HTTP inspection to block
all HTTP traffic until you remove and reconfigure the attached service policy so that all the match
commands are reprocessed.
Hello Panagiotis,
Thanks for posting this article, it is very interesting. I wonder if you had a chance to try this feature with HTTPS (SSL/TLS encrypted) traffic. All examples I found are for clear text HTTP traffic.
Another question – does this feature work if the client is configured to use a web proxy? This is probably standard config in many companies, the users have to pass through a Microsoft IIS proxy server in order to access the Internet.
I will test the configs in the lab. You obviously spent time testing this feature, you may have the answers.
Thank you,
Cristian
Hello Cristian,
HTTPS filtering is not supported on ASA. ASA cannot do deep packet inspection or inspection based on regular expression for HTTPS traffic, because in HTTPS, content of packet is encrypted (SSL).
You can also use URL filtering to direct specific traffic to an external filtering server, such an Secure Computing SmartFilter (formerly N2H2) or Websense filtering server. Long URL, HTTPS, and FTP filtering can now be enabled using both Websense and Secure Computing SmartFilter for URL filtering. Filtering servers can block traffic to specific sites or types of sites, as specified by the security policy.
For more information on URL filters, refer the following URLs:
http://www.cisco.com/en/US/docs/security/asa/asa81/config/guide/filter.html#wp1042606
http://www.cisco.com/en/US/products/ps6120/products_configuration_example09186a0080940e04.shtml
Thanks
Sathesh
It is a god scenario, but, there is a bug for this solution which is attachments for any hotmail account only can be accessed either to be opened nor to be saved to the desktop.
If you please, if you have any idea how to solve this issue
hmmm, i'm a bit confused on the url filtering. Would the communication from an internal user be on the resolved IP address of the url instread of the url ? ( assuming i have an internal dns server) So when user1 is going to www.cnn.com, the firewall sees the url ? or the IP address of cnn.com (which was internalyl resolved) ?
ahmedchohan,
I was wondering the same thing. I did a simple packet captue on my machine.. Note the "Host: www.microsoft.com". I am by no means an expert, but it would make sense if this is what we're matching with "request header host"
GET / HTTP/1.1
Accept: image/gif, image/jpeg, image/pjpeg, image/pjpeg, application/x-shockwave-flash, application/xaml+xml, application/vnd.ms-xpsdocument, application/x-ms-xbap, application/x-ms-application, application/x-silverlight, application/vnd.ms-excel, application/vnd.ms-powerpoint, application/msword, application/csm-desktop, */*
Accept-Language: en-us
User-Agent: Mozilla/4.0 (compatible; MSIE 8.0; Windows NT 5.1; Trident/4.0; .NET CLR 2.0.50727; .NET CLR 1.1.4322; .NET CLR 3.0.04506.30; .NET CLR 3.0.04506.648; InfoPath.2; .NET CLR 3.0.4506.2152; .NET CLR 3.5.30729)
Accept-Encoding: gzip, deflate
Host: www.microsoft.com: Keep-Alive
Connection
Hey Ahmed,
I couldn't figure out how to stop it from letting you save the attachment. However, I did figure out how to block access to all attachments from hotmail (even the little picture preview thing). Use at your own risk:
regex GET_ATTACH ".*([Gg][Ee][Tt][Aa][Tt][Tt][Aa][Cc][Hh][Mm][Ee][Nn][Tt])\.([Aa][Ss][Pp][Xx]).*"
regex SCAN_ATTACH ".*([Ss][Cc][Aa][Nn][Aa][Tt][Tt][Aa][Cc][Hh][Mm][Ee][Nn][Tt])\.([Aa][Ss][Pp][Xx]).*"
regex HOTMAIL_URL ".*([Mm][Aa][Ii][Ll])\.([Ll][Ii][Vv][Ee])\.([Cc][Oo][Mm])"
class-map type inspect http match-all HOTMAIL_SMACKDOWN
match request header host regex HOTMAIL_URL
match request uri regex SCAN_ATTACH
class-map type inspect http match-all HOTMAIL_SMACKDOWN_THE_SEQUEL
match request header host regex HOTMAIL_URL
match request uri regex GET_ATTACH
policy-map type inspect http HOTMAIL_SMACKDOWN
parameters
class HOTMAIL_SMACKDOWN_THE_SEQUEL
drop-connection log
class HOTMAIL_SMACKDOWN
drop-connection log
policy-map global_policy
class inspection_default
inspect http HOTMAIL_SMACKDOWN
First, I must warn everyone, I'm a Storage CCIE with little background in R&S and Security. If it doesn't deal with FCIP or iSCSI with static routes, I'm at a loss but I'm reading and trying to understand all of this.
Q1. As for the document at the start of this post, I'm having a hard time understanding the statement:
access-list user-acl extended deny tcp host 192.168.1.2 any eq www. Doesn't this statement prevent the administrator at 192.168.1.2 from going anywhere via http?
Q2. I'm doing volunteer work at my children's local school (pre-K to 8th grade) and trying to clean up the mess the previous administrator installed. I've acquired 4 - Cisco 3548XL (I know they are EoL/EoS) and I want to secure them behind a secure router/firewall with URL filtering. I've been looking at a Cisco PIX 501 (its free, that's why I'm looking at it), Cisco SR520, Cisco ASA5505, and a Cisco 851. The staff has asked me to deny ALL student PC's access to anything on the Internet except ~10 URL's but allow the teachers and staff access to anywhere they want to go (with certain exceptions of course). The cost of a Websense or N2H2 server and licensing is beyond what they can afford and whatever secure router/firewall is deemed a correct fit, I'll end up purchasing it myself and donating it to the school. I keep bouncing between the SR520 and the ASA5505. Does anyone have any advice for me as I'm looking at trying to move on this fairly quickly? I've tapped a couple of local Cisco resources with no response yet.
Thanks.
Gary
Hello Gary,
The URL filtering may be overkill for your requirements. For 10 URLs, it may be simple to limit the access based on destination IP addresses. For example, if you want to allow access to www.wikipedia.org use a rule like “access-list acl-inside permit tcp student_ip_address host 208.80.152.2 eq 80”. This solution will not allow you to limit the access to certain URLs per destination (allow access to wikipedia.org/school, but deny access to wikipedia.org/politics). Enterprise solutions like N2H2 allow restrictions for a large number of sites, with URLs that change (you pay for updates with lists of restricted sites).
How do you separate the teachers from students? The most flexible solution would be to create two VLANs, let’s say network 10.1.1.0/24 for teachers and network 10.1.2.0/24 for students. In your PIX, define three interfaces: outside facing the Internet, inside-teacher for network 10.1.1.1/24, and inside-student network 10.1.2.1/24.
The config would be something like this:
access-list acl-inside-student remark à Allow access to DNS
access-list acl-inside-student permit udp 10.1.2.0 255.255.255.0 host dns_server eq 53
access-list acl-inside-student remark à Allow access to wikipedia
access-list acl-inside-student permit tcp 10.1.2.0 255.255.255.0 host 208.80.152.2 eq 80
access-list acl-inside-student remark à Allow access to cnn.com
access-list acl-inside-student permit tcp 10.1.2.0 255.255.255.0 host 157.166.226.25 eq 80
access-list acl-inside-student permit tcp 10.1.2.0 255.255.255.0 host 157.166.226.26 eq 80
access-list acl-inside-student permit tcp 10.1.2.0 255.255.255.0 host 157.166.255.18 eq 80
…
[implied deny at the end]
access-list acl-inside-teacher remark à Allow access to DNS
access-list acl-inside- teacher permit udp 10.1.1.0 255.255.255.0 host dns_server eq 53
access-list acl-inside- teacher remark à no restrictions for web browsing
access-list acl-inside- teacher permit tcp 10.1.1.0 255.255.255.0 any eq 80
…
[implied deny at the end]
The PIX can be configured as DHCP servers for both internal networks.
The PIX may not support three interfaces at the same time, depending on what license you have. If this is the case, a solution would be to use only one network for teachers and students and assign static IP addresses to the teacher’s workstations.
access-list acl-inside remark à DNS
access-list acl-inside permit udp 10.1.1.0 255.255.255.0 host dns_server eq 53
access-list acl-inside remark à teacher 1
access-list acl-inside permit tcp host 10.1.1.254 any eq 80
access-list acl-inside remark à teacher 2
access-list acl-inside permit tcp host 10.1.1.253 any eq 80
access-list acl-inside remark à Students access to wikipedia
access-list acl-inside permit tcp 10.1.2.0 255.255.255.0 host 208.80.152.2 eq 80
access-list acl-inside remark à Students allow access to cnn.com
access-list acl-inside permit tcp 10.1.1.0 255.255.255.0 host 157.166.226.25 eq 80
access-list acl-inside permit tcp 10.1.1.0 255.255.255.0 host 157.166.226.26 eq 80
access-list acl-inside permit tcp 10.1.1.0 255.255.255.0 host 157.166.255.18 eq 80
…
[implied deny at the end]
A tech savy student may go around this restrictions by configuring a static IP address on his workstation (hopefully the pre-K kids won't do it).
Regards,
Cristian
Cristian,
Thanks for the reply. It was very helpful!! Yes, I'm planning and creating 2 VLAN's. One for students and one for teachers and staff making my job easier. I'm assuming if I put the students on their own VLAN that even if they set their IP address statically, they would still be restricted. The only way I can see them getting around the restriction is to swap network cables with the teachers PC or use the teachers PC.
Thanks again!
Gary
Sadly, this can't be used to block visiting websites by IP address (and thus ensureing every site is resolved by DNS for security reasons), tried...
regex blockex3 "\d*\.\d*\.\d*\.\d*"
regex blockex4 "(0|1|2|3|4|5|6|7|8|9)*\.(0|1|2|3|4|5|6|7|8|9)*\.(0|1|2|3|4|5|6|7|8|9)*\.(0|1|2|3|4|5|6|7|8|9)*\."
regex blockex6 "^http://\d*\.\d*\.\d*\.\d*(/|$)"
regex blockex7 "http://\d*\.\d*\.\d*\.\d*"
regex blockex8 "\d+\.\d+\.\d+\.\d+"
...either it had no effect, or it ended up blocking everything :-(
Hi,
I'm very newbie - and vey dumm - in cisco stuff...
Can anyone explain me or give me any link to the appropriate
documentation about how to implement the point "allow every url for specific host, block specific urls for the rest" using ASDM 6.x tool. I've found a useful document:
about url filtering
but no hints about how to grant evey url for some ip and I believe that it's a very common scenario.
Thank you very much for your help
filippo
Hi,
If I want to block this url: www.123.nl/index.html www\.123\.nl\/index\.html does not work!
How do I block this then, what will be the expression?
thx,
Marc
Hi Gary,
Probably this topic is history but if you have similar issues in the future, you can have a look at SquidGuard - I know it is Linux and it is both not ok to speak about it on a CISCO forum and it requires some strong knowledge but if you combine this with your CISCO PIX cappability to allow egress www connections only from the squid machine - you may achieve a strong web filtering.
I know this is may be outdated discussion, but I could not understand how
access-list user-acl extended deny tcp host 192.168.1.2 any eq www
Does allow user access to internet when it is a deny not permit
The ACL 'user-acl' is used in a class-map. By using a 'deny' we prevent that traffic from matching that class-map and as a result prevent that traffic (sourced from 192.168.1.2) from being subjected to the http inspection policy defined.
Find answers to your questions by entering keywords or phrases in the Search bar above. New here? Use these resources to familiarize yourself with the community: