ASA URL filtering without a Websense or N2H2/Smartfilter server

[Panos Kampanakis]

• Introduction
• Block specific urls
• Allow every url for specific hosts, block specific urls for the rest
• Allow only cisco.com
• Allow specific urls
• Allow every url for specific hosts, allow only specific urls for the rest
• Same domain allow/deny

Introduction

One of the ASA features is url filtering. It can be used to block or allow users from going to certain urls/websites. This article aims to educate the user on how to use this feature. After reading it carefully someone should be able to take full advantage of url filtering and use it for his needs.

In this article we will either block or allow domains in urls and words in the uri. Of course the ASA can match on other things too. They can be found in the ASA configuration guides. From now and onwards we will allow or block the cisco.com domain. In other words, any user browsing to any page that is behind cisco.com will be subject to url filtering. Such pages would be www.cisco.com/index.html or cisco.com/exampledir/page.html. Also, we will allow or block "/test/" in the uri. In other words, any page path that contains "/test/" will be url filtered. Examples are www.examplesite.com/exampledir/test/page.html or www.anyurl.com/test/examplepage.jsp or www.anything.com/onedir/seconddir/test

The mechanism used to apply url filtering is Modular Policy Framework (MPF). We will create regular expressions (regex) that will be matched in class-maps of type http. These class-maps will be used in policy-maps to define the drop action. Then the policy-maps will be applied with an http inspection in another policy-map that will be applied to an interface. In that way the http inspection action will be applied to the traffic that hits an interface.

NOTE: Though, we need to highlight that for Enterprise URL Filtering, customers should be steered toward using WebSense or N2H2 integration with the ASA. Such web filtering engines can provide much more robust filtering based on classes of sites. URL filtering directly on the ASA using regex, should be used only sparsely when broad classifications can be applied, with limited regex patterns. The ASA will not scale being used in an enterprise with large regex matches and large volumes of HTTP traffic.

Block specific urls

Lets assume that we want to block some specific websites. For example we want to block specific anything under cisco.com and uris that contain "/test/". We will create the regexes and match them in a class-map. Note that if ANY regex is matched then the class-map will actually be met. Then in the policy-map block-url-policy whatever meets the class-map (cisco.com OR uri containing "/test/") is reset. The rest are allowed (not cisco.com and not uri containing "/test/"). The policy-map block-url-policy is used for http inspection in another policy-map (global_policy) and applied with a service-policy.

 
regex blockex1 "/test/"
regex blockex2 "cisco\.com"

class-map type inspect http match-any block-url-class
 match request uri regex blockex1
 match request header host regex blockex2

policy-map type inspect http block-url-policy
 parameters
 class block-url-class
  drop-connection log
policy-map global_policy
 class inspection_default
  inspect http block-url-policy

service-policy global_policy global

Allow every url for specific hosts, block specific urls for the rest

Finally lets say that we have a few hosts (administrators, i.e. ip address 192.168.1.2) that need access to any website and the rest of the users need to be blocked from specific websites (Note that you need to understand the example above to be able to follow the process of this example). We will use approximately the same configuration as in the example above but this time we will need an extra access-list, a class-map and a policy-map. This access-list (user-acl) will match all the users with the exception of the ones that need unrestricted access. It will be matched in a new class-map (block-user-class) which in turn will be used in a separate policy-map (block-user-url-policy). That policy-map will do the http inspection for the allowed websites policy-map block-url-policy (that part is the same as above). Thus the block-user-url-policy will be applied to an interface with a service-policy. What this policy-map is actually doing is to match on all the users except the unrestricted ones (class block-user-class) and block them from going to the specified websites (inspect http block-url-policy. The rest of the users (not matching/denied in the access-list) will be able to go anywhere.

 
regex blockex1 "/test/"
regex blockex2 "cisco\.com"

access-list user-acl extended deny tcp host 192.168.1.2 any eq www
access-list user-acl extended permit tcp any any eq www


class-map type inspect http match-any block-url-class
 match request uri regex blockex1
 match request header host regex blockex2
class-map block-user-class
 match access-list user-acl

policy-map type inspect http block-url-policy
 parameters
 class block-url-class
  drop-connection
policy-map block-user-url-policy
 class block-user-class
  inspect http block-url-policy

service-policy block-user-url-policy interface inside

Allow only cisco.com

Lets assume that we want to allow only cisco.com. We create the regex and match them (match statement) in a class-map. Note that if the match statements is matched the class-map is met. The match statement is met the regex is NOT matched by the url. Then in the policy-map allow-url-policy the connection is reset. The rest is allowed (cisco.com). In other words, if the page you browse is not cisco.com it will be reset. The policy map is used for http inspection in another policy-map (global_policy) and applied with a service-policy.

regex allowex2 "cisco\.com"

class-map type inspect http match-all allow-url-class
 match not request header host regex allowex2

policy-map type inspect http allow-url-policy
 parameters
 class allow-url-class
  drop-connection log
policy-map global_policy
 class inspection_default
  inspect http allow-url-policy

service-policy global_policy global

Allow specific urls

Lets assume that we want to allow only specific websites. For example we want to allow only cisco.com pages and only uris that contain "/test/". We create the regexes and match them in a class-map. Note that if both match statements are matched the class-map is met. Both the match statements are matched if and only if NEITHER regex is matched by the url browsed by the user. Then in the policy-map allow-url-policy the connection is reset. The rest is allowed (cisco.com or uri containing "/test/"). In other words, if the page you browse is neither cisco.com nor contains "/test/" in the url, it will be reset. The policy map is used for http inspection in another policy-map (global_policy) and applied with a service-policy.

regex allowex1 "/test/"
regex allowex2 "cisco\.com"

class-map type inspect http match-all allow-url-class
 match not request uri regex allowex1
 match not request header host regex allowex2

policy-map type inspect http allow-url-policy
 parameters
 class allow-url-class
  drop-connection log
policy-map global_policy
 class inspection_default
  inspect http allow-url-policy

service-policy global_policy global

Allow every url for specific hosts, allow only specific urls for the rest

Now lets say that we have a few hosts (administrators, i.e. ip address 192.168.1.2) that need access to any website and the rest of the users need to be able to go only to specific websites (Note that you need to understand the example above to be able to follow the process of this example). We will use approximately the same configuration as in the example above but this time we will need an extra access-list, a class-map and a policy-map. This access-list (user-acl) will match all the users with the exception of the ones that need unrestricted access. It will be matched in a new class-map (allow-user-class) which in turn will be used in a separate policy-map (allow-user-url-policy). That policy-map will do the http inspection for the allowed websites policy-map allow-url-policy (that part is the same as above). Thus the allow-user-url-policy will be applied to an interface with a service-policy. What this policy-map is actually doing is to match on all the users except the unrestricted ones (class allow-user-class) and allow them only to go to the specified websites (inspect http allow-url-policy. The rest of the users (not matching/denied in the access-list) will be able to go anywhere.

 
regex allowex1 "/test/"
regex allowex2 "cisco\.com"

access-list user-acl extended deny tcp host 192.168.1.2 any eq www
access-list user-acl extended permit tcp any any eq www

class-map type inspect http match-all allow-url-class
 match not request uri regex allowex1
 match not request header host regex allowex2
class-map allow-user-class
 match access-list user-acl

policy-map type inspect http allow-url-policy
 parameters
 class allow-url-class
  drop-connection
policy-map allow-user-url-policy
 class allow-user-class
  inspect http allow-url-policy

service-policy allow-user-url-policy interface inside

Same domain allow/deny

The requirements are as follows:

1. deny facebook.com

2. allow developer.facebook.com which gets redirected automatically to developers.facebook.com

3. and all other domains like yahoo.com and google.com

regex block-fb ".*facebook\.com"
regex allow-fb "developer[s]*\.facebook\.com"

!
class-map type regex match-any block-fb
   match regex block-fb
class-map type regex match-any allow-regex-class
   match regex allow-fb 
!
policy-map type inspect http http-inspect-pol
   parameters
match not request header host regex class allow-regex-class
match request header host regex class block-fb
    reset log

class-map http-class
   match port tcp eq www

policy-map http-traffic
   class http-class
    inspect http http-inspect-pol
!
service-policy http-traffic interface inside

Alternatively, one could also tie the http policy-map to the default inspection under the global_policy.

Bear in mind, every time you add remove or change regex, you need to 
remove the service-policy applied to the interface and add it again.

http://www.cisco.com/en/US/docs/security/asa/asa82/configuration/guide/inspect_basic.html#wp1514315

If you need to change a match command for HTTP inspection after configuring the inspection, you must 
remove the attached service policy by using the no service policy command and then reconfigure the 
service policy. Changing the class map by removing a match command causes HTTP inspection to block 
all HTTP traffic until you remove and reconfigure the attached service policy so that all the match 
commands are reprocessed.

[r.walthall]

Hi, Thanks for the post!  We have a Cisco ASA 5516 running ver 9.8(2). My manager only wants to permit http traffic to a couple of URLs. Will this approach still work?   Allow specific URLs

[tubani]

Hi, am not so good with the Cisco ASA. I am trying to block the specific sites listed below and allow everything else. however nothing is blocked. But  when I change the acl to match https, it blocks everything including legitimate traffic. below is what i have configured. please help

regex urllist1 ".*\.([Ee][Xx][Ee]|[Cc][Oo][Mm]|[Bb][Aa][Tt])"
regex urllist2 ".*\.([Pp][Ii][Ff]|[Vv][Bb][Ss]|[Ww][Ss][Hh])"
regex urllist3 ".*\.([Dd][Oo][Cc]|[Xx][Ll][Ss]|[Pp][Pp][Tt])"
regex urllist4 ".*\.([Zz][Ii][Pp]|[Tt][Aa][Rr]|[Tt][Gg][Zz])"
regex domainlist1 "\.piratebay\.com"
regex domainlist2 "\.kickasstorrent\.com"
regex domainlist3 "\.1337x\.com"
regex contenttype "Content-Type"
regex applicationheader "application/.*"

class-map type inspect http match-all BlockDomainsClass
match request header host regex class DomainBlockList
exit
class-map type regex match-any URLBlockList
no match regex urllist1
no match regex urllist2
no match regex urllist3
no match regex urllist4
exit

class-map type inspect http match-all AppHeaderClass
match response header regex contenttype regex applicationheader
exit
class-map type inspect http match-all BlockDomainsClass
match request header host regex class DomainBlockList
exit
class-map type inspect http match-all BlockURLsClass
match request uri regex class URLBlockList
exit

policy-map type inspect http http_inspection_policy
parameters
match request method connect
drop-connection log
class AppHeaderClass
drop-connection log
class BlockDomainsClass
reset log
class BlockURLsClass
reset log
exit
exit

access-list Guest_mpc extended permit tcp any any eq www
access-list Guest_mpc extended permit tcp any any eq 8080
class-map httptraffic
match access-list Guest_mpc
exit
policy-map Guest-policy
class httptraffic
inspect http http_inspection_policy
exit
exit
service-policy Guest-policy interface Guest