We worked a case on this with Sourcefire support before they were fully assimilated by Cisco.
My conclusion.......The URL Filtering is pretty much garbage.
Here is the response from support:
"The feature is working as designed, with no plans to change this. However, it's possible they could make this an enhancement. In order to consider that, they would need to understand the specific use case and clarify why you want the change."
"It appears the reason the first attempt passes is because the AC rule category will not match when a cloud lookup has to be done on the URL and the result has not yet been received. It instead matches the later "Allow" rule, so further requests to the same host will be permitted as long as the browser keeps the connection open."
"Note that each Snort instance (appliance) keeps its own individual cache of URL cloud lookups, so a different connection from a different IP address could still go through, if it is load-balanced to a different Snort instance."
"The maximum size of each Snort instance's cache for URL cloud lookups is fixed. When the limit is reached, the least recently used entries are discarded."
"So essentially, the first attempt may pass because a cloud lookup has to be done on the URL and the result has not yet been received."
