Use Wildcard to Limit TLD (and block Google search images)

[jmerichards]

Hi Folks,

I'm new, so please take it easy.

I know this has been canvassed to death over the years on OpenDNS forums, but I have found no recent discussion on the subject.

I want to be able to block all Google search sites, without having to list and maintain every TLD they prefix "google" to.  For example, I'd like to be able to enter "google.*" into my always block list, then allow certain Google domains (e.g. translate.google.com).  I've already got explicit blocking on google.com, google.ca, etc., but the list is too extensive to reasonably maintain.  I'm allowing translate.google.com through and it all seems to work and I have not noticed any performance issues.

For those people that are wondering why, it's because Google do not support blocking porn from their search results, but Bing do (via explicit.bing.com). I am blocking access to all Google search sites in favour of Bing for this reason.

If you'd like to suggest installing other software on computers, that is not feasible because of the number of devices accessing the Internet through my network (tablets, computers, TV's, etc.).

Any help or suggestions welcome, thanks in advance.

EDIT: Please see the document attached to this original post (below) for a summary solution from this thread.

Use Wildcard to Limit TLD (and block Google search images).pdf

[rotblitz]

The preferred method for selective Google content filtering is Google's SafeSearch feature.

You catch most Google domains with blocking the Search Engines category.  Then you had to whitelist the few you want to allow.

[jmerichards]

Hi Rotblitz, Appreciating the feedback, thanks. Client based solutions aren't much use I'm afraid, as there are browser built into other devices that also need blocking. I've used your suggestion to block the search engine category, and it seems to do the trick. "Safe" image queries on Bing yield pictures, "unsafe" ones yield placeholders only; that's great. I've added maps.google.com to the "never block" list, but it is still being blocked. I've tested this on numerous devices and computers - I cleared the cache and flushed dns but still no go. Any suggestions on that? TIA

[kpatullo1]

It looks like the following domains are also accessed when querying maps.google.com: google.com, apis.google.com, maps.google.com, plus.google.com

Since you want google.com to remain blocked, you may want to try unblocking apis.google.com to see if that allows you to access maps.google.com.  

[rotblitz]

"Client based solutions aren't much use I'm afraid"

Hey, Google SafeSearch is not only a client based solution, but also a server based one:

https://support.google.com/websearch/answer/144686?hl=en
If you deploy a proxy on your web traffic, it may be possible to configure your proxy to append &safe=strict to all search requests sent to Google. This parameter enables strict SafeSearch for all searches, regardless of the setting on the Google Preferences page.

https://support.google.com/websearch/answer/186669?hl=en
To enable SafeSearch throughout a school network, you can use a proxy server to append &safe=active directly to all search URLs. This will enable strict SafeSearch.

"I've added maps.google.com to the "never block" list, but it is still being blocked."

Not everything is hosted on maps.google.com, but also other (Google) domains are involved.  Check with a tool like http://www.nirsoft.net/utils/dns_query_sniffer.html or with your OpenDNS domain stats to see what domains still need to be whitelisted.

[jmerichards]

@Kristy - many thanks. I did manage to get maps to load (actually I did nothing, it just started to work), but the map background did not load. I'd been trying to find out what other domains served map data, so I'll try your suggestion and post back. @rotblitz - again, thanks for the detailed help. I don't use a proxy as this is a home network and a proxy always seemed like added hassle and maybe expense. It also seemed to be taking a sledge hammer to a peanut. However, I may have to consider this option again if it will help to keep my kids safe. A general question relating to the various domains with maps; I'd noticed that the sub domain l.google.com us used with maps and many other Google sites. What is the general rule with OpenDNS when it comes to blocking - use the actual sub domain name, or is it sufficient to block alias names? Thanks again.

[rotblitz]

Normally both methods will work, domain name and/or aliases (CNAMEs).  There may be exceptions.
Also, example.com covers this and all its subdomains and almost possible aliases.

[jmerichards]

First off, Kristy: tried adding apis.google.com to the whitelist - no joy. Map page loads but actual map background (the important bit) does not. @rotblitz Since I blocked the search engine category I have also not been able to get youtube to load, despite whitelisting that and a variety of sub domains. I'm no longer getting the OpenDNS block logo for YouTube, but it only partially loads the page. Interestingly, whitelisting only youtube.com still resulted in the OpenDNS block message. I am only seeing this behaviour with Google maps and youtube. I was able to whitelist yahoo.com and it started working straight away. Anyone else have similar issues with Google domains? Perhaps I should start a new thread?

[jmerichards]

I think I have the solution to my problem. Just to recap - I wanted to block the google search engine because of porn. Rotblitz suggested (quite rightly) blocking the entire search engine category. I did so, which had the effect of breaking my access to youtube and google maps, which I wanted to keep. The answer is to add, believe it or not, google.com to the whitelist, but still leave the search engine category blocked. Doing this, plus whitelisting Bing means I have all search engines other than Bing blocked, but still have access to other desirable Google services. Thank you both all for your help.

[rotblitz]

Excellent!  You're welcome.

[jmerichards]

Actually, it didn't fix it. :( Ok, the problem I'm having is that I've blocked the search engine category, which also blocks YouTube and Google Maps - I don't want YouTube and Google Maps to be blocked. So, I whitelist youtube.com, s.ytimg.com, ytimg.com, maps.google.com and maps.google.com.au (as I'm in Australia). I've also white listed bing.com and yahoo.com. Here's the rub - bing and yahoo load, as you'd expect as they're white listed. Google Maps and YouTube do not load, despite being white listed. Do we have an idea why this is and what the best way to approach it is? It doesn't seem to be a problem with OpenDNS, it must be Google trickery?

[cervezafria]

Couple of things to do... 1) look through your stats over the past day and see what is getting blocked... not everything is obvious but trial/error whitelisting can help. or...2) run Fiddler in the background and see what comes up when you try to load maps, for example. You'll see immediately which connections are being blocked. Again... whitelist trial & error (as some of the blocks may be legitimate, for example for adware.

For youtube... I also would whitelist gstatic.com (loads with both maps and youtube) You can delete s.ytimg.com, since that is already available through your whitelisted ytimg.com

[jmerichards]

Cheers Cervezafria. I'll try all those things tonight (my time) and report back.

[jmerichards]

I think I need some help interpreting Fiddler (looks awesome though).

Several other domains listed in Fiddler results that I have not whitelisted - I have to go out now so can't elaborate until tomorrow.

Interestingly (to me) there is a sub-domain called "safesearch.google.com".  Could this be a key to helping resolve this issue?

More tomorrow...

[cervezafria]

Did whitelisting gstatic.com help? For youtube, I also had to whitelist "youtube.be" and "youtube-nocookie.com". Good luck!