Blocked Bots And IP’s
Running a server can be a daunting task for some. Limiting access to your files can make this task a little bit easier.
I do currently limit access to this domain by Client, IP and Worldwide Location. Located below is the list of limits that I am currently using. It should be noted that overall, and due to the nature of the micro-niche information on this web site I am not allowing traffic from non-English speaking countries. The countries that I am allowing traffic from is:
United States
United Kingdom
Australia
Canada
[sc:adsense]
It isn’t like I don’t like the rest of the world. It is more akin to using my server resources where they will be more needed. Since this is a micro-niche web site I think the information on this web site will be more enjoyed by those on the list above.
Here is the list of Browser Client Strings I block:
BrowserMatchNoCase Silk bad_bot
BrowserMatchNoCase Jakarta bad_bot
BrowserMatchNoCase python-urllib bad_bot
BrowserMatchNoCase YandexBot bad_bot
BrowserMatchNoCase mj12bot bad_bot
BrowserMatchNoCase Ezooms bad_bot
BrowserMatchNoCase ZmEu bad_bot
BrowserMatchNoCase Whoismindbot bad_bot
BrowserMatchNoCase X11 bad_bot
BrowserMatchNoCase SWEBot bad_bot
BrowserMatchNoCase Baiduspider bad_bot
BrowserMatchNoCase curl bad_bot
BrowserMatchNoCase webcollage bad_bot
BrowserMatchNoCase Sosospider bad_bot
BrowserMatchNoCase SmartLinksAddon bad_bot
BrowserMatchNoCase Transcoder bad_bot
BrowserMatchNoCase SmartLinksAddon bad_bot
BrowserMatchNoCase AhrefsBot bad_bot
BrowserMatchNoCase JS-Kit bad_bot
BrowserMatchNoCase appengine bad_bot
BrowserMatchNoCase Moreoverbot bad_bot
BrowserMatchNoCase rv:1.9.0.13 bad_bot
BrowserMatchNoCase yandexbot bad_bot
BrowserMatchNoCase butterfly bad_bot
BrowserMatchNoCase ia_archiver bad_bot
BrowserMatchNoCase zh-cn bad_bot
BrowserMatchNoCase idbot bad_bot
BrowserMatchNoCase w3c_validator bad_bot
BrowserMatchNoCase python-requests bad_bot
BrowserMatchNoCase wget bad_bot
BrowserMatchNoCase ccbot bad_bot
BrowserMatchNoCase spbot bad_bot
BrowserMatchNoCase PHP bad_bot
BrowserMatchNoCase linkdexbot bad_bot
BrowserMatchNoCase MetaURI bad_bot
BrowserMatchNoCase JS-Kit bad_bot
BrowserMatchNoCase archive.org_bot bad_bot
BrowserMatchNoCase kraken bad_bot
BrowserMatchNoCase rogerbot bad_bot
BrowserMatchNoCase ADmantX bad_bot
BrowserMatchNoCase A6-Indexer bad_bot
BrowserMatchNoCase ContentScan bad_bot
BrowserMatchNoCase WordPressPollBot bad_bot
BrowserMatchNoCase BIXOCRAWLER bad_bot
BrowserMatchNoCase SemrushBot bad_bot
BrowserMatchNoCase Apache-HttpClient bad_bot
BrowserMatchNoCase MaxPointCrawler bad_bot
BrowserMatchNoCase Nutch bad_bot
BrowserMatchNoCase Moreover bad_bot
BrowserMatchNoCase wscheck bad_bot
The listed client strings above are blocked because they really add no value to this web site. They are mostly web site scrapers and content rip-offs. I have yet to get any returned traffic from the use of the bots above. If I am incorrect, let me know by posting a comment below or using the Contact Us link located at the top of the page.
Located below is the list of IP’s I am blocking:
deny from 208.43.225.84
deny from 162.243.45.13
deny from 54.241.7.7
deny from 198.143.144.29
deny from 172.56.5.91
deny from 192.187.110.130
deny from 68.232.187.219
deny from 184.22.183.114
deny from 174.92.37.86
deny from 88.208.217.45
deny from 76.164.216.114
deny from 209.190.49.39
deny from 209.190.57.132
deny from 208.53.152.21
deny from 64.64.30.118
deny from 192.99.9.108
deny from 209.190.81.216
deny from 67.192.255.68
deny from 198.61.225.98
deny from 142.54.180.123
deny from 176.61.142.175
deny from 204.12.239.228
deny from 64.237.45.114
deny from 192.99.11.146
deny from 50.115.166.208
deny from 173.209.207.210
deny from 192.187.118.114
deny from 176.61.141.95
deny from 60.241.8.7
deny from 54.219.26.116
deny from 54.219.70.147
deny from 142.54.184.142
deny from 23.90.5.77
deny from 172.56.17.133
deny from 209.190.49.82
deny from 198.27.69.78
deny from 184.169.245.99
deny from 54.213.70.186
deny from 204.236.185.82
deny from 23.19.155.20
deny from 108.62.64.196
deny from 192.3.22.242
deny from 173.236.54.253
deny from 184.154.9.186
deny from 198.143.144.52
deny from 75.101.234.136
deny from 192.99.3.58
deny from 205.237.88.158
deny from 216.244.83.218
deny from 216.152.243.167
deny from 198.27.75.80
deny from 192.99.12.128
deny from 199.19.213.228
deny from 69.197.160.98
deny from 198.23.60.180
deny from 66.119.41.34
This list of IP’s is mostly from other Internet servers, content scrapers, and general pains in the Asses.
I can’t understand the value of having an Internet Server visiting my site when the information here is designed for the human eye and not for distillation in some database on the ‘net.
Again, if I am wrong, let me know about it.
This list will be updated as I make changes to it.
The last update to this list was:
12-10-2013
Comments: