Inktomi (Yahoo!, AOL, MSN, etc.) banned (part 2)
The Ponemon Institute polled a random sample of 1,017 Internet users with eight questions on Google and warrentless bulk query disclosure to the USG. The results are here (pdf).
In comments to Inktomi (Yahoo!, AOL, MSN, etc.) banned Chris Clarke provided the generic answer for how to block Inktomi for bloggers (and other Apache users) who don't have access to the httpd.conf file (usually in /usr/local/apache/conf and requiring superuser privilages), but who can create an access control file called .htaccess in their home directory. This is the .htaccess version:
order allow,deny
Deny from 66.196.64.0/18
Deny from 68.142.192.0/18
Deny from 72.30.0.0/16
allow from all
The self-hosting (you are the superuser, and all your mistakes are supermistakes) version for httpd.conf is:
<Directory /some_name_here >
# deny INKT CIDRs
Deny from 66.196.64.0/18
Deny from 68.142.192.0/18
Deny from 72.30.0.0/16
< /Directory >
Wampum readers who want to block Inktomi's Slurp spider (and it is a non-benign spider for resource consumption reasons) please drop me a line. If you want to test it, block 65.99.1.129 and send me the URL of some file in your directory, e.g. http://myblog.myhost.com/wget_me.html, and I'll use the wget pseudo-spider to attempt to fetch the file. If it is there and I can't get it, you win (and Inktomi, Yahoo!, AOL, MSN, ... loses). I'll mail you back the bytecount of your file, some number other than zero, or zero if you've got a working block.
The Search Engine Industry is watching this. You can't impeach Bush, but you can impeach Gonzales one blog at a time. You can also spread the "This site blocks collaborators" meme. Anyone with gif/jpg skills should use them, I'll host the bits. Notes outstanding to the accomplished Julia of SisyphusShrugged (link fixed) and Kevin of The American Street and the Perranoski Prize for graphical bits...