Magic Main
Random Lines
Indy Lib Block

Get my PGP Key

Main Flarp!Bar ShirtWear WebServer Magic Software Misc.

How to block any client with the Indy Library user-agent from your server using PHP:

As the China-based spambot which seems to be the primary source of Indy Library abuse appears to always start trawling at the "root" of a site (in my case the front page at, it should be 100% effective to just put the contents of this file at the top of your highest-level index.php page (above the initial <html> tag).

If you want to be more thorough and paranoid, put the script stuff in a .txt file somewhere and then use the Include instruction (<? include("/home/YOURNAME/indylib.txt"); ?> ) at the top of every page to be protected. You'll need to modify that file path to suit how your server is set up.

It's not really necessary to protect every page, as if the spambot is thwarted at the first page it tries to read (usually the site root in every instance I've spotted in my logs) it won't see any links to follow and will give up.

If you'd rather lock these clients out directly with .htaccess (so they get a 403 Forbidden error and really know they're not welcome), incorporate the contents of this file into your .htaccess file (or if you don't have one, make a text file called .htaccess in your web root directory and just add this material).

An alternative approach from the fine folks at is to detect and block any client which follows a normally invisible link to a page which is forbidden in a robots.txt file. Thanks to the webmaster of the International Campaign to Ban Landmines for that link, and also thanks to everyone else who has chipped in ideas or expressed support.