Last night I implemented the strategy that Mark Pilgrim recommends to stop badly behaved robots.

Some of his recommendations, particularly the IP-address blocking, if widely followed could cause a "previous owner" problem if they were to get reassigned from a spammer to a person, like, say, me. All of a sudden I'm getting banned from 5% of internet sites because one year ago my IP address was owned by a really bad spammer.

It beats maxing out my download quota because of robotic downloaders, though. My site's been up for less than a week and already has been hit by a good number of these guys, as well as 36 worms and vulnerability-hunters, plus 33 requests from robots.txt-friendly bots.

:)

Posted by: led on January 12, 2005 02:03 AM

:)

Posted by: led on January 12, 2005 02:04 AM

Hi_please_visit_http://www.your-furniture.biz

Posted by: furniture on January 21, 2006 01:51 PM

Hi_please_visit_http://www.loan-for-you.net

Posted by: secured 0an on January 23, 2006 10:11 PM