Someone at Picsearch replied to my complaint about PSBot’s naughty behavior. He apologized and said they’d fixed it so this site wouldn’t be indexed again.
There was this curious remark:
A general note however: computertoaster.com/robots.txt redirects to www.computertoaster.com/robots.txt, which would prevent bots from stopping to request URL:s on computertoaster.com (I do not know if this is what you are seeing from us or not).
Eh? I want PSBot to stop requesting for sure. Or am I being told the redirection would prevent the bot from being stopped?
Related posts:
- Robots.txt : A Webmaster’s Guide ...
- Robots.Txt Explained How to write your site's robots text file. ...
- Cut Search Robots Drain on Your Bandwidth ...
- How To Control Search Engine Robots ...
- Playing in Googlebot’s Sandbox with Slurp, Teoma & MSNbot ...
Related posts brought to you by Yet Another Related Posts Plugin.
0 comments ↓
There are no comments yet...Kick things off by filling out the form below.
Leave a Comment