MSNBot Ignores Robots.txt

I think I might have accidentally started the MSNBot Boycott when I made some comments on on JWZ’s blog. I posted the exclusion rules to put in your robots.txt, to keep Microsoft from crawling your site. I set it up on my site immediately, but I discovered that the MSNBot ignores the robots.txt rules, they scanned my site anyway. MS claims it’s a bug. Yeah right. As usual, Microsoft thinks the rules don’t apply to them.

2 thoughts on “MSNBot Ignores Robots.txt”

Leave a Reply to Charles Cancel reply

Your email address will not be published. Required fields are marked *

© Copyright 2016 Charles Eicher