# See http://www.robotstxt.org/wc/norobots.html for documentation on how to use the robots.txt file User-Agent: * Disallow: /stylesheets/ Disallow: /javascripts/ Disallow: /files/download Disallow: /files/clipper Disallow: /pages/ Disallow: /message/ Disallow: /simple_captcha/ Disallow: /login Disallow: /register Disallow: /suggestion Disallow: /errors/ Disallow: /topics/*/posts/ Disallow: /s? Disallow: /groups/*/topics