# This file is used to tell search engine robots what NOT # to index on your site. Controling your indexing can be # VERY important with your affecting your ranking and even # your indexing with Google. # # See http://www.robotstxt.org/wc/norobots.html # for more information on what you can do with this file # These commands are set to tell Googlebot NOT to index # your dyannmically generated pages search_results.asp and # detail.asp page links as Google could penalize your ranking # if it finds "substantially duplicate content" under different # page names. # If your your store is in a subdirectory of your web, you # will need to add that subdirectory below. For example: # # Disallow: /myStore/detail.asp?* # At the time of this publication, we don't see the need to # control any other search engine robot in the way it is # necessary to control for this with Googlebot. User-Agent: Googlebot Disallow: /detail.asp?* Disallow: /search_results.asp?* Disallow: /manufacturer.asp?mfgid=* User-agent: ShopWiki Disallow: /search_results.asp?* Disallow: /search_subcat.asp?* User-agent: Microsoft URL Control - 6.00.8169 Disallow: / User-agent: Microsoft URL Control Disallow: / User-Agent: * Disallow: /myaccount/ Disallow: /smd/ Disallow: /smd/links/*.html Disallow: /smd/*.html Disallow: /myaccount/*.html Disallow: /smd/links/smd_slink.asp