# don't let search engines see the RSS feed, it's just confusing. User-agent: Googlebot User-agent: InfoNaviRobot User-agent: TV33_Mercator User-agent: AVSearch User-agent: Mercator User-agent: Scooter User-agent: Slurp User-agent: SearchengineLicenceSheep User-agent: shadow User-agent: MultiText User-agent: FAST-WebCrawler User-agent: Lycos_Spider User-agent: Atomz User-agent: htdig User-agent: spider00.logika.net User-agent: NetMechanic User-agent: libwww-perl User-agent: Teleport Pro Disallow: /searchtools-rss.xml User-agent: * Disallow: /admin/ Disallow: /daryl/ Disallow: /CGI-BIN/ Disallow: /database/ Disallow: /filemanager/ Disallow: /fm/ Disallow: /fileexplorer/ Disallow: /FileManager_20060317/ Disallow: /forms/ Disallow: /dev/ Disallow: /shopping_cart/ Disallow: /XYINZNWSK/ # updated 2006-03-09 (ADDED FILE BASED ON http://www.searchtools.com/robots/robots-meta.html RECOMENDATION)