# block image index robots User-Agent: Googlebot-Mobile Disallow: / User-Agent: Yahoo-MMCrawler Disallow: / User-Agent: psbot Disallow: / # Crawl limitation for 300sec ( 5 min) # Live Search User-agent: msnbot Crawl-delay: 300 # Yahoo User-agent: Slurp Crawl-delay: 5 # baidu User-agent: baiduspider Crawl-delay: 300