# Check with http://tool.motoricerca.info/robots-checker.phtml # Crawl delay is seconds User-agent: Yandex Disallow: / User-agent: psbot Disallow: / User-agent: Slurp Crawl-delay: 3600 User-agent: VoilaBot* Crawl-delay: 3600 User-agent: * Disallow: /bot_trap.php