# Robots.txt file for www.oulu.fi, robots welcome User-agent: * Crawl-delay: 10 Disallow: /cgi-bin Disallow: /stats Disallow: /yliopisto/node/*/* Disallow: /english/node/*/* User-agent: UnivOfOuluNutch