# /robots.txt file for http://www.arc.net.au/ # mail webmaster@arc.net.au.com for constructive criticism User-agent: * # directed to all spiders, not just Scooter Disallow: /cgi-bin Disallow: /arc/stats Disallow: /arc/images Disallow: /logs Disallow: /arc/logs