# Robots.txt file from http://ssidegr.org/ # # Problem Sites # cuill.com 208.36.144.7 twiceler # Disallow: /cgi-bin/ User-agent: twiceler Disallow: / User-agent: Wget Disallow: / User-agent: HTTrack Disallow: / User-agent: WebCopier Disallow: / User-agent: WebSauger Disallow: / User-agent: WebReaper Disallow: / User-agent: WebStripper Disallow: / User-agent: Web Downloader Disallow: / User-agent: libwww-perl Disallow: / User-agent: Python-urllib Disallow: / User-agent: * Disallow: /cgi-bin/ Disallow: /axsgrandrapidscity/ Disallow: /img/ Disallow: /1old/ Disallow: /css/ Disallow: /fpdb/ Disallow: /inc/ Disallow: /mail/ Disallow: /stats/