User-Agent: * Disallow: /cgi-bin/ User-Agent: sitecheck.internetseer.com Disallow: /file-to-disallow