# Allow isn't supported in Robots.txt :-( # See: http://www.robotstxt.org/robotstxt.html # So I'm disallowing all, by changing Disallow: /* to Disallow: / # Because one bot was generating 'Unable to read beyond the end of the stream' errors User-agent: * Disallow: / Allow: /Sitemap.ashx* Allow: /sitemap.ashx* Allow: /?