Edit the robots to stop the readthedocs page from being indexed

matkuliak · amotl · commit bbd7df362bb7 · 2023-04-17T10:58:07.000-07:00
diff --git a/docs/_extra/robots.txt b/docs/_extra/robots.txt
@@ -1,2 +1,4 @@
-Sitemap: https://crate.io/docs/python/en/latest/site.xml
 User-agent: *
+Disallow: /
+
+Sitemap: https://crate.io/docs/python/en/latest/site.xml
