diff options
| author | bnewbold <bnewbold@archive.org> | 2020-08-20 21:17:59 +0000 |
|---|---|---|
| committer | bnewbold <bnewbold@archive.org> | 2020-08-20 21:17:59 +0000 |
| commit | daf91b137483b7345448b597289c78f8fb3f9969 (patch) | |
| tree | 712c27d902235d8d007763b512c57eaecd8045ad /python/fatcat_web/static/robots.deny_all.txt | |
| parent | 5007ee299ce07b31db6d48cd4ab2587f87af53ab (diff) | |
| parent | 2a98d10be1cc1368f9510745bff07c343974d4a7 (diff) | |
| download | fatcat-daf91b137483b7345448b597289c78f8fb3f9969.tar.gz fatcat-daf91b137483b7345448b597289c78f8fb3f9969.zip | |
Merge branch 'bnewbold-sitemap' into 'master'
basic sitemap setup
See merge request webgroup/fatcat!79
Diffstat (limited to 'python/fatcat_web/static/robots.deny_all.txt')
| -rw-r--r-- | python/fatcat_web/static/robots.deny_all.txt | 7 |
1 files changed, 7 insertions, 0 deletions
diff --git a/python/fatcat_web/static/robots.deny_all.txt b/python/fatcat_web/static/robots.deny_all.txt new file mode 100644 index 00000000..b88274b1 --- /dev/null +++ b/python/fatcat_web/static/robots.deny_all.txt @@ -0,0 +1,7 @@ +# Hello friends! + +# You have found a QA/development instance of the Fatcat catalog. The canonical +# location is https://fatcat.wiki, please crawl and index that location instead. + +User-agent: * +Disallow: / |