diff options
Diffstat (limited to 'proposals')
| -rw-r--r-- | proposals/2021-04-02_crawlability.md | 10 |
1 files changed, 9 insertions, 1 deletions
diff --git a/proposals/2021-04-02_crawlability.md b/proposals/2021-04-02_crawlability.md index 6b9ef66c..ee9f3c5b 100644 --- a/proposals/2021-04-02_crawlability.md +++ b/proposals/2021-04-02_crawlability.md @@ -1,9 +1,16 @@ -status: wip +status: not-implemented Crawlability Improvements -------------------------- +NOTE: After some back and forth on this topic, we have decided for now to focus +on having scholar.archive.org indexed, not fatcat.wiki. This proposal document +document is being kept as documentation of that decision. + + +## Original Intro + We are interested in making the fatcat corpus more crawlable/indexable by aggregators and academic search enginges. For example, CiteseerX, Google Scholar, or Microsoft Academic (when themselves get used by other projects). @@ -13,6 +20,7 @@ Some open questions: - is the web.archive.org iframe for PDFs ok, or should we redirect to PDFs with `id_` in the datetime? + ## Redirect URLs and `citation_pdf_url` We suspect that some crawlers do not like that fatcat.wiki landing pages have |