Commit message (Expand) | Author | Age | Files | Lines | |
---|---|---|---|---|---|
* | robots: deny SemanticScholarBot search result pages | Bryan Newbold | 2022-06-06 | 1 | -0/+1 |
* | robots.txt: block /search for large crawlers | Bryan Newbold | 2022-01-11 | 1 | -5/+13 |
* | robots: add commented out reference to sitemap-index-access.xml | Bryan Newbold | 2021-06-30 | 1 | -0/+3 |
* | web: use absolute URLs to sitemaps and in citation_pdf_url | Bryan Newbold | 2021-04-30 | 1 | -2/+2 |
* | web: sitemap.xml serving | Bryan Newbold | 2021-04-29 | 1 | -2/+14 |
* | web robots.txt | Bryan Newbold | 2020-09-15 | 1 | -0/+5 |