Commit message (Collapse) | Author | Age | Files | Lines | ||
---|---|---|---|---|---|---|
... | ||||||
| * | address spammy datacite titles | Martin Czygan | 2020-09-23 | 2 | -0/+25 | |
|/ | | | | | | | | | seemingly from zenodo: * https://fatcat.wiki/release/rzcpjwukobd4pj36ipla22cnoi * https://doi.org/10.5281/zenodo.4041777 About 3400 records with "FULL MOVIE" in title, currently. | |||||
* | homepage: small grammar tweaks (The/the) | Bryan Newbold | 2020-09-11 | 1 | -3/+3 | |
| | ||||||
* | ingest: default to crawl protocols.io DOIs | Bryan Newbold | 2020-09-10 | 1 | -0/+2 | |
| | ||||||
* | Merge branch 'bnewbold-datacite-not-empty-version' into 'master' | bnewbold | 2020-09-11 | 3 | -2/+3 | |
|\ | | | | | | | | | datacite: handle case of empty-string version See merge request webgroup/fatcat!83 | |||||
| * | datacite: handle case of empty-string version | Bryan Newbold | 2020-09-10 | 3 | -2/+3 | |
|/ | | | | | Includes a tiny tweak to the datacite import sample file to test this code path. | |||||
* | file_meta import notes | Bryan Newbold | 2020-09-04 | 1 | -0/+75 | |
| | ||||||
* | update stats snapshot | Bryan Newbold | 2020-09-03 | 2 | -0/+47 | |
| | ||||||
* | remove spurious print statement | Bryan Newbold | 2020-09-03 | 1 | -1/+0 | |
| | ||||||
* | Merge branch 'bnewbold-file-meta-cleanups' into 'master' | Martin Czygan | 2020-09-03 | 3 | -0/+149 | |
|\ | | | | | | | | | generic file entity clean-ups as part of file_meta importer See merge request webgroup/fatcat!82 | |||||
| * | generic file entity clean-ups as part of file_meta importer | Bryan Newbold | 2020-09-02 | 3 | -0/+149 | |
|/ | ||||||
* | Merge branch 'bnewbold-filemeta' | Bryan Newbold | 2020-08-27 | 5 | -0/+162 | |
|\ | ||||||
| * | fix comment typo (thanks martin) | Bryan Newbold | 2020-08-27 | 1 | -1/+1 | |
| | | ||||||
| * | fixes and test coverage for file_meta importer | Bryan Newbold | 2020-08-21 | 4 | -6/+82 | |
| | | ||||||
| * | initial implementation of file_meta importer | Bryan Newbold | 2020-08-21 | 3 | -0/+86 | |
| | | ||||||
* | | Merge branch 'bnewbold-meta-tags' into 'master' | Martin Czygan | 2020-08-25 | 1 | -2/+1 | |
|\ \ | |/ |/| | | | | | meta tags See merge request webgroup/fatcat!81 | |||||
| * | remove typo (isbn:) from metadata DC.language field | Bryan Newbold | 2020-08-21 | 1 | -1/+1 | |
| | | ||||||
| * | remove placeholder description meta tag | Bryan Newbold | 2020-08-20 | 1 | -1/+0 | |
|/ | ||||||
* | Merge branch 'bnewbold-sitemap' into 'master' | bnewbold | 2020-08-20 | 10 | -7/+206 | |
|\ | | | | | | | | | basic sitemap setup See merge request webgroup/fatcat!79 | |||||
| * | fix SearchAction nesting in WebSite (schema.org) | Bryan Newbold | 2020-08-20 | 1 | -5/+2 | |
| | | | | | | | | | | This is not related to sitemap changes, but I was reminded in google search tools when validating site. | |||||
| * | sitemap fixes from testing | Bryan Newbold | 2020-08-19 | 4 | -9/+20 | |
| | | ||||||
| * | update robots.txt and sitemap.xml | Bryan Newbold | 2020-08-19 | 4 | -2/+52 | |
| | | | | | | | | | | | | - show minimal robots/sitemap if not in prod environment - default to allow all in robots.txt; link to sitemap index files - basic sitemap.xml without entity-level links | |||||
| * | iterate on sitemap generation | Bryan Newbold | 2020-08-19 | 6 | -7/+119 | |
| | | ||||||
| * | initial sitemap.xml notes/template | Bryan Newbold | 2020-08-19 | 2 | -0/+29 | |
|/ | ||||||
* | bulk edit log: add notes on recent chocula import | Bryan Newbold | 2020-08-17 | 1 | -0/+17 | |
| | ||||||
* | entity updater: handle doi=None case better | Bryan Newbold | 2020-08-14 | 1 | -1/+1 | |
| | ||||||
* | entity updater: es['publisher_type'] not always set | Bryan Newbold | 2020-08-14 | 1 | -1/+1 | |
| | | | | This is a small bugfix for a production issue. | |||||
* | Merge branch 'bnewbold-ingest-improvements' into 'master' | Martin Czygan | 2020-08-13 | 8 | -38/+120 | |
|\ | | | | | | | | | ingest behavior changes; some datacite metadata tweaks See merge request webgroup/fatcat!78 | |||||
| * | entity update: change big5 ingest behavior | Bryan Newbold | 2020-08-11 | 1 | -9/+15 | |
| | | | | | | | | | | | | | | | | | | In addition to changing the OA default, this was the main intended behavior change in this group of commits: want to ingest fewer attempts that we *expect* to fail, but default to ingest/crawl attempt if we are uncertain. This is because there is a long tail of journals that register DOIs and are defacto OA (fulltext is available), but we don't have metadata indicating them as such. | |||||
| * | datacite importer: update test cases for 'Additional file' as component, not ↵ | Bryan Newbold | 2020-08-11 | 5 | -5/+5 | |
| | | | | | | | | stub | |||||
| * | entity update: default to ingest non-OA works | Bryan Newbold | 2020-08-11 | 1 | -9/+10 | |
| | | ||||||
| * | entity update: skip ingest of figshare+zenodo 'group' DOIs | Bryan Newbold | 2020-08-11 | 1 | -0/+15 | |
| | | ||||||
| * | datacite import: figshare-specific hacks | Bryan Newbold | 2020-08-11 | 2 | -3/+4 | |
| | | ||||||
| * | datacite import: refactor release_type detection into static method | Bryan Newbold | 2020-08-11 | 1 | -14/+51 | |
| | | ||||||
| * | datacite import: refactor publisher-specific hacks into static method | Bryan Newbold | 2020-08-11 | 1 | -15/+29 | |
| | | | | | | | | Also tweak title/publisher detection to use DOI prefixes | |||||
| * | update crawl blocklist for SPNv2 requests which mostly fail | Bryan Newbold | 2020-08-10 | 1 | -2/+10 | |
| | | ||||||
* | | Merge branch 'martin-datacite-json-decode-err-sentry-38625' into 'master' | bnewbold | 2020-08-10 | 1 | -1/+8 | |
|\ \ | |/ |/| | | | | | harvest: datacite API yields HTTP 200 with broken JSON See merge request webgroup/fatcat!77 | |||||
| * | harvest: datacite API yields HTTP 200 with broken JSON | Martin Czygan | 2020-08-10 | 1 | -1/+8 | |
|/ | | | | As a first step: log response body for debugging. | |||||
* | release ES transform tweaks | Bryan Newbold | 2020-08-07 | 1 | -3/+5 | |
| | | | | | | | | pass-through publisher_type from container extra metadata (ES field already existed; this is from newer chocula metadata) count arxiv and PMCID papers which haven't been crawled (by IA) as "dark", not "bright" | |||||
* | Merge branch 'bnewbold-work-dumps' into 'master' | bnewbold | 2020-08-05 | 6 | -19/+237 | |
|\ | | | | | | | | | release dumps grouped by work_id See merge request webgroup/fatcat!75 | |||||
| * | fatcat export: flush after batch, not per-line | Bryan Newbold | 2020-08-05 | 1 | -1/+1 | |
| | | | | | | | | Good catch, thanks Martin | |||||
| * | proposal for work grouping | Bryan Newbold | 2020-08-04 | 1 | -0/+60 | |
| | | ||||||
| * | include releases_by_work in ident tarball | Bryan Newbold | 2020-08-04 | 1 | -1/+2 | |
| | | ||||||
| * | update SQL dump docs with group-by-work command (by default) | Bryan Newbold | 2020-08-04 | 1 | -1/+1 | |
| | | ||||||
| * | group-by-work mode for fatcat-export | Bryan Newbold | 2020-08-04 | 1 | -15/+157 | |
| | | ||||||
| * | rust Makefile: fix test command | Bryan Newbold | 2020-08-04 | 1 | -2/+1 | |
| | | ||||||
| * | WIP: sorted release ident dumps | Bryan Newbold | 2020-08-04 | 1 | -0/+16 | |
| | | ||||||
* | | Merge branch 'bnewbold-chocula-import-tweaks' into 'master' | bnewbold | 2020-08-05 | 1 | -12/+22 | |
|\ \ | |/ |/| | | | | | chocula import tweaks See merge request webgroup/fatcat!74 | |||||
| * | chocula import update tweaks | Bryan Newbold | 2020-08-04 | 1 | -10/+14 | |
| | | ||||||
| * | more update keys and cases for chocula importer | Bryan Newbold | 2020-08-04 | 1 | -5/+11 | |
| | | ||||||
| * | fix key name mismatch in chocula importer | Bryan Newbold | 2020-08-04 | 1 | -1/+1 | |
|/ | | | | chocula 'export-fatcat' uses 'ident', not 'fatcat_ident' |