Commit message (Collapse) | Author | Age | Files | Lines | ||
---|---|---|---|---|---|---|
... | ||||||
* | | datacite: skip empty abstracts | Martin Czygan | 2021-10-01 | 4 | -2/+95 | |
|/ | | | | | Do not add abstracts where `clean` results in the empty string - this violates a constraint: `either abstract_sha1 or content is required` | |||||
* | default ingest request topic now '-daily'; configurable for ingest_tool.py | Bryan Newbold | 2021-09-30 | 4 | -4/+9 | |
| | ||||||
* | Merge branch 'martin-pubmed-ftp-extramuros' into 'master' | Martin Czygan | 2021-09-09 | 1 | -24/+21 | |
|\ | | | | | | | | | pubmed: workaround a networking issue See merge request webgroup/fatcat!118 | |||||
| * | pubmed: workaround a networking issue | Martin Czygan | 2021-09-09 | 1 | -24/+21 | |
| | | | | | | | | | | | | use an http proxy (https://github.com/miku/ftpup) to fetch files from FTP, keep some retry logic; also, hardcoding the proxy path as this should be a temporary workaround | |||||
* | | trivial blank line lint | Bryan Newbold | 2021-09-08 | 1 | -1/+0 | |
|/ | ||||||
* | pubmed: add option to ftp download with lftp | Martin Czygan | 2021-09-08 | 1 | -2/+31 | |
| | | | | | lftp is a classic command line ftp client, and we hope that its retry capabilities are enough of a workaround for the current networking issue | |||||
* | pubmed harvester: add basic retry logic | Martin Czygan | 2021-08-20 | 1 | -8/+21 | |
| | | | | | | | | Related to a previous issue with seemingly random EOFError from FTP connections, this patch wrap "ftpretr" helper function with a basic retry. Refs: fatcat-workers/issues/92151, fatcat-workers/issues/91102 | |||||
* | web: fix stats rowspan (oops) | Bryan Newbold | 2021-08-12 | 1 | -1/+1 | |
| | ||||||
* | web: remove confusing 'references' row from stats table | Bryan Newbold | 2021-08-12 | 1 | -3/+0 | |
| | | | | Now that we have refcat, which is a different number | |||||
* | refs: default to *not* consolidating works | Bryan Newbold | 2021-08-06 | 1 | -1/+1 | |
| | | | | | | | We don't handle counts for consolidated refs yet, so just don't consolidate. This should fix, eg, "Showing 1-18 of 19" type UX confusion, with the trade-off that some works will be duplicated in inbound ref tables. | |||||
* | web: update front-page static stats | Bryan Newbold | 2021-08-06 | 1 | -3/+3 | |
| | ||||||
* | refs: format (commas) large refs hit counts | Bryan Newbold | 2021-08-06 | 1 | -1/+1 | |
| | ||||||
* | refs web: correct URL to refs section of guide | Bryan Newbold | 2021-08-04 | 1 | -1/+1 | |
| | ||||||
* | refs: web UI tweaks for iterated CSL schema | Bryan Newbold | 2021-08-03 | 2 | -6/+26 | |
| | ||||||
* | refs: fix typo preventing CSL from rendering in refs output | Bryan Newbold | 2021-07-27 | 1 | -1/+1 | |
| | ||||||
* | refs: start the most basic/minimal web refs test coverage ('integration' level) | Bryan Newbold | 2021-07-27 | 4 | -0/+1094 | |
| | ||||||
* | refs: revert fatcat-pubmed -> pubmed truncation | Bryan Newbold | 2021-07-27 | 1 | -4/+1 | |
| | | | | This was just going to be confusing | |||||
* | refs: lint fixes | Bryan Newbold | 2021-07-27 | 2 | -2/+3 | |
| | ||||||
* | refs: several small improvements to web UI | Bryan Newbold | 2021-07-27 | 5 | -35/+71 | |
| | ||||||
* | refs: slightly better match form (will change) | Bryan Newbold | 2021-07-27 | 1 | -42/+46 | |
| | ||||||
* | refs: show up to 8 authors in summary tables | Bryan Newbold | 2021-07-27 | 1 | -4/+4 | |
| | ||||||
* | refs: support for wikipedia outbound refs, and display in tables | Bryan Newbold | 2021-07-27 | 4 | -8/+69 | |
| | ||||||
* | refs: fix offset/limit bug | Bryan Newbold | 2021-07-27 | 1 | -1/+1 | |
| | ||||||
* | refs: generalize web endpoints; JSON content negotiation; openlibrary ↵ | Bryan Newbold | 2021-07-23 | 4 | -41/+166 | |
| | | | | inbound view; etc | |||||
* | refs: change mind about URL structure again | Bryan Newbold | 2021-07-23 | 1 | -2/+2 | |
| | ||||||
* | web: refactor refs table into separate refs_macros file | Bryan Newbold | 2021-07-23 | 3 | -74/+127 | |
| | ||||||
* | refs: small refactors/tweaks | Bryan Newbold | 2021-07-23 | 1 | -11/+17 | |
| | ||||||
* | remove unused imports (lint) | Bryan Newbold | 2021-07-23 | 3 | -8/+4 | |
| | ||||||
* | web: always log upstream errors (may be redundant) | Bryan Newbold | 2021-07-23 | 1 | -0/+2 | |
| | ||||||
* | pylint: skip pydantic import check (dynamic/extensions) | Bryan Newbold | 2021-07-23 | 2 | -8/+4 | |
| | ||||||
* | refs: refactor web paths; enrich refs as generic; remove old refs link | Bryan Newbold | 2021-07-23 | 4 | -66/+52 | |
| | ||||||
* | refs fetch: add some hacks; sort hits | Bryan Newbold | 2021-07-23 | 1 | -6/+16 | |
| | ||||||
* | release view: improve biblio metadata display in central column | Bryan Newbold | 2021-07-23 | 1 | -13/+14 | |
| | ||||||
* | match UI: improve form layout | Bryan Newbold | 2021-07-23 | 1 | -13/+16 | |
| | ||||||
* | improvements to fuzzy refs view | Bryan Newbold | 2021-07-23 | 3 | -47/+75 | |
| | | | | | | | | - fixes to release summary macro - show tab counts correctly by re-using generic entity get helper - table styling; 'prev' link - openlibrary access links - parse-and-match button for unmatched+unstructured refs | |||||
* | fixes for newer ref index | Bryan Newbold | 2021-07-23 | 2 | -50/+11 | |
| | ||||||
* | web: inbound/outbound refs as links (temporarily); change URL names | Bryan Newbold | 2021-07-23 | 3 | -3/+7 | |
| | ||||||
* | web: initial implementation of fuzzy citation parsing and matching tool | Bryan Newbold | 2021-07-23 | 3 | -0/+173 | |
| | ||||||
* | references: refactor to point to access_options transform; comment out CSL ↵ | Bryan Newbold | 2021-07-23 | 1 | -57/+8 | |
| | | | | fields | |||||
* | partial access options transform for releases | Bryan Newbold | 2021-07-23 | 1 | -0/+58 | |
| | ||||||
* | web: template macro to display release entry summary | Bryan Newbold | 2021-07-23 | 1 | -0/+52 | |
| | ||||||
* | first iteration of basic citation inbound/outbound views | Bryan Newbold | 2021-07-23 | 3 | -1/+146 | |
| | ||||||
* | initial inbound/outbound reference query helpers | Bryan Newbold | 2021-07-23 | 1 | -0/+450 | |
| | ||||||
* | pubmed: update docs | Martin Czygan | 2021-07-17 | 1 | -2/+3 | |
| | ||||||
* | pubmed: do not fail when accessing missing file | Martin Czygan | 2021-07-17 | 1 | -2/+8 | |
| | | | | | | | after a sync gap (e.g. 06/07 2021) harvester wanted to fetch a file, that was not on the server (any more) - do not fail in this case we'll need to backfill missing records via full data dump | |||||
* | pubmed: reconnect on error | Martin Czygan | 2021-07-16 | 1 | -4/+30 | |
| | | | | | | | | | ftp retrieval would run but fail with EOFError on /pubmed/updatefiles/pubmed21n1328_stats.html - not able to find the root cause; using a fresh client, the exact same file would work just fine. So when we retry, we reconnect on failure. Refs: sentry #91102. | |||||
* | web: fix flask/werkzeug encoding for mediawiki oauth | Bryan Newbold | 2021-07-13 | 1 | -1/+4 | |
| | ||||||
* | web: fix missing ext_ids default for deleted entity view | Bryan Newbold | 2021-07-13 | 1 | -1/+1 | |
| | ||||||
* | web: fix 'file' entity edit form links | Bryan Newbold | 2021-07-02 | 1 | -1/+1 | |
| | ||||||
* | web: missing trailing parens | Bryan Newbold | 2021-07-02 | 1 | -1/+1 | |
| |