Commit message (Collapse) | Author | Age | Files | Lines | |
---|---|---|---|---|---|
* | Merge branch 'bnewbold-ingest-tweaks' into 'master' | bnewbold | 2021-10-02 | 5 | -39/+142 |
|\ | | | | | | | | | ingest importer behavior tweaks See merge request webgroup/fatcat!120 | ||||
| * | update changelog with notable ingest importer tweaks | Bryan Newbold | 2021-10-01 | 1 | -0/+3 |
| | | |||||
| * | kafka import: optional 'force-flush' mode for some importers | Bryan Newbold | 2021-10-01 | 2 | -0/+16 |
| | | | | | | | | Behavior and motivation described in the kafka json import comment. | ||||
| * | new SPN web (html) importer | Bryan Newbold | 2021-10-01 | 3 | -27/+111 |
| | | |||||
| * | ingest importer behavior tweaks | Bryan Newbold | 2021-10-01 | 1 | -8/+8 |
| | | | | | | | | | | - change order of 'want()' checks, so that result counts are clearer - don't require GROBID success for file imports with SPN | ||||
| * | importer common: more verbose logging (with counts) | Bryan Newbold | 2021-10-01 | 1 | -4/+4 |
| | | |||||
* | | Merge branch 'martin-datacite-emtpy-abstract-sentry-94639' into 'master' | bnewbold | 2021-10-02 | 4 | -2/+95 |
|\ \ | |/ |/| | | | | | datacite: skip empty abstracts See merge request webgroup/fatcat!119 | ||||
| * | datacite: skip empty abstracts | Martin Czygan | 2021-10-01 | 4 | -2/+95 |
|/ | | | | | Do not add abstracts where `clean` results in the empty string - this violates a constraint: `either abstract_sha1 or content is required` | ||||
* | default ingest request topic now '-daily'; configurable for ingest_tool.py | Bryan Newbold | 2021-09-30 | 4 | -4/+9 |
| | |||||
* | Merge branch 'martin-pubmed-ftp-extramuros' into 'master' | Martin Czygan | 2021-09-09 | 1 | -24/+21 |
|\ | | | | | | | | | pubmed: workaround a networking issue See merge request webgroup/fatcat!118 | ||||
| * | pubmed: workaround a networking issue | Martin Czygan | 2021-09-09 | 1 | -24/+21 |
| | | | | | | | | | | | | use an http proxy (https://github.com/miku/ftpup) to fetch files from FTP, keep some retry logic; also, hardcoding the proxy path as this should be a temporary workaround | ||||
* | | trivial blank line lint | Bryan Newbold | 2021-09-08 | 1 | -1/+0 |
| | | |||||
* | | Merge branch 'master' of git.archive.org:webgroup/fatcat | Bryan Newbold | 2021-09-08 | 1 | -2/+31 |
|\| | |||||
| * | Merge branch 'martin-pubmed-use-lftp' into 'master' | Martin Czygan | 2021-09-08 | 1 | -2/+31 |
| |\ | | | | | | | | | | | | | pubmed: add option to ftp download with lftp See merge request webgroup/fatcat!117 | ||||
| | * | pubmed: add option to ftp download with lftp | Martin Czygan | 2021-09-08 | 1 | -2/+31 |
| |/ | | | | | | | | | lftp is a classic command line ftp client, and we hope that its retry capabilities are enough of a workaround for the current networking issue | ||||
* / | sql_dumps: set collection at upload time | Bryan Newbold | 2021-09-02 | 1 | -2/+5 |
|/ | |||||
* | Merge branch 'martin-pubmed-eof-sentry-92151' into 'master' | Martin Czygan | 2021-08-21 | 1 | -8/+21 |
|\ | | | | | | | | | pubmed harvester: add basic retry logic See merge request webgroup/fatcat!116 | ||||
| * | pubmed harvester: add basic retry logic | Martin Czygan | 2021-08-20 | 1 | -8/+21 |
|/ | | | | | | | | Related to a previous issue with seemingly random EOFError from FTP connections, this patch wrap "ftpretr" helper function with a basic retry. Refs: fatcat-workers/issues/92151, fatcat-workers/issues/91102 | ||||
* | guide: remove accidental duplicated background section | Bryan Newbold | 2021-08-18 | 1 | -9/+0 |
| | |||||
* | cgraph -> refcat | Bryan Newbold | 2021-08-13 | 2 | -2/+2 |
| | |||||
* | web: fix stats rowspan (oops) | Bryan Newbold | 2021-08-12 | 1 | -1/+1 |
| | |||||
* | web: remove confusing 'references' row from stats table | Bryan Newbold | 2021-08-12 | 1 | -3/+0 |
| | | | | Now that we have refcat, which is a different number | ||||
* | Merge branch 'martin-guide-ref-minor-tweaks' into 'master' | bnewbold | 2021-08-09 | 1 | -3/+4 |
|\ | | | | | | | | | guide: reference graph, minor tweaks See merge request webgroup/fatcat!115 | ||||
| * | guide: reference graph, minor tweaks | Martin Czygan | 2021-08-07 | 1 | -3/+4 |
|/ | |||||
* | guide: expand on refcat | Bryan Newbold | 2021-08-06 | 3 | -31/+160 |
| | |||||
* | guide: update and rename search index page | Bryan Newbold | 2021-08-06 | 3 | -13/+16 |
| | |||||
* | guide: update bibliography, blog links | Bryan Newbold | 2021-08-06 | 1 | -11/+11 |
| | |||||
* | Merge branch 'martin-guide-ref' into 'master' | bnewbold | 2021-08-06 | 1 | -2/+22 |
|\ | | | | | | | | | guide: draft notes on background and mode of operatio See merge request webgroup/fatcat!114 | ||||
| * | guide: draft notes on background and mode of operatio | Martin Czygan | 2021-08-06 | 1 | -2/+22 |
|/ | |||||
* | refs: default to *not* consolidating works | Bryan Newbold | 2021-08-06 | 1 | -1/+1 |
| | | | | | | | We don't handle counts for consolidated refs yet, so just don't consolidate. This should fix, eg, "Showing 1-18 of 19" type UX confusion, with the trade-off that some works will be duplicated in inbound ref tables. | ||||
* | web: update front-page static stats | Bryan Newbold | 2021-08-06 | 1 | -3/+3 |
| | |||||
* | prod stats snapshot | Bryan Newbold | 2021-08-06 | 4 | -0/+47 |
| | |||||
* | Merge branch 'bnewbold-refs-apis' | Bryan Newbold | 2021-08-06 | 24 | -79/+2313 |
|\ | |||||
| * | refs: format (commas) large refs hit counts | Bryan Newbold | 2021-08-06 | 1 | -1/+1 |
| | | |||||
| * | refs web: correct URL to refs section of guide | Bryan Newbold | 2021-08-04 | 1 | -1/+1 |
| | | |||||
| * | refs: web UI tweaks for iterated CSL schema | Bryan Newbold | 2021-08-03 | 2 | -6/+26 |
| | | |||||
| * | start CHANGELOG for refs work | Bryan Newbold | 2021-07-27 | 4 | -0/+45 |
| | | |||||
| * | refs: fix typo preventing CSL from rendering in refs output | Bryan Newbold | 2021-07-27 | 1 | -1/+1 |
| | | |||||
| * | refs: start the most basic/minimal web refs test coverage ('integration' level) | Bryan Newbold | 2021-07-27 | 4 | -0/+1094 |
| | | |||||
| * | refs: revert fatcat-pubmed -> pubmed truncation | Bryan Newbold | 2021-07-27 | 1 | -4/+1 |
| | | | | | | | | This was just going to be confusing | ||||
| * | refs: lint fixes | Bryan Newbold | 2021-07-27 | 2 | -2/+3 |
| | | |||||
| * | refs: several small improvements to web UI | Bryan Newbold | 2021-07-27 | 5 | -35/+71 |
| | | |||||
| * | refs: slightly better match form (will change) | Bryan Newbold | 2021-07-27 | 1 | -42/+46 |
| | | |||||
| * | refs: show up to 8 authors in summary tables | Bryan Newbold | 2021-07-27 | 1 | -4/+4 |
| | | |||||
| * | refs: support for wikipedia outbound refs, and display in tables | Bryan Newbold | 2021-07-27 | 4 | -8/+69 |
| | | |||||
| * | refs: fix offset/limit bug | Bryan Newbold | 2021-07-27 | 1 | -1/+1 |
| | | |||||
| * | refs: generalize web endpoints; JSON content negotiation; openlibrary ↵ | Bryan Newbold | 2021-07-23 | 4 | -41/+166 |
| | | | | | | | | inbound view; etc | ||||
| * | refs: change mind about URL structure again | Bryan Newbold | 2021-07-23 | 2 | -7/+7 |
| | | |||||
| * | web: refactor refs table into separate refs_macros file | Bryan Newbold | 2021-07-23 | 3 | -74/+127 |
| | | |||||
| * | refs: small refactors/tweaks | Bryan Newbold | 2021-07-23 | 1 | -11/+17 |
| | |