summaryrefslogtreecommitdiffstats
Commit message (Collapse)AuthorAgeFilesLines
* another vanished content exampleBryan Newbold2021-10-071-0/+7
|
* Merge branch 'bnewbold-ingest-tweaks' into 'master'bnewbold2021-10-025-39/+142
|\ | | | | | | | | ingest importer behavior tweaks See merge request webgroup/fatcat!120
| * update changelog with notable ingest importer tweaksBryan Newbold2021-10-011-0/+3
| |
| * kafka import: optional 'force-flush' mode for some importersBryan Newbold2021-10-012-0/+16
| | | | | | | | Behavior and motivation described in the kafka json import comment.
| * new SPN web (html) importerBryan Newbold2021-10-013-27/+111
| |
| * ingest importer behavior tweaksBryan Newbold2021-10-011-8/+8
| | | | | | | | | | - change order of 'want()' checks, so that result counts are clearer - don't require GROBID success for file imports with SPN
| * importer common: more verbose logging (with counts)Bryan Newbold2021-10-011-4/+4
| |
* | Merge branch 'martin-datacite-emtpy-abstract-sentry-94639' into 'master'bnewbold2021-10-024-2/+95
|\ \ | |/ |/| | | | | datacite: skip empty abstracts See merge request webgroup/fatcat!119
| * datacite: skip empty abstractsMartin Czygan2021-10-014-2/+95
|/ | | | | Do not add abstracts where `clean` results in the empty string - this violates a constraint: `either abstract_sha1 or content is required`
* default ingest request topic now '-daily'; configurable for ingest_tool.pyBryan Newbold2021-09-304-4/+9
|
* Merge branch 'martin-pubmed-ftp-extramuros' into 'master'Martin Czygan2021-09-091-24/+21
|\ | | | | | | | | pubmed: workaround a networking issue See merge request webgroup/fatcat!118
| * pubmed: workaround a networking issueMartin Czygan2021-09-091-24/+21
| | | | | | | | | | | | use an http proxy (https://github.com/miku/ftpup) to fetch files from FTP, keep some retry logic; also, hardcoding the proxy path as this should be a temporary workaround
* | trivial blank line lintBryan Newbold2021-09-081-1/+0
| |
* | Merge branch 'master' of git.archive.org:webgroup/fatcatBryan Newbold2021-09-081-2/+31
|\|
| * Merge branch 'martin-pubmed-use-lftp' into 'master'Martin Czygan2021-09-081-2/+31
| |\ | | | | | | | | | | | | pubmed: add option to ftp download with lftp See merge request webgroup/fatcat!117
| | * pubmed: add option to ftp download with lftpMartin Czygan2021-09-081-2/+31
| |/ | | | | | | | | lftp is a classic command line ftp client, and we hope that its retry capabilities are enough of a workaround for the current networking issue
* / sql_dumps: set collection at upload timeBryan Newbold2021-09-021-2/+5
|/
* Merge branch 'martin-pubmed-eof-sentry-92151' into 'master'Martin Czygan2021-08-211-8/+21
|\ | | | | | | | | pubmed harvester: add basic retry logic See merge request webgroup/fatcat!116
| * pubmed harvester: add basic retry logicMartin Czygan2021-08-201-8/+21
|/ | | | | | | | Related to a previous issue with seemingly random EOFError from FTP connections, this patch wrap "ftpretr" helper function with a basic retry. Refs: fatcat-workers/issues/92151, fatcat-workers/issues/91102
* guide: remove accidental duplicated background sectionBryan Newbold2021-08-181-9/+0
|
* cgraph -> refcatBryan Newbold2021-08-132-2/+2
|
* web: fix stats rowspan (oops)Bryan Newbold2021-08-121-1/+1
|
* web: remove confusing 'references' row from stats tableBryan Newbold2021-08-121-3/+0
| | | | Now that we have refcat, which is a different number
* Merge branch 'martin-guide-ref-minor-tweaks' into 'master'bnewbold2021-08-091-3/+4
|\ | | | | | | | | guide: reference graph, minor tweaks See merge request webgroup/fatcat!115
| * guide: reference graph, minor tweaksMartin Czygan2021-08-071-3/+4
|/
* guide: expand on refcatBryan Newbold2021-08-063-31/+160
|
* guide: update and rename search index pageBryan Newbold2021-08-063-13/+16
|
* guide: update bibliography, blog linksBryan Newbold2021-08-061-11/+11
|
* Merge branch 'martin-guide-ref' into 'master'bnewbold2021-08-061-2/+22
|\ | | | | | | | | guide: draft notes on background and mode of operatio See merge request webgroup/fatcat!114
| * guide: draft notes on background and mode of operatioMartin Czygan2021-08-061-2/+22
|/
* refs: default to *not* consolidating worksBryan Newbold2021-08-061-1/+1
| | | | | | | We don't handle counts for consolidated refs yet, so just don't consolidate. This should fix, eg, "Showing 1-18 of 19" type UX confusion, with the trade-off that some works will be duplicated in inbound ref tables.
* web: update front-page static statsBryan Newbold2021-08-061-3/+3
|
* prod stats snapshotBryan Newbold2021-08-064-0/+47
|
* Merge branch 'bnewbold-refs-apis'Bryan Newbold2021-08-0624-79/+2313
|\
| * refs: format (commas) large refs hit countsBryan Newbold2021-08-061-1/+1
| |
| * refs web: correct URL to refs section of guideBryan Newbold2021-08-041-1/+1
| |
| * refs: web UI tweaks for iterated CSL schemaBryan Newbold2021-08-032-6/+26
| |
| * start CHANGELOG for refs workBryan Newbold2021-07-274-0/+45
| |
| * refs: fix typo preventing CSL from rendering in refs outputBryan Newbold2021-07-271-1/+1
| |
| * refs: start the most basic/minimal web refs test coverage ('integration' level)Bryan Newbold2021-07-274-0/+1094
| |
| * refs: revert fatcat-pubmed -> pubmed truncationBryan Newbold2021-07-271-4/+1
| | | | | | | | This was just going to be confusing
| * refs: lint fixesBryan Newbold2021-07-272-2/+3
| |
| * refs: several small improvements to web UIBryan Newbold2021-07-275-35/+71
| |
| * refs: slightly better match form (will change)Bryan Newbold2021-07-271-42/+46
| |
| * refs: show up to 8 authors in summary tablesBryan Newbold2021-07-271-4/+4
| |
| * refs: support for wikipedia outbound refs, and display in tablesBryan Newbold2021-07-274-8/+69
| |
| * refs: fix offset/limit bugBryan Newbold2021-07-271-1/+1
| |
| * refs: generalize web endpoints; JSON content negotiation; openlibrary ↵Bryan Newbold2021-07-234-41/+166
| | | | | | | | inbound view; etc
| * refs: change mind about URL structure againBryan Newbold2021-07-232-7/+7
| |
| * web: refactor refs table into separate refs_macros fileBryan Newbold2021-07-233-74/+127
| |
generated by cgit v1.2.3 (git 2.39.1) at 2024-12-15 20:46:05 +0000