aboutsummaryrefslogtreecommitdiffstats
path: root/python
Commit message (Collapse)AuthorAgeFilesLines
* WIP: more fileset ingestBryan Newbold2021-10-181-13/+21
|
* python: gitignore moreBryan Newbold2021-10-151-0/+2
|
* WIP: rel fixesBryan Newbold2021-10-141-6/+6
|
* fileset ingest small tweaksBryan Newbold2021-10-141-21/+36
|
* initial implementation of fileset ingest importersBryan Newbold2021-10-143-3/+298
|
* ingest: handle datasets, components, other ingest typesBryan Newbold2021-10-141-1/+15
|
* generic fileset importer class, with test coverageBryan Newbold2021-10-146-0/+169
|
* web: minor typo correctionBryan Newbold2021-10-131-1/+1
|
* web: editor username /u/<username> helperBryan Newbold2021-10-132-0/+16
|
* web: container lookup and display featuresBryan Newbold2021-10-133-7/+13
|
* python: additional test coverage for v0.4 changesBryan Newbold2021-10-132-2/+19
|
* dblp import: basic support for handles as identifiersBryan Newbold2021-10-131-1/+5
|
* python: normalization/validation support for handle identifiers (hdl)Bryan Newbold2021-10-131-0/+33
|
* dblp import: fix typos in identifier parsingBryan Newbold2021-10-131-2/+1
|
* python: partial importer utilization of new schema changesBryan Newbold2021-10-133-6/+18
|
* python: test coverage of rust schema changesBryan Newbold2021-10-134-2/+59
|
* python: implement ES schema changesBryan Newbold2021-10-131-4/+17
|
* web: implement new schema changesBryan Newbold2021-10-136-11/+45
|
* Merge branch 'bnewbold-ingest-tweaks' into 'master'bnewbold2021-10-024-39/+139
|\ | | | | | | | | ingest importer behavior tweaks See merge request webgroup/fatcat!120
| * kafka import: optional 'force-flush' mode for some importersBryan Newbold2021-10-012-0/+16
| | | | | | | | Behavior and motivation described in the kafka json import comment.
| * new SPN web (html) importerBryan Newbold2021-10-013-27/+111
| |
| * ingest importer behavior tweaksBryan Newbold2021-10-011-8/+8
| | | | | | | | | | - change order of 'want()' checks, so that result counts are clearer - don't require GROBID success for file imports with SPN
| * importer common: more verbose logging (with counts)Bryan Newbold2021-10-011-4/+4
| |
* | datacite: skip empty abstractsMartin Czygan2021-10-014-2/+95
|/ | | | | Do not add abstracts where `clean` results in the empty string - this violates a constraint: `either abstract_sha1 or content is required`
* default ingest request topic now '-daily'; configurable for ingest_tool.pyBryan Newbold2021-09-304-4/+9
|
* Merge branch 'martin-pubmed-ftp-extramuros' into 'master'Martin Czygan2021-09-091-24/+21
|\ | | | | | | | | pubmed: workaround a networking issue See merge request webgroup/fatcat!118
| * pubmed: workaround a networking issueMartin Czygan2021-09-091-24/+21
| | | | | | | | | | | | use an http proxy (https://github.com/miku/ftpup) to fetch files from FTP, keep some retry logic; also, hardcoding the proxy path as this should be a temporary workaround
* | trivial blank line lintBryan Newbold2021-09-081-1/+0
|/
* pubmed: add option to ftp download with lftpMartin Czygan2021-09-081-2/+31
| | | | | lftp is a classic command line ftp client, and we hope that its retry capabilities are enough of a workaround for the current networking issue
* pubmed harvester: add basic retry logicMartin Czygan2021-08-201-8/+21
| | | | | | | | Related to a previous issue with seemingly random EOFError from FTP connections, this patch wrap "ftpretr" helper function with a basic retry. Refs: fatcat-workers/issues/92151, fatcat-workers/issues/91102
* web: fix stats rowspan (oops)Bryan Newbold2021-08-121-1/+1
|
* web: remove confusing 'references' row from stats tableBryan Newbold2021-08-121-3/+0
| | | | Now that we have refcat, which is a different number
* refs: default to *not* consolidating worksBryan Newbold2021-08-061-1/+1
| | | | | | | We don't handle counts for consolidated refs yet, so just don't consolidate. This should fix, eg, "Showing 1-18 of 19" type UX confusion, with the trade-off that some works will be duplicated in inbound ref tables.
* web: update front-page static statsBryan Newbold2021-08-061-3/+3
|
* refs: format (commas) large refs hit countsBryan Newbold2021-08-061-1/+1
|
* refs web: correct URL to refs section of guideBryan Newbold2021-08-041-1/+1
|
* refs: web UI tweaks for iterated CSL schemaBryan Newbold2021-08-032-6/+26
|
* refs: fix typo preventing CSL from rendering in refs outputBryan Newbold2021-07-271-1/+1
|
* refs: start the most basic/minimal web refs test coverage ('integration' level)Bryan Newbold2021-07-274-0/+1094
|
* refs: revert fatcat-pubmed -> pubmed truncationBryan Newbold2021-07-271-4/+1
| | | | This was just going to be confusing
* refs: lint fixesBryan Newbold2021-07-272-2/+3
|
* refs: several small improvements to web UIBryan Newbold2021-07-275-35/+71
|
* refs: slightly better match form (will change)Bryan Newbold2021-07-271-42/+46
|
* refs: show up to 8 authors in summary tablesBryan Newbold2021-07-271-4/+4
|
* refs: support for wikipedia outbound refs, and display in tablesBryan Newbold2021-07-274-8/+69
|
* refs: fix offset/limit bugBryan Newbold2021-07-271-1/+1
|
* refs: generalize web endpoints; JSON content negotiation; openlibrary ↵Bryan Newbold2021-07-234-41/+166
| | | | inbound view; etc
* refs: change mind about URL structure againBryan Newbold2021-07-231-2/+2
|
* web: refactor refs table into separate refs_macros fileBryan Newbold2021-07-233-74/+127
|
* refs: small refactors/tweaksBryan Newbold2021-07-231-11/+17
|
generated by cgit v1.2.3 (git 2.39.1) at 2024-11-24 00:44:43 +0000