summaryrefslogtreecommitdiffstats
path: root/python/fatcat_tools/harvest/oaipmh.py
Commit message (Collapse)AuthorAgeFilesLines
* harvest: log state on startup and use stderr for diagnosticsMartin Czygan2020-02-141-7/+8
|
* review/fix all confluent-kafka produce codeBryan Newbold2019-09-201-5/+8
|
* small fixes to confluent-kafka importers/workersBryan Newbold2019-09-201-1/+1
| | | | | | | | - decrease default changelog pipeline to 5.0sec - fix missing KafkaException harvester imports - more confluent-kafka tweaks - updates to kafka consumer configs - bump elastic updates consumergroup (again)
* bump max message size to ~20 MBytesBryan Newbold2019-09-201-0/+1
|
* fixes to confluent-kafka harvestersBryan Newbold2019-09-201-8/+8
|
* first draft harvesters using confluent-kafkaBryan Newbold2019-09-201-11/+30
|
* MEDLINE/Pubmed noteBryan Newbold2019-03-151-2/+6
| | | | Also, arXivRaw, not arXiv (though see WIP on more-importers branch)
* bunch of lint/whitespace cleanupsBryan Newbold2019-02-221-6/+5
|
* clean up harvester comments/docsBryan Newbold2018-11-211-44/+29
|
* use isoformat() to format datesBryan Newbold2018-11-211-2/+2
| | | | This shouldn't change behavior; it's just more consistent.
* fix loop_sleep typoBryan Newbold2018-11-211-1/+1
|
* fix OAI-PMH name/finished messageBryan Newbold2018-11-211-1/+6
|
* fix oai-pmh issue againBryan Newbold2018-11-211-13/+14
|
* oaipmh: handle NoRecordsMatchBryan Newbold2018-11-211-5/+8
|
* initial OAI-PMH harvestersBryan Newbold2018-11-191-0/+157