summaryrefslogtreecommitdiffstats
path: root/python/fatcat_tools/harvest/harvest_common.py
Commit message (Collapse)AuthorAgeFilesLines
* lint (flake8) tool python filesBryan Newbold2020-07-011-3/+5
|
* rename HarvestState.next() to HarvestState.next_span()Bryan Newbold2020-05-261-2/+2
| | | | | | | | | "span" short for "timespan" to harvest; there may be a better name to use. Motivation for this is to work around a pylint erorr that .next() was not callable. This might be a bug with pylint, but .next() is also a very generic name.
* Correct spelling mistakesEdward Betts2020-03-271-2/+2
|
* harvest: log state on startup and use stderr for diagnosticsMartin Czygan2020-02-141-3/+7
|
* review/fix all confluent-kafka produce codeBryan Newbold2019-09-201-4/+33
|
* fixes to confluent-kafka harvestersBryan Newbold2019-09-201-2/+2
|
* first draft harvesters using confluent-kafkaBryan Newbold2019-09-201-11/+29
|
* increase default harvest window to 14 daysBryan Newbold2019-04-011-2/+2
|
* HACK: force pylint to ignore urllib3 Retry importBryan Newbold2019-03-151-1/+3
| | | | | | As the code comment mentions, not sure why pylint throws this error. requests and urllib3 are recent, and this code runs fine in tests and QA, and pylint is running (in CI) within pipenv.
* retry/backoff for Crossref harvesterBryan Newbold2019-03-061-0/+22
|
* bunch of lint/whitespace cleanupsBryan Newbold2019-02-221-1/+0
|
* clean up harvester comments/docsBryan Newbold2018-11-211-1/+1
|
* better DOI registrar harvestersBryan Newbold2018-11-191-0/+124