summaryrefslogtreecommitdiffstats
path: root/python/fatcat_tools/harvest
Commit message (Expand)AuthorAgeFilesLines
* harvest: fail on HTTP 400Martin Czygan2020-05-291-4/+0
* rename HarvestState.next() to HarvestState.next_span()Bryan Newbold2020-05-264-5/+5
* HACK: skip pylint errors on lines that seem to be fineBryan Newbold2020-05-223-3/+3
* crossref: switch from index-date to update-dateBryan Newbold2020-03-301-1/+1
* crossref: longer comment about crossref API date fieldsBryan Newbold2020-03-301-2/+22
* Merge pull request #53 from EdwardBetts/spellingbnewbold2020-03-271-2/+2
|\
| * Correct spelling mistakesEdward Betts2020-03-271-2/+2
* | pubmed: log to stderrMartin Czygan2020-03-101-1/+1
* | pubmed: move mapping generation out of fetch_dateMartin Czygan2020-03-101-7/+8
* | harvest: fix imports from HarvestPubmedWorker cleanupMartin Czygan2020-03-101-2/+2
* | pubmed: citations is a bit more preciseMartin Czygan2020-03-091-1/+1
* | pubmed: we sync from FTPMartin Czygan2020-03-091-1/+1
* | oaipmh: HarvestPubmedWorker obsoleted by PubmedFTPWorkerMartin Czygan2020-03-091-34/+0
* | more pubmed adjustmentsMartin Czygan2020-02-222-70/+118
* | pubmed ftp: fix urlMartin Czygan2020-02-191-4/+6
* | pubmed ftp harvest and KafkaBs4XmlPusherMartin Czygan2020-02-192-0/+214
|/
* harvest: log state on startup and use stderr for diagnosticsMartin Czygan2020-02-143-17/+22
* datacite: extend range search queryMartin Czygan2019-12-271-1/+1
* avoid usage of short linksMartin Czygan2019-12-271-2/+2
* Datacite API v2 throws 400, we cannot recover from, currently.Martin Czygan2019-12-271-0/+4
* datacite: update documentation, add links to issuesMartin Czygan2019-12-271-10/+5
* datacite: use v2 of the API (flaky)Martin Czygan2019-12-271-5/+28
* refactor kafka producer in crossref harvesterBryan Newbold2019-12-061-21/+26
* crossref is_update isn't what I thoughtBryan Newbold2019-12-031-6/+2
* review/fix all confluent-kafka produce codeBryan Newbold2019-09-203-14/+49
* small fixes to confluent-kafka importers/workersBryan Newbold2019-09-202-2/+2
* small kafka tweaks for robustnessBryan Newbold2019-09-201-0/+2
* bump max message size to ~20 MBytesBryan Newbold2019-09-202-0/+2
* fixes to confluent-kafka harvestersBryan Newbold2019-09-203-20/+21
* first draft harvesters using confluent-kafkaBryan Newbold2019-09-203-48/+104
* increase default harvest window to 14 daysBryan Newbold2019-04-011-2/+2
* HACK: force pylint to ignore urllib3 Retry importBryan Newbold2019-03-151-1/+3
* MEDLINE/Pubmed noteBryan Newbold2019-03-151-2/+6
* fix harvester session.get() paramsBryan Newbold2019-03-061-5/+8
* retry/backoff for Crossref harvesterBryan Newbold2019-03-062-2/+24
* bunch of lint/whitespace cleanupsBryan Newbold2019-02-223-9/+6
* check request status codes idiomaticallyBryan Newbold2018-12-291-2/+2
* clean up harvester comments/docsBryan Newbold2018-11-213-50/+31
* use isoformat() to format datesBryan Newbold2018-11-212-4/+4
* fix loop_sleep typoBryan Newbold2018-11-212-2/+2
* fix datacite DOI extractionBryan Newbold2018-11-211-1/+1
* fix OAI-PMH name/finished messageBryan Newbold2018-11-211-1/+6
* fix oai-pmh issue againBryan Newbold2018-11-211-13/+14
* oaipmh: handle NoRecordsMatchBryan Newbold2018-11-211-5/+8
* initial OAI-PMH harvestersBryan Newbold2018-11-193-5/+167
* better DOI registrar harvestersBryan Newbold2018-11-193-48/+145
* bunch of pylint cleanupBryan Newbold2018-11-151-7/+12
* refactoring harvestersBryan Newbold2018-11-155-196/+210
* initial work on metadata harvest botsBryan Newbold2018-11-144-0/+197
generated by cgit v1.2.3 (git 2.39.1) at 2024-11-09 08:23:19 +0000