Commit message (Collapse) | Author | Age | Files | Lines | |
---|---|---|---|---|---|
* | refactor kafka producer in crossref harvester | Bryan Newbold | 2019-12-06 | 1 | -21/+26 |
| | | | | | | | | producer creation/configuration should be happening in __init__() time, not 'daily' call. This specific refactor motivated by mocking out the producer in unit tests. | ||||
* | crossref is_update isn't what I thought | Bryan Newbold | 2019-12-03 | 1 | -6/+2 |
| | | | | | | | | I thought this would filter for metadata updates to an existing DOI, but actually "updates" are a type of DOI (eg, a retraction). TODO: handle 'updates' field. Should both do a lookup and set work_ident appropriately, and store in crossref-specific metadata. | ||||
* | review/fix all confluent-kafka produce code | Bryan Newbold | 2019-09-20 | 1 | -5/+8 |
| | |||||
* | small fixes to confluent-kafka importers/workers | Bryan Newbold | 2019-09-20 | 1 | -1/+1 |
| | | | | | | | | - decrease default changelog pipeline to 5.0sec - fix missing KafkaException harvester imports - more confluent-kafka tweaks - updates to kafka consumer configs - bump elastic updates consumergroup (again) | ||||
* | small kafka tweaks for robustness | Bryan Newbold | 2019-09-20 | 1 | -0/+2 |
| | |||||
* | bump max message size to ~20 MBytes | Bryan Newbold | 2019-09-20 | 1 | -0/+1 |
| | |||||
* | fixes to confluent-kafka harvesters | Bryan Newbold | 2019-09-20 | 1 | -10/+11 |
| | |||||
* | first draft harvesters using confluent-kafka | Bryan Newbold | 2019-09-20 | 1 | -26/+45 |
| | |||||
* | fix harvester session.get() params | Bryan Newbold | 2019-03-06 | 1 | -5/+8 |
| | |||||
* | retry/backoff for Crossref harvester | Bryan Newbold | 2019-03-06 | 1 | -2/+2 |
| | |||||
* | bunch of lint/whitespace cleanups | Bryan Newbold | 2019-02-22 | 1 | -2/+1 |
| | |||||
* | check request status codes idiomatically | Bryan Newbold | 2018-12-29 | 1 | -2/+2 |
| | |||||
* | clean up harvester comments/docs | Bryan Newbold | 2018-11-21 | 1 | -5/+1 |
| | |||||
* | use isoformat() to format dates | Bryan Newbold | 2018-11-21 | 1 | -2/+2 |
| | | | | This shouldn't change behavior; it's just more consistent. | ||||
* | fix loop_sleep typo | Bryan Newbold | 2018-11-21 | 1 | -1/+1 |
| | |||||
* | fix datacite DOI extraction | Bryan Newbold | 2018-11-21 | 1 | -1/+1 |
| | |||||
* | initial OAI-PMH harvesters | Bryan Newbold | 2018-11-19 | 1 | -5/+8 |
| | |||||
* | better DOI registrar harvesters | Bryan Newbold | 2018-11-19 | 1 | -48/+20 |
| | |||||
* | bunch of pylint cleanup | Bryan Newbold | 2018-11-15 | 1 | -7/+12 |
| | |||||
* | refactoring harvesters | Bryan Newbold | 2018-11-15 | 1 | -0/+209 |