| Commit message (Collapse) | Author | Age | Files | Lines | |
|---|---|---|---|---|---|
| * | add some more domain/rel URL mappings | Bryan Newbold | 2020-02-22 | 1 | -0/+9 |
| | | |||||
| * | fix KafkaError worker reporting for partition errors | Bryan Newbold | 2020-01-29 | 1 | -1/+1 |
| | | |||||
| * | importers: control update behavior with more-standard flag | Bryan Newbold | 2020-01-06 | 1 | -0/+1 |
| | | |||||
| * | write diagnostic messages to stderr | Martin Czygan | 2019-12-16 | 1 | -2/+2 |
| | | | | | | During debugging, it can be helpful to keep stdout (e.g. processing results) and dignostic messages separate. | ||||
| * | Merge branch 'martin-importers-common-doc-fix' into 'master' | Martin Czygan | 2019-12-14 | 1 | -13/+10 |
| |\ | | | | | | | | | Update EntityImporter docstring. See merge request webgroup/fatcat!9 | ||||
| | * | complete parse_record docstring | Martin Czygan | 2019-12-14 | 1 | -0/+6 |
| | | | |||||
| | * | Update EntityImporter docstring. | Martin Czygan | 2019-12-13 | 1 | -13/+4 |
| | | | | | | | | | I believe the required method is `parse_record`, not `parse`. | ||||
| * | | revert accidentally commited test timing | Bryan Newbold | 2019-12-13 | 1 | -2/+2 |
| | | | | | | | | | Also fix a spurious typo. | ||||
| * | | ensure importer description arg isn't clobbered | Bryan Newbold | 2019-12-12 | 1 | -1/+3 |
| | | | |||||
| * | | flush importer editgroups every few minutes | Bryan Newbold | 2019-12-12 | 1 | -5/+20 |
| | | | |||||
| * | | EntityImporter: submit (not accept) mode | Bryan Newbold | 2019-12-12 | 1 | -2/+14 |
| |/ | | | | | For use with bots that don't have admin privileges, or where human follow-up review is desired. | ||||
| * | crude support for 'sandcrawler' kafka topics | Bryan Newbold | 2019-11-15 | 1 | -2/+3 |
| | | |||||
| * | refactor duplicated b32_hex function in importers | Bryan Newbold | 2019-10-08 | 1 | -0/+9 |
| | | |||||
| * | review/fix all confluent-kafka produce code | Bryan Newbold | 2019-09-20 | 1 | -1/+0 |
| | | |||||
| * | small fixes to confluent-kafka importers/workers | Bryan Newbold | 2019-09-20 | 1 | -10/+24 |
| | | | | | | | | | - decrease default changelog pipeline to 5.0sec - fix missing KafkaException harvester imports - more confluent-kafka tweaks - updates to kafka consumer configs - bump elastic updates consumergroup (again) | ||||
| * | small kafka tweaks for robustness | Bryan Newbold | 2019-09-20 | 1 | -0/+3 |
| | | |||||
| * | convert importers to confluent-kafka library | Bryan Newbold | 2019-09-20 | 1 | -19/+71 |
| | | |||||
| * | refactor all python source for client lib name | Bryan Newbold | 2019-09-05 | 1 | -3/+3 |
| | | |||||
| * | fix Importer editgroup_extra pass-through | Bryan Newbold | 2019-09-05 | 1 | -2/+1 |
| | | |||||
| * | file rel: social -> academicsocial | Bryan Newbold | 2019-09-03 | 1 | -2/+2 |
| | | |||||
| * | better importer 'total' counting | Bryan Newbold | 2019-09-03 | 1 | -4/+2 |
| | | |||||
| * | make importer extid lookups faster by hiding | Bryan Newbold | 2019-05-29 | 1 | -2/+2 |
| | | |||||
| * | is_cjk() handles kanji better | Bryan Newbold | 2019-05-29 | 1 | -4/+6 |
| | | |||||
| * | faster LargeFile XML importer for PubMed | Bryan Newbold | 2019-05-29 | 1 | -0/+50 |
| | | |||||
| * | more MARC languages, and less verbose reporting | Bryan Newbold | 2019-05-24 | 1 | -3/+14 |
| | | |||||
| * | missing MARC/ISO languages | Bryan Newbold | 2019-05-22 | 1 | -0/+2 |
| | | |||||
| * | Gaelic! | Bryan Newbold | 2019-05-22 | 1 | -0/+3 |
| | | |||||
| * | creative importer for bulk JSTOR imports | Bryan Newbold | 2019-05-22 | 1 | -0/+22 |
| | | |||||
| * | bs4 XML parse cleanup | Bryan Newbold | 2019-05-22 | 1 | -0/+2 |
| | | |||||
| * | JALC bulk file importer | Bryan Newbold | 2019-05-21 | 1 | -0/+20 |
| | | |||||
| * | updates to pubmed importer | Bryan Newbold | 2019-05-21 | 1 | -1/+20 |
| | | |||||
| * | tweaks to new imports/tests | Bryan Newbold | 2019-05-21 | 1 | -5/+78 |
| | | |||||
| * | initial flesh out of JALC parser | Bryan Newbold | 2019-05-21 | 1 | -0/+36 |
| | | |||||
| * | python impl | Bryan Newbold | 2019-05-14 | 1 | -3/+3 |
| | | |||||
| * | add limits to match importers | Bryan Newbold | 2019-04-23 | 1 | -0/+3 |
| | | |||||
| * | archive.org isn't really a repository | Bryan Newbold | 2019-04-22 | 1 | -1/+3 |
| | | |||||
| * | mechanism to not double-update entities | Bryan Newbold | 2019-04-18 | 1 | -0/+3 |
| | | |||||
| * | update URL rel list | Bryan Newbold | 2019-04-18 | 1 | -1/+10 |
| | | |||||
| * | add SqlitePusher importer option | Bryan Newbold | 2019-04-12 | 1 | -0/+20 |
| | | |||||
| * | bunch of lint/whitespace cleanups | Bryan Newbold | 2019-02-22 | 1 | -2/+2 |
| | | |||||
| * | fix bug in clean() resulting in many consistency check fails | Bryan Newbold | 2019-01-29 | 1 | -2/+3 |
| | | |||||
| * | add stub parse_record() to make pylint happy | Bryan Newbold | 2019-01-28 | 1 | -0/+4 |
| | | |||||
| * | don't allow empty or single-character clean strings | Bryan Newbold | 2019-01-28 | 1 | -1/+1 |
| | | |||||
| * | transform and import fixes/tweaks | Bryan Newbold | 2019-01-25 | 1 | -2/+2 |
| | | |||||
| * | refactor _get_editgroup => get_editgroup_id | Bryan Newbold | 2019-01-24 | 1 | -4/+5 |
| | | |||||
| * | refactor make_rel_url | Bryan Newbold | 2019-01-24 | 1 | -0/+60 |
| | | |||||
| * | clean() checks if it returns null-length string | Bryan Newbold | 2019-01-23 | 1 | -1/+5 |
| | | |||||
| * | ftfy all over (needs Pipfile.lock) | Bryan Newbold | 2019-01-23 | 1 | -0/+31 |
| | | |||||
| * | more tests; fix some importer behavior | Bryan Newbold | 2019-01-23 | 1 | -10/+20 |
| | | |||||
| * | improve changelog tests | Bryan Newbold | 2019-01-23 | 1 | -1/+0 |
| | | |||||
 |
index : fatcat | |
| [no description] |
| summaryrefslogtreecommitdiffstats |