Commit message (Collapse) | Author | Age | Files | Lines | |
---|---|---|---|---|---|
* | small fixes to confluent-kafka importers/workers | Bryan Newbold | 2019-09-20 | 1 | -1/+1 |
| | | | | | | | | - decrease default changelog pipeline to 5.0sec - fix missing KafkaException harvester imports - more confluent-kafka tweaks - updates to kafka consumer configs - bump elastic updates consumergroup (again) | ||||
* | convert importers to confluent-kafka library | Bryan Newbold | 2019-09-20 | 1 | -2/+3 |
| | |||||
* | start chocula importer | Bryan Newbold | 2019-09-03 | 1 | -0/+14 |
| | |||||
* | support extids in matched importer | Bryan Newbold | 2019-06-20 | 1 | -0/+4 |
| | |||||
* | faster LargeFile XML importer for PubMed | Bryan Newbold | 2019-05-29 | 1 | -1/+1 |
| | |||||
* | make pubmed ref lookups configurable | Bryan Newbold | 2019-05-22 | 1 | -1/+8 |
| | |||||
* | creative importer for bulk JSTOR imports | Bryan Newbold | 2019-05-22 | 1 | -0/+18 |
| | |||||
* | pubmed importer command and tweaks | Bryan Newbold | 2019-05-22 | 1 | -0/+25 |
| | |||||
* | arxiv importer robustification and CLI impl | Bryan Newbold | 2019-05-21 | 1 | -0/+21 |
| | |||||
* | JALC bulk file importer | Bryan Newbold | 2019-05-21 | 1 | -0/+21 |
| | |||||
* | fix default mimetype (impacted pre-1923 files) | Bryan Newbold | 2019-05-15 | 1 | -1/+5 |
| | |||||
* | editgroup description override | Bryan Newbold | 2019-04-22 | 1 | -1/+11 |
| | |||||
* | minor arabesque tweaks | Bryan Newbold | 2019-04-18 | 1 | -12/+22 |
| | |||||
* | arabesque importer using crawl-bot creds | Bryan Newbold | 2019-04-18 | 1 | -1/+1 |
| | |||||
* | arabesque import tweaks | Bryan Newbold | 2019-04-18 | 1 | -0/+4 |
| | |||||
* | early version of arabesque importer | Bryan Newbold | 2019-04-12 | 1 | -0/+28 |
| | |||||
* | importer for CDL/DASH dat pilot dweb datasets | Bryan Newbold | 2019-03-19 | 1 | -1/+29 |
| | |||||
* | new importer: wayback_static | Bryan Newbold | 2019-03-19 | 1 | -0/+48 |
| | |||||
* | reduce default import batch size to 50 | Bryan Newbold | 2019-01-29 | 1 | -1/+1 |
| | |||||
* | batch size as a general import param | Bryan Newbold | 2019-01-28 | 1 | -13/+4 |
| | |||||
* | add missing bezerk-mode flag to GROBID import | Bryan Newbold | 2019-01-28 | 1 | -3/+8 |
| | |||||
* | fix typo in crossref importer | Bryan Newbold | 2019-01-28 | 1 | -1/+1 |
| | |||||
* | update journal meta import/transform | Bryan Newbold | 2019-01-25 | 1 | -3/+3 |
| | |||||
* | more import script fixes | Bryan Newbold | 2019-01-23 | 1 | -1/+4 |
| | |||||
* | update importer script | Bryan Newbold | 2019-01-23 | 1 | -33/+24 |
| | |||||
* | pubmed+datacite tokens; no journal,grobid,matched tokens | Bryan Newbold | 2019-01-22 | 1 | -2/+2 |
| | |||||
* | issn => journal_metadata in several places | Bryan Newbold | 2019-01-17 | 1 | -9/+9 |
| | |||||
* | start refactoring API object passing | Bryan Newbold | 2019-01-08 | 1 | -13/+36 |
| | |||||
* | crossref importer checks for existing DOIs | Bryan Newbold | 2018-11-21 | 1 | -3/+7 |
| | |||||
* | correct kafka topic names | Bryan Newbold | 2018-11-20 | 1 | -1/+1 |
| | |||||
* | start supporting kafka importers | Bryan Newbold | 2018-11-19 | 1 | -3/+17 |
| | | | | A nice feature would be some/any log output as to progress. | ||||
* | bunch of pylint cleanup | Bryan Newbold | 2018-11-15 | 1 | -1/+1 |
| | |||||
* | large refactor of python names/paths | Bryan Newbold | 2018-11-15 | 1 | -39/+37 |
| | | | | | | | - Add __init__.py files for fatcat_tools submodules, and use them in imports - Add a bunch of comments to files. - rename a number of classes and functions to be less verbose | ||||
* | shuffle around fatcat_tools layout | Bryan Newbold | 2018-11-13 | 1 | -5/+5 |
| | |||||
* | more python module refactoring | Bryan Newbold | 2018-11-12 | 1 | -5/+5 |
| | |||||
* | remove more old python cruft | Bryan Newbold | 2018-11-12 | 1 | -8/+0 |
| | |||||
* | fixes for grobid metadata importer | Bryan Newbold | 2018-09-28 | 1 | -0/+15 |
| | |||||
* | fix issues with extid mapping in crossref-importer | Bryan Newbold | 2018-09-20 | 1 | -1/+1 |
| | |||||
* | switch manifest importer to be json-based | Bryan Newbold | 2018-09-14 | 1 | -16/+2 |
| | |||||
* | add insert counting to importers | Bryan Newbold | 2018-09-14 | 1 | -0/+22 |
| | |||||
* | extid support for crossref importer | Bryan Newbold | 2018-09-12 | 1 | -2/+5 |
| | |||||
* | rename python scripts | Bryan Newbold | 2018-07-26 | 1 | -0/+94 |