Commit message (Collapse) | Author | Age | Files | Lines | |
---|---|---|---|---|---|
* | python: isort everything | Bryan Newbold | 2021-11-02 | 1 | -2/+9 |
| | |||||
* | default ingest request topic now '-daily'; configurable for ingest_tool.py | Bryan Newbold | 2021-09-30 | 1 | -1/+1 |
| | |||||
* | container ES index worker: support for querying status | Bryan Newbold | 2021-04-06 | 1 | -0/+5 |
| | |||||
* | entity updater: new work update feed (ident and changelog metadata only) | Bryan Newbold | 2020-10-16 | 1 | -0/+2 |
| | |||||
* | lint (flake8) top-level python files | Bryan Newbold | 2020-07-01 | 1 | -1/+0 |
| | |||||
* | derive changelog worker from release worker | Martin Czygan | 2020-04-17 | 1 | -1/+18 |
| | | | | | Early versions of changelog entries may not have all the fields required for the current transform. | ||||
* | improve argparse usage | Bryan Newbold | 2019-12-11 | 1 | -8/+10 |
| | | | | | | | | | | | | | | Use --fatcat-api-url instead of (ambiguous) --host-url for commands that aren't deployed/running via systemd. TODO: update the other --host-url usage, and either roll-out change consistently or support the old arg as an alias during cut-over Use argparse.ArgumentDefaultsHelpFormatter (thanks Martin!) Add help messages for all sub-commands, both as documentation and as a way to get argparse to print available commands in a more readable format. | ||||
* | add ingest request feature to entity_updates worker | Bryan Newbold | 2019-11-15 | 1 | -0/+2 |
| | | | | | | | | | | | | | Initially was going to create a new worker to consume from the release update channel, but couldn't get the edit context ("is this a new release, or update to an existing") from that context. Currently there is a flag in source code to control whether we only do OA releases or all releases. Starting with OA only to start slow, but should probably default to all, and make this a config flag. Should probably also have a config flag to control this entire feature. Tested locally in dev. | ||||
* | small fixes to confluent-kafka importers/workers | Bryan Newbold | 2019-09-20 | 1 | -1/+1 |
| | | | | | | | | - decrease default changelog pipeline to 5.0sec - fix missing KafkaException harvester imports - more confluent-kafka tweaks - updates to kafka consumer configs - bump elastic updates consumergroup (again) | ||||
* | make default kafka env 'dev', not 'qa' | Bryan Newbold | 2019-09-20 | 1 | -2/+2 |
| | |||||
* | fix syntax bugs (container elastic worker) | Bryan Newbold | 2019-05-30 | 1 | -1/+1 |
| | |||||
* | add container update elastic worker | Bryan Newbold | 2019-05-30 | 1 | -0/+16 |
| | |||||
* | file and container update kafka topics | Bryan Newbold | 2019-05-30 | 1 | -2/+8 |
| | |||||
* | push elastic release updates to new (schema) index | Bryan Newbold | 2019-05-24 | 1 | -1/+1 |
| | |||||
* | new kafka topic for release obj w/ v0.3 schema | Bryan Newbold | 2019-05-23 | 1 | -2/+2 |
| | |||||
* | elastic-release worker w/o API | Bryan Newbold | 2019-03-04 | 1 | -1/+1 |
| | | | | | Forgot that this worker really doesn't want/need any API connection at all; just an ApiClient to deserialize objects from Kafka. | ||||
* | fix (untested) ElasticsearchReleaseWorker invocation | Bryan Newbold | 2019-03-04 | 1 | -1/+1 |
| | |||||
* | bunch of lint/whitespace cleanups | Bryan Newbold | 2019-02-22 | 1 | -1/+2 |
| | |||||
* | more ES index fixes | Bryan Newbold | 2019-01-28 | 1 | -1/+1 |
| | |||||
* | workers do API-passing (not URI-passing) | Bryan Newbold | 2019-01-08 | 1 | -5/+5 |
| | |||||
* | start refactoring API object passing | Bryan Newbold | 2019-01-08 | 1 | -2/+9 |
| | |||||
* | bunch of pylint cleanup | Bryan Newbold | 2018-11-15 | 1 | -1/+1 |
| | |||||
* | large refactor of python names/paths | Bryan Newbold | 2018-11-15 | 1 | -12/+12 |
| | | | | | | | - Add __init__.py files for fatcat_tools submodules, and use them in imports - Add a bunch of comments to files. - rename a number of classes and functions to be less verbose | ||||
* | refactoring harvesters | Bryan Newbold | 2018-11-15 | 1 | -36/+0 |
| | |||||
* | initial work on metadata harvest bots | Bryan Newbold | 2018-11-14 | 1 | -6/+43 |
| | |||||
* | elasticsearch config for worker | Bryan Newbold | 2018-11-13 | 1 | -1/+8 |
| | |||||
* | shuffle around fatcat_tools layout | Bryan Newbold | 2018-11-13 | 1 | -2/+2 |
| | |||||
* | more python module refactoring | Bryan Newbold | 2018-11-12 | 1 | -2/+2 |
| | |||||
* | elastic release worker | Bryan Newbold | 2018-11-04 | 1 | -0/+10 |
| | |||||
* | first-draft kafka workers (changelog, release_update) | Bryan Newbold | 2018-11-04 | 1 | -0/+52 |