Commit message (Collapse) | Author | Age | Files | Lines | |
---|---|---|---|---|---|
* | move container_status ES query code from fatcat_web to fatcat_tools | Bryan Newbold | 2022-02-09 | 1 | -2/+2 |
| | | | | | | The main motivation is to never have fatcat_tools import from fatcat_web, only vica-versa. Some code in fatcat_tools needs container stats, so starting with that code path (plus some generic helpers). | ||||
* | small default config typo fixes for elasticsearch workers | Bryan Newbold | 2021-12-15 | 1 | -2/+2 |
| | |||||
* | file elasticsearch index worker | Bryan Newbold | 2021-12-15 | 1 | -1/+34 |
| | |||||
* | typing: add annotations to remaining fatcat_tools code | Bryan Newbold | 2021-11-03 | 1 | -31/+32 |
| | | | | | Again, these are just annotations, no changes made to get type checks to pass | ||||
* | re-fix some lint issues after big 'fmt' | Bryan Newbold | 2021-11-02 | 1 | -2/+3 |
| | |||||
* | fmt (black): fatcat_tools/ | Bryan Newbold | 2021-11-02 | 1 | -61/+110 |
| | |||||
* | hacks to work around new pylint false positives | Bryan Newbold | 2021-11-02 | 1 | -2/+3 |
| | |||||
* | cleanup imports after fatcat_tools.transforms change | Bryan Newbold | 2021-11-02 | 1 | -5/+8 |
| | |||||
* | es worker: ensure kafka messages get cleared | Bryan Newbold | 2021-04-12 | 1 | -0/+2 |
| | |||||
* | es indexing: more 'wip' fixes | Bryan Newbold | 2021-04-12 | 1 | -1/+5 |
| | |||||
* | ES indexing: skip 'wip' entities with a warning | Bryan Newbold | 2021-04-12 | 1 | -11/+16 |
| | |||||
* | container ES index worker: support for querying status | Bryan Newbold | 2021-04-06 | 1 | -5/+32 |
| | |||||
* | indexing: don't use document names | Bryan Newbold | 2021-04-06 | 1 | -14/+4 |
| | |||||
* | lint (flake8) tool python files | Bryan Newbold | 2020-07-01 | 1 | -2/+0 |
| | |||||
* | more changelog ES fixes | Bryan Newbold | 2020-04-17 | 1 | -4/+6 |
| | |||||
* | ES changelog worker: fixes for ident; fetch update from API if needed | Bryan Newbold | 2020-04-17 | 1 | -2/+9 |
| | | | | | The API fetch update may be needed for old changelog entries in the kafka feed. | ||||
* | derive changelog worker from release worker | Martin Czygan | 2020-04-17 | 1 | -1/+22 |
| | | | | | Early versions of changelog entries may not have all the fields required for the current transform. | ||||
* | fix KafkaError worker reporting for partition errors | Bryan Newbold | 2020-01-29 | 1 | -1/+1 |
| | |||||
* | review/fix all confluent-kafka produce code | Bryan Newbold | 2019-09-20 | 1 | -8/+14 |
| | |||||
* | small fixes to confluent-kafka importers/workers | Bryan Newbold | 2019-09-20 | 1 | -3/+30 |
| | | | | | | | | - decrease default changelog pipeline to 5.0sec - fix missing KafkaException harvester imports - more confluent-kafka tweaks - updates to kafka consumer configs - bump elastic updates consumergroup (again) | ||||
* | convert pipeline workers from pykafka to confluent-kafka | Bryan Newbold | 2019-09-20 | 1 | -29/+66 |
| | |||||
* | refactor all python source for client lib name | Bryan Newbold | 2019-09-05 | 1 | -1/+1 |
| | |||||
* | start new ES container worker kafka group | Bryan Newbold | 2019-07-31 | 1 | -0/+2 |
| | | | | | | | | The previous group seems to have gotten corrupted; my hypothesis is that this is due to pykafka being somewhat flakey, and am planning to move to librdkafka anyways. Re-indexing all the containers is pretty small/easy, so starting a new consumer group works find in this case; release indexer would be a bigger problem. | ||||
* | more links on new homepage | Bryan Newbold | 2019-06-19 | 1 | -1/+1 |
| | | | | | matching produce sizes. may want to tweak this config in the future for throughput. | ||||
* | fix and workaround container entities in release topic | Bryan Newbold | 2019-05-30 | 1 | -0/+6 |
| | |||||
* | fix syntax bugs (container elastic worker) | Bryan Newbold | 2019-05-30 | 1 | -5/+5 |
| | |||||
* | add container update elastic worker | Bryan Newbold | 2019-05-30 | 1 | -5/+25 |
| | |||||
* | elastic-release worker w/o API | Bryan Newbold | 2019-03-04 | 1 | -4/+4 |
| | | | | | Forgot that this worker really doesn't want/need any API connection at all; just an ApiClient to deserialize objects from Kafka. | ||||
* | fix elastic research worker api arg | Bryan Newbold | 2019-03-04 | 1 | -4/+3 |
| | |||||
* | fatcat -> fatcat_release ES index | Bryan Newbold | 2019-01-28 | 1 | -2/+3 |
| | |||||
* | check request status codes idiomatically | Bryan Newbold | 2018-12-29 | 1 | -1/+1 |
| | |||||
* | large refactor of python names/paths | Bryan Newbold | 2018-11-15 | 1 | -0/+52 |
- Add __init__.py files for fatcat_tools submodules, and use them in imports - Add a bunch of comments to files. - rename a number of classes and functions to be less verbose |