diff options
| author | Bryan Newbold <bnewbold@robocracy.org> | 2019-11-12 21:14:32 -0800 |
|---|---|---|
| committer | Bryan Newbold <bnewbold@robocracy.org> | 2019-11-15 16:46:26 -0800 |
| commit | 9d2bde08e9f43942b76ca6d0f6c8845c33c02515 (patch) | |
| tree | 38f54a2ebe210b97289a6ed6463ef07be8122a9b /python/fatcat_worker.py | |
| parent | 06cf64414d4c7fa497c9ddb83b7c066b3779c4d2 (diff) | |
| download | fatcat-9d2bde08e9f43942b76ca6d0f6c8845c33c02515.tar.gz fatcat-9d2bde08e9f43942b76ca6d0f6c8845c33c02515.zip | |
add ingest request feature to entity_updates worker
Initially was going to create a new worker to consume from the release
update channel, but couldn't get the edit context ("is this a new
release, or update to an existing") from that context.
Currently there is a flag in source code to control whether we only do
OA releases or all releases. Starting with OA only to start slow, but
should probably default to all, and make this a config flag. Should
probably also have a config flag to control this entire feature.
Tested locally in dev.
Diffstat (limited to 'python/fatcat_worker.py')
| -rwxr-xr-x | python/fatcat_worker.py | 2 |
1 files changed, 2 insertions, 0 deletions
diff --git a/python/fatcat_worker.py b/python/fatcat_worker.py index 628312be..c2120bae 100755 --- a/python/fatcat_worker.py +++ b/python/fatcat_worker.py @@ -23,11 +23,13 @@ def run_entity_updates(args): release_topic = "fatcat-{}.release-updates-v03".format(args.env) file_topic = "fatcat-{}.file-updates".format(args.env) container_topic = "fatcat-{}.container-updates".format(args.env) + ingest_file_request_topic = "sandcrawler-{}.ingest-file-requests".format(args.env) worker = EntityUpdatesWorker(args.api, args.kafka_hosts, changelog_topic, release_topic=release_topic, file_topic=file_topic, container_topic=container_topic, + ingest_file_request_topic=ingest_file_request_topic, ) worker.run() |