diff options
author | Bryan Newbold <bnewbold@robocracy.org> | 2020-01-21 10:48:16 -0800 |
---|---|---|
committer | Bryan Newbold <bnewbold@robocracy.org> | 2020-01-21 10:52:43 -0800 |
commit | 2fcc59388a4eb53a7e2370275366272459874e99 (patch) | |
tree | 7d092d73dae564f39f68ca4cfb91b9c245ae7eca /proposals/20191018_bigger_db.md | |
parent | f7b6b8b3cda32b258ea21518a42bea41bd87532d (diff) | |
download | fatcat-2fcc59388a4eb53a7e2370275366272459874e99.tar.gz fatcat-2fcc59388a4eb53a7e2370275366272459874e99.zip |
refactor fatcat_import kafka group names
My current understanding is that consumer group names should be
one-to-one with topic names. I previously though offsets were stored on
a {topic, group} key, but they seem to be mixed and having too many
workers in the same group is bad. In particular, we don't want
cross-talk or load between QA and prod.
All these topics are caught up in prod, so deploying this change and
restarting workers should be safe.
This commit does not update the elasticsearch or entity updates workers.
Diffstat (limited to 'proposals/20191018_bigger_db.md')
0 files changed, 0 insertions, 0 deletions