sandcrawler/kafka, branch master [no description] https://git.bnewbold.net/sandcrawler/atom?h=master 2021-10-26T18:51:35+00:00 kafka monitoring commands 2021-10-26T18:51:35+00:00 Bryan Newbold bnewbold@archive.org 2021-10-26T18:51:35+00:00 urn:sha1:fb36ee6e0f5582695a385b0c106b9c48486eb233 new 'daily' and 'priority' ingest request topics 2021-09-30T22:24:24+00:00 Bryan Newbold bnewbold@archive.org 2021-09-30T22:24:22+00:00 urn:sha1:e4800fc4d0d0467d0e34a4059b941d001916e232 The old ingest request queue was always getting lopsided, suspect because it was scaled up (additional partitions) at some point in the past, hoping new topics will fix this. New '-priority' queue is like '-bulk', but for smaller-volume SPN-like requests. Eg, interactive mode. kafka: delete unused work-updates topic 2021-09-14T02:40:11+00:00 Bryan Newbold bnewbold@archive.org 2021-09-14T02:40:11+00:00 urn:sha1:1c43b0d2a663815c7cb43c918933588f5184c714 kafka re-balancing tweaks 2021-09-03T19:23:41+00:00 Bryan Newbold bnewbold@archive.org 2021-09-03T19:23:41+00:00 urn:sha1:cf1bf8001d426c41143436cb578dc64d67d1ca0f kafka docs for rolling back a consumer group 2020-11-20T19:02:54+00:00 Bryan Newbold bnewbold@archive.org 2020-11-20T19:02:54+00:00 urn:sha1:ed5ef9801fc94ab20defc77ade53b585f695aa6c Merge branch 'bnewbold-html-ingest' 2020-11-07T02:32:35+00:00 Bryan Newbold bnewbold@archive.org 2020-11-07T02:32:35+00:00 urn:sha1:175019c96fced3e21d0f60ea1a4a37da6b8872ac kafka: new XML+HTML topics 2020-11-05T01:07:46+00:00 Bryan Newbold bnewbold@archive.org 2020-11-05T01:07:46+00:00 urn:sha1:2fdba24da0e0bf3d300cfb959514bf57a3cf6701 kafka topics for fatcat -> scholar pipeline 2020-10-27T22:54:10+00:00 Bryan Newbold bnewbold@archive.org 2020-10-27T22:54:10+00:00 urn:sha1:8e30d5ff73703a74c939b398e8c73b6f43c87fe0 PDF extraction kafka topics 2020-06-25T20:00:23+00:00 Bryan Newbold bnewbold@archive.org 2020-06-25T20:00:23+00:00 urn:sha1:0c0585fc83bb155519c6e00c5c67920d2972116f kafka: more reblance notes 2020-04-24T23:12:13+00:00 Bryan Newbold bnewbold@archive.org 2020-04-24T23:12:13+00:00 urn:sha1:a8d76d29c23b9aaf32fe531e56244bb3422a23aa