fatcat/python/fatcat_tools/workers, branch v0.3.4 [no description] https://git.bnewbold.net/fatcat/atom?h=v0.3.4 2021-04-12T22:27:34+00:00 es worker: ensure kafka messages get cleared 2021-04-12T22:27:34+00:00 Bryan Newbold bnewbold@robocracy.org 2021-04-12T22:27:34+00:00 urn:sha1:83ae1d85d093f1f37bba74283b1d689bb1f1c346 es indexing: more 'wip' fixes 2021-04-12T17:51:31+00:00 Bryan Newbold bnewbold@robocracy.org 2021-04-12T17:51:31+00:00 urn:sha1:389f391f42442c5df1f91d7cc3243e7c7d561909 ES indexing: skip 'wip' entities with a warning 2021-04-12T17:35:53+00:00 Bryan Newbold bnewbold@robocracy.org 2021-04-12T17:35:53+00:00 urn:sha1:b61e2e020a9249ab04be7484f4159a9dc1ca4a83 container ES index worker: support for querying status 2021-04-07T04:58:54+00:00 Bryan Newbold bnewbold@robocracy.org 2021-04-07T04:48:40+00:00 urn:sha1:9f110393b90d5b9e95a39b4f83d3e864434dd189 indexing: don't use document names 2021-04-07T04:53:49+00:00 Bryan Newbold bnewbold@robocracy.org 2021-02-27T04:13:07+00:00 urn:sha1:913f1b845ea9763e301db924199c7db8356624aa entity update worker: treat fileset and webcapture updates like file updates 2020-12-16T23:00:18+00:00 Bryan Newbold bnewbold@robocracy.org 2020-12-16T22:58:07+00:00 urn:sha1:f60ba0ea04081ac0095c12d8ecbaa48b3da74aee When webcapture or fileset entities are updated, then the release entities associated with them also need to be updated (and work entities, recursively). A TODO is to handle the case where a release_id is *removed* as well as *added*, and reprocess the releases in that case as well. entity updates: don't ingest JSTOR DOI prefixes 2020-10-23T20:17:34+00:00 Bryan Newbold bnewbold@robocracy.org 2020-10-23T20:17:34+00:00 urn:sha1:bf77adc854022213951daec14bd904f483f21202 entity updater: new work update feed (ident and changelog metadata only) 2020-10-16T23:41:16+00:00 Bryan Newbold bnewbold@robocracy.org 2020-10-16T03:53:11+00:00 urn:sha1:99c54764f0cbbb05409001b4af83182842f4e52d ingest: default to crawl protocols.io DOIs 2020-09-11T01:49:35+00:00 Bryan Newbold bnewbold@robocracy.org 2020-09-11T01:49:35+00:00 urn:sha1:3bac4de14219b5d8a3ef8f5919c4f004663424c0 entity updater: handle doi=None case better 2020-08-14T23:09:32+00:00 Bryan Newbold bnewbold@robocracy.org 2020-08-14T23:09:32+00:00 urn:sha1:e024a38bd29686eaa687e54cde1dc255ba6af52b