fatcat/python/fatcat_tools/transforms, branch bnewbold-doaj-article-harvest [no description] https://git.bnewbold.net/fatcat/atom?h=bnewbold-doaj-article-harvest 2022-07-12T23:17:57+00:00 ingest: DOAJ article URLs 2022-07-12T23:17:57+00:00 Bryan Newbold bnewbold@robocracy.org 2022-07-12T23:17:57+00:00 urn:sha1:1975b3267aa20925bb28598a57517a530458b538 container ES transform: include old extra.issne/p fields 2022-02-04T01:25:50+00:00 Bryan Newbold bnewbold@robocracy.org 2022-02-04T01:23:20+00:00 urn:sha1:301abbce765cbe5b905b7d1751819668d95801d0 These were removed prematurely. Not all containers have been updated to use these fields yet. ES container schema: add 'sim_pubid' and `ia_sim_collection` fields 2021-12-04T01:22:43+00:00 Bryan Newbold bnewbold@robocracy.org 2021-12-04T01:20:45+00:00 urn:sha1:9ad198b7a1e12504e3d4718e56edd0a7e8b5e61a ES transform: remove prototype microfilm links 2021-12-04T01:19:55+00:00 Bryan Newbold bnewbold@robocracy.org 2021-12-04T01:19:48+00:00 urn:sha1:b0ccb0b027fa3ee75f21f4e7b8d3037f04499961 This ended up being a feature in scholar.archive.org, not fatcat. ES release transform: handle redirected containers better 2021-11-25T02:16:36+00:00 Bryan Newbold bnewbold@robocracy.org 2021-11-25T02:16:34+00:00 urn:sha1:0584499e7887d8c1ff216b27652d28c8377c3a17 Despite the inline comment, we were not actually grabbing the "redirected" ident correctly, meaning some counts would not be accurate. content_scope: include in file ES schema and transform 2021-11-18T00:36:16+00:00 Bryan Newbold bnewbold@robocracy.org 2021-11-18T00:36:16+00:00 urn:sha1:09a1829a10caf9759762b41c36169c0b88cd5ed7 typing: first batch of python bulk type annotations 2021-11-03T19:31:07+00:00 Bryan Newbold bnewbold@robocracy.org 2021-11-03T19:29:39+00:00 urn:sha1:10a2374051568edf3d872988e730328d899a0fdd While these changes are more delicate than simple lint changes, this specific batch of edits and annotations was *relatively* simple, and resulted in few code changes other than function signature additions. lint: resolve existing mypy type errors 2021-11-03T02:51:51+00:00 Bryan Newbold bnewbold@robocracy.org 2021-11-03T02:51:48+00:00 urn:sha1:4c77bdb8d92523935454f1c406c954913f923c01 Adds annotations and re-workes dataflow to satisfy existing mypy issues, without adding any additional type annotations to, eg, function signatures. There will probably be many more type errors when annotations are all added. fmt (black): fatcat_tools/ 2021-11-03T01:14:59+00:00 Bryan Newbold bnewbold@robocracy.org 2021-11-03T01:14:59+00:00 urn:sha1:31d1a6a713d177990609767d508209ced19ca396 python: isort everything 2021-11-03T01:05:23+00:00 Bryan Newbold bnewbold@robocracy.org 2021-11-03T01:05:23+00:00 urn:sha1:78f08280edea4ff65ca613ad30005c45cc48dea6