fatcat/python/tests, branch v0.3.3 [no description] https://git.bnewbold.net/fatcat/atom?h=v0.3.3 2020-12-18T07:03:08+00:00 improve dblp release import 2020-12-18T07:03:08+00:00 Bryan Newbold bnewbold@robocracy.org 2020-12-17T09:56:27+00:00 urn:sha1:9451b3063c2d446748db74027c40c13ee69c24fb very simple dblp container importer 2020-12-18T07:03:08+00:00 Bryan Newbold bnewbold@robocracy.org 2020-12-17T09:55:30+00:00 urn:sha1:58ff361eb481bee9d2ef7249f48f94729d2a830d basic test coverage of dblp release importer 2020-12-18T07:03:08+00:00 Bryan Newbold bnewbold@robocracy.org 2020-12-02T19:30:03+00:00 urn:sha1:c66f9b2d98de88a98d3a1737d415bdab4e89027c add 'lxml' mode for large XML file import, and multi-tags 2020-12-18T07:03:08+00:00 Bryan Newbold bnewbold@robocracy.org 2020-12-02T18:49:35+00:00 urn:sha1:4e332e9037530ebc62836acfa78896dc76700c9c fix sloppy is_preserved ES transfom test failure 2020-12-18T07:03:00+00:00 Bryan Newbold bnewbold@robocracy.org 2020-12-18T07:03:00+00:00 urn:sha1:53b17b96c61ecfe14a30da523e993a9902d7f375 Merge branch 'bnewbold-doaj-fuzzy' into 'master' 2020-12-18T02:13:47+00:00 bnewbold bnewbold@archive.org 2020-12-18T02:13:47+00:00 urn:sha1:443243e8cccba3e779b7c56d0cdb6dcd992a3100 DOAJ import fuzzy match filter See merge request webgroup/fatcat!92 update fuzzy helper to pass 'reason' through to import code 2020-12-18T00:01:06+00:00 Bryan Newbold bnewbold@robocracy.org 2020-12-18T00:01:04+00:00 urn:sha1:5eeb7a9d61beb8cb40fd89bd91fcd9dd820035aa The motivation for this change is to enable passing the 'reason' through to edit extra metadata, in cases where we merge or cluster releases. add fuzzy match filtering to DOAJ importer 2020-12-17T04:16:09+00:00 Bryan Newbold bnewbold@robocracy.org 2020-12-17T03:56:01+00:00 urn:sha1:6d5811693c36b9e73dedf0205c40f2aed63e2870 In this default configuration, any entities with a fuzzy match (even "ambiguous") will be skipped at import time, to prevent creating duplicates. This is conservative towards not creating new/duplicate entities. In the future, as we get more confidence in fuzzy match/verification, we can start to ignore AMBIGUOUS, handle EXACT as same release, and merge STRONG (and WEAK?) matches under the same work entity. add fuzzy matching helper to importer base class 2020-12-17T04:16:09+00:00 Bryan Newbold bnewbold@robocracy.org 2020-12-17T03:54:47+00:00 urn:sha1:38328c25674fee7781a8d8601e1d47de04186f19 Using fuzzycat. Add basic test coverage. improve release elasticsearch transform test coverage 2020-12-16T22:33:52+00:00 Bryan Newbold bnewbold@robocracy.org 2020-12-16T22:33:52+00:00 urn:sha1:ebcc86561dabf3974ca11151445e66c0df4431f1