From 6a97a067aa967d681c112f7d2ea1e02e038189ee Mon Sep 17 00:00:00 2001 From: Martin Czygan Date: Thu, 8 Jul 2021 17:48:39 +0200 Subject: notes on matching metrics --- notes/matching_metrics.md | 16 ++++++++++++++++ 1 file changed, 16 insertions(+) create mode 100644 notes/matching_metrics.md diff --git a/notes/matching_metrics.md b/notes/matching_metrics.md new file mode 100644 index 0000000..d37240f --- /dev/null +++ b/notes/matching_metrics.md @@ -0,0 +1,16 @@ +# Matching Metrics + +## Precision/Recall + +For fuzzy matching we want to understand precision and recall. Options for test datasets: + +* manually curated (100s of examples); could determine +* autogenerate slightly different set of real-world metadata (e.g. crossref vs. doaj) converted to releases +* automatically distorted set of records; 1 original, plus N distorted (synthetic) + +## Overall numbers + +* number of clusters per clustering method: "title", "lowercase", "nysiis", + "sandcrawler", a few more - contrastive comparison of these cluster, e.g. how +many more matches/non-matches we get for the various methods +* take N docs from non-clusters and run verify; we would want 100% different/ambiguous results -- cgit v1.2.3