notes on search query parsing (WIP)

1 files changed, 22 insertions, 0 deletions

diff --git a/proposals/20190911_search_query_parsing.md b/proposals/20190911_search_query_parsing.md
new file mode 100644
index 00000000..1e656fef
--- /dev/null
+++ b/proposals/20190911_search_query_parsing.md
@@ -0,0 +1,22 @@
+
+status: work-in-progress
+
+The default "release" search on fatcat.wiki currently uses the elasticsearch
+built-in `query_string` parser, which is explicitly not recommended for
+public/production use.
+
+The best way forward is likely a custom query parser (eg, PEG-generated parser)
+that generates a complete elasticsearch query JSON structure.
+
+A couple search issues this would help with:
+
+- better parsing of keywords (year, year-range, DOI, ISSN, etc) in complex
+  queries and turning these in to keyword term sub-queries
+- queries including terms from multiple fields which aren't explicitly tagged
+  (eg, "lovelace computer" vs. "author:lovelace title:computer")
+- avoiding unsustainably expensive queries (eg, prefix wildcard, regex)
+- handling single-character mispellings and synonyms
+- collapsing multiple releases under the same work in search results
+
+In the near future, we may also create a fulltext search index, which will have
+it's own issues.