From d5e2af24cb6563eca91407425cea9b808a7d691c Mon Sep 17 00:00:00 2001 From: Bryan Newbold Date: Tue, 12 Nov 2019 18:57:24 -0800 Subject: notes on search query parsing (WIP) --- proposals/20190911_search_query_parsing.md | 22 ++++++++++++++++++++++ 1 file changed, 22 insertions(+) create mode 100644 proposals/20190911_search_query_parsing.md diff --git a/proposals/20190911_search_query_parsing.md b/proposals/20190911_search_query_parsing.md new file mode 100644 index 00000000..1e656fef --- /dev/null +++ b/proposals/20190911_search_query_parsing.md @@ -0,0 +1,22 @@ + +status: work-in-progress + +The default "release" search on fatcat.wiki currently uses the elasticsearch +built-in `query_string` parser, which is explicitly not recommended for +public/production use. + +The best way forward is likely a custom query parser (eg, PEG-generated parser) +that generates a complete elasticsearch query JSON structure. + +A couple search issues this would help with: + +- better parsing of keywords (year, year-range, DOI, ISSN, etc) in complex + queries and turning these in to keyword term sub-queries +- queries including terms from multiple fields which aren't explicitly tagged + (eg, "lovelace computer" vs. "author:lovelace title:computer") +- avoiding unsustainably expensive queries (eg, prefix wildcard, regex) +- handling single-character mispellings and synonyms +- collapsing multiple releases under the same work in search results + +In the near future, we may also create a fulltext search index, which will have +it's own issues. -- cgit v1.2.3