Preserve Meilisearch’s result ranking #772

Merged
floatingghost merged 4 commits from Oneric/akkoma:search-meili-order into develop 2024-05-31 14:12:06 +00:00
Member

Supersedes

Meilisearch was easier to setup than initially presumed so i was now able to test this myself. Previously top results were reorderd by date, now Meilisearch’s ranking is respected; note how reordering pushed the top Meilisearch result down to the end of the list:

# BEFORE PATCH
MeiliRanking:
["http://localhost:4000/objects/3b72d9bd-e821-466a-8ace-c5244bc03bce", "http://localhost:4000/objects/7bd90971-2ffc-477a-b0d9-649a82766c7c", "http://localhost:4000/objects/3e1e7b34-ee22-4592-ba0c-6bded82ba6cc", "http://localhost:4000/objects/bb92b7c4-2ce6-4e83-b2e7-b21b40239153"]

RetrivedRes:
["http://localhost:4000/objects/7bd90971-2ffc-477a-b0d9-649a82766c7c", "http://localhost:4000/objects/3e1e7b34-ee22-4592-ba0c-6bded82ba6cc", "http://localhost:4000/objects/bb92b7c4-2ce6-4e83-b2e7-b21b40239153", "http://localhost:4000/objects/3b72d9bd-e821-466a-8ace-c5244bc03bce"]


# AFTER PATCH
MeiliRanking:
["http://localhost:4000/objects/3b72d9bd-e821-466a-8ace-c5244bc03bce", "http://localhost:4000/objects/7bd90971-2ffc-477a-b0d9-649a82766c7c", "http://localhost:4000/objects/3e1e7b34-ee22-4592-ba0c-6bded82ba6cc", "http://localhost:4000/objects/bb92b7c4-2ce6-4e83-b2e7-b21b40239153"]

RetrivedRes:
["http://localhost:4000/objects/3b72d9bd-e821-466a-8ace-c5244bc03bce", "http://localhost:4000/objects/7bd90971-2ffc-477a-b0d9-649a82766c7c", "http://localhost:4000/objects/3e1e7b34-ee22-4592-ba0c-6bded82ba6cc", "http://localhost:4000/objects/bb92b7c4-2ce6-4e83-b2e7-b21b40239153"]

Compared to this avoids a costly O(n²) resort in Elixir by just retrieving objects in the right order to begin with.

Side note: Meilisearch’s ranking for this particular query almost matches what ORDER BY ts_rank_cd(vector, query, 16) would yield if it was used in the default GIN search.

Supersedes #579 Meilisearch was easier to setup than initially presumed so i was now able to test this myself. Previously top results were reorderd by date, now Meilisearch’s ranking is respected; note how reordering pushed the top Meilisearch result down to the end of the list: ``` # BEFORE PATCH MeiliRanking: ["http://localhost:4000/objects/3b72d9bd-e821-466a-8ace-c5244bc03bce", "http://localhost:4000/objects/7bd90971-2ffc-477a-b0d9-649a82766c7c", "http://localhost:4000/objects/3e1e7b34-ee22-4592-ba0c-6bded82ba6cc", "http://localhost:4000/objects/bb92b7c4-2ce6-4e83-b2e7-b21b40239153"] RetrivedRes: ["http://localhost:4000/objects/7bd90971-2ffc-477a-b0d9-649a82766c7c", "http://localhost:4000/objects/3e1e7b34-ee22-4592-ba0c-6bded82ba6cc", "http://localhost:4000/objects/bb92b7c4-2ce6-4e83-b2e7-b21b40239153", "http://localhost:4000/objects/3b72d9bd-e821-466a-8ace-c5244bc03bce"] # AFTER PATCH MeiliRanking: ["http://localhost:4000/objects/3b72d9bd-e821-466a-8ace-c5244bc03bce", "http://localhost:4000/objects/7bd90971-2ffc-477a-b0d9-649a82766c7c", "http://localhost:4000/objects/3e1e7b34-ee22-4592-ba0c-6bded82ba6cc", "http://localhost:4000/objects/bb92b7c4-2ce6-4e83-b2e7-b21b40239153"] RetrivedRes: ["http://localhost:4000/objects/3b72d9bd-e821-466a-8ace-c5244bc03bce", "http://localhost:4000/objects/7bd90971-2ffc-477a-b0d9-649a82766c7c", "http://localhost:4000/objects/3e1e7b34-ee22-4592-ba0c-6bded82ba6cc", "http://localhost:4000/objects/bb92b7c4-2ce6-4e83-b2e7-b21b40239153"] ``` Compared to #579 this avoids a costly O(n²) resort in Elixir by just retrieving objects in the right order to begin with. Side note: Meilisearch’s ranking for this particular query almost matches what `ORDER BY ts_rank_cd(vector, query, 16)` would yield if it was used in the default GIN search.
norm reviewed 2024-05-15 03:02:59 +00:00
@ -59,2 +59,3 @@
You will see a "Default Admin API Key", this is the key you actually put into your configuration file.
Depending on your Meilisearch version, you will either see a "Default Admin API Key" or some line starting with "Use it for anything that is not a search operation. Caution! Do not expose it on a public frontend:".
This is the key you actually put into your configuration file.
Contributor

Wondering if it would make more sense to limit the admin API key to just admin tasks and use a separate key for just searches done via our APIs..

Wondering if it would make more sense to limit the admin API key to just admin tasks and use a separate key for just searches done via our APIs..
Author
Member

repusehd, using separate search and admin keys is now possible and instead of changing documentation to match the output of our show-keys task, show-keys is changed to display Meilisearch's name parameter if present instead of description.

repusehd, using separate search and admin keys is now possible and instead of changing documentation to match the output of our `show-keys` task, `show-keys` is changed to display Meilisearch's `name` parameter if present instead of `description`.
Oneric marked this conversation as resolved
Oneric force-pushed search-meili-order from 560c5a4b5d to e38fa81518 2024-05-15 05:10:54 +00:00 Compare
Oneric force-pushed search-meili-order from e38fa81518 to dba848032c 2024-05-15 05:17:27 +00:00 Compare
Oneric force-pushed search-meili-order from dba848032c to 17f49efb97 2024-05-20 16:40:15 +00:00 Compare
floatingghost requested changes 2024-05-28 02:01:53 +00:00
floatingghost left a comment
Owner

PR looks ok, fails lint though

although this is notably lacking a test - can you add one? get_presorted_create_by_object_ap_id/1 seems a good one to test to check the main component of this works

PR looks ok, fails lint though although this is notably lacking a test - can you add one? `get_presorted_create_by_object_ap_id/1` seems a good one to test to check the main component of this works
Oneric force-pushed search-meili-order from 17f49efb97 to fc7e07f424 2024-05-30 00:18:45 +00:00 Compare
Author
Member

oops, forgot to rerun lint after adding search_key
Fixed now and test added

oops, forgot to rerun lint after adding `search_key` Fixed now and test added

all good now, thanks!

all good now, thanks!
floatingghost merged commit 8f97c15b07 into develop 2024-05-31 14:12:06 +00:00
floatingghost deleted branch search-meili-order 2024-05-31 14:12:06 +00:00
Sign in to join this conversation.
No description provided.