AkkomaGang/akkoma
AkkomaGang/akkoma
13
63
Fork 84
Code Issues 162 Pull requests 13 Packages Projects 2 Releases 14 Wiki Activity

Preserve Meilisearch’s result ranking #772

Merged
floatingghost merged 4 commits from Oneric/akkoma:search-meili-order into develop 2024-05-31 14:12:06 +00:00
Conversation 2 Commits 4 Files changed 6 +85 -22
Oneric commented 2024-05-07 00:22:42 +00:00
Member
Copy link

Supersedes #579

Meilisearch was easier to setup than initially presumed so i was now able to test this myself. Previously top results were reorderd by date, now Meilisearch’s ranking is respected; note how reordering pushed the top Meilisearch result down to the end of the list:

# BEFORE PATCH
MeiliRanking:
["http://localhost:4000/objects/3b72d9bd-e821-466a-8ace-c5244bc03bce", "http://localhost:4000/objects/7bd90971-2ffc-477a-b0d9-649a82766c7c", "http://localhost:4000/objects/3e1e7b34-ee22-4592-ba0c-6bded82ba6cc", "http://localhost:4000/objects/bb92b7c4-2ce6-4e83-b2e7-b21b40239153"]

RetrivedRes:
["http://localhost:4000/objects/7bd90971-2ffc-477a-b0d9-649a82766c7c", "http://localhost:4000/objects/3e1e7b34-ee22-4592-ba0c-6bded82ba6cc", "http://localhost:4000/objects/bb92b7c4-2ce6-4e83-b2e7-b21b40239153", "http://localhost:4000/objects/3b72d9bd-e821-466a-8ace-c5244bc03bce"]


# AFTER PATCH
MeiliRanking:
["http://localhost:4000/objects/3b72d9bd-e821-466a-8ace-c5244bc03bce", "http://localhost:4000/objects/7bd90971-2ffc-477a-b0d9-649a82766c7c", "http://localhost:4000/objects/3e1e7b34-ee22-4592-ba0c-6bded82ba6cc", "http://localhost:4000/objects/bb92b7c4-2ce6-4e83-b2e7-b21b40239153"]

RetrivedRes:
["http://localhost:4000/objects/3b72d9bd-e821-466a-8ace-c5244bc03bce", "http://localhost:4000/objects/7bd90971-2ffc-477a-b0d9-649a82766c7c", "http://localhost:4000/objects/3e1e7b34-ee22-4592-ba0c-6bded82ba6cc", "http://localhost:4000/objects/bb92b7c4-2ce6-4e83-b2e7-b21b40239153"]

Compared to #579 this avoids a costly O(n²) resort in Elixir by just retrieving objects in the right order to begin with.

Side note: Meilisearch’s ranking for this particular query almost matches what ORDER BY ts_rank_cd(vector, query, 16) would yield if it was used in the default GIN search.

Supersedes #579 Meilisearch was easier to setup than initially presumed so i was now able to test this myself. Previously top results were reorderd by date, now Meilisearch’s ranking is respected; note how reordering pushed the top Meilisearch result down to the end of the list: ``` # BEFORE PATCH MeiliRanking: ["http://localhost:4000/objects/3b72d9bd-e821-466a-8ace-c5244bc03bce", "http://localhost:4000/objects/7bd90971-2ffc-477a-b0d9-649a82766c7c", "http://localhost:4000/objects/3e1e7b34-ee22-4592-ba0c-6bded82ba6cc", "http://localhost:4000/objects/bb92b7c4-2ce6-4e83-b2e7-b21b40239153"] RetrivedRes: ["http://localhost:4000/objects/7bd90971-2ffc-477a-b0d9-649a82766c7c", "http://localhost:4000/objects/3e1e7b34-ee22-4592-ba0c-6bded82ba6cc", "http://localhost:4000/objects/bb92b7c4-2ce6-4e83-b2e7-b21b40239153", "http://localhost:4000/objects/3b72d9bd-e821-466a-8ace-c5244bc03bce"] # AFTER PATCH MeiliRanking: ["http://localhost:4000/objects/3b72d9bd-e821-466a-8ace-c5244bc03bce", "http://localhost:4000/objects/7bd90971-2ffc-477a-b0d9-649a82766c7c", "http://localhost:4000/objects/3e1e7b34-ee22-4592-ba0c-6bded82ba6cc", "http://localhost:4000/objects/bb92b7c4-2ce6-4e83-b2e7-b21b40239153"] RetrivedRes: ["http://localhost:4000/objects/3b72d9bd-e821-466a-8ace-c5244bc03bce", "http://localhost:4000/objects/7bd90971-2ffc-477a-b0d9-649a82766c7c", "http://localhost:4000/objects/3e1e7b34-ee22-4592-ba0c-6bded82ba6cc", "http://localhost:4000/objects/bb92b7c4-2ce6-4e83-b2e7-b21b40239153"] ``` Compared to #579 this avoids a costly O(n²) resort in Elixir by just retrieving objects in the right order to begin with. Side note: Meilisearch’s ranking for this particular query almost matches what `ORDER BY ts_rank_cd(vector, query, 16)` would yield if it was used in the default GIN search.
norm reviewed 2024-05-15 03:02:59 +00:00
docs/docs/configuration/search.md Outdated
@ -59,2 +59,3 @@
You will see a "Default Admin API Key", this is the key you actually put into your configuration file.
Depending on your Meilisearch version, you will either see a "Default Admin API Key" or some line starting with "Use it for anything that is not a search operation. Caution! Do not expose it on a public frontend:".
This is the key you actually put into your configuration file.
norm commented 2024-05-15 03:02:51 +00:00
Contributor
Copy link

Wondering if it would make more sense to limit the admin API key to just admin tasks and use a separate key for just searches done via our APIs..

Wondering if it would make more sense to limit the admin API key to just admin tasks and use a separate key for just searches done via our APIs..
Oneric commented 2024-05-15 05:13:39 +00:00
Author
Member
Copy link

repusehd, using separate search and admin keys is now possible and instead of changing documentation to match the output of our show-keys task, show-keys is changed to display Meilisearch's name parameter if present instead of description.

repusehd, using separate search and admin keys is now possible and instead of changing documentation to match the output of our `show-keys` task, `show-keys` is changed to display Meilisearch's `name` parameter if present instead of `description`.
👍 1
Oneric marked this conversation as resolved
Oneric force-pushed search-meili-order from 560c5a4b5d to e38fa81518 2024-05-15 05:10:54 +00:00 Compare
Oneric force-pushed search-meili-order from e38fa81518 to dba848032c 2024-05-15 05:17:27 +00:00 Compare
Oneric force-pushed search-meili-order from dba848032c to 17f49efb97 2024-05-20 16:40:15 +00:00 Compare
floatingghost requested changes 2024-05-28 02:01:53 +00:00
floatingghost left a comment
Owner
Copy link

PR looks ok, fails lint though

although this is notably lacking a test - can you add one? get_presorted_create_by_object_ap_id/1 seems a good one to test to check the main component of this works

PR looks ok, fails lint though although this is notably lacking a test - can you add one? `get_presorted_create_by_object_ap_id/1` seems a good one to test to check the main component of this works
Oneric force-pushed search-meili-order from 17f49efb97 to fc7e07f424 2024-05-30 00:18:45 +00:00 Compare
Oneric commented 2024-05-30 00:24:20 +00:00
Author
Member
Copy link

oops, forgot to rerun lint after adding search_key
Fixed now and test added

oops, forgot to rerun lint after adding `search_key` Fixed now and test added
floatingghost commented 2024-05-31 14:12:01 +00:00
Owner
Copy link

all good now, thanks!

all good now, thanks!
floatingghost merged commit 8f97c15b07 into develop 2024-05-31 14:12:06 +00:00
floatingghost deleted branch search-meili-order 2024-05-31 14:12:06 +00:00
Sign in to join this conversation.
Reviewers
No reviewers
floatingghost
Labels
Clear labels   
approved, awaiting change

   
bug

Something is not working

  
configuration

This requires config changes

  
documentation

This is about human-readable docs

  
duplicate

This issue or pull request already exists

  
enhancement

New feature

  
extremely low priority

   
feature request

Something new?

  
Fix it yourself

Support has been attempted

  
help wanted

Need some help

  
invalid

Something is wrong

  
mastodon_api

   
needs docs

   
needs tests

   
not a bug

   
planned

   
pleroma_api

   
privacy

   
question

More information is needed

  
static_fe

   
triage

   
wontfix

This won't be fixed

No labels
approved, awaiting change
bug
configuration
documentation
duplicate
enhancement
extremely low priority
feature request
Fix it yourself
help wanted
invalid
mastodon_api
needs docs
needs tests
not a bug
planned
pleroma_api
privacy
question
static_fe
triage
wontfix
Milestone
Clear milestone
No items
No milestone
Projects
Clear projects
No items
No project
Assignees
Clear assignees
No assignees
3 participants
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference: AkkomaGang/akkoma#772
No description provided.
Delete branch "Oneric/akkoma:search-meili-order"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?