ShariVegas/akkoma
1
0
Fork 0
forked from AkkomaGang/akkoma
Code Issues Pull requests Projects Releases Packages Wiki Activity
14582 commits 14 branches 60 tags 272 MiB
Commit graph

14582 commits

This branch
This branch
All branches
Author SHA1 Message Date
FloatingGhost
bc9e76cce7 Add documentation for ES search 2022-06-30 17:36:57 +01:00
FloatingGhost
635a3c223a Add elasticsearch tests 2022-06-30 16:53:21 +01:00
FloatingGhost
1ecdb19de5 Refactor ES on top of search behaviour 2022-06-30 16:28:31 +01:00
Ekaterina Vaartis
69d5d1a01b Update meilisearch docs 2022-06-29 20:49:45 +01:00
Ekaterina Vaartis
563b964690 Change updateId to uid because apparently that's the new name 2022-06-29 20:49:45 +01:00
Ekaterina Vaartis
b7462040cc Change the meilisearch key auth to conform to 0.25.0 2022-06-29 20:49:45 +01:00
Ekaterina Vaartis
09ea3bb694 Actually, unlisted posts are indexed 2022-06-29 20:49:45 +01:00
Ekaterina Vaartis
774b4e165a Change search_indexing = 10 and retries for indexing = 2 2022-06-29 20:49:45 +01:00
Ekaterina Vaartis
7f53aa400b Don't try removing deleted users and such from index as posts 2022-06-29 20:49:45 +01:00
Ekaterina Vaartis
7aebff799b Fix meilisearch tests and jobs for oban 2022-06-29 20:49:45 +01:00
Ekaterina Vaartis
5def4a7d49 Use oban for search indexing 2022-06-29 20:49:45 +01:00
Ekaterina Vaartis
54b2a86f47 Add a search backend behaviour 2022-06-29 20:49:45 +01:00
Ekaterina Vaartis
80e52f4d86 Add description for initial_indexing_chunk_size 2022-06-29 20:49:45 +01:00
Ekaterina Vaartis
a4914add8c Don't support meilisearch < 0.24.0, since it breaks things 2022-06-29 20:49:45 +01:00
Ekaterina Vaartis
cc3319ac1d Make chunk size configurable 2022-06-29 20:49:45 +01:00
Ekaterina Vaartis
426cff3372 Update search.md documentation with meilisearch indexing steps 2022-06-29 20:49:45 +01:00
Ekaterina Vaartis
58cc5d13a2 Add config description for meilisearch 2022-06-29 20:49:45 +01:00
Ekaterina Vaartis
bac70a2bc1 Implement suggestions from the Meilisearch MR 
- Index unlisted posts
- Move version check outside of the streaming and only do it once
- Use a PUT request instead of checking manually if there is need to insert
- Add error handling, sort of
 2022-06-29 20:49:45 +01:00
Ekaterina Vaartis
c128798418 Rename Activity.Search to Search.DatabaseSearch 2022-06-29 20:49:44 +01:00
Ekaterina Vaartis
ea582fbf93 Rename search.ex to database_search.ex and add search/2 2022-06-29 20:49:00 +01:00
Ekaterina Vaartis
0769f06bd1 Style fixes 2022-06-29 20:49:00 +01:00
Ekaterina Vaartis
5dd908552c Move the search.ex file so credo doesn't complain 2022-06-29 20:48:59 +01:00
Ekaterina Vaartis
7a9d9cf457 Fix a typo in search docs 2022-06-29 20:48:44 +01:00
Ekaterina Vaartis
86971fceaa Support reindexing meilisearch >=0.24.0 
It has has a different error code key
 2022-06-29 20:48:44 +01:00
Ekaterina Vaartis
9e7d7ebd48 Add a reindex option 
Signed-off-by: Ekaterina Vaartis <vaartis@kotobank.ch>
 2022-06-29 20:48:44 +01:00
Ekaterina Vaartis
d99a2be351 Only add local posts to index in activity_pub 
Remote ones are already added in another place
 2022-06-29 20:48:44 +01:00
Ekaterina Vaartis
51faa28568 Set content-type to application/json 2022-06-29 20:48:44 +01:00
Ekaterina Vaartis
afcdd2f437 Modify some meilisearch variables 2022-06-29 20:48:44 +01:00
Ekaterina Vaartis
37913c8ddf Use proper deleted object for removing from index 2022-06-29 20:48:44 +01:00
Ekaterina Vaartis
a937a98df5 Don't try removing from index again in common_api 
It's already removed in the side effects of the pipeline
 2022-06-29 20:48:44 +01:00
Ekaterina Vaartis
9ab0f130cc Add more documentation about rum to meilisearch docs 2022-06-29 20:48:44 +01:00
Ekaterina Vaartis
0802c0666b Move add_to_index / remove_from_index to Pleroma.Actitivy.Search 2022-06-29 20:48:44 +01:00
Ekaterina Vaartis
5ed1759091 Reorder ranking rules for (maybe) better results 2022-06-29 20:48:44 +01:00
Ekaterina Vaartis
b519726571 Add private_key: nil to default meilisearch options 2022-06-29 20:48:43 +01:00
Ekaterina Vaartis
88891e9d76 Add tests for local post indexing for meilisearch 2022-06-29 20:48:29 +01:00
Ekaterina Vaartis
41f3572152 Fix activity being passed to objec_to_search_data 2022-06-29 20:48:29 +01:00
Ekaterina Vaartis
375154e5de Add search/meilisearch documentation 2022-06-29 20:48:29 +01:00
Ekaterina Vaartis
d1079f1aa3 Add the meilisearch.stats command 2022-06-29 20:48:29 +01:00
Ekaterina Vaartis
d5cc272a91 Add a message with a count of posts to index 2022-06-29 20:48:29 +01:00
Ekaterina Vaartis
dbf556cdcf Implement meilisearch auth 2022-06-29 20:48:29 +01:00
Ekaterina Vaartis
5360cc1097 Make indexing logs rewrite themselves 2022-06-29 20:48:29 +01:00
Ekaterina Vaartis
0cf3654907 Rework task indexing to share code with the main module 
The code in the main module now scrubs new posts too
 2022-06-29 20:48:29 +01:00
Ekaterina Vaartis
117f525fd6 Adjust content indexing to skip more unneeded stuff 2022-06-29 20:48:29 +01:00
Ekaterina Vaartis
14ef6ce80f Mark only content as searchable for meilisearch 2022-06-29 20:48:29 +01:00
Ekaterina Vaartis
b3401ba7bd Also index incoming federated posts 2022-06-29 20:48:29 +01:00
Ekaterina Vaartis
52a872432d Make the chunk size smaller 2022-06-29 20:48:29 +01:00
Ekaterina Vaartis
a586ce0ddd Use content instead of source and scrub it 2022-06-29 20:48:29 +01:00
Ekaterina Vaartis
c3a04166a0 Tweak search ordering to hopefully return newer results 2022-06-29 20:48:29 +01:00
Ekaterina Vaartis
7b3701e6b9 Make meilisearch sort on publish date converted to unix time 2022-06-29 20:48:29 +01:00
Ekaterina Vaartis
abf82a63ec Make the indexing batch differently and more, show number indexed 2022-06-29 20:48:29 +01:00