Commit graph

8981 commits

Author SHA1 Message Date
8683252fc5 Metadata/Utils: use summary as description if set
When generating OpenGraph and TwitterCard metadata for a post, the
summary field will be used first if it is set to generate the post
description.
2022-09-11 19:55:38 +01:00
0b14f02ed2 User: generate private keys on user creation
This fixes a race condition bug where keys could be regenerated
post-federation, causing activities and HTTP signatures from an user to
be dropped due to key differences.
2022-09-11 19:54:37 +01:00
e88f36f72b ObjectView: do not fetch an object for its ID
Non-Create/Listen activities had their associated object field
normalized and fetched, but only to use their `id` field, which is both
slow and redundant. This also failed on Undo activities, which delete
the associated object/activity in database.

Undo activities will now render properly and database loads should
improve ever so slightly.
2022-09-11 19:52:59 +01:00
a6d85003fe Remote interaction with posts (#198)
Grabbed from https://git.pleroma.social/pleroma/pleroma/-/merge_requests/3587

Co-authored-by: Tusooa Zhu <tusooa@kazv.moe>
Reviewed-on: AkkomaGang/akkoma#198
Co-authored-by: Norm <normandy@biribiri.dev>
Co-committed-by: Norm <normandy@biribiri.dev>
2022-09-08 10:19:22 +00:00
2641dcdd15 Post editing (#202)
Rebased from #103

Co-authored-by: Tusooa Zhu <tusooa@kazv.moe>
Co-authored-by: FloatingGhost <hannah@coffee-and-dreams.uk>
Reviewed-on: AkkomaGang/akkoma#202
2022-09-06 19:24:02 +00:00
6c80977b06 turn inlineQuotePolicy on by default 2022-09-05 17:22:33 +01:00
f6304cfd78 add extra tests for builder 2022-09-05 01:24:40 +01:00
1c7d7845c3 fix compilation warnings 2022-09-05 00:39:32 +01:00
1b826eea54 Allow reacting with remote emoji when they exist on the post (#200)
Co-authored-by: FloatingGhost <hannah@coffee-and-dreams.uk>
Reviewed-on: AkkomaGang/akkoma#200
2022-09-04 23:31:41 +00:00
7a90d71e8d ensure .exs config is used before default (#197)
Co-authored-by: FloatingGhost <hannah@coffee-and-dreams.uk>
Reviewed-on: AkkomaGang/akkoma#197
2022-09-02 22:05:39 +00:00
8e4de118c1 Don't persist local undone follow (#194)
same deal but backwards this time

Co-authored-by: FloatingGhost <hannah@coffee-and-dreams.uk>
Reviewed-on: AkkomaGang/akkoma#194
2022-08-31 18:00:36 +00:00
decbca0c91 add seperate source and dest entries in language listing (#193)
Co-authored-by: FloatingGhost <hannah@coffee-and-dreams.uk>
Reviewed-on: AkkomaGang/akkoma#193
2022-08-30 16:59:33 +00:00
c3fde9577d Allow listing languages, setting source language (#192)
Co-authored-by: FloatingGhost <hannah@coffee-and-dreams.uk>
Reviewed-on: AkkomaGang/akkoma#192
2022-08-30 14:58:54 +00:00
df39cab9c1 Automatic status translation (#187)
Fixes #115

Co-authored-by: FloatingGhost <hannah@coffee-and-dreams.uk>
Reviewed-on: AkkomaGang/akkoma#187
2022-08-29 19:42:22 +00:00
Tusooa Zhu
95e4018c1a Disconnect streaming sessions when token is revoked
Use Websockex to replace websocket_client

Test that server will disconnect websocket upon token revocation

Lint

Execute session disconnect in background

Refactor streamer test

allow multi-streams

rebase websocket change
2022-08-27 19:07:48 +01:00
772c209914 GTS: cherry-picks and collection usage (#186)
https://git.pleroma.social/pleroma/pleroma/-/merge_requests/3725?commit_id=61254111e59f02118cad15de49d1e0704c07030e

what is this, a yoink of a yoink? good times

Co-authored-by: Hélène <pleroma-dev@helene.moe>
Co-authored-by: FloatingGhost <hannah@coffee-and-dreams.uk>
Reviewed-on: AkkomaGang/akkoma#186
2022-08-27 18:05:48 +00:00
85137f591f Add ability to obfuscate domains in MRF transparency 2022-08-27 11:57:57 +01:00
e4f2251e0f Add support for setting language in instance metadata (#183)
Reviewed-on: AkkomaGang/akkoma#183
2022-08-25 16:11:21 +00:00
618cf7ff7f reuse valid oauth tokens (#182)
Reviewed-on: AkkomaGang/akkoma#182
2022-08-25 14:37:51 +00:00
92ba2802fb generate-keys-at-registration-time (#181)
Reviewed-on: AkkomaGang/akkoma#181
2022-08-24 14:36:33 +00:00
8d7b63a766 Revert "Fix oauth2 (for real) (#179)"
This reverts commit aa681d7e15.
2022-08-21 17:52:02 +01:00
aa681d7e15 Fix oauth2 (for real) (#179)
Reviewed-on: AkkomaGang/akkoma#179
2022-08-21 16:24:37 +00:00
b0130bfa7b Revert "oauth2 fixes (#177)"
This reverts commit 429e2ac832.
2022-08-21 16:22:15 +01:00
d72f9e39d9 add visibility check on quote (#178)
Reviewed-on: AkkomaGang/akkoma#178
2022-08-21 15:17:01 +00:00
429e2ac832 oauth2 fixes (#177)
Reviewed-on: AkkomaGang/akkoma#177
2022-08-21 14:46:52 +00:00
e9f1897cfd parser MFM server-side (#172)
Reviewed-on: AkkomaGang/akkoma#172
2022-08-18 03:14:48 +00:00
aaf78e2b52 only put linked mfm in source (#171)
Reviewed-on: AkkomaGang/akkoma#171
2022-08-17 09:35:11 +00:00
11ec9daa5b API compatibility with fedibird, frontend config (#163)
Reviewed-on: AkkomaGang/akkoma#163
2022-08-17 00:22:59 +00:00
89ffc01c23 only return create objects for ES search (#165)
Reviewed-on: AkkomaGang/akkoma#165
2022-08-16 23:24:19 +00:00
61641957cb fix compatibility with meilisearch (#164)
Reviewed-on: AkkomaGang/akkoma#164
2022-08-16 22:56:49 +00:00
37a1001b97 add finch outbound proxy support (#158)
Reviewed-on: AkkomaGang/akkoma#158
2022-08-14 23:13:49 +00:00
d3b9cfb03f use :discard instead of cancel 2022-08-11 19:17:50 +01:00
55179d4214 set soapbox-fe v2 by default
fixes #157
2022-08-11 10:25:03 +01:00
1245141779 treat rejections in MRF as a reject in federator (#155)
Reviewed-on: AkkomaGang/akkoma#155
2022-08-08 15:47:57 +00:00
5d23df84c9 Mix format 2022-08-07 20:49:56 +01:00
b3e4d81362 StatusView: implement pleroma.context field
This field replaces the now deprecated conversation_id field, and now
exposes the ActivityPub object `context` directly via the MastoAPI
instead of relying on StatusNet-era data concepts.
2022-08-07 20:48:08 +01:00
b9bb093600 StatusView: clear MSB on calculated conversation_id
This field seems to be a left-over from the StatusNet era.
If your application uses `pleroma.conversation_id`: this field is
deprecated.

It is currently stubbed instead by doing a CRC32 of the context, and
clearing the MSB to avoid overflow exceptions with signed integers on
the different clients using this field (Java/Kotlin code, mostly; see
Husky and probably other mobile clients.)

This should be removed in a future version of Pleroma. Pleroma-FE
currently depends on this field, as well.
2022-08-07 20:47:59 +01:00
62e179f446 make conversation-id deterministic (#154)
Reviewed-on: AkkomaGang/akkoma#154
2022-08-06 20:59:15 +00:00
ec162b496b /notice signing checks on redirect (#150)
Reviewed-on: AkkomaGang/akkoma#150
2022-08-05 19:31:32 +00:00
0ec3a11895 don't persist undo of follows (#149)
Reviewed-on: AkkomaGang/akkoma#149
2022-08-05 13:28:56 +00:00
a82fb2acc1 Merge pull request 'Update default paths' (#141) from norm/akkoma:update-default-paths into develop
Reviewed-on: AkkomaGang/akkoma#141
2022-08-05 11:42:14 +00:00
c1e15ff6f8 Transmogrifier: fix reply context fixing
Incoming Pleroma replies to a Misskey thread were rejected due to a
broken context fix, which caused them to not be visible until a
non-Pleroma user interacted with the replies.

This fix properly sets the post-fix object context to its parent Create
activity as well, if it was changed.
2022-08-04 12:57:48 +01:00
456c97fda9 Merge pull request 'remove unneeded function' (#143) from compile-fix into develop
Reviewed-on: AkkomaGang/akkoma#143
2022-08-03 11:12:05 +00:00
842ab82ef0 Merge pull request 'Allow users to create backups without providing email address' (#140) from norm/akkoma:backup-without-email into develop
Reviewed-on: AkkomaGang/akkoma#140
2022-08-03 11:11:53 +00:00
2c40d565fa Fix config path lookup (#139)
Reviewed-on: AkkomaGang/akkoma#139
Co-authored-by: Norm <normandy@biribiri.dev>
Co-committed-by: Norm <normandy@biribiri.dev>
2022-08-03 10:52:21 +00:00
359510eebc remove unneeded function 2022-08-03 11:50:48 +01:00
8bfd01b9c7
Update default paths 2022-08-03 01:05:53 -04:00
Tusooa Zhu
f08241c8ab
Allow users to create backups without providing email address
Ref: backup-without-email
2022-08-02 22:16:54 -04:00
c9600dbbbf local-only-fixed (#138)
Reviewed-on: AkkomaGang/akkoma#138
2022-08-02 14:46:46 +00:00
ca000f8301 Merge mrf_simple-reject with quarantine (#137)
Reviewed-on: AkkomaGang/akkoma#137
2022-08-02 14:19:24 +00:00
e26388a01c Support reaching user@sub.domain.tld at user@domain.tld (#134)
Reviewed-on: AkkomaGang/akkoma#134
Co-authored-by: Joel Beckmeyer <joel@beckmeyer.us>
Co-committed-by: Joel Beckmeyer <joel@beckmeyer.us>
2022-08-02 13:54:22 +00:00
c3eea8dc7d expose bubble instances via nodeinfo (#136)
Reviewed-on: AkkomaGang/akkoma#136
2022-08-02 09:11:22 +00:00
19a27ff006 allow small/center tags in misskeymarkdown (#132)
Reviewed-on: AkkomaGang/akkoma#132
2022-08-01 12:46:52 +00:00
38659e5610 Use uppercase HTTP HEAD method for media preview proxy request (#128)
Reviewed-on: AkkomaGang/akkoma#128
Co-authored-by: Yukkuri <iamtakingiteasy@eientei.org>
Co-committed-by: Yukkuri <iamtakingiteasy@eientei.org>
2022-07-30 21:58:14 +00:00
2033d7d4fc ensure extra info in fix_follow_state prints 2022-07-29 19:50:26 +01:00
db99edacfe do the same for soapbox 2022-07-29 10:10:12 +01:00
4f6caae209 ensure we can't run the same clause of fix_quote_url more than once 2022-07-29 10:08:40 +01:00
bf3f934275 add guards around fix misskey content 2022-07-29 10:04:04 +01:00
405406601f Fix emoji qualification (#124)
Reviewed-on: AkkomaGang/akkoma#124
2022-07-28 12:02:36 +00:00
2796a9acaf backend-i18n (#121)
Reviewed-on: AkkomaGang/akkoma#121
2022-07-27 21:56:59 +00:00
645f0390bc Prepare for ubuntu22 murdering openssl (#120)
Reviewed-on: AkkomaGang/akkoma#120
2022-07-27 21:48:13 +00:00
a3501cab86 ensure quote fetching obeys max thread distance (#119)
Reviewed-on: AkkomaGang/akkoma#119
2022-07-26 17:28:47 +00:00
0a55c37182 don't error out if the featured collection has a string ID 2022-07-26 15:08:35 +01:00
1f6deb0ef4 include local instance in bubble timeline (#117)
Reviewed-on: AkkomaGang/akkoma#117
2022-07-26 12:22:49 +00:00
90c4785b89 remove public post quarantine exception (#114)
Reviewed-on: AkkomaGang/akkoma#114
2022-07-26 11:09:13 +00:00
1419eee5df Quote posting (#113)
Reviewed-on: AkkomaGang/akkoma#113
2022-07-25 16:30:06 +00:00
516d155558 open up functions in user 2022-07-24 17:56:48 +01:00
c4e9c4bc95 extend custom runtime system (#108)
Reviewed-on: AkkomaGang/akkoma#108
2022-07-24 16:42:43 +00:00
4c47992686 bugfix/follow-state (#104)
Reviewed-on: AkkomaGang/akkoma#104
2022-07-23 18:58:45 +00:00
cb6e7359af add bubble timeline (#100)
Reviewed-on: AkkomaGang/akkoma#100
2022-07-22 14:55:38 +00:00
0c542e58aa Remove instrumentors (#98)
Reviewed-on: AkkomaGang/akkoma#98
2022-07-21 11:32:17 +00:00
0f132b802d purge chat and shout endpoints 2022-07-21 11:29:28 +01:00
07ea4d73e1 update mastofe paths (#95)
Reviewed-on: AkkomaGang/akkoma#95
2022-07-20 20:13:50 +00:00
3b8bf8464f update features array 2022-07-20 15:43:41 +01:00
729f45ccd2 purge ldap authenticator (#92)
Reviewed-on: AkkomaGang/akkoma#92
2022-07-20 12:49:13 +00:00
dc9f66749c remove all endpoints marked as deprecated (#91)
Reviewed-on: AkkomaGang/akkoma#91
2022-07-20 12:00:58 +00:00
cf0ad02ea9 Remove scrobbling support 2022-07-19 15:07:45 +01:00
85e2e64c82 fix resolution of GTS user keys 2022-07-18 15:21:27 +01:00
54ed8760ff Merge branch 'from/upstream-develop/tusooa/server-announcements' into 'develop' (#85)
Reviewed-on: AkkomaGang/akkoma#85
2022-07-18 13:08:36 +00:00
5b4d77eaa7 maintenance: dependency upgrade (#81)
Reviewed-on: AkkomaGang/akkoma#81
2022-07-18 00:56:35 +00:00
d2a185c013 Documentation updates for stable release (#73)
Reviewed-on: AkkomaGang/akkoma#73
2022-07-15 12:27:16 +00:00
d598c7a834 remove anonymous function from plug 2022-07-14 11:17:14 +01:00
4aee900ae8 fix compilation 2022-07-14 10:48:27 +01:00
37ae047e16 Add swaggerUI options (#66)
Reviewed-on: AkkomaGang/akkoma#66
2022-07-13 15:09:35 +00:00
7c1a2092ca Merge pull request 'Change default Postgres user/DB to akkoma' (#63) from norm/akkoma:default-db-name into develop
Reviewed-on: AkkomaGang/akkoma#63
2022-07-13 12:16:40 +00:00
Tusooa Zhu
4c5bc3e9f7 Pass remote follow avatar into media proxy 2022-07-12 15:44:38 -04:00
becf7dced6 Merge branch 'develop' into default-db-name 2022-07-12 16:44:10 +00:00
7dfc3f3d0e Change default Postgres user/DB to akkoma 2022-07-12 12:41:30 -04:00
8215434c65 also resolve @full@tags in mfm (#62)
Reviewed-on: AkkomaGang/akkoma#62
2022-07-11 13:48:43 +00:00
ff6c8455fb [#58] ensure all users are linked in MFM content (#61)
Reviewed-on: AkkomaGang/akkoma#61
2022-07-11 11:48:29 +00:00
5ad256f170 [#58] pre-link MFM content (#59)
Reviewed-on: AkkomaGang/akkoma#59
2022-07-10 17:06:25 +00:00
82fa766ed7 Remove precompiled javascript (#55)
Reviewed-on: AkkomaGang/akkoma#55
2022-07-08 13:03:18 +00:00
bc6bfe383f Add configurable theme color (#53)
Reviewed-on: AkkomaGang/akkoma#53
2022-07-06 20:00:43 +00:00
a9c82b62f2 Fixes for elasticsearch 8 (#52)
Reviewed-on: AkkomaGang/akkoma#52
2022-07-06 18:57:00 +00:00
3d9dabd914 Fix config path lookup (#51)
Reviewed-on: AkkomaGang/akkoma#51
Co-authored-by: norm <normandy@biribiri.dev>
Co-committed-by: norm <normandy@biribiri.dev>
2022-07-06 18:28:41 +00:00
1486ad0c6e import logger 2022-07-05 14:45:28 +01:00
a036a01a1e mix format 2022-07-04 17:38:16 +01:00
364b6969eb Use finch everywhere (#33)
Reviewed-on: AkkomaGang/akkoma#33
2022-07-04 16:30:38 +00:00
sfr
058bf96798 implement Move activities (#45)
Reviewed-on: AkkomaGang/akkoma#45
Co-authored-by: sfr <sol@solfisher.com>
Co-committed-by: sfr <sol@solfisher.com>
2022-07-04 16:29:39 +00:00
95ef3a8b1e Use Akkoma modification for collections 2022-07-03 19:36:30 +01:00
05081cd81b Add collection fetching module 2022-07-03 19:20:59 +01:00
0a3a552696 Add support for a first reference in pinned objects 2022-07-03 17:25:20 +01:00
3f7c64f30b Fix broken attachments from owncast (#31)
Reviewed-on: AkkomaGang/akkoma#31
2022-07-01 11:14:55 +00:00
2937495712 fix ES import from live (#29)
Reviewed-on: AkkomaGang/akkoma#29
2022-06-30 18:44:31 +00:00
bc9e76cce7 Add documentation for ES search 2022-06-30 17:36:57 +01:00
635a3c223a Add elasticsearch tests 2022-06-30 16:53:21 +01:00
1ecdb19de5 Refactor ES on top of search behaviour 2022-06-30 16:28:31 +01:00
Ekaterina Vaartis
563b964690 Change updateId to uid because apparently that's the new name 2022-06-29 20:49:45 +01:00
Ekaterina Vaartis
b7462040cc Change the meilisearch key auth to conform to 0.25.0 2022-06-29 20:49:45 +01:00
Ekaterina Vaartis
7f53aa400b Don't try removing deleted users and such from index as posts 2022-06-29 20:49:45 +01:00
Ekaterina Vaartis
7aebff799b Fix meilisearch tests and jobs for oban 2022-06-29 20:49:45 +01:00
Ekaterina Vaartis
5def4a7d49 Use oban for search indexing 2022-06-29 20:49:45 +01:00
Ekaterina Vaartis
54b2a86f47 Add a search backend behaviour 2022-06-29 20:49:45 +01:00
Ekaterina Vaartis
a4914add8c Don't support meilisearch < 0.24.0, since it breaks things 2022-06-29 20:49:45 +01:00
Ekaterina Vaartis
cc3319ac1d Make chunk size configurable 2022-06-29 20:49:45 +01:00
Ekaterina Vaartis
bac70a2bc1 Implement suggestions from the Meilisearch MR
- Index unlisted posts
- Move version check outside of the streaming and only do it once
- Use a PUT request instead of checking manually if there is need to insert
- Add error handling, sort of
2022-06-29 20:49:45 +01:00
Ekaterina Vaartis
c128798418 Rename Activity.Search to Search.DatabaseSearch 2022-06-29 20:49:44 +01:00
Ekaterina Vaartis
ea582fbf93 Rename search.ex to database_search.ex and add search/2 2022-06-29 20:49:00 +01:00
Ekaterina Vaartis
0769f06bd1 Style fixes 2022-06-29 20:49:00 +01:00
Ekaterina Vaartis
5dd908552c Move the search.ex file so credo doesn't complain 2022-06-29 20:48:59 +01:00
Ekaterina Vaartis
86971fceaa Support reindexing meilisearch >=0.24.0
It has has a different error code key
2022-06-29 20:48:44 +01:00
Ekaterina Vaartis
9e7d7ebd48 Add a reindex option
Signed-off-by: Ekaterina Vaartis <vaartis@kotobank.ch>
2022-06-29 20:48:44 +01:00
Ekaterina Vaartis
d99a2be351 Only add local posts to index in activity_pub
Remote ones are already added in another place
2022-06-29 20:48:44 +01:00
Ekaterina Vaartis
51faa28568 Set content-type to application/json 2022-06-29 20:48:44 +01:00
Ekaterina Vaartis
afcdd2f437 Modify some meilisearch variables 2022-06-29 20:48:44 +01:00
Ekaterina Vaartis
37913c8ddf Use proper deleted object for removing from index 2022-06-29 20:48:44 +01:00
Ekaterina Vaartis
a937a98df5 Don't try removing from index again in common_api
It's already removed in the side effects of the pipeline
2022-06-29 20:48:44 +01:00
Ekaterina Vaartis
0802c0666b Move add_to_index / remove_from_index to Pleroma.Actitivy.Search 2022-06-29 20:48:44 +01:00
Ekaterina Vaartis
5ed1759091 Reorder ranking rules for (maybe) better results 2022-06-29 20:48:44 +01:00
Ekaterina Vaartis
41f3572152 Fix activity being passed to objec_to_search_data 2022-06-29 20:48:29 +01:00
Ekaterina Vaartis
d1079f1aa3 Add the meilisearch.stats command 2022-06-29 20:48:29 +01:00
Ekaterina Vaartis
d5cc272a91 Add a message with a count of posts to index 2022-06-29 20:48:29 +01:00
Ekaterina Vaartis
dbf556cdcf Implement meilisearch auth 2022-06-29 20:48:29 +01:00
Ekaterina Vaartis
5360cc1097 Make indexing logs rewrite themselves 2022-06-29 20:48:29 +01:00
Ekaterina Vaartis
0cf3654907 Rework task indexing to share code with the main module
The code in the main module now scrubs new posts too
2022-06-29 20:48:29 +01:00
Ekaterina Vaartis
117f525fd6 Adjust content indexing to skip more unneeded stuff 2022-06-29 20:48:29 +01:00
Ekaterina Vaartis
14ef6ce80f Mark only content as searchable for meilisearch 2022-06-29 20:48:29 +01:00
Ekaterina Vaartis
b3401ba7bd Also index incoming federated posts 2022-06-29 20:48:29 +01:00
Ekaterina Vaartis
52a872432d Make the chunk size smaller 2022-06-29 20:48:29 +01:00
Ekaterina Vaartis
a586ce0ddd Use content instead of source and scrub it 2022-06-29 20:48:29 +01:00
Ekaterina Vaartis
c3a04166a0 Tweak search ordering to hopefully return newer results 2022-06-29 20:48:29 +01:00
Ekaterina Vaartis
7b3701e6b9 Make meilisearch sort on publish date converted to unix time 2022-06-29 20:48:29 +01:00
Ekaterina Vaartis
abf82a63ec Make the indexing batch differently and more, show number indexed 2022-06-29 20:48:29 +01:00
Ekaterina Vaartis
e5ac2ffa07 Ensure only indexing public posts and implement clearing and delete 2022-06-29 20:48:29 +01:00
Ekaterina Vaartis
41db5c8653 Add logging to milisiearch index and make it use desc(id) 2022-06-29 20:48:29 +01:00
Ekaterina Vaartis
e961cf2689 Initial meilisearch implementation, doesn't delete posts yet 2022-06-29 20:48:28 +01:00
8f140deb8f StealEmojiPolicy: fix String rejected_shortcodes
* rejected_shortcodes is defined as a list of strings in the
  configuration description. As such, database-based configuration was
  led to handle those settings as strings, and not as the actually
  expected type, Regex.
* This caused each message passing through this MRF, if a rejected
  shortcode was set and the emoji did not exist already on the instance,
  to fail federating, as an exception was raised, swiftly caught and
  mostly silenced.
* This commit fixes the issue by introducing new behavior: strings are
  now handled as perfect matches for an emoji shortcode (meaning that if
  the emoji-to-be-pulled's shortcode is in the blacklist, it will be
  rejected), while still supporting Regex types as before.
2022-06-29 20:47:45 +01:00
Ilja
661d0ba481 Also use actor_type to determine if an account is a bot in antiFollowbotPolicy 2022-06-29 20:47:44 +01:00
Tusooa Zhu
3fd87b6a75 Skip cache when /objects or /activities is authenticated
Ref: fix-local-public
2022-06-29 20:47:27 +01:00
Tusooa Zhu
932e5df19e Allow to skip cache in Cache plug
Ref: fix-local-public
2022-06-29 20:47:26 +01:00