Commit graph

15053 commits

Author SHA1 Message Date
ilja
8b2adc4fb4 Rename users_ap_id_COALESCE_follower_address_index for faster db restoration
By default Postgresql first restores the data and then the indexes when dumping and restoring the database.
Restoring index activities_visibility_index took a very long time.
users_ap_id_COALESCE_follower_address_index was later added because having this could speed up the restoration tremendously.
The problem now is that restoration apparently happens in alphabetical order, so this new index wasn't created yet
by the time activities_visibility_index needed it.
There were several work-arounds which included more complex steps during backup/restore.
By renaming this index, it should be restored first and thus activities_visibility_index can make use of it.
This speeds up restoration significantly without requiring more complex or unexpected steps from people.
2023-02-26 14:33:17 +01:00
9f34294332 Add changelog entry 2023-02-23 11:07:59 +00:00
d3089ec399 Ensure we can update contentMap on update 2023-02-23 11:00:55 +00:00
f22c6e4108 Merge pull request 'Update docs about clients' (#484) from mahito1594/akkoma:docs/update-clients-info into develop
Reviewed-on: AkkomaGang/akkoma#484
2023-02-22 15:44:00 +00:00
3f03f1df9c docs: update URL hosting husky's sourse code 2023-02-22 22:50:18 +09:00
9dc3f8fcdc docs: remark that Pinafore is now unmaintained 2023-02-22 22:43:17 +09:00
3744789710 Merge pull request 'update healthcheck route in locale string' (#475) from flisk/akkoma:update-healthcheck-route-reference into develop
Reviewed-on: AkkomaGang/akkoma#475
2023-02-21 09:44:55 +00:00
ea30d22dfe Merge pull request 'Interpret \n as newline for MFM' (#478) from ilja/akkoma:newline_remains_newline_in_mfm into develop
Reviewed-on: AkkomaGang/akkoma#478
2023-02-21 09:43:15 +00:00
ilja
b4952a81fe Interpret \n as newline for MFM
Markdown doesn't generally consider `\n` a newline,
but Misskey does for MFM.

Now we do to for MFM (and not for Markdown) :)
2023-02-18 19:56:11 +01:00
292f0444d0 update healthcheck route in locale string 2023-02-18 14:59:46 +01:00
a18b5755b4 run mix gettext.extract
seems like this hasn't been run in a while. putting this in a separate
commit so the changes i'm about to make don't get drowned in all of this
stuff.
2023-02-18 14:58:35 +01:00
ef279ac53f build musl on develop 2023-02-15 22:09:31 +00:00
b4e37b03d8 use a slightly earlier alpine version 2023-02-15 22:08:55 +00:00
f92484fd01 add version in changelog 2023-02-11 11:19:33 +00:00
6e07ed6ea9 fix docker tag 2023-02-11 10:58:53 +00:00
d2b57a7f9e bump elixir version in CI 2023-02-11 10:56:53 +00:00
439e915531 bump version 2023-02-11 10:48:52 +00:00
ilja
b71db2f82d create_service_actor is now type Application
This is used for internal fetch and for relay. Both represent the instance and therefore are an aplication.
2023-02-04 21:00:21 +00:00
aeb68a0ad1 paginate follow requests (#460)
matches https://docs.joinmastodon.org/methods/follow_requests/#get mostly

Co-authored-by: FloatingGhost <hannah@coffee-and-dreams.uk>
Reviewed-on: AkkomaGang/akkoma#460
2023-02-04 20:51:17 +00:00
56c37dc6b3 Merge pull request 'DOCS: Add info on fe to setup of dev env' (#456) from ilja/akkoma:docs_add_fe_to_akkoma_dev into develop
Reviewed-on: AkkomaGang/akkoma#456
2023-01-30 11:28:28 +00:00
3405623d46 Merge pull request 'Docs: Add more info to the development index file' (#451) from ilja/akkoma:docs_add_some_developer_docs into develop
Reviewed-on: AkkomaGang/akkoma#451
2023-01-30 11:27:23 +00:00
702979bca3 Merge pull request 'Docs: Changes to pgtune docs' (#449) from ilja/akkoma:docs_small_adition_to_pgtune into develop
Reviewed-on: AkkomaGang/akkoma#449
2023-01-30 11:25:46 +00:00
ilja
7e3ede02f7 Add info on fe to setup of dev env
I added info about installing front ends from the development branch

I also rearanged the list of exceptions (what's different than "normal" installation)
so the order is closer to how you'd encounter things in the installation docs + small fixes
2023-01-29 08:29:06 +01:00
d601ddeb91 Merge pull request 'Make default outgoing-blocks setting off' (#454) from Seirdy/akkoma:outgoing-blocks-default-off into develop
Reviewed-on: AkkomaGang/akkoma#454
2023-01-27 10:06:49 +00:00
676cc0d0d7
Make default outgoing-blocks setting off
This should help mitigate negative impacts related to block-retaliation
and block-circumvention when blocks become visible to the blocked party.
Instances interested in broadcasting blocks can turn this on if they
wish. This should have always been the default.

See also: AkkomaGang/akkoma-fe#274
2023-01-26 22:01:22 -08:00
ilja
ce6f652a9a Add more info to the development index file
Credit where credit is due; I inspired myself by looking at the yunohost docs
* https://yunohost.org/en/dev
* https://yunohost.org/en/packaging_apps_start

I try to be inviting to new developers and guide them in their first steps into Akkoma development.
I try to keep the page itself as short as possible and link to relevant places.
That way people can quickly skim over parts that they don't need, while people who do need more can simply follow the links.
2023-01-26 13:40:51 +01:00
ilja
377527ea03 Changes to pgtune docs
I experienced that it may be better to tell pgtune you have lower resoures than what you have when you have other services running.
I added that now.

I also moved the examples as part of the pgtune section.
2023-01-26 10:11:29 +01:00
153539a246 add changelog entry 2023-01-15 19:02:58 +00:00
d394ab0a8a Merge branch 'develop' of akkoma.dev:AkkomaGang/akkoma into develop 2023-01-15 18:58:26 +00:00
90088cce11 Support TLD wildcards in MRF matches
Fixes #431
2023-01-15 18:57:49 +00:00
63ce25f32c Merge pull request 'Correct og:description tag in static-fe' (#373) from sfr/akkoma:fix/og-description into develop
Reviewed-on: AkkomaGang/akkoma#373
2023-01-15 18:15:20 +00:00
sfr
20cd8a0fc4 URL encode remote emoji pack names (#362)
fix #246

Co-authored-by: Sol Fisher Romanoff <sol@solfisher.com>
Reviewed-on: AkkomaGang/akkoma#362
Co-authored-by: sfr <sol@solfisher.com>
Co-committed-by: sfr <sol@solfisher.com>
2023-01-15 18:14:04 +00:00
0fb2042f2c Merge pull request 'Remove refences to ARM OTP builds in install guide' (#432) from norm/akkoma:remove-arm-otp into develop
Reviewed-on: AkkomaGang/akkoma#432
2023-01-15 18:11:53 +00:00
0c8da6466e Merge pull request 'Update OpenBSD docs' (#414) from lechindianer/akkoma:update-openbsd-docs into develop
Reviewed-on: AkkomaGang/akkoma#414
2023-01-15 18:09:32 +00:00
975bc6d7e8 Merge pull request 'fix: Give error message to users when address has already been validated' (#435) from cheerfulstoic/akkoma:develop into develop
Reviewed-on: AkkomaGang/akkoma#435
2023-01-15 18:06:12 +00:00
2fc5fb7f5a Add changelog entry 2023-01-15 18:05:02 +00:00
f3c118ca23 Mix format 2023-01-15 18:00:03 +00:00
0d342a35e3 add contentMap to schema 2023-01-15 17:59:19 +00:00
7ca9ce9d67 fix: Give error message to users when address has already been validated
Plus other errors.
2023-01-12 22:08:10 +01:00
65e8e8fb6d Merge branch 'language-on-posts' into develop 2023-01-11 15:44:15 +00:00
ff5793198f add inbound language test 2023-01-11 15:42:13 +00:00
78c44f31ca fix no-language-specified federation 2023-01-11 15:25:34 +00:00
260c87006e revert ae54c06bb4
revert Language code on mastoapi statuses (#433)

Co-authored-by: FloatingGhost <hannah@coffee-and-dreams.uk>
Reviewed-on: AkkomaGang/akkoma#433
2023-01-10 15:00:47 +00:00
ae54c06bb4 Language code on mastoapi statuses (#433)
Co-authored-by: FloatingGhost <hannah@coffee-and-dreams.uk>
Reviewed-on: AkkomaGang/akkoma#433
2023-01-10 13:45:43 +00:00
22068f0853 fix unused variable warnings 2023-01-10 10:58:17 +00:00
7eebaa7a18 bump mock 2023-01-10 10:48:14 +00:00
cc63a89b5d Fix tests 2023-01-10 10:29:17 +00:00
f86bf16430 Add language support on /api/v1/statuses 2023-01-10 10:29:17 +00:00
6965a2f163 Merge pull request 'use postgres 15 in CI' (#411) from ci-postgres-15 into develop
Reviewed-on: AkkomaGang/akkoma#411
2023-01-09 22:16:43 +00:00
7695010268 Prune Objects --keep-threads option (#350)
This adds an option to the prune_objects mix task.
The original way deleted all non-local public posts older than a certain time frame.
Here we add a different query which you can call using the option --keep-threads.

We query from the activities table all context id's where
    1. the newest activity with this context is still old
    2. none of the activities with this context is is local
    3. none of the activities with this context is bookmarked
and delete all objects with these contexts.

The idea is that posts with local activities (posts, replies, likes, repeats...) may be interesting to keep.
Besides that, a post lives in a certain context (the thread), so we keep the whole thread as well.

Caveats:
* ~~Quotes have a different context. Therefore, when someone quotes a post, it's possible the quoted post will still be deleted.~~ fixed in AkkomaGang/akkoma#379
* Although undocumented (in docs/docs/administration/CLI_tasks/database.md/#prune-old-remote-posts-from-the-database), the 'normal' delete action still kept old remote non-public posts. I added an option to keep this behaviour, but this also means that you now have to explicitly provide that option. **This could be considered a breaking change!**
* ~~Note that this removes from the objects table, but not from the activities.~~ See AkkomaGang/akkoma#427 for that.

Some statistics from explain analyse:
(cost=1402845.92..1933782.00 rows=3810907 width=62) (actual time=2562455.486..2562455.495 rows=0 loops=1)
 Planning Time: 505.327 ms
 Trigger for constraint chat_message_references_object_id_fkey: time=651939.797 calls=921740
 Trigger for constraint deliveries_object_id_fkey: time=52036.009 calls=921740
 Trigger for constraint hashtags_objects_object_id_fkey: time=20665.778 calls=921740
 Execution Time: 3287933.902 ms

***
**TODO**
1. [x] **Question:** Is it OK to keep it like this in regard to quote posts? If not (ie post quoted by local users should also be kept), should we give quotes the same context as the post they are quoting? (If we don't want to give them the same context, I'll have to see how/if I can do it without being too costly)
    * See AkkomaGang/akkoma#379
2. [x] **Question:** the "original" query only deletes public posts (this is undocumented, but you can check the code). This new one doesn't care for scope. From the docs I get that the idea is that posts can be refetched when needed. But I have from a trusted source that Pleroma can't refetch non-public posts. I assume that's the reason why they are kept here. I see different options to deal with this
    1. ~~We keep it as currently implemented and just don't care about scope with this option~~
    2. ~~We add logic to not delete non-public posts either (I'll have to see how costly that becomes)~~
    3. We add an extra --keep-non-public parameter. This is technically speaking breakage (you didn't have to provide a param before for this, now you do), but I'm inclined to not care much because it wasn't documented nor tested in the first place.
3. [x] See if we can do the query using Elixir
4. [x] Test on a bigger DB to see that we don't run into a timeout
5. [x] Add docs

Co-authored-by: ilja <git@ilja.space>
Reviewed-on: AkkomaGang/akkoma#350
Co-authored-by: ilja <akkoma.dev@ilja.space>
Co-committed-by: ilja <akkoma.dev@ilja.space>
2023-01-09 22:15:41 +00:00