migrate to conversation API (replaces "direct" timeline)

Oneric commented

2025-12-20 02:38:41 +00:00

Owner

The "direct" timeline is a poor interface since it mixes different DM threads together and has been deprecated for ages. The newer conversation API improves on this by offering a separate TL for each conversation and an overview over all active conversations and whether they’ve been read yet.

Example image of the overview list in the current draft:

Since we plan to soon drop api/v1/timelines/direct from the backend (as it will be the only remaining users of the known-buggy visibility index; by extension this TL itself is buggy too under the wrong circumstances), this immediately drops the legacy interface from the frontend too rather than offering both for a while.

As it turns out though, while we long supported the conversations API, it is a bit buggy in the backend so we’ll have to fix this too. Potentially also add some more small extensions to make it more convenient.

Remaining tasks

Frontend (this PR)

fix title in timeline menu for conversation TLs
make sure all text buttons use the theme styling by setting `class="btn button-default" (only found out about this late)
sort new keys in i18n alphabetically
allow editing core participants
(manually controlling the set of core particpants is a pre-exsiting, intentional *oma extension; Mastodon changes it automatically based on who’s addressed in the last(?)/all(?)/any(?) message)
fix the visual read indicator not automatically updating when marking as read etc. This used to work but was regressed by some update
ideally, show a total unread conversations count in the nav menu, similar to pending follow requests
This total count is returned when fetching ones own account as pleroma.unread_conversation_count

Backend (AkkomaGang/akkoma#1039)

fix broken pagination; this requires DB changes
accept with_muted in conversation timeline
pre-sort context index; ALL exisiting context queries (it is also used outside of dm conversations) sort it, so this can save quite a bit of time for large threads / long-lived conversations. It’s unfortunately also a fair bit bigger than the unsorted index, but should still be less than what we recently removed and will soon remove
accept JSON bodies in PATCH /api/v1/pleroma/conversation, not just query parameters
(getting it to work with current backend versions necessitated fixing the FEs handling of arrays in query parameter encoding)
check if it is feasible to also automatically mark a conversation as read when faving or emoji reacting to the last_status of it. Currently only replies / new posts to the conversations auto-marks it as read, but going to the conv list and marking stuff as read manually if there’s nothing to reply with is a bit cumbersome and fav/reacts are a clear "read" indicator
despite the improved index and the recent-participations-of-user query now being very fast, the conversations list API endpoint is still sluggish. Might be due to the preloads all being individual queries plus the extra queries for last activities creating delays from many elixir<->db roundtrips. Experiment with reducing preloads (why are conversation users preloaded?) using preloads by join for the remaining and if it doesn't run in a transaction yet anyway, maybe checking out a repo connection for the request so it doesn't need to repeatedly wait for a free connection.
After some testing, there are indeedsome unecessary preloads we can get rid of which speed things up measurably, but it remains sluggish. A Repo.checkout did not significantly improve things further. From manually running the operation in a console and watching the flood of queries (presuming the logging in itself didn’t skew the relative time consumption too much) it seems the most costly part is, even with a pres-sorted index, fetching and rendering the statuses per conversation. Each call to fetch_context_query re-queries a bunch of additional info like mutes, followers, filters, etc of the current user, before then making the query against the context index. De- and serializing this back and forth between the db and elixir comes with more overhead beyond the query times themselves.

(the won’t necessarily all be implemented, or even if not necessarily as part of this initial PR but I’m throwing any idea which seems interesting enough out here for discussion)

expose Mastodon-compatible, auto-updating participants in the top-level key and push manually managed core members down into pleroma.participants as the docs used to claim was already case (it wasn’t and we recently dropped the mention of it from docs)
allow naming conversations
This currently uses "Conversation ${id}" for the display title, which is kinda meh. Husky uses a list of all participants but that’s also not great and scales poorly for many participants. Being able to name long-lived conversations might be neat and easy to implement
add a context_of parameter to status creation, which copies the context of the given status ID for the new root status (parameter should be ignored for replies or cause an error)
We already have explicit addressing; together this would allow showing a compose box at the top of the conversation to add a new message without having to reply to a particular pre-existing post (and without needing to @ all participants)
This can in principle also be used for regular, public threads if anyone finds that useful.
allow to opt-in into manual addressing via to to being additive rather than completely replacing addressing via in-line mentions
Would be convenient to explicitly add a non-core-participant to a subthread without needing to manually add all core-participants too

The "direct" timeline is a poor interface since it mixes different DM threads together and has been deprecated for ages. The newer conversation API improves on this by offering a separate TL for each conversation and an overview over all active conversations and whether they’ve been read yet. Example image of the overview list in the current draft: ![convlist_draft](/attachments/ef7af289-cdf8-4f21-bcb2-990f42cdb044) Since we plan to soon drop `api/v1/timelines/direct` from the backend *(as it will be the only remaining users of the known-buggy visibility index; by extension this TL itself is buggy too under the wrong circumstances)*, this immediately drops the legacy interface from the frontend too rather than offering both for a while. As it turns out though, while we long supported the conversations API, it is a bit buggy in the backend so we’ll have to fix this too. Potentially also add some more small extensions to make it more convenient. ## Remaining tasks ### Frontend (this PR) - [x] fix title in timeline menu for conversation TLs - [x] make sure all text buttons use the theme styling by setting `class="btn button-default" *(only found out about this late)* - [x] sort new keys in i18n alphabetically - [x] allow editing core participants *(manually controlling the set of core particpants is a pre-exsiting, intentional \*oma extension; Mastodon changes it automatically based on who’s addressed in the last(?)/all(?)/any(?) message)* - [x] fix the visual read indicator not automatically updating when marking as read etc. This used to work but was regressed by some update - [x] ideally, show a total unread conversations count in the nav menu, similar to pending follow requests This total count is returned when fetching ones own account as `pleroma.unread_conversation_count` ### Backend (https://akkoma.dev/AkkomaGang/akkoma/pulls/1039) - [X] fix broken pagination; this requires DB changes - [X] accept `with_muted` in conversation timeline - [X] pre-sort context index; ALL exisiting context queries (it is also used outside of dm conversations) sort it, so this can save quite a bit of time for large threads / long-lived conversations. It’s unfortunately also a fair bit bigger than the unsorted index, but should still be less than what we recently removed and will soon remove - [X] accept JSON bodies in `PATCH /api/v1/pleroma/conversation`, not just query parameters *(getting it to work with current backend versions necessitated fixing the FEs handling of arrays in query parameter encoding)* - [ ] check if it is feasible to also automatically mark a conversation as read when faving or emoji reacting to the `last_status` of it. Currently only replies / new posts to the conversations auto-marks it as read, but going to the conv list and marking stuff as read manually if there’s nothing to reply with is a bit cumbersome and fav/reacts are a clear "read" indicator - [ ] despite the improved index and the recent-participations-of-user query now being very fast, the conversations list API endpoint is still sluggish. Might be due to the preloads all being individual queries plus the extra queries for last activities creating delays from many elixir<->db roundtrips. Experiment with reducing preloads *(why are conversation users preloaded?)* using preloads by join for the remaining and if it doesn't run in a transaction yet anyway, maybe checking out a repo connection for the request so it doesn't need to repeatedly wait for a free connection. After some testing, there are indeedsome unecessary preloads we can get rid of which speed things up measurably, but it remains sluggish. A `Repo.checkout` did not significantly improve things further. From manually running the operation in a console and watching the flood of queries *(presuming the logging in itself didn’t skew the relative time consumption too much)* it seems the most costly part is, even with a pres-sorted index, fetching and rendering the statuses per conversation. Each call to `fetch_context_query` re-queries a bunch of additional info like mutes, followers, filters, etc of the current user, before then making the query against the context index. De- and serializing this back and forth between the db and elixir comes with more overhead beyond the query times themselves. ### Extensions? (and related) *(the won’t necessarily all be implemented, or even if not necessarily as part of this initial PR but I’m throwing any idea which seems interesting enough out here for discussion)* - [ ] expose Mastodon-compatible, auto-updating participants in the top-level key and push manually managed core members down into `pleroma.participants` as the docs used to claim was already case *(it wasn’t and we recently dropped the mention of it from docs)* - [ ] allow naming conversations This currently uses "Conversation ${id}" for the display title, which is kinda meh. Husky uses a list of all participants but that’s also not great and scales poorly for many participants. Being able to name long-lived conversations might be neat and easy to implement - [ ] add a `context_of` parameter to status creation, which copies the context of the given status ID for the new root status *(parameter should be ignored for replies or cause an error)* We already have explicit addressing; together this would allow showing a compose box at the top of the conversation to add a new message without having to reply to a particular pre-existing post (and without needing to `@` all participants) This can in principle also be used for regular, public threads if anyone finds that useful. - [ ] allow to opt-in into manual addressing via `to` to being _additive_ rather than completely replacing addressing via in-line mentions Would be convenient to explicitly add a non-core-participant to a subthread without needing to manually add all core-participants too

convlist_draft.webp

67 KiB

Oneric added 7 commits

2025-12-20 02:38:43 +00:00

Remove DM timeline d64995b0f5

It has been long deprecated and even already removed from Mastodon
and our existing implementation suffers from bugs (at least on
large/some instances), see:
   AkkomaGang/akkoma#798

Except for pleroma-derived web frontends, other clients generally don't
seem to make use of this timeline either, often also omitting an
interface for the conversations API.

Even if it worked properly, this isn’t the best way to present
DMs as all threads with different participants or topics are mixed
together in one linear timeline. The conversations API which suceeded
it in Mastodon and our backend already supports offers a much better
interface.

Add basic UI for conversation API 10f617f225

This replaces the removed "direct" timeline.
Curtnetly this is a read-only interface missing ways to
mark conversations as read, dismiss/delete conversations
or modify the core members of a conversation.

Future work may also add QoL stuff like automatic implicit addressing of
core members or (provided another backend extension) add messages to a
conversation without replying to something particular.

Allow marking conversations as read 0d2d2d7e6e

Allow deleting conversations from overview a458b6a3e9

By default require confirmation to delete a dm conversation 2afef1a4c5

components/dm_conv_card: make compactness and status preview configurable cd80a7fce8

components/dm_conv_timeline: show conversation details in header 6ab6d71b32

This also allows marking the conversation
as read etc from its timeline view.

Oneric force-pushed conversations-api from 6ab6d71b32 to 7a16cf2d58

ci/woodpecker/pull_request_metadata/woodpecker Pipeline was successful

Details

2025-12-21 03:39:05 +00:00

Compare

Oneric force-pushed conversations-api from 7a16cf2d58

ci/woodpecker/pull_request_metadata/woodpecker Pipeline was successful

Details

to 7a913f50d1

ci/woodpecker/pr/woodpecker Pipeline failed

Details

ci/woodpecker/pull_request_metadata/woodpecker Pipeline was successful

Details

2025-12-22 18:13:25 +00:00

Compare

Oneric added 3 commits

2025-12-22 21:38:09 +00:00

components/side_drawer: drop call to non-existent action 55ffd1e029

This is the only place such an action is referenced

Automatically update follow request count shown in sidebar 3cdb20ba57

The current user info fetcher will also be used for
total unread DM counts in the future.

To avoid any future mishaps with improperly stopped fetchers etc,
this  does not use the existing 'setCurrentUser' method but a new
update method with a safeguard against accidentally changing the
identity.

Show total count of unread DM conversations in sidebar

ci/woodpecker/pr/woodpecker Pipeline failed

Details

ci/woodpecker/pull_request_metadata/woodpecker Pipeline was successful

Details

687461a94c

In preparation to an eventual switch to native Masto API format,
as well as flexibility to backends without the extension,
the entity normaliser just copies the count paramter to the same path

Oneric changed title from ~~WIP: migrate to conversation API (replaces "direct" timeline)~~ to migrate to conversation API (replaces "direct" timeline)

2025-12-22 21:38:43 +00:00

Oneric commented

2025-12-22 21:41:00 +00:00

Author

Owner

All frontends TODOs are dealt with and it is already (mostly) usable. Remaining (known) issues are backend bugs (or future™ frontend changes depending on not yet existing backend API extensions)

AS a bonus this now also automatically updates the pending follow request count shown in the sidebar (without needing a page reload as was the case up until now)

All frontends TODOs are dealt with and it is already (mostly) usable. Remaining (known) issues are backend bugs (or future™ frontend changes depending on not yet existing backend API extensions) AS a bonus this now also automatically updates the pending follow request count shown in the sidebar (without needing a page reload as was the case up until now)

Oneric force-pushed conversations-api from 687461a94c

ci/woodpecker/pr/woodpecker Pipeline failed

Details

ci/woodpecker/pull_request_metadata/woodpecker Pipeline was successful

Details

to 0531420797

ci/woodpecker/pr/woodpecker Pipeline failed

Details

ci/woodpecker/pull_request_metadata/woodpecker Pipeline was successful

Details

2025-12-23 16:32:34 +00:00

Compare