Provided the quote is private too.
Ideally we’d inline the quoted, private status since not all
remotes may already know the old post and some implmentations
(ourselves included) have trouble fetching private posts.
In practice, at least we cannot yet make use of such an inlined post
anyway defeating the point. Implementing the inlining and ability to
make use of the inlined copy is thus deferred to a future patch.
Resolves: #952
The quote creation interface still isn’t exactly drop-in compatible for
masto-only clients since we do not provide or otherwise deal with
quote-authorization objects which clients are encouraged to check before
even offering the possibility of attempting a quote. Still, having a
consistent paramter name will be easier on clients.
Also dropped unused quote_id parameter from ActivityDraft struct
The only kown user is akkoma-fe and it only ever
accesses the software information. For *oma instances
the full, unfiltered nodeinfo data can be quite large
adding unneeded bloat to API responses.
This would have become worse with the duplication of
account data needed for Masto quote post interop.
In case a client we’re not aware of actually uses more fields from
nodeinfo, a new but temporary config setting is provided as a workaround.
Fixes: #827
This upgrade pulls in a fix to better avoid killing re-actived pools,
obsoletes the need for our own HTTP2 server push workaround and allows
us to use system CA certs without breaking plain HTTP connections.
We tried to to the latter before on a per request basis, but this didn’t
actually do anything and we actually relied on the CAStore package
fallback the entire time. The broken attempt was removed in
ed5d609ba4.
Resolves: #880
This is the first release containg fixes making DecompressResponse
stable enough and suitable to be used by default allowing us to
profit from transport compression in obtained responses.
(Note: no compression is used in bodies we send out, i.e.
ActivityPub documents federated to remote inboxes, since
this will likely break signatures depending on whether
the checksum is generated and checked before or after compression)
Ref: 5bc9b82823
Ref: 288699e8f5
On fedi.absturztau.be the planner did not utilise the context
index for the original version leading to a query plan
100× worse than with this tweaked version.
With PostgreSQL 18 the relative difference is much smaller
but also in favour for the new version with the best observed
instance resulting in nearly half the estimated cost.
Before 32a2a0e5fa the context index
acted as a (based on the name surprising) type index too.
Type-based queries are used in the daily pruning of old e.g. Delete
activities by PruneDatabaseWorker, when querying reports for the admin
API and inferring by the significant change in average execution time
a mysterious COUNT query we couldn’t associated with any code so far:
SELECT count(a0."id") FROM "activities" AS a0 WHERE (a0."data"->>$2 = $1);
Having this as a separate index without pre-sorting results in
an overall smaller index size than merging this into the context index
again and based on it not having been sorted before non-context queries
appear to not significantly profit from presorting.
Metrics showed this endpoint taking unexpectedly long processing times
for a simple readout of an ETS table. Further profiling with
fprof revealed almost all time was spent in URI.merge.
Endpoint.url() is per its documented API contract guaranteed to have a
blank path and all our emoji file paths start with a slash.
Thus, a simple binary concat is equivalent to the result of URI.merge.
This cuts down the profiled time of just the fetching and
rendering of all emoji to a tenth for me. For the overall API request
with overhead for handling of the incoming response as well as encoding
and seding out of the data, the overall time as reported by phoenix
metrics dropped down by a factor of about 2.5.
With a higher count of installed emoji the overall relative time
reduction is assumed to get closer to the relative time reduction of
the actual processing in the controller + view alone.
For reference, this was measured with 4196 installed emoji:
(keep in mind enabling fprof slows down the overall execution)
fprof'ed controller Phoenix stop duration
BEFORE: (10 ± 0.3)s ~250 ms
AFTER: (0.9 ± 0.06)s ~100 ms
Note: akkoma-fe does not use this endpoint, but /api/v1/pleroma/emoji
defined in Pleroma.Web.TwitterAPI.UtilController:emoji/2 which only
emits absolute-path references and thus had no use for URI.merge anyway.
It is never used outside tests
and even there its correctness and/or
worth was questionable. The participations recipients
or just testing over the particiaptions’ user_id seem
like a better fit.
In particular this was always preloaded for API responses
needlessly slowing them down
The handling of remote and local posts should really be unified.
Currently new remote posts did not bump conversations
(notably though, until before the previous commit remote
edits (and only edits), did in fact bump conversations due to being
the sole caller of notify_and_stream outside its parent module)
And consistently treat muted notifications.
Before, when Notifications where immediately sent out, it correctly
skipped streaming nad web pushes for muted notifications, but the
distinction was lost when notification sending was deferred.
Furthermore, for users which are muted, but are still allowed to
create notifications, it previously (sometimes) automatically marked
them as read. Now notifications are either always silent, meaning
- will not be streamed or create web pushes
- will automatically be marked as read on creation
- should not show up unless passing with_muted=true
or active, meaning
- will be streamed to active websockets and cause web pushes if any
- must be marked as read manually
- show up without with_muted=true
Deferring sending out of notifications isdesirable to avoid duplicate
sending and ghost notifications when processing of the actiity fails
later in the pipeline and avoids stalling the ongoing db transaction.
Inspired by https://git.pleroma.social/pleroma/pleroma/-/merge_requests/4032
but the actual implementation differs to not break with_muted=true.
Previously the view received pre-filtered results and therefore
pagination headers were also only generated with pre-filtered
results in mind. Thus the next page would again traverse over
previously seen but filtered out entries.
Curucially, since this last_activity filtering is happening _after_
the SQL query, it can lead to fewer results being returned than
requested and in particular even zero results remaining.
Sufficiently large blocks of filtered participations thus caused
pagination to get stuck and abort early.
Ideally we would like for this filtering to occur as part of the SQL
query ensuring we will laways return as many results as are allowed as
long as there are more to be found. This would oboslete this issue.
However, for the reasons discussed in the previous commit’s message
this is currently not feasible.
The view is the only caller inside the actual application of the
for_user_with_pagiantion* functions, thus we can simply move filtering
inside the view allowing the full results set to be used when generating
pagination headers.
This still means there might be empty results pages, but now with
correct pagination headers one can follow to eventually get the full set
The old pagination logic was inconsistent and thus broken.
It used to order conversations based on updated_at but generated
pagination HTTP Link headers based on participation IDs.
Thus entries could repeat or be left out entirely.
Notably using updated_at also lead to bumps when
merely marking an individual conversation as read,
though not when marking _all_ conversations as read in bulk
since Repo.update_all does not touch date fields autoamtically.
For consistent and sensible "last active" ordering this is replaced
by using the flake ID (which contains the date) of the last status
which bumped the conversation for both ordering and pagination
parameters. This takes into account whether the status was
visible to the participation owner at the time of posting.
Notably however, it does not care about whether the status continues to
exist or continues to be visible to the owner. Thus this is not marked
as a foreign key and MUST NOT be blindly used as such!
For the most part, this should be considered as just a funny timestamp,
which is more resiliant against spurious bumps than updated_at, offers
deterministic sorting for simulateanously bumped conversations and
better usability in pagination HTTP headers and requests.
Implementing this as a proper foreign key with continuously enforced
visibility restrictions was considered. This would allow to just load
the "last activity" by joining on this foreign key, to immediately
delete participations once they become empty and obsoleting the
pre-existing post-query result filtering.
However, maintaining this such that visibility restrictions are
respected at all times is challenging and incurs overhead.
E.g. unfollows may retroactively change whether the participation owner
is allowed to access activities in the context.
This may be reconsidered in the future once we are clearer
on how we want to (not) extend conversation features.
This also improves on query performance by now using a multi-row,
pre-sorted index such that one user querying their latest conversations
(a frequent operation) only needs to perform an index lookup (and
loading full details from the heap).
Before queries from rarely active users needed to traverse newer
conversations of all other users to collect results.
While we already use wrapped return lists for
HTML pagination headers, currently SQL queries
from the SQL pagination helper use the primary key "id"
of some given table binding. However, for participations
we would like to be able to sort by a field which is not
a primary key. Thus allow custom field names.
Since side effect handling of local notes currently can only immediately
stream out changes and notifications (eventhough it really shouldn’t for
many more reasons then this) the transaction inserting the various
created objects into the database is not finished yet when StreamerView
tries to render the conversation.
This would be fine were it using the same db connection as the
inserting transaction. However, the streamer is a different process
and gets sent the in-memory version of the participation.
In the case of newly created threads the preloaded the streamer process
will not be able to load preload it itself since it uses a different db
connection and thus cannot see any effects of the unfinished transaction
yet. Thus it must be preloaded before passing it to Streamer.
Notably, the same reasoning applies to the new status activity itself.
Though fetching the activity takes more time with several prepatory
queries and in practice it appears rare for the actual activity query to
occur before the transaction finished.
Nonetheless, this too should and will be fixed in a future commit.
Fixes: #887
They can never query the participation via API nor edit the recipients,
thus such particiaptions would just forever remain unused and are
a waste of space.
For unfathomable reasons, this did not actually use recipiepent (ship)
information in any way, but compared against all users owning a
participation within the same context.
This is obviously contradicting *oma’s manually managed recipient data
and even without this extension breaks if there are multiple distinct
subthreads under the same (possibly public) root post.
Presorting the index will speed up all context queries
and avoids having to load the full context before sorting
and limiting can even start.
This is relevant when get the context of a status
and for dm conversation timelines.
Including the id column adds to the index size,
but dropping non-Creates also brings it down again
for an overall only moderate size increase.
The speedup when querying large threads/conversations
is worth it either way.
Otherwise, the context attribute is only used as a condition in
queries related to notifications, but only to filter an otherwise
pre-selected set of notifications and this index will not be relevant.
It does not make sense to check for thread mutes here though.
Even if this thread was muted, it being explicitly fetched
indicates it is desired to be displayed.
While this used to load thread mutes, it didn’t actually apply them
until now, so in regard it does not change behaviour either and we
can optimise the query by not loading thread mutes at all.
This does change behavior of conversation timelines however
by now omitting posts of muted users by default, aligning it
with all other timelines.
