The ban on redirects was based on a misreading of FEP-2c59’s
requirements. It is only meant to forbid addresses other than
the canonical ActivityPub ID being advertised as such in the
returned WebFinger data.
This does not meaningfully lessen security and verification still
remains stricter than without FEP-2c59.
Notably this allows Mastodon with its backwards WebFinger redirect
(redirecting from the canonical WebFinger domain to the AP domain)
to adopt FEP-2c59 without causing issues or extra effort to existing
deplyoments which already adopted the Mastodon-recommended setup.
One asserted the response format of finger_actor on a finger_mention
call as a previous iteration of the implementation mistakenly returned.
The other didn’t actually test anything WebFinger but fundamental id
containment and verification for generic AP fetches. Now it does.
It is used in security checks and only due to an abudance of
existing tests lacking it, _only_ the test env is allowed to
fallback to the query URL for theses tests as a temporary
(well, ... it’s been a while now) measure.
We really shouldn’t be adding more deficient tests like that.
Notably user follow* collections faked a zero totalItem count
rather than simply omitting the field and included a link to a first
page even when knowing this page cannot be fetched while most others
omitted it. Omitting it will spare us receiving requests doomed to
failure for this page and matches the format used by GtS and Mastodon.
Such requests occur e.g. when other *oma servers try to determine
whether follow* relationships should be publicly shown. Other
implementations like Mastodon¹ simply treat the presence of a (link to)
a first page aready as an indicator for a public collection. By
omitting the link Mastodon servers will now too honour our users
request to hide follow* details.
The "featured" user collection are typically quite small and thus
the sole occurence of the alternative form where all items are directly
inlined into the root without an intermediate page. Thus it is not
converted but also no helper for this format created.
1: eb848d082a/app/services/activitypub/process_account_service.rb (L303)
The count is precomputed as a user property.
Masto API already relies on this cached value.
This let’s us skip actually querying follow* details unless
follow(ing/ed) users are publicly exposed and thus will be served.
In fact this could now be optimised further by using keyset pagination
to only fetch what’s actually needed for the current page. This would
also completely obsolete the need for the _offset collection page
helpers. However, for this pagination to be efficient it needs to happen
o the follow relation table, not users. This is left to a future commit.
Due to an ambiguity with PhoenixHtmlHelpers the Ecto.Query
select import was unusable without extra qualification,
therfore it is converted to a require expression.
While "is_active" is a property of users, it is not a recognised keyword
for the User.Query helper; instead "deactivated" with negated value must
be used.
This arose because originally the user property was also called
"deactivated" until it was renamed adn negated five years ago
in 860b5c7804. This included renaiming
the parameter in most but not all User.Query usages.
Notably the parameter in User.get_followers_query was renamed
but not in User.get_friends_query (friends == following).
The accepted query parameter in User.Query however was not changed.
This lead to the former mistakenly including deleted users causing
too large values to be reported in the ActivityPub follower, but not
following collection as reported in #1078.
In Masto API responses filtering by `User.is_visible` weeded out
the extra accounts before they got displayed to API users.
On the surface it might seem logical to align the name of the User.Query
parameter with the actual property name. However, User.Query already
accepts an "active" parameter which is an alias for limiting to accounts
which are neither deleted nor deactivated by moderators (both indicated
by is_active) and also are not newly created account requests still
pending an admin approval (is_approved) or necessary email confirmation
(is_confirmed); in short as the alias suggests whether the account is
active. Two highly similar parameter names like this would be much too
confusing.
The renamed "is_active" on the other hand does not actually suffice to
say whether an account is actually active, only whether it has (not yet)
ceased to be active (by its own volition or moderator action).
Meaning its "new" name is actively misleading. Arguably the rename
made things worse for no reason whatsoever and should not ever have
happened.
For now, we’ll just revert the incorrect query helper parameter renames.
Fixes: https://akkoma.dev/AkkomaGang/akkoma/ issues/ 1078
With the follow info update now actually running after being fixed
a bunch of errors about :not_found and :forbidden collection fetches
started to be logged
Even when the "always force revalidation" option is not enabled
while avoiding unnecessary revalidations if nothing changed.
With this heuristic we should be able to change the default to "false"
soon, but for now keep it enabled to help amend recent bugs.
The real intent behind the commit introducing this seemed to have been
avoiding running this when the actor does not expose follow collection ids
ec24c70db8.
This is already taken care of with the :collections_available check.
Some Implementations use other actor type like Group etc for visible,
followable actors making skipping undesirable.
Notably though, this actually has _always_ skipped counter updates
as even when this check was introduced, the user changeset data and
struct used the :actor_type key not :type.
In some situations fetch_follow_information_for_user is called directly
from other modules thus occasionally counters still got updated
for accounts with closer federation relationships masking the issue.
Resolves all security issues discussed in 5a120417fd86bbd8d1dd1ab720b24ba02c879f09
and thus reactivates skipped tests.
Since the necessary logic significantly differs for WebFinger handle dicovery/validation
and fetching of actors from just the webfinger handle the relevant public function was split
necessitating also a partial rewrite of the user fetch logic.
This works with all of the following:
- ActivityPub domain is identical to WebFinger handle domain
- AP domain set up host-meta LRDD link to WebFinger domain
- AP domain set up a HTTP redirect on /.well-known/webfinger
to the WebFinger domain
- Mastodon style: WebFinger domain set up a HTTP redirect
on its well-known path to AP backend (only for discovery
from nickname though until Mastodon supports FEP-2c59)
This intentionally does not work for setups where FEP-2c59 is not
supported and the initially queried domain simply directly responds with
data containing a nickname from another domain’s authority without any
redirecty. (This includes the setup currently recommended by Mastodon,
when enriching an AP actor. Once Mastodon supports FEP-2c59 though its
setup will start to work again too automatically).
While technically possible to cross-verify the data with the nickname
domain, the existing validation logic is already complex enough and
such cross-validation needs extra safety measures to not get trapped
into infinite loops. Such setups are considered broken.
It doesn’t make sense in general (many implementations use ids not nicks in ap_id)
and just wastes time by making additional, unnecessary, failing network requests.
This arguably should have never been committed.
The ActivityPub module is already overloaded and way too large.
Logic for fetching users and user information is isolated from
all other parts of the ActivityPub module, so let’s split it out.
It makes discovery and validation of the desired webfinger address
much easier. Future commits will actually use it for validation
and nick change discovery.
Proper validation of nicknames must consider both the domain
the nickname is associated with _and_ the actor to be assigned this
nickname consent to this.
Prior attempts at securing this wer emade in
a953b1d927 and
0d66237205 but they do not suffice.
The existing code attempted to validate webfinger responses independent
of the actual ActivityPub actor data and only ever consider the former
(yet failed to properly validate even this).
When resolving a user via a user-provided nickname the assignment
done by the provided URL was simply trusted regardless of the actors
AP host or data. When the nresolving the AP id, the nickname from this
original WebFinger query was passed along as pre-trusted data overriding
any discovery or validation from the actual actors side.
This allowed malicious actors to serve rogue WebFinger data associating
arbitrary actors with any nicknames they wished. Prompting servers to
resolve this rogue nickname then creates this nonconsensual assignment.
Notably, the existing code’s attempt at verification (only for domain
consent) used the originally requested URL for comparison against the
domain in the nickname handle. This effectively disabled custom
WebFinger domains for honest servers unless using an host-meta LRDD link.
(While LRDD links were recommend in the past by both *oma nad Mastodon,
today most implementations other than *oma instead
recommend setups emplyoing HTTP redirects.)
Still, this strictness did not prevent spoofing by malicious server.
It did however mean that rogue nickname assignments from an initial
nickname-based discovery were at least undone on the next user refresh
provided :pleroma, Pleroma.Web.WebFinger, :update_nickname_on_user_fetch
was not changed from its default truthy value.
(A renewed fetch via the rogue nickname would re-establish it though)
When enriching an already resolved ActivityPub actor to discover its
nickname the WebFinger query was not done with the unique AP id as a
resource, but a guessed nickname handle.
Furthermore, the received WebFinger response was not validated
to ensure the ActivityPub ID the WebFinger server pointed to
for the final nickname matched the actual ID in the considered
AP actor data.
While the faulty request URI check described above provides some
friction for malicious actors, it is still possible for mischiveous
AP instances to setup a rogue LRDD link poitning to a third-party
domain’s WebFInger and using the freedom provided by the LRDD link
to overwrite the resource value we provide in the lookup. Thus usurping
existing nicknames in another domains authority.
Proposed tweaks to the existing, faulty checks to work with
HTTP-redirect-based custom WebFInger domains would have made it
even easier to usurp nicknames from foreign domains.
For now simply disable custom WebFinger domains as a quick hotfix.
Subsequent commits will partially de-spaghettify the relevant code
and completely overhaul webfinger and nickname handling and validation.
Deleting a whole pack at once didn’t remove its emoji from memory
and ewly updated or added emoji used a wrong path. While the pack
also contains a path, this is the filesystem path while in-memory
Emoji need the URL path constructed from a constant prefix,
pack name and just the filename component of the filesystem file.
Test used to not check for this at all.
Fixes oversight in 318ee6ee17
Provided the quote is private too.
Ideally we’d inline the quoted, private status since not all
remotes may already know the old post and some implmentations
(ourselves included) have trouble fetching private posts.
In practice, at least we cannot yet make use of such an inlined post
anyway defeating the point. Implementing the inlining and ability to
make use of the inlined copy is thus deferred to a future patch.
Resolves: #952
The quote creation interface still isn’t exactly drop-in compatible for
masto-only clients since we do not provide or otherwise deal with
quote-authorization objects which clients are encouraged to check before
even offering the possibility of attempting a quote. Still, having a
consistent paramter name will be easier on clients.
Also dropped unused quote_id parameter from ActivityDraft struct
The only kown user is akkoma-fe and it only ever
accesses the software information. For *oma instances
the full, unfiltered nodeinfo data can be quite large
adding unneeded bloat to API responses.
This would have become worse with the duplication of
account data needed for Masto quote post interop.
In case a client we’re not aware of actually uses more fields from
nodeinfo, a new but temporary config setting is provided as a workaround.
Fixes: #827
This upgrade pulls in a fix to better avoid killing re-actived pools,
obsoletes the need for our own HTTP2 server push workaround and allows
us to use system CA certs without breaking plain HTTP connections.
We tried to to the latter before on a per request basis, but this didn’t
actually do anything and we actually relied on the CAStore package
fallback the entire time. The broken attempt was removed in
ed5d609ba4.
Resolves: #880
This is the first release containg fixes making DecompressResponse
stable enough and suitable to be used by default allowing us to
profit from transport compression in obtained responses.
(Note: no compression is used in bodies we send out, i.e.
ActivityPub documents federated to remote inboxes, since
this will likely break signatures depending on whether
the checksum is generated and checked before or after compression)
Ref: 5bc9b82823
Ref: 288699e8f5
On fedi.absturztau.be the planner did not utilise the context
index for the original version leading to a query plan
100× worse than with this tweaked version.
With PostgreSQL 18 the relative difference is much smaller
but also in favour for the new version with the best observed
instance resulting in nearly half the estimated cost.
Before 32a2a0e5fa the context index
acted as a (based on the name surprising) type index too.
Type-based queries are used in the daily pruning of old e.g. Delete
activities by PruneDatabaseWorker, when querying reports for the admin
API and inferring by the significant change in average execution time
a mysterious COUNT query we couldn’t associated with any code so far:
SELECT count(a0."id") FROM "activities" AS a0 WHERE (a0."data"->>$2 = $1);
Having this as a separate index without pre-sorting results in
an overall smaller index size than merging this into the context index
again and based on it not having been sorted before non-context queries
appear to not significantly profit from presorting.
Metrics showed this endpoint taking unexpectedly long processing times
for a simple readout of an ETS table. Further profiling with
fprof revealed almost all time was spent in URI.merge.
Endpoint.url() is per its documented API contract guaranteed to have a
blank path and all our emoji file paths start with a slash.
Thus, a simple binary concat is equivalent to the result of URI.merge.
This cuts down the profiled time of just the fetching and
rendering of all emoji to a tenth for me. For the overall API request
with overhead for handling of the incoming response as well as encoding
and seding out of the data, the overall time as reported by phoenix
metrics dropped down by a factor of about 2.5.
With a higher count of installed emoji the overall relative time
reduction is assumed to get closer to the relative time reduction of
the actual processing in the controller + view alone.
For reference, this was measured with 4196 installed emoji:
(keep in mind enabling fprof slows down the overall execution)
fprof'ed controller Phoenix stop duration
BEFORE: (10 ± 0.3)s ~250 ms
AFTER: (0.9 ± 0.06)s ~100 ms
Note: akkoma-fe does not use this endpoint, but /api/v1/pleroma/emoji
defined in Pleroma.Web.TwitterAPI.UtilController:emoji/2 which only
emits absolute-path references and thus had no use for URI.merge anyway.
It is never used outside tests
and even there its correctness and/or
worth was questionable. The participations recipients
or just testing over the particiaptions’ user_id seem
like a better fit.
In particular this was always preloaded for API responses
needlessly slowing them down
The handling of remote and local posts should really be unified.
Currently new remote posts did not bump conversations
(notably though, until before the previous commit remote
edits (and only edits), did in fact bump conversations due to being
the sole caller of notify_and_stream outside its parent module)
And consistently treat muted notifications.
Before, when Notifications where immediately sent out, it correctly
skipped streaming nad web pushes for muted notifications, but the
distinction was lost when notification sending was deferred.
Furthermore, for users which are muted, but are still allowed to
create notifications, it previously (sometimes) automatically marked
them as read. Now notifications are either always silent, meaning
- will not be streamed or create web pushes
- will automatically be marked as read on creation
- should not show up unless passing with_muted=true
or active, meaning
- will be streamed to active websockets and cause web pushes if any
- must be marked as read manually
- show up without with_muted=true
Deferring sending out of notifications isdesirable to avoid duplicate
sending and ghost notifications when processing of the actiity fails
later in the pipeline and avoids stalling the ongoing db transaction.
Inspired by https://git.pleroma.social/pleroma/pleroma/-/merge_requests/4032
but the actual implementation differs to not break with_muted=true.
Previously the view received pre-filtered results and therefore
pagination headers were also only generated with pre-filtered
results in mind. Thus the next page would again traverse over
previously seen but filtered out entries.
Curucially, since this last_activity filtering is happening _after_
the SQL query, it can lead to fewer results being returned than
requested and in particular even zero results remaining.
Sufficiently large blocks of filtered participations thus caused
pagination to get stuck and abort early.
Ideally we would like for this filtering to occur as part of the SQL
query ensuring we will laways return as many results as are allowed as
long as there are more to be found. This would oboslete this issue.
However, for the reasons discussed in the previous commit’s message
this is currently not feasible.
The view is the only caller inside the actual application of the
for_user_with_pagiantion* functions, thus we can simply move filtering
inside the view allowing the full results set to be used when generating
pagination headers.
This still means there might be empty results pages, but now with
correct pagination headers one can follow to eventually get the full set
The old pagination logic was inconsistent and thus broken.
It used to order conversations based on updated_at but generated
pagination HTTP Link headers based on participation IDs.
Thus entries could repeat or be left out entirely.
Notably using updated_at also lead to bumps when
merely marking an individual conversation as read,
though not when marking _all_ conversations as read in bulk
since Repo.update_all does not touch date fields autoamtically.
For consistent and sensible "last active" ordering this is replaced
by using the flake ID (which contains the date) of the last status
which bumped the conversation for both ordering and pagination
parameters. This takes into account whether the status was
visible to the participation owner at the time of posting.
Notably however, it does not care about whether the status continues to
exist or continues to be visible to the owner. Thus this is not marked
as a foreign key and MUST NOT be blindly used as such!
For the most part, this should be considered as just a funny timestamp,
which is more resiliant against spurious bumps than updated_at, offers
deterministic sorting for simulateanously bumped conversations and
better usability in pagination HTTP headers and requests.
Implementing this as a proper foreign key with continuously enforced
visibility restrictions was considered. This would allow to just load
the "last activity" by joining on this foreign key, to immediately
delete participations once they become empty and obsoleting the
pre-existing post-query result filtering.
However, maintaining this such that visibility restrictions are
respected at all times is challenging and incurs overhead.
E.g. unfollows may retroactively change whether the participation owner
is allowed to access activities in the context.
This may be reconsidered in the future once we are clearer
on how we want to (not) extend conversation features.
This also improves on query performance by now using a multi-row,
pre-sorted index such that one user querying their latest conversations
(a frequent operation) only needs to perform an index lookup (and
loading full details from the heap).
Before queries from rarely active users needed to traverse newer
conversations of all other users to collect results.
While we already use wrapped return lists for
HTML pagination headers, currently SQL queries
from the SQL pagination helper use the primary key "id"
of some given table binding. However, for participations
we would like to be able to sort by a field which is not
a primary key. Thus allow custom field names.
Since side effect handling of local notes currently can only immediately
stream out changes and notifications (eventhough it really shouldn’t for
many more reasons then this) the transaction inserting the various
created objects into the database is not finished yet when StreamerView
tries to render the conversation.
This would be fine were it using the same db connection as the
inserting transaction. However, the streamer is a different process
and gets sent the in-memory version of the participation.
In the case of newly created threads the preloaded the streamer process
will not be able to load preload it itself since it uses a different db
connection and thus cannot see any effects of the unfinished transaction
yet. Thus it must be preloaded before passing it to Streamer.
Notably, the same reasoning applies to the new status activity itself.
Though fetching the activity takes more time with several prepatory
queries and in practice it appears rare for the actual activity query to
occur before the transaction finished.
Nonetheless, this too should and will be fixed in a future commit.
Fixes: #887
They can never query the participation via API nor edit the recipients,
thus such particiaptions would just forever remain unused and are
a waste of space.
For unfathomable reasons, this did not actually use recipiepent (ship)
information in any way, but compared against all users owning a
participation within the same context.
This is obviously contradicting *oma’s manually managed recipient data
and even without this extension breaks if there are multiple distinct
subthreads under the same (possibly public) root post.
Presorting the index will speed up all context queries
and avoids having to load the full context before sorting
and limiting can even start.
This is relevant when get the context of a status
and for dm conversation timelines.
Including the id column adds to the index size,
but dropping non-Creates also brings it down again
for an overall only moderate size increase.
The speedup when querying large threads/conversations
is worth it either way.
Otherwise, the context attribute is only used as a condition in
queries related to notifications, but only to filter an otherwise
pre-selected set of notifications and this index will not be relevant.
It does not make sense to check for thread mutes here though.
Even if this thread was muted, it being explicitly fetched
indicates it is desired to be displayed.
While this used to load thread mutes, it didn’t actually apply them
until now, so in regard it does not change behaviour either and we
can optimise the query by not loading thread mutes at all.
This does change behavior of conversation timelines however
by now omitting posts of muted users by default, aligning it
with all other timelines.
This was supposed to be already handled for both XML and JSON
with d1f6ecf607 though the code
failed to consider variable scopes and thus was broken and
actually just a noop for XML.
For inexplicable reasons 1a250d65af
just outright removed both the failed attempt to parse the canonical
type in XML documents and also serving of the canonical type in our own
XML (and only XML) response.
With the refactor in 28f7f4c6de
the canoncial type was again served in both our own JSON and XML
responses but parsing of the canonical type remained missing
from XML until now.
It did not use the same Emoji object template as other occurrences.
This also fixes an issue with the icon URL not being properly encoded
as well as an inconsistency regarding the domain part of remote
reactions in retractions. All places use the image URL domain
except the query to find the activity to retract relie on the id.
Even before this change this made it impossible to retract remote
emoji reactions if the remote either doesn't send emoji IDs or
doesn't store images on the ActivityPub domain.
Addresses omission in 4ff5293093
Fixes: #1042
Based on the Pleroma patch linked below, but not needlessly
hiding the count if only the listing of the specific follow* accounts is
hidden while counts are still public.
https://git.pleroma.social/pleroma/pleroma/-/merge_requests/4205
Co-authored-by: nicole mikołajczyk <git@mkljczk.pl>
We had a feww files matching neither inclusion nor exclusion rules.
With elixir 1.19 this creates a warning.
Most were intended to be ignored and thus we now override the default
rules in mix.exs to be explicit about this. However, two test files were
simply misnamed and intended to run, but untill now skipped.
Notably, the signing key test for shared key ids currently fails due to
a missing mock. For now this faulty test is simply disabled, the next
commit will fix it
Elixir 1.19 now requires (with a deprecation warning) return_diagnostics
to be set to true for parallel compiles. However, this parameter was only
added in Elixir 1.15, so this raises our base requirement.
Since Elixir 1.14 is EOL anyway now this shoulöd be fine though.
When feasible actually enforce the to-be-updated data being the desired struct type.
For ActivityDraft this would add too much clutter and isn't necessary
since all affected functions are private functions we can ensure only
get correct data, thus just use simple map-update syntax here.
Currently the test is still correctly sensitive to the visibility issue
it wants to test, but it easy to see how this could chane in the future
if it starts considering whether a mute existed in the first place.
Inserting a mute first ensures the test will keep working as intended.
If a user successfully favourited a post in the past (implying they once
had access), but now no longer are alllowed to see the (potentially
since edited) post, the request would still process and leak the current
status data in the response.
As a compromise to still allow retracting past favourites (if IDs are
still cached), the unfavouriting operation will still be processed, but
at the end lie to the user and return a "not found" error instead of
a success with forbidden data.
This was originally found by Phantasm and fixed in Pleroma as part of
https://git.pleroma.social/pleroma/pleroma/-/merge_requests/4400
but by completely preventing the favourite retraction.
Also fixes Bookmark.destroy crashing when called with
parameters not mapping to any existing bookmark.
Partially-based-on: fe7108cbc2
Co-authored-by: Phantasm <phantasm@centrum.cz>
Often raised errors get logged automatically,
but not always and here it doesn't seem to happen.
I’m not sure what the criteria for it being logged or not are tbh.
When a user tried to unpin a status not belonging to them, a full
MastoAPI response was sent back even if status was not visible to them.
Ditto with (un)mutting except ownership.
Cherry-picked-from: 2b76243ec8
As discussed on IRC this is already buggy on large instnaces
and conflicts with future cleanup and optimisation work.
The only remaining users currently appear to be Pleroma-derived web
frontends, namely akkoma-fe, pleroma-fe, pl-fe and Mangane.
We will switch akkoma-fe over to conversations before the actual
removal.
This was originally added in a97b642289 as
part of https://git.pleroma.social/pleroma/pleroma/-/merge_requests/1818
with the hope/intent of using it to hide DMs by default from timelines.
This never came to be with thisd parameter still being unused today in
al of akkoma-fe, pleroma-fe, Husky, pl-fe, Mangange, fedibird-fe and
masto-fe. Given thypically only a small percentage of posts are DMs,
as well as issues with the SQL function and index used to implement this
a removal likely won't bother anyone while allowing us to clean up
bugged code and improve actually used functions.
While unlikely, we don't have a way to ascertain for sure whether there
are still users so for now only drop it from API Spec which leads to
the parameter being stripped out before processing and keep the actual
logic. If any users exist and complain we can easily revert this, if not
we will follow up with a proper cleanup.
This reduces both the size the indexes take on the disk
and the overhead of maintaining it since it now only
needs to be updated for some entries.
For the email index restriction queries needed to be updated,
the last_active_at index is used only by one query which already
also restricted by "local" and for the remaining restrictions
no query adaptations are needed.
This was replaced by the always-on exclusion from the inspect protocol
implemented in the previous commit which is far more robus.
This partially reverts commit 2b4b68eba7;
the more detailed response for generic HTTP errors is retained.
Duped code just means double the chance to mess up. This would have
prevented the leak of confidential info more minimally fixed in
6a8b8a14999f3ed82fdaedf6a53f9a391280df2f and now furthermore
fixes the representation of Update activites which _need_ to have their
object inlined, as well as better interop for follow Accept and Reject
activities and all other special cases already handled in Transmogrifier.
It also means we get more thorough tests for free.
This also already adds JSON-LD context and does not add bogus Note-only
fields as happened before due to this views misuse of prepare_object
for activities. The doc of prepare_object clearly states it is only
intended for creatable objects, i.e. (for us) Notes and Questions.
When the object associated with the activity was preloaded
(which happens automatically with Activity.normalize used in the
controller) Object.normalize’s "id_only" option did not actually work.
This option and it’s usage were introduced to fix display of Undo
activities in e88f36f72b.
For "Undo"s (and "Delete"s) there is no object preloaded
(since it is already gone from the database) thus this appeared
to work and for the particular case considered there in fact did.
Create activities use different rendering logic and thus remained
unaffected too.
However, for all other types of Activities (yes, including Update
which really _should_ include a properly sanitised, full object)
this new attempt at including "just the id", lead to it instead
including the full, unsanitised data of the referenced object.
This is obviously bad and can get worse due to access restrictions
on the activity being solely performed based on the addressing
of the activity itself, not of the (unintentionally) embedded
object.
Starting with the obvious, this leaks all "internal" fields
but as already mentioned in 8243fc0ef4
all current "internal" fields from Constants.object_internal_fields
are already publicised via MastoAPI etc anyway. Assuming matching
addressing of the referenced object and activity this isn't problematic
with regard to confidentiality.
Except, the internal "voters" field recording who voted for a poll
is currently just omitted from Constants.object_internal_fields
and indeed confidential information (fix in subsequent commit).
Fortunately this list is for the poll as a whole and there are no
inlined lists for individual choices. While this thus leaks _who_
voted for a poll, it at least doesn't directly expose _what_ each voter
chose if there are multiple voters.
As alluded to before, the access restriction not being aware
of the misplaced object data into account makes the issue worse.
If the activity addressing is not a subset of the referenced object’s
addressing, this will leak private objects to unauthorised users.
This begs the question whether such mismatched addressing can occur.
For remote activities the answer is ofc a resounding YES,
but we only serve local ActivityPub objects and for the latter
it currently(!) seems like a "no".
For all intended interactions, the user interacting must already have
access to the object of interest and our ActivityPub Builder
already uses a subset of the original posts addressing for
posts not publicly accessible. This addressing creation logic
was last touched six years ago predating the introduction of this
exposure blunder.
The rather big caveat her being, until it was fixed just yesterday in
dff532ac72 it was indeed possible to
interact with posts one is not allowed to actually see. Combined, this
allowed unauthorised access to private posts. (The API ID of such
private posts can be obtained e.g. from replies one _is_ allowed to see)
During the time when ActivityPub C2S was supported there might have been
more ways to create activities with mismatched addressing and sneak a
peek on private posts. (The AP id can be obtained in an analogous way)
Replaces and fixes e88f36f72b.
Since there never were any users of the
bugged "id_only" option it is removed.
This was reported by silverpill <silverpill@firemail.cc> as an
ActivityPub interop issue, since this blunder of course also
leads to invalid AP documents by adding an additional layer
in form of the "data" key and directly exposing the internal
Pleroma representation which is not always identical to valid AP.
Fixes: #1017
In our CI we sometimes fail assert_receive statmenets even in retry
making CI results unreliabl'ish. The default is 100ms, now it is 1s.
See e.g: https://ci.akkoma.dev/repos/1/pipeline/504/5
It was mistakenly removed as part of the bundled frontend in
82fa766ed7 but actually it
is used as the default value for user profile banners.
Mastodon API typespecs define the relevant field as non-nullable in
585545d0d5/app/javascript/mastodon/api_types/accounts.ts (L30)
and clients do in fact rely on it being non-null, meaning we can't just
omit the value if a user didn't setup a bespoke banner.
Signed-off-by: nicole mikołajczyk <git@mkljczk.pl>
For future use. Mark all current frontends as first-party.
The key is named 'blind_trust' instead of 'first_party' to (hopefully)
increase the chances someone notices if some random, malicous frontend
adds the key into their install instructions
It doesn't make sense to like, react, reply, etc to something you cannot
see and is unexpected for the author of the interacted with post and
might make them believe the reacting user actually _can_ see the post.
Wrt to fav, reblog, reaction indexes the missing visibility check was
also leaking some (presumably/hopefully) low-severity data.
Add full-API test for all modes of interactions with private posts.
If "thread containment" isn’t skipped the API will only return posts
whose entire ancestor chain is also visible to the current user.
For example if this contianment is active and the current user
follows A but not B and A replies to a followers-only post of B
the current user will not be able to see A’s post eventhough
per ActivityPub-semantics (an behaviour of other implementations)
the current user was addressed and has read permissions to A’s post.
(Though this semantics frequently surprise some users.)
There is a user-level option to control whether or not to perform
this kind of filtering and a server-wide config option under :instance.
If this containment is _disabled_ (i.e. skip_thread_containment: true)
server-wide, user-level options are ignored an filtering never takes
place.
This is cheked via the database function "thread_visibility" which
recursively calls itself on the ancestor of the currently inspected post
and for each post performs additional queries to check the follow
relationship between author and current user.
While this implementation introduced in
https://git.pleroma.social/pleroma/pleroma/-/merge_requests/971
performs better than the previous elixir-side iteration due to
less elixir-database roundtrips the perf impact is still ridiculously
large and when fetching and entire conversation / context at once there
are many redundnat checks and queries.
Due to this an option to dis/enable the "containment" was added and
promptly defaulted to disabling "containment" in
593b8b1e6a six years ago.
This default remained unchanged since and the implementation wasn’t
overhauled for improved performance either. Essentially this means
the feature has already been entirely disabled oout-of-the box
without too much discoverability for the last six years. It is thus
not too unlikely that no actual users of it exist today.
The user-level setting also didn’t made its way into any known clients.
Surveying current versions of akkoma-fe, husky, pleroma-fe, pl-fe,
Mangane and just to be sure also admin-fe, fedibird-fe and masto-fe
none of them appears to expose a control for the user-level setting.
pl-fe’s pl-api acknowledges the existance of the setting in the API
definition but the parameter appears unused in any actual logic.
Similarly Mangane and pl-fe have a few matches for
"skip_thread_visibility" in test samples of user setting responses
but again no actual uses in active code.
While the idea behind the feature is sensible (though care must be taken
to not mislead users into believing _all_ software would apply the same
restrictions!), the current implementation is very much not sensible.
With the added code complexity and apparent lack of users it is unclear
whether keeping the feature around and attempting to overhaul the
implementation is even worth it.
Thus start pritning a big fat warning for any potentially existing users
prompting for feedback. If there are no responses two releases from now
on it will presumably be safe to just entirely drop it.
It only served for a niche, admin nice-to-have informational stat
without too much value but was unreasonably costly to maintain
adding overhead with multiple queries added to all modifications
to the fairly busy activities table.
The database schema of the counter table and the activity_visibility function
used for counter updates also did not know about "local" visibility (nor the
recently removed "list" visibility) and misattributed them to the "direct" counter.
On my small instance this nearly halved the average
insert time for activiteis from 0.926 ms to 0.465 ms.
No known client ever used this. Currently among akkoma-fe, pleroma-fe,
Husky, Mangane and pl-fe only the latter acknowledes the existence of
the in_reply_to_conversation_id paramter in its API definitions,
but even pl-fe does never actually use the parameter anywhere.
Since the API parameter already was converted to DMs internally,
we do not need to make special considerations for already existing
old conversation-addressed posts. Since they share the context they
should also continue to show up in the intended thread anyway.
The pleroma.participants subkey mentioned in docs did already not exist
prior to this commit. Instead the accounts key doesn’t automatically update
and this affects conversations retrieved from the Mastodon API endpoint too
(which may be considered a pre-existing bug).
If desired clients can already avoid unintended participant additions
by using the explicit-addressing feature originally introduced in
https://git.pleroma.social/pleroma/pleroma/-/merge_requests/1239.
With the above-mentioned feature/bug of conversation participants
not updating automatically it can replace almost everything
conversation addressing was able to do. The sole exception being
creating new non-reply posts in the same context.
Neither conversation addressing nor explicit addressing
achieves robust, federated group chats though.
Resolves: #812
This feature was both conceptually broken and through bitrotting
the implementation was also buggy with the handling of certain
list-post interactions just crashing.
Remote servers had no way to know who belongs to a list and thus
posts basically showed just up as weird DM threads with different
participants on each instance. And while on the local instance
addition and removal from a listed grated and revoked post
access retroactively, it never acted retroactively on remotes.
Notably our "activity_visibility" database function also didn’t
know about "list visibility" instead treating them as direct messages.
Furthermore no known client actualy allows creating such messages
and the lack of complaints about the accumulutaed bugs supports
the absence of any users.
Given this there seems no point in fixing the implementation.
To reduce complexity of visibility handling it will be dropped instead.
Note, a similar effect with less federation weirdness can already be achieved
client-side using the explicit-addressing feature originally introduced in
https://git.pleroma.social/pleroma/pleroma/-/merge_requests/1239.
Ref: #812
While *oma, *key, GtS and even Mastodon federate a full media type for attachments,
posts from Bridgy only contain a generic type and the URLs also appear to never end
with a file exstension. This lead to our old type detection always classifying them
as "unknown" and it showing up like a generic document attachment in frontends.
We can (for vanilla Masto API clients) avoid this by falling back to the
federated generic type.
(Note: all other software mentioned at the start appears to always use "Document"
for the generic type of attachments regardless of the also federated actual full type)
For clients relying on the full mime type provided by an *oma extension,
like currently akkoma-fe, this in itself does not fix the display
but it is a necessary prerequisite to handling this more gracefully.
A MIME type MUST always contain both the type and subtype part.
Also, we already add this binary type for new incoming attachments
without a pre-existing MIME type entry anyway.
This caches query results from our own database, but does not respect
transaction boundaries or syncs to transaction success/failure. Long
lifetimes increase the chance of such desync occuring and being written
back to the database, see: #956
Until 1d02a9a35d recently fixed it
this used a 3 second lifetime anyway, so this won’t result in
performance degradations but hopefully prevents a rise in desyncs.
This avoids the importing the heavy Gettext backend in some places
and makes it clearer what’s actually used in the project and what’s
used by the Gettext library.
With the to-be-pulled-in Gettext API change this split
will be even more helpful for code clarity.
As a bonus documentation is improved and
the unused locale_or_default function removed.
Only the default parameters changed from 3.x to 4.x.
It now matches the proposed defaults suggested in the RFC
for constrained environments.
The prior defaults have worse latency, but probably slightly better
security while spawning fewer threads, so let’s stick with them.
By setting the defaults in the config instance owners
can (continue to) tweak these for the specific setup.
Or at least they were supposed to be compatible.
Due to an ecto optimisation change it is now illegal to use
replace_all_except if the union of conflict_targets and the fields
exempted from updating are equal to _all_ fields of this table.
See: https://github.com/elixir-ecto/ecto/issues/4633
Mastodon 3.3 added support for temproary mutes but uses "duration"
instead of our older "expires_in". Even Husky only sets "duration"
nowadays.
Signed-off-by: marcin mikołajczak <git@mkljczk.pl>
Cherry-picked-from: 5d3d6a58f7
Of course the aprent post might still be deleted after the reply was
already created, but in this case the reply will still show up as a
reply and be federated as a reply with a reference to the parent post.
If the parent was already deleted before the reply gets created however
it used to be indistinguishable from a root post both in Masto API and
ActivityPub.
From a UX perspective, users likely will like to know if the post
they’re replying to no longer exists by the time they finished writing.
The natural language error will show up in akkoma-fe without clearing
the post form, meaning users can decide to discard the reply or copy it
to post as a new root post. It seems sensibly to for other clients to
behave like this too, but so far no more clients were actually tested.
Furthermore, this used to allow replying to all sorts of activities not
just posts which was rather non-sensical (and after all processsing
steps turned into a reply to the object referenced by the activity).
In particular this allowed replying to an user object by specifying the
db ID of a follow request activity (if the latter was somehow obtained).
Note: empty-string in_reply_to parameters are explicitly ignored since
45ebc8dd9a to workaround one buggy client;
see: https://git.pleroma.social/pleroma/pleroma/-/issues/355.
It’s not clear if this workaround is still necessary,
but it is preserved by this commit.
Resolves: #522
While the function signature allows returning many errors at once,
OpenApiSpex.cast_and_validate currently only ever returns the first
invalid field it encounters. Thus we need to retry multiple times to
clean up all offenders.
Fixes: #992 (comment)
Markdown requires an indentation of 4 for a following paragraph to
continue a list item. Previously, the continuing paragraphs were only
indented by 2 spaces, leading to the list being interrupted and
numbering restarted each time.
No caller of `reload` actually uses the result in any way
so there’s no need to wait for a response and risk running
into a timeout (by default 5 seconds).
Discovered-by: sn0w <me@sn0w.re>
Based-on: 1fb54d5c2c
Reloading the entire emoji set from disk, reparsing all pack JSON files,
etc is unnecessarily costly for large emoji sets. We already know which
single or few changes we want to apply, so do just that instead.
No caller cares about the order
(and although, rare with concurrent reads at same time like a write
the table might return unordered results anyway).
Unordered sets have a constant read time,
ordered sets logarithmic times, but there’s no benfit for us
Display will fail for all but Create and Announce anyway since
0c9bb0594a. We exclude Announce
activities from redirects here since they are not identical
with the announced post and akkoma-fe stripping the repeat header
on he /notice/ page might lead to confusion about which is which.
In particular those redirects exiting breaks the assumptions from
the above commit’s commit message and made it possible to obtain
database IDs for activities other than one’s own likes allowing
slightly more mischief with the rendering bug it fixed.
Note: while 0c9bb0594a speculated about
public likes also leaking IDs to other users, the public like endpoint
is actually paginated by post id/date not like id/date like the private
endpoint. Thus it does not allow getting database IDs of others’ likes.
Happens commonly for e.g. replies to follower-only posts
if no one one your instance follows the replied-to account
or replies/quotes of deleted posts.
Before this change Masto API response would treat those
replies as root posts, making it hard to automatically or
mentally filter them out.
With this change replies already show up sensibly as
recognisable replies in akkoma-fe.
Quotes of unavailable posts however still show up as if they
weren’t quotes at all, but this can only be improved client-side.
Fixes: #715
They do nothing. As documented[1] only three specific
options regarding timeouts are parsed for individual request
and none of them is set by AdapterHelper, only pool-specific options.
In particular this means we always relied on Mint’s default CA cert
verification based on queries to the CAStore package (which we include).
[1]: https://hexdocs.pm/finch/Finch.html#request/3-options
It required a bunch of and even call-specific boilerplate
and is not necessary since we can just capture the real logger
as laready done in other tests.
While its data was included in healthcheck responses,
it was not used to determine the healthy status
and for informational purposes Prometheus metrics,
ObanWeb dashboard or the Phoenix live dashboard are all better fits.
In particular, the data shown in healtcheck responses had no temporal
information, but there’s quite a difference between X failures scattered
across many days of uptime and X failures within a couple minutes.
If the id of another activity type was used
it would show the post referenced by the activity
but wrongly attributing it to the activity actor
instead of the actual author.
E.g. ids of like activities can be obtained from
pagiantion info of the favourites endpoint.
For all other activity types the id would need to be guessed
which is considered practically infeasible for Flake UUIDs.
This should’ve been mostly harmless in practice, since:
- since the activity has the same context as the original post,
both the original and misattributed duplicate will show up in the
same thread
- only posts liked by a user can be misattributed to them,
presumably making it hard/impossible to associate someone with
content they disagree with
- by default only the liking user themself can view their like history
and therefore obtain IDs for their like activities.
Notably though, there is a user seting to allow anyone to browse
ones like history and therefore obtain like IDs. However, since
akkoma-fe has no support for actually displaying those, there might
be no actual users of this features.
This is more intuitive for users.
On the flip side, this makes the API more confusing
since now min_id/max_id no longer correspond to ids returned in the
response and only link headers can be used to traverse all response
pages. However, Mastodon already does this according to its
documentation, so clients should already handle this well.
The only other usage of get_follow_requests_query is only interested
in the total count of requests (from active users), thus changing its
select part is safe.
Also gets rid of (outside tests unused) User.get_follow_requests.
Fixes: #380
The old approach required adding a special virtual field
to any table potentially needing such foreign-id pagination and
also still required manually sorting according to pagiantion settings
since the pagination helper does not know whether
this virtual field was set or not.
Using lists with each entry containing the pagination id and the actual
entry insterad allows any table to use this mechanism unchanged and
does not require manually sorting.
Since it was unused, this also drops the pagination mode paramter from
fetch_favourited_with_fav_id.
Furthermore, as a side effect of this change a bug in the favourite
benchmark is fixed. It used to incorrectly attempt to use IDs of
the liked objects for pagination instead of the like IDs as advertised
in Link headers.
Split into scheduled (intentionally delayed until a later trigger date)
and available (eligible for immediate processing but did not yet start).
This will help in diagnosing overloaded instances or too-low queue
limits as well as expose configuration mishaps like
#924.
(The latter by violently crashing the telemetry poller process while
attempting put_in for a non-configured queue creating well visible logs)
Instancce stats are cached only renewed every 5 minutes anyway
and IO stats are cumulative over the entire runtime so no info
is lost.
Polling those every 10s is wasteful and the next commit will add a
periodic measurement which is (comparetively) more costly to compute.
Overall fairly minor changes, but most importantly the Host header passed to the backend is set to `$host` instead of `$http_host` as HTTP/3 no longer uses the Host header, leaving it unset. `$host` will fallback to the `server_name` value in that situation while `$http_host` doesn't.
Ref: https://community.nginx.org/t/proxy-params-by-default-blocks-reverse-proxy-for-http-3/6792
It’s only used in one place and there not even all of
its functionality is needed. It’s not only simpler and shorter,
but easier to understand if Tesla’s keyword list is just inlined.
The only useful bit which is now migrated to Pleroma.HTTP is
addition of the user-agent header (except, sometimes, in tests)
The web_push_encryption lib assumes HTTPoison semantics
which is why we also need to convert the header format.
Inspecting the libraries source shows that Tesla won’t
understand the options anyway and its only used to enable TLS/SSL.
When this was ported from Pleroma in
5da9cbd8a5
we did not take into acount that Akkoma’s and Pleroma’s
HTTP backend take different options.
There’s no need for the :pool option
and enforcing a body limit on download
is currently not possible with Finch
Ideally we’d use a single common HTTP request error format handling
for _all_ HTTP requests (including non-ActivityPub requests, e.g. NodeInfo).
But for the purpose of this commit this would create too much noise
and it is significant effort to go through all error pattern matches etc
too ensure it is still all correct or update as needed.
The Signature module now handles interaction with the HTTPSignature library
and the plug everything related to HTTP itself. It now also no longer needs to be public.
To achieve this signatures are now generated by a custom
Tesla Middleware placed after the FollowRedirects Middleware.
Any requests which should be signed needs
to pass the signing key via opts.
This also unifies the associated header logic between fetching and
publishing, notably resolving a divergence wrt the "host" header.
Relevant spec demands the host header shall include a port
identification if not using the protocols standard port.
Fixes: #731
While the previous version was already plenty fast for smaller instances
there seems to be a more-than-linear slowdown for large instances.
These prefilters should (hopefully) not exclude anything in need of
fixing, but seem to already cut down significantly on the overall
processing time.
The remote part is included in federated emoji names by e.g.
Iceshrimp.NET ever since remote emoji support was added in
4d21aa1670
and as of writing it still continues to do so.
It adds no value for us though; we add the remote part automatically
based on the URL and it makes it more difficult to correctly coalesce
the original reaction (from a user for whom the moji was local)
and the subsequent reactions with the identical emoji from users of
other instances. Additionally the remote part can cause issues when
later used with our REST API.
For non-reactions this is unproblematic and thus
there’s no need to change anything there.
Use a migration to fix up existing activities.
This will cause some (further) desync from the inlined reactions
array, but will be fixable with the resync mix task and avoids
issues when running the resync without first fixing existing activities.
This was already removed from mix.exs in
ea5a2a9f21
but as it turns out it was also re-set
during runtime.
Since we never set it outside of CI in
the first place there’s no need to
force-disable it here.
Even if --keep-threads is used, replies of
pinned posts might still be pruned as documented
for this option.
Thus keep-threads is no reason to skip reply counter recalculation.
ActivityPub spec demands each actor has at least an inbox and outbox.
Furthermore, the current representation wouldn’t even be accepted by
ourselves, since our processing requires objects to be flagged with a
sensible type else we don't know what to do with it.
Including the nickname is just a peemptive measure.
There were no reports of this causing problems in real-world deployments
and at least for federation with other Akkoma instances we should have
never run into this, since we _always_ expose the full representation of
the instance actor and atm also always use the latter for fetching
remote content (which prevents us from fetching followers-only content).
Nonetheless, serving something which violates spec and we wouldn’t even
accept ourselves seems obviously bad, so fix it and add tests to prevent
this from reoccuring.
Fixes bug introduced in 8f322456a0
Presumably those inlined copies were added to avoid the need for queries
each time the info is needed. However, they tend to desync from the actual
activities for not yet fully understood reasons; see:
#956
As a workaround until the root cause is identified and fixed and/or
we no longer rely on the inliend copies add a mix task to regenerate
the inlined "cache" from the authorative activity data.
Does not yet deal with inlined emoji reactions
since its format is a pain to deal with.
This emulates the previous effective behaviour of only
running tests on pull requests and skipping straight
to building artefacts and docs after merge.
Pull requests which do not pass tests shouldn’t be merged in the first place.
Most HTTPS requests actually fall into the single-digit millisecond
range or below on average. Even the more costly endpoints almost always
average around the lower third of the millisecond magnitude.
Only endpoints doing synchronous remote HTTP fetches (e.g. for signing
keys) occasionally spike into the order of seconds.
As is, the bucket resolution is completely unfit to reason about
anything and even just averages are better indications.
Most database queries take less than a millisecond and even in total
almost all take less than 50ms for me. Decode time is but a tiny
fraction of that and queue time usually only takes a small part of total
time too (but may spike on high load).
Shift the buckets down to be able to
give insight into all relevant cases.
In particular this allows to determine whether high averages
are the result of generally high processing times or just a few
outliers lifting the whole average up (e.g. slow network fetches).
Exact numbers are biased towards my setup for lack of other comparison
data, but at least the order of magnitude should be ok everywhere.
The old code was unnecessarily complicated, full of unused and/or
duplicated functions making it hard to understand what will actually
happen and for whom at runtime.
Since we only support a single HTTP backend this can be greatly simplified.
Now everything gets default options from a single place and only
functions to modify parts actually difffering across calls are exposed.
No HTTP3/QUIC support yet.
Note, allowing both here means we don't actually profit from HTTP2 multiplexing
due to Finch(? or maybe a dependency of Finch?) limitations. But it means we can
now interact with HTTP2-only instances (if such exist) and still may get minor
gains from header compression etc
Adventurous admins can change the config to allow only HTTP2,
thus profiting from multiplexing (but breaking federation with
HTTP1-only instances which are in fact observed to exist).
The next Tusky release is going to remove support for the v1 filters API,
see: https://codeberg.org/tusky/Tusky/pulls/5215.
Since Akkoma doesn't support the v2 API this
could cause significant issues for Akkoma users.
And improve monitoring documentation in particular
more detailed instructions for setting up Prometheus
metric scraping and the reference Grafana dashboard.
Currently translated at 0.1% (2 of 1004 strings)
Translated using Weblate (Turkish)
Currently translated at 100.0% (0 of 0 strings)
Added translation using Weblate (Turkish)
Co-authored-by: Anonymous <noreply@weblate.org>
Co-authored-by: Hasan Yıldız <hasanyildiz0@yaani.com>
Co-authored-by: Weblate <noreply@weblate.org>
Translate-URL: http://translate.akkoma.dev/projects/akkoma/akkoma-backend-config-descriptions/tr/
Translation: Pleroma fe/Akkoma Backend (Config Descriptions)
the :do_not_federate checks were omitted from the undo pipeline,
which could lead to them federating.
this commit enforces :outgoing_blocks on undos as well - and
any existing value of :do_not_federate is preserved, if one exists
Fixes#957
Resolves interop issue with a (reverted but possibly returning) bridgy change
as was reported in the comments of
#831.
This won't change anything for the problem originally reported there.
Notably we now always fetch the full collection (up to the configured
item count limit) instead of only using the first page if its link was
inlined.
If only steps are conditional the whole CI workflow
will be held up waiting until a slot is available to start them
just to then not do anything at all.
This allows us to drop "when" condition from individual steps
whenever it is now redundant with the top-level condition.
The lint pipeline spent ~7 minutes downloading and compiling
and only a few seconds actually checking the style.
The former is fully redundandt with what’s done during test anyway.
- This adds extra tests to be sure that scrubbing still happens.
- When doing this I notices that the htmlMfm key wasn't stored in the database when comming through the federator. This has been now been fixed too.
- We also test that values true, false or no attribute all work for incomming messages.
Previously all such requests led to '401 Unauthorized'
whih might have triggered retries.
Now, to not leak any MRF info, we just indicate an
accept for POST requests without actually processing the object
and indiscriminately return "not found" for GET requests.
Notably this change also now causes all signed fetch requests from
blocked domains to be rejected even if authorized_fetch isn’t enabled.
Fixes: #929
To make it usable in scenarios without a draft.
The next commit adds a user for the new function.
This does technically change behaviour a bit, since
"private" relies to "direct" messages no longer implicitly
address the parent post’s actor, but this seems like a contrived
scenario and was likely never intended to actually occur anyway
as cocorroborated by the absence of tests for it.
A pool timeout shorter than the receive timeout
makes race conditions leading to active connections
being killed more likely and laso just doesn’t make
much sense in general.
See: https://github.com/sneako/finch/pull/292
This was added in a924e117fd
with its name mirroring Finch’s own config option, but with
pool_timeout such a setting already existed since
2fe1484ed3 and the new one
was never actually used.
It may still crash due to a race condition between checking for file
existence and opening/streaming, but File.stream! has no safe version
we can use to avoid this completely.
Just not deleting such files during a reload is easy enough.
Until now only a limited number of self-replies were inlined as an
anonymous, unordered ActivityPub collection. Notably the advertised
replies might be private posts.
However, providing all (non-private) replies allows for better thread
consistency across instances if the remote server cooperates.
The collection existing as a stndalone object has two advantages
for this. For one, if it was still anonymous, _all_ replies would need
to be inlined, which might be too bloated in pathological cases.
Secondly, it allows remote servers to update the thread by traversing
the reply collection independent of the original post. (If the remote
part knows about chronological ordering, it can in theory
even efficiently resume from where it previously stopped)
An OrderedCollection uses orderedItems instead of the items key.
So far inlined orderedCollections thus failed to be processed.
Ordered replies collections are used e.g. by IceShrimp.NET and Sharkey,
while Mastodon uses a partially inlined plain Collection.
Not all endpoints use OpenAPISpex’ string-to-atom mapping
and they’ll end up with path params being promoted to
query params in pagination next/prev links.
Fix this by never including path params in the first place
Ecto.cast is will convert valid string keys to atoms, but can
only deal with inputs which use either string keys everywhere
or atom keys everywhere.
Since :id_type is used before the case it must be an atom,
thus it was impossible to use it with string paramteres before.
Up until now queries were always forced into descending ID order
(reverse chronological order with our ID schemes).
Now it’s possible to request the reverse by passing `oder_asc: true`.
The initial info message listing all found packs ought to be sufficient
and with many packs installed thiscan create multiple pages of log
messages on each emoji reload or server start.
Any errors or non-indexed packs are still logged to higher levels.
This requirement was originally added together with splicing the
inbox owner into the non b* addressing fields to make bcc transports
work in https://git.pleroma.social/pleroma/pleroma/-/merge_requests/390.
Later on this was relaxed to always allow deliveries devoid of any
addressing at all in f6cb963df2
and always allow deliveries from actors the owner is following in
750b369d04 to fix interop issues with
Mastodon and Honk respectively.
The justification for both the filtering and splicing comes from
one sentence in AP spec’s inbox section:
> In general, the owner of an inbox is likely
> to be able to access all of their inbox contents.
While this may provide plausible justification for splicing the owner
into cc, it is less clear how this requires or justifies the set of
filtering rules employed here.
Surveying a few other implementations no similar
filtering or splicing appears to be employed.
Furthermore, spec-compliant servers will strip bto/bcc _before_
delivery to remote servers, meaning any compliant bcc transport
out there will NOT contain any explicit addressing of the inbox owner.
Thus the addressing requirement directly opposes
the goal of the original patch.
Currently the requirement for the owner to be addressed once again
is causing interop issues. It turns out to be the root cause of
a long-standing (2+ years) bug preventing meaningful federation.
Bridgy sends e.g. Follow activities and Accepts for Follows directly
to the affected user’s personal inbox while solely addressing
the public scope in the to field. Notably follow relations never
getting established prevented the "accept if followed" allow rule
to ever come into effect.
To make matters worse non-addressed messages simply lead to a
vague "internal server error" response being sent back
which likely slowed down locating the issue.
Furthermore additional issues wrt to signatures cropped up after
the 500-response issues wa first reported, but they seem to have
already been fixed in the meantime, possibly with the signature
handling overhaul in Akkoma.
Given it repeatedly caused issues, does not appear to align with common
practice in the wider fedi ecosystem and apparently contradicts its
original intention, simply remove the requirement.
This is confirmed to fix bridgy interop.
The addressing splicing actually should also add the inbox owner to bto
or bcc instead of cc, but for now this is not changed and in practice
bto/bcc delivery appears to be basically unused anyway.
Most headers are automatically checked by the library after this
upgrade. But since digest is only required for requests with a body
and body processing is handled outside the lib atm, we need to
explicity pass the presence or absence along or not get feedback
about creating broken signatures.
This makes bugs in our signatures more apparent
allowing faster discovery and fixing
This property was introduced as a way to gauge whether and
how much enabling authfetch might break passive federation in
#312.
However, with the db field defaulting to false, there’s no distinction
between instances without valid signatures and those which just never
attempted to fetch anything from the local instance.
Furthermore, this was never exposed anywhere and required manually
checking the database or cachex state via a remote shell.
Given the above it appears this doesn't actually
provide anything useful, thus drop it.
The most common permanent receiver error arises for likes/boosts
when we don’t yet know the rlevant object and can't fetch it
due to the remote being overwhelmed or otherwise down.
Before this changes all retries were rather rapid
thus not giving the remote enough time to recover
and usually all failing. Now the remote has about 20
minutes to recover before we give up.
Transient errors from race conditions and (presumably)
weird database-cache interactions also occur regularly.
However, they resolve within the first one or two retries
and those intial retries still happen relatively quickly.
Only scrubbing "content" leads to differences between
"content" and "contentMap" eventhough the latter should
ideally match the former exactly for the primary language’s entry.
While ideally, for locally generated posts there should be no difference
between applying the scrubber or not, as it turns out automatically
generated attachment links didn't match the form expected by our default
scrubber.
Currently Akkoma never uses nor exposes the value of contentMap entries,
thus this oversight was harmless wrt to safety and at most pertubed
the language detection for our posts perfomed by remote servers.
Fixes: #928
Despite its name this property is not supposed to be a full URI,
but just a bare domain witout protocol. Furthermore, it’s supposed
to be the WebFinger domain used in userhandles and NOT the domain used
for API and ActivityPub objects (which every caller will already know
anyway).
Not following this caused issues for Pachli and Tusky.
Reported-by: nikclayton
Added in Mastodon 2.9.2 (June 2019) this is plain-text-only and supposed
to be shorter compared to the older description field.
Some clients were reported to require this field to properly function.
Reported-by: https://akkoma.dev/paulyd
The queue was not actually registered leading to
jobs getting scheduled each week but never processed
forever lingering in 'available' state.
Fixes omission in: a4f834a687
And use the project’s preferred spelling when
not referring to specific distro package.
Spelling of RedHat packages was left as is
as I can’t access RedHat’s package repository.
This allows to retain posts and boosts of remote actors with local
follows regardless of age.
With the "full" setting this can be taken further treating such
followed actors just like local users even keeping all posts they
liked or reacated to.
Pinned objects and their threads will be refetched
on user refresh which by default happens after a day
once a user is encountered again in any form including a mention.
We observed pruning pinned objects usually results in heavy load for
hours after a database prune due to a clogged up remote fetch queue as
pinned posts and their threads of many (most?) users get refetched.
Thus do not prune pinned posts by default.
Keeping closer to earlier behaviour this will still prune threads of
pinned posts regardless of --keep-threads if nothing else prevenets it.
Statmenets for keeping and breaking threads vastly differ
and the whole if block doesn't even fit on one screen.
Thus move each version out into its own function to
improve readability
Since bcfbfbcff5 (part of
<#789>), the overhead for
cleaning up orphaned attachments has been drattically reduced.
Most admins are unaware of this option even existing, but may notice an
increase in the size of the uploads directory (or S3 bucket size if used
instead) even if auto-expiring posts are used. This should hopefully
make this problem more manageable.
For those that still encounter performance issues, the setting can still
be disabled if needed.
In theory a pedantic reading of the spec indeed suggests
DMs must only be delivered to personal inboxes. However,
in practice the normative force of real-world implementations
disagrees. Mastodon, Iceshrimp.NET and GtS (the latter notably has a
config option to never use sharedInboxes) all unconditionally prefer
sharedInbox for everything without ill effect. This saves on duplicate
deliveries on the sending and processing on the receiving end.
(Typically the receiving side ends up rejecting
all but the first copy as duplicates)
Furthermore current determine_inbox logic also actually needs up
forcing personal inboxes for follower-only posts, unless they
additionally explicitly address at least one specific actor.
This is even much wasteful and directly contradicts
the explicit intent of the spec.
There’s one part where the use of sharedInbox falls apart,
namely spec-compliant bcc and bto addressing. AP spec requires
bcc/bto fields to be stripped before delivery and then implicitly
reconstructed by the receiver based on the addressed personal inbox.
In practice however, this addressing mode is almost unused. Neither of
the three implementations brought up above supports it and while *oma
does use bcc for list addressing, it does not use it in a spec-compliant
way and even copies same-host recipients into cc before delivery.
Messages with bcc addressing are handled in another function clause,
always force personal inboxes for every recipient and not affected by
this commit.
In theory it would be beneficial to use sharedInbox there too for all
but bcc recipients. But in practice list addressing has been broken for
quite some time already and is not actually exposed in any frontend,
as discussed in #812.
Therefore any changes here have virtually no effect anyway
and all code concerning it may just be outright removed.
Currently FlakeId.flake_id crashes if receiving non-UTF-8 binaries,
but we use it e.g. in the /:nick_or_id path used in akkoma-fe user
profiles.
With the upgrade such invalid binaries simply fail the id check.
Reported-in: https://meta.akkoma.dev/t/frontend-unicodeconversionerror/847
This allows discovering a page represents an ActivityPub object
and also where to find the underlying representation.
Other servers already implement this and some tools
came to rely or profit from it.
The alternate link is provided both with the "application/activity+json"
format as used by Mastodon and the standard-compliant media type.
Just like the feed provider, ActivityPub links are always enabled
unless access to local posts is restricted and not configurable.
The commit is based on earlier work by Charlotte 🦝 Deleńkec
but with fixes and some tweaks.
Co-authored-by: Charlotte 🦝 Deleńkec <lotte@chir.rs>
:discard marks jobs as "discarded", i.e. jobs which permanently failed
due to e.g. exhausting all retries or explicitly being discared due to a
fatal error.
:cancel marks jobs as "cancelled" which does not imply failure.
While neither method counts as a job "exception" in the set of
telemetries we currently export via Prometheus, the different state
is visible in the (not-exported) metadata of oban job telemetry.
We can use handlers of those events to build bespoke statistics.
Ideally we'd like to distinguish in the receiver worker between
"invalid" and "already present or delete of unknown" documents,
but this is cumbersome to get get right with a list of
free-form, human-readable descriptions oof the violated constraints.
For now, just count both as an fatal error.
# but that is cumbersome to get right with a list of string error descriptions
We now add the htmlMfm key when relevant, store this in the database, and we see it when fetching using e.g.
curl -L -H 'Accept: application/activity+json' "$ap_id"
The `@context` of the Activity Pub message now also contains `htmlMfm: https://w3id.org/fep/c16b#htmlMfm`.
When an incomming post has `htmlMfm: true`, we will not re-parse the content.
FEDERATION.md is adapted to show the `htmlMfm` term is used.
CUrrently internal actors are supposed to be identified in the database
by either a NULL nickname or a nickname prefixed by "internal.". For old
installations this is true, but only if they were created over five
years ago before 70410dfafd.
Newer installations will use "relay" as the nickname of the realy actor
causing ii to be treated as a regular user.
In particular this means all installations in the last five years never
made use of the reduced endpoint case, thus it is dropped.
Simplify this distinction by properly marking internal actors asa an
Application type in the database. This was already implemented before by
ilja in #457 but accidentally
reverted during a translation update in
eba3cce77b. This commit effectively
restores this patch together with further changes.
Also service actors unconditionally expose follow* collections atm,
eventhough the internal fetch actor doesn't actually implement them.
Since they are optional per spec and with Mastodon omitting them too
for its instance actor proving the practical viability, we should just
omit them. The relay actor however should continue to expose such
collections and they are properly implemented here.
Here too we now just use the values or their absence in the database.
We do not have any other internal.* actors besides fetch atm.
Fixes: #855
Co-authored-by: ilja space <git@ilja.space>
E.g. \*oma federates (most) follower-only posts multiple times
to each personal inbox. This commonly leads to race conditions
with jobs of several copies running at the same time and getting
past the initial "already known" check but then later all but
one will crash with an exception from the unique db index.
Since the only special thing we do with copies anyway is to discard them,
just don't create such duplicate jobs in the first place.
For the same reason and since failed jobs don't count towards
duplicates, this should have virtually no effect on federation.
Since we later only consider the Create activity for
access permission checks, but the semantically more
sensible set of fields are the object’s.
Changing the check itself to use the object may have unintended
consequences on already existing legacy posts as the old code
which processed it when it arrived may have never considered
effects on the objects addressing fields.
While the object itself has the expected adressing for an
"unlisted" post, we always use the Create activity’s
adressing fields for permission checks.
To avoid unintended effects on legacy objects
we will continue to use the activity for access perm checks,
but fix its addressing fields based on its object data.
Ref: https://git.pleroma.social/pleroma/pleroma/-/issues/3323
With the current strategy the individual
and cumulative backoff looks like this
(the + part denotes max extra random delay):
attempt backoff_single cumulative
1 16+30 16+30
2 47+60 63+90
3 243+90 ≈ 4min 321+180
4 1024+120 ≈17min 1360+300 ≈23+5min
5 3125+150 ≈20min 4500+450 ≈75+8min
6 7776+180 ≈ 2.1h 12291+630 ≈3.4h
7 16807+210 ≈ 4.6h 29113+840 ≈8h
8 32768+240 ≈ 9.1h 61896+1080 ≈17h
9 59049+270 ≈16.4h 120960+1350 ≈33h
10 100000+300 ≈27.7h 220975+1650 ≈61h
We default to 5 retries meaning the least backoff runs with attempt=4.
Therefore outgoing activiities might already be permanently dropped by a
downtime of only 23 minutes which doesn't seem too implausible to occur.
Furthermore it seems excessive to retry this quickly this often at the
beginning.
At the same time, we’d like to have at least one quick'ish retry to deal
with transient issues and maintain reasonable federation responsiveness.
If an admin wants to tolerate one -day downtime of remotes,
retries need to be almost doubled.
The new backoff strategy implemented in this commit instead
switches to an exponetial after a few initial attempts:
attempt backoff_single cumulative
1 16+30 16+30
2 143+60 159+90
3 2202+90 ≈37min 2361+180 ≈40min
4 8160+120 ≈ 2.3h 10521+300 ≈ 3h
5 77393+150 ≈21.5h 87914+450 ≈24h
Initial retries are still fast, but the same amount of retries
now allows a remote downtime of at least 40 minutes. Customising
the retry count to 5 allows for whole-day downtimes.
This was accidentally broken in c8e0f7848b
due to a one-letter mistake in the plug option name and an absence of
tests. Therefore it was once again possible to serve e.g. Javascript or
CSS payloads via uploads and emoji.
However due to other protections it was still NOT possible for anyone to
serve any payload with an ActivityPub Content-Type. With the CSP policy
hardening from previous JS payload exloits predating the Content-Type
sanitisation, there is currently no known way of abusing this weakened
Content-Type sanitisation, but should be fixed regardless.
This commit fixes the option name and adds tests to ensure
such a regression doesn't occur again in the future.
Reported-by: Lain Soykaf <lain@lain.com>
It is due to be removed in one of the next releases if no strong arguments for keeping it are brought up.
It is already semi-broken for large threads and conflicts with pending optimisation and cleanup work.
- support for `exclude_visibilities` in timeline and notification endpoints has been dropped
- support for list visibility / list addressing has been dropped due to lack of usage, maintenance burden and redundancy with the still supported explicit-addressing feature
- support for conversations addressing has been dropped due to lack of usage, maintenance burden and being mostly redundant with explicit addressing
- per-visibility status counters have been dropped from `/api/v1/pleroma/admin/stats`
due to unreasonably perf costs added on most database operations.
For now, the response still contains the fields, but with stubbed-out values.
### Added
- status responses include two new fields for ActivityPub cross-referencing: `akkoma.quote_apid` and `akkoma.in_reply_to_apid`
- attempting to reply to an already deleted post will return an error
(in akkoma-fe the error will be shown and your draft message retained so you can decide
for yourself whether to discard it or copy and repost as a, now intentional, new thread)
- the notification endpoint now supports the `types` parameter for filtering added in vanilla Mastodon
- the mute endpoint now supports the `duration` parameter added in vanilla Mastodon
(fixes temporary mutes created via e.g. Husky)
### Fixed
- replies and quotes to unresolvable posts now fill out IDs for replied to
status, user or quoted status with a 404-ing ID to make them recognisable as
replies/quotes instead of pretending they’re root posts
- querying a status using the ID of a non-post AP activity no longer displays
a duplicate of the post referenced by said activity with mangled author information
- fix users being able to interact (like, emoji react, ...) with posts they cannot access
- fix AP fetches of local non-Create, non-Undo activities exposing the raw, unsanitised content of the referenced object
- the above two combined allowed local users to gain access to private posts
of user they do not follow, but follow a follower of the author.
(remote users and other scenarios were to our knowledge not able to achieve this due to other restrictions)
- fix RSS and Atom feeds of hashtag timelines potentially exposing more information than Mastodon API when restricting unauthenticated API access
- fix mentioning and sending DMs to users with non-ASCII-alphanumerical usernames
- correctly hide and show inlined fallback links for quotes from Mastodon instances
- API requests with multiple unsupported parameters now will ignore all of them up to a certain limit.
If there are too many unsupported parameters this is indicated in the returned error message.
- expose generic type of attachment via Masto API if remote did not send a full MIME type but indicated a generic one
(the \*oma-specific full mime type field in the API response remains generic however, since we don't have this info)
- add back the default banner image we advertise in Masto API
- correctly redirect `/users/:nickname.rss` to the RSS instead of Atom feed
### Changed
- depreacted the `included_types` parameter in the notification endpoint; replaced by `types`
- depreacted the `expires_in` parameter in the mute endpoint; replaced by `duration`
- optimised emoji addition and removal
- emoji reloading now happens asynchronously so you won't run into timeout issues with many emoji and/or a slow disk
- upgraded all of our dependencies; this should reduce issues when running akoma with OTP28
- prefer "summary" over "name" for the attachment alt text of incoming ActivityPub documents;
this fixes alt text federation from GtS and Honk
- slightly improve index overhead for the users table
## 2025.10
### REMOVED
- Dropped `accepts_chat_messages` column from users table in database;
it has been unused for almost 3 years
- Healthcheck responses no longer contain job queue data;
it was useless anyway due to lacking any temporal information about failures
and more complete data can be obtained from Prometheus metrics.
### Added
- We mark our MFM posts as FEP-c16b compliant, and retain remote HTML representations for incoming posts marked as FEP-c16b-compliant. (Safety scrubbers are still applied)
- Prometheus stats now exposes failed ActivityPub deliveries
which failed all attempts and the failure reason
- status and user HTML pages now provide ActivityPub alternate links
- the `prune_objects` mix task no longer deletes pinned posts by default
- added `--prune-pinned` and `--keep-followed {posts,full,none}` options to the `prune_objects` mix task
- timestamps of incoming HTTP signatures are now verified.
By default up to two hour old signatures and a maximal clock skew
of 40 min for future timestamps or explicit expiry deadlines are accepted
- Added `short_description` field to `api/v1/instance` for Mastodon compatibility; the corresponding
new setting `:pleroma, :instance, :short_description` is also preferred for nodeinfo use
- Note AP objects now expose full `replies` collections and those collections can be accessed on their own;
previously only self-replies were inlined as an anonymous collection into the Note object
- Added a reference Grafana dashboard and improved documentation for Prometheus metrics
- New mix task `clean_inlined_replies` to delete some unused data from objects
- New mix task `resync_inlined_caches` to retroactively fix various issues with e.g. boosts, emoji reacts and likes
- It is now possible to allow outgoing requests to use HTTP2 via config option,
but due to bugs in the relevant backend this is not the default nor recommended.
- Prometheus metrics now expose count of scheduled and pending jobs per queue
### Fixed
- Internal actors no longer pretend to have unresolvable follow(er|ing) collections
- fixed user-level default post expiry duration overriding `expires_in` values explicitly passed during post creation
- fix crashes on non-UTF8 usernames for the API paths taking both nicknames and IDs
- fixed divergences in fields used to determine visibility;
this lead e.g. to unlisted replies from Pleroma instances being partially treated as private posts
- fixed our fetch actor advertising bogus follower and following collection ActivityPub IDs
- fix network-path references not being handled by media proxy
- federation with bridgy now works
- remote signing keys are no longer refreshed multiple times per incoming request
- fix digest emails never being sent and clogging the job queue even if not enabled
- `api/v1/instance`’s `uri` field now correctly shows the bare WebFinger domain
- fixed bug leading to `content` and the `contentMap` entry of the primary language to sometimes diverge
- reloading emoji with a broken `pack.json` file being on disk no longer crashes the whole server
- fixed blocked servers being able to access local objects when authorized fetch isn’t enabled
even when the remote server identifies itselfs
- fixed handling of inlined "featured" collections
- fixed user endpoint serving invalid ActivityPub for minimal, authfetch-fallback responses
- remote emoji reacts from IceShrimp.NET instances are now handled consistently and always merged with identical other emoji reactions
- ActivityPub requests signatures are now renewed when following redirects making sure path and host actually match the final URL
- private replies no longer increase the publicly visible reply counter
- unblock activities are no longer federated when block federation is disabled (the default)
- fix like activity database IDs rendering as misattributed posts
### Changed
- Internal and relay actors are now again represented with type "Application"
- `cleanup_attachments` is now enabled by default
- shared inboxes are now generally preferred over personal inboxes, cutting down on duplicate publishing churn
- instance actors are now really of type `Service`
- ActivityPub delivery attempts are spaced out more and increased by one
now giving up after 24h instead of ~20min by default before
- inboxes now fake a succcess reply on incoming Delete documents whose signing key is unknown but gone;
this prevents older Mastodon from repeatedly trying to deliver Deletes of actors we never knew anyway
- The config option `config :pleroma, :http, :pool_max_idle_time` was removed; it never actually
did anything and was redundant with `config :pleroma, :http, :pool_timeout` which actually works.
- repeated attempt to process incoming ActivityPub objects are spaced out more, allowing unreachable remotes
more time to come back up when e.g. processing repeats of a post not yet locally known
- `/api/v1/statuses/:id/reblog` now honours all possible visibilities except `list` and `conversation`
instead of mapping them down to a boolean private/public
- we no longer repeatedly try to deliver to explicitly deleted inboxes
- Config option `Pleroma.Web.MediaProxy.Invalidation.Http, :options` and
the `:http` subkey of `:media_proxy, :proxy_opts` now only accept
adapter-related settings inside the `:adapter` subkey, no longer on the top-level
- follow requests are now ordered reverse chronologically
## 2025.03
## Added
### Added
- Oban (worker) dashboard at `/akkoma/oban`
## Fixed
### Fixed
- fixed some holes in SigningKey verification potentially allowing they key-user mapping to be poisoned
- frontend ZIP files can no longer traverse to paths outside their install dir
- fixed user updates trying but failing to renew signing key information
- fixed signing key refresh on key rotation
## Changed
### Changed
- Dropped obsolete `ap_enabled` indicator from user table and associated buggy logic
- The remote user count in prometheus metrics is now an estimate instead of an exact number
since the latter proved unreasonably costly to obtain for a merely nice-to-have statistic
@ -31,12 +214,12 @@ Hotfix: Federation could break if a null value found its way into `should_federa
## 2025.01
## Added
### Added
- New config option `:instance, :cleanup_attachments_delay`
- It is now possible to display custom source URLs in akkoma-fe;
the settings are part of the frontend configuration
## Fixed
### Fixed
- Media proxy no longer attempts to proxy embedded images
- Fix significant uneccessary overhead of attachment cleanup;
it no longer attempts to cleanup attachments of deleted remote posts
@ -45,24 +228,24 @@ Hotfix: Federation could break if a null value found its way into `should_federa
- ObjectAge policy no longer leaks belated DMs and follower-only posts
- the NodeINfo endpoint now uses the correct content type
## Changed
### Changed
- Anonymous objects now federate completely without an id
adopting a proposed AP spec errata and restoring federation
with e.g. IceShrimp.NET and fedify-based implementations
## 3.13.3
## BREAKING
### BREAKING
- Minimum PostgreSQL version is raised to 12
- Swagger UI moved from `/akkoma/swaggerui/` to `/pleroma/swaggerui/`
- Meilisearch: it is now possible to use separate keys for search and admin actions
- New standalone `prune_orphaned_activities` mix task with configurable batch limit
- The `prune_objects` mix task now accepts a `--limit` parameter for initial object pruning
## Fixed
### Fixed
- Meilisearch: order of results returned from our REST API now actually matches how Meilisearch ranks results
- Emoji are now federated as anonymous objects, fixing issues with
some strict servers e.g. rejecting e.g. remote emoji reactions
@ -70,25 +253,25 @@ Hotfix: Federation could break if a null value found its way into `should_federa
- Single-selection polls no longer expose the voter_count; MastoAPI demands it be null
and this confused some clients leading to vote distributions >100%
## Changed
### Changed
- Refactored Rich Media to cache the content in the database. Fetching operations that could block status rendering have been eliminated.
## 2024.04.1 (Security)
## Fixed
### Fixed
- Issue allowing non-owners to use media objects in posts
- Issue allowing use of non-media objects as attachments and crashing timeline rendering
- Issue allowing webfinger spoofing in certain situations
## 2024.04
## Added
### Added
- Support for [FEP-fffd](https://codeberg.org/fediverse/fep/src/branch/main/fep/fffd/fep-fffd.md) (proxy objects)
- Verified support for elixir 1.16
- Uploadfilter `Pleroma.Upload.Filter.Exiftool.ReadDescription` returns description values to the FE so they can pre fill the image description field
NOTE: this filter MUST be placed before `Exiftool.StripMetadata` to work
## Changed
### Changed
- Inbound pipeline error handing was modified somewhat, which should lead to less incomprehensible log spam. Hopefully.
- Uploadfilter `Pleroma.Upload.Filter.Exiftool` was replaced by `Pleroma.Upload.Filter.Exiftool.StripMetadata`;
the latter strips all non-essential metadata by default but can be configured.
@ -97,7 +280,7 @@ Hotfix: Federation could break if a null value found its way into `should_federa
- MRF.InlineQuotePolicy now prefers to insert display URLs instead of ActivityPub IDs
- Old accounts are no longer listed in WebFinger as aliases; this was breaking spec
## Fixed
### Fixed
- Issue preventing fetching anything from IPv6-only instances
- Issue allowing post content to leak via opengraph tags despite :estrict\_unauthenticated being set
- Move activities no longer operate on stale user data
@ -113,17 +296,17 @@ Hotfix: Federation could break if a null value found its way into `should_federa
JSON-LD-compacted forms of public scope; affected e.g. federation with bovine
- Ratelimits encountered when fetching objects are now respected; 429 responses will cause a backoff when we get one.
## Removed
### Removed
- ActivityPub Client-To-Server write API endpoints have been disabled;
read endpoints are planned to be removed next release unless a clear need is demonstrated
## 2024.03
## Added
### Added
- CLI tasks best-effort checking for past abuse of the recent spoofing exploit
- new `:mrf_steal_emoji, :download_unknown_size` option; defaults to `false`
## Changed
### Changed
- `Pleroma.Upload, :base_url` now MUST be configured explicitly if used;
use of the same domain as the instance is **strongly** discouraged
- `:media_proxy, :base_url` now MUST be configured explicitly if used;
@ -139,7 +322,7 @@ Hotfix: Federation could break if a null value found its way into `should_federa
- Uploads, emoji and media proxy now restrict Content-Type headers to a safe subset
- Akkoma will no longer fetch and parse objects hosted on the same domain
## Fixed
### Fixed
- Critical security issue allowing Akkoma to be used as a vector for
(depending on configuration) impersonation of other users or creation
of bogus users and posts on the upload domain
@ -152,7 +335,7 @@ Hotfix: Federation could break if a null value found its way into `should_federa
- our litepub JSON-LD schema is now served with the correct content type
- remote APNG attachments are now recognised as images
## Upgrade Notes
### Upgrade Notes
- As mentioned in "Changed", `Pleroma.Upload, :base_url`**MUST** be configured. Uploads will fail without it.
- Akkoma will refuse to start if this is not set.
@ -160,20 +343,20 @@ Hotfix: Federation could break if a null value found its way into `should_federa
## 2024.02
## Added
### Added
- Full compatibility with Erlang OTP26
- handling of GET /api/v1/preferences
- Akkoma API is now documented
- ability to auto-approve follow requests from users you are already following
- The SimplePolicy MRF can now strip user backgrounds from selected remote hosts
## Changed
### Changed
- OTP builds are now built on erlang OTP26
- The base Phoenix framework is now updated to 1.7
- An `outbox` field has been added to actor profiles to comply with AP spec
- User profile backgrounds do now federate with other Akkoma instances and Sharkey
## Fixed
### Fixed
- Documentation issue in which a non-existing nginx file was referenced
- Issue where a bad inbox URL could break federation
- Issue where hashtag rel values would be scrubbed
@ -181,7 +364,7 @@ Hotfix: Federation could break if a null value found its way into `should_federa
## 2023.08
## Added
### Added
- Added a new configuration option to the MediaProxy feature that allows the blocking of specific domains from using the media proxy or being explicitly allowed by the Content-Security-Policy.
- Please make sure instances you wanted to block media from are not in the MediaProxy `whitelist`, and instead use `blocklist`.
@ -194,7 +377,7 @@ Hotfix: Federation could break if a null value found its way into `should_federa
- OTP26 is currently "unsupported". It will probably work, but due to the way
it handles map ordering, the test suite will not pass for it as yet.
## Changed
### Changed
- Alpine OTP builds are now from alpine 3.18, which is OpenSSLv3 compatible.
If you use alpine OTP builds you will have to update your local system.
@ -205,19 +388,19 @@ Hotfix: Federation could break if a null value found its way into `should_federa
- Blocks/Mutes now return from max ID to min ID, in line with mastodon.
- The AnonymizeFilename filter is now enabled by default.
## Fixed
### Fixed
- Deactivated users can no longer show up in the emoji reaction list
- Embedded posts can no longer bypass `:restrict\_unauthenticated`
- GET/HEAD requests will now work when requesting AWS-based instances.
## Security
### Security
- Add `no_new_privs` hardening to OpenRC and systemd service files
- XML parsers cannot load any entities (thanks @Mae@is.badat.dev!)
- Reduced permissions of config files and directories, distros requiring greater permissions like group-read need to pre-create the directories
## Removed
### Removed
- Builds for debian oldstable (bullseye)
- If you are on oldstable you should NOT attempt to update OTP builds without
@ -225,7 +408,7 @@ Hotfix: Federation could break if a null value found its way into `should_federa
## 2023.05
## Added
### Added
- Custom options for users to accept/reject private messages
- options: everybody, nobody, people\_i\_follow
- MRF to reject notes from accounts newer than a given age
@ -233,16 +416,16 @@ Hotfix: Federation could break if a null value found its way into `should_federa
post gets boosted outside of your local bubble and people your instance
does not know about reply to it.
## Fixed
### Fixed
- Support for `streams` public key URIs
- Bookmarks are cleaned up on DB prune now
## Security
### Security
- Fixed mediaproxy being a bit of a silly billy
## 2023.04
## Added
### Added
- Nodeinfo keys for unauthenticated timeline visibility
- Option to disable federated timeline
- Option to make the bubble timeline publicly accessible
@ -256,7 +439,7 @@ Hotfix: Federation could break if a null value found its way into `should_federa
## 2023.03
## Fixed
### Fixed
- Allowed contentMap to be updated on edit
- Filter creation now accepts expires\_at
@ -316,7 +499,7 @@ Hotfix: Federation could break if a null value found its way into `should_federa
## 2022.12
## Added
### Added
- Config: HTTP timeout options, :pool\_timeout and :receive\_timeout
- Added statistic gathering about instances which do/don't have signed fetches when they request from us
- Ability to set a default post expiry time, after which the post will be deleted. If used in concert with ActivityExpiration MRF, the expiry which comes _sooner_ will be applied.
@ -326,7 +509,7 @@ Hotfix: Federation could break if a null value found its way into `should_federa
- Option to extend `reject` in MRF-Simple to apply to entire threads, where the originating instance is rejected
- Extra information to failed HTTP requests
## Changed
### Changed
- MastoAPI: Accept BooleanLike input on `/api/v1/accounts/:id/follow` (fixes follows with mastodon.py)
- Relays from akkoma are now off by default
- NormalizeMarkup MRF is now on by default
@ -335,30 +518,30 @@ Hotfix: Federation could break if a null value found its way into `should_federa
- Overhauled static-fe view for logged-out users
- Blocked instances will now not be sent _any_ requests, even fetch ones that would get rejected by MRF anyhow
## Removed
### Removed
- FollowBotPolicy
- Passing of undo/block into MRF
## Upgrade Notes
### Upgrade Notes
- If you have an old instance, you will probably want to run `mix pleroma.database prune_task` in the foreground to catch it up with the history of your instance.
## 2022.11
## Added
### Added
- Officially supported docker release
- Ability to remove followers unilaterally without a block
- Scraping of nodeinfo from remote instances to display instance info
- `requested_by` in relationships when the user has requested to follow you
## Changed
### Changed
- Follows no longer override domain blocks, a domain block is final
- Deletes are now the lowest priority to publish and will be handled after creates
- Domain blocks are now subdomain-matches by default
## Fixed
### Fixed
- Registrations via ldap are now compatible with the latest OTP24
## Update notes
### Update notes
- If you use LDAP and run from source, please update your elixir/erlang
to the latest. The changes in OTP24.3 are breaking.
- You can now remove the leading `*.` from domain blocks, but you do not have to.
- `<path>` - where to save migrated config. E.g. `--path=/tmp`. If file saved into non standart folder, you must manually copy file into directory where Pleroma can read it. For OTP install path will be `PLEROMA_CONFIG_PATH` or `/etc/akkoma`. For installation from source -`config` directory in the akkoma folder.
- `<env>` - environment, for which is migrated config. By default is `prod`.
- To delete transferred settings from database optional flag `-d` can be used
- `<path>` - where to save migrated config. E.g. `--path=/tmp`. If the file was saved into a non-standard directory, you must manually copy it into a location where Pleroma can read it. For OTP the install path will be `PLEROMA_CONFIG_PATH` or `/etc/akkoma`. For installation from source it’s the`config` directory in the akkoma folder.
- `<env>` - environment, for which is migrated config. By default, it’s `prod`.
- To delete transferred settings from database the optional flag `-d` can be used
Replaces embedded objects with references to them in the `objects` table. Only needs to be ran once if the instance was created before Pleroma 1.0.5. The reason why this is not a migration is because it could significantly increase the database size after being ran, however after this `VACUUM FULL` will be able to reclaim about 20% (really depends on what is in the database, your mileage may vary) of the db size before the migration.
Replaces embedded objects with references to them in the `objects` table. Only needs to be ran once if the instance was created before Pleroma 1.0.5. The reason why this is not a migration is because it could significantly increase the database size after being ran, however after this `VACUUM FULL` will be able to reclaim about 20% (really depends on what is in the database, your mileage may vary) of the database size before the migration.
=== "OTP"
@ -26,13 +26,19 @@ Replaces embedded objects with references to them in the `objects` table. Only n
## Prune old remote posts from the database
This will prune remote posts older than 90 days (configurable with [`config :pleroma, :instance, remote_post_retention_days`](../../configuration/cheatsheet.md#instance)) from the database. Pruned posts may be refetched in some cases.
This will selectively prune remote posts older than 90 days (configurable with [`config :pleroma, :instance, remote_post_retention_days`](../../configuration/cheatsheet.md#instance)) from the database. Pruned posts may be refetched in some cases.
!!! note
The disk space will only be reclaimed after a proper vacuum. By default Postgresql does this for you on a regular basis, but if your instance has been running for a long time and there are many rows deleted, it may be advantageous to use `VACUUM FULL` (e.g. by using the `--vacuum` option).
The disk space used up by deleted rows only becomes usable for new data after a vaccum.
By default, Postgresql does this for you on a regular basis, but if you delete a lot at once
it might be advantageous to also manually kick off a vacuum and statistics update using `VACUUM ANALYZE`.
**However**, the freed up space is never returned to the operating system unless you run
the much more heavy `VACUUM FULL` operation. This epensive but comprehensive vacuum mode
can be schedlued using the `--vacuum` option.
!!! danger
You may run out of disk space during the execution of the task or vacuuming if you don't have about 1/3rds of the database size free. Vacuum causes a substantial increase in I/O traffic, and may lead to a degraded experience while it is running.
You may run out of disk space during the execution of the task or full vacuuming if you don't have about 1/3rds of the database size free. `VACUUM FULL` causes a substantial increase in I/O traffic, needs full table locks and thus renders the instance basically unusable while its running.
=== "OTP"
@ -48,10 +54,41 @@ This will prune remote posts older than 90 days (configurable with [`config :ple
### Options
- `--keep-threads` - Don't prune posts when they are part of a thread where at least one post has seen local interaction (e.g. one of the posts is a local post, or is favourited by a local user, or has been repeated by a local user...). It also wont delete posts when at least one of the posts in that thread is kept (e.g. because one of the posts has seen recent activity).
The recommended starting point and configuration for small and medium-sized instances is:
This almost certainly won’t delete stuff your interested in and
makes sure the database is immediately utilising the newly freed up space.
If you need more aggressive database size reductions or if this proves too costly to run for you
you can drop restrictions and/or use the `--limit` option.
In the opposite case if everything goes through quickly,
you can combine the three CLI tasks into one for future runs using `--prune-orphaned-activities`
and perhaps even using a full vacuum (which implies a reindex) using `--vacuum` too.
Full details below:
- `--no-fix-replies-count` - Skip recalculating replies count for posts.
When using multiple batches of prunes with `--limit`, all but the last batch
should set this option to avoid unnecessary overhead.
- `--keep-followed <mode>` - If set to `posts` all posts and boosts of users with local follows will be kept.
If set to `full` it will additionally keep any posts such users interacted with; this requires `--keep-threads`.
By default this is set to `none` and followed users are not treated special.
- `--keep-threads` - Don't prune posts when they are part of a thread where at least one post has seen local interaction (e.g. one of the posts is a local post, or is favourited by a local user, or has been repeated by a local user...). It also won’t delete posts when at least one of the posts in the thread has seen recent activity or is kept due to `--keep-followed`.
- `--keep-non-public` - Keep non-public posts like DM's and followers-only, even if they are remote.
- `--limit` - limits how many remote posts get pruned. This limit does **not** apply to any of the follow up jobs. If wanting to keep the database load in check it is thus advisable to run the standalone `prune_orphaned_activities` task with a limit afterwards instead of passing `--prune-orphaned-activities` to this task.
See documentation of other options for futher hints.
- `--prune-orphaned-activities` - Also prune orphaned activities afterwards. Activities are things like Like, Create, Announce, Flag (aka reports)... They can significantly help reduce the database size.
- `--prune-pinned` - Also prune pinned posts; keeping pinned posts does not suffice to protect their threads from pruning, even when using `--keep-threads`.
Note, if using this option and pinned posts are pruned, they and their threads will just be refetched on the next user update. Therefore it usually doesn't bring much gain while incurring a heavy fetch load after pruning.
One exception to this is if you already need to use a relatively small `--limit` to keep downtime mangeable or even being able to run it without downtime. Retaining pinned posts adds a mostly constant overhead which will impact repeated runs with small limit much more than one full prune run.
- `--vacuum` - Run `VACUUM FULL` after the objects are pruned. This should not be used on a regular basis, but is useful if your instance has been running for a long time before pruning.
## Prune orphaned activities from the database
@ -130,7 +167,7 @@ Can be safely re-run
## Vacuum the database
!!! note
By default Postgresql has an autovacuum deamon running. While the tasks described here can help in some cases, they shouldn't be needed on a regular basis. See [the Postgresql docs on vacuuming](https://www.postgresql.org/docs/current/sql-vacuum.html) for more information on this.
By default, Postgresql has an autovacuum daemon running. While the tasks described here can help in some cases, they shouldn't be needed on a regular basis. See [the Postgresql docs on vacuuming](https://www.postgresql.org/docs/current/sql-vacuum.html) for more information on this.
### Analyze
@ -150,7 +187,7 @@ Running an `analyze` vacuum job can improve performance by updating statistics u
### Full
Running a `full` vacuum job rebuilds your entire database by reading all of the data and rewriting it into smaller
Running a `full` vacuum job rebuilds your entire database by reading all data and rewriting it into smaller
and more compact files with an optimized layout. This process will take a long time and use additional disk space as
it builds the files side-by-side the existing database files. It can make your database faster and use less disk space,
but should only be run if necessary. **It is safe to cancel this.**
@ -183,7 +220,7 @@ but should only be run if necessary. **It is safe to cancel this.**
## Change Text Search Configuration
Change `default_text_search_config` for database and (if necessary) text_search_config used in index, then rebuild index (it may take time).
Change `default_text_search_config` for database and (if necessary) text_search_config used in index, then rebuild index (it may take time).
=== "OTP"
@ -202,9 +239,9 @@ See [PostgreSQL documentation](https://www.postgresql.org/docs/current/textsearc
## Pruning old activities
Over time, transient `Delete` activities and `Tombstone` objects
can accumulate in your database, inflating its size. This is not ideal.
There is a periodic task to prune these transient objects,
but on first run this may take a while on older instances to catch up
can accumulate in your database, inflating its size. This is not ideal.
There is a periodic task to prune these transient objects,
but on the first run this may take a while on older instances to catch up
to the current day.
=== "OTP"
@ -218,3 +255,48 @@ to the current day.
```sh
mix pleroma.database prune_task
```
## Clean inlined replies lists
Those inlined arrays of replies AP IDs are not used (anymore).
Delete them to free up a little bit of space.
=== "OTP"
```sh
./bin/pleroma_ctl database clean_inlined_replies
```
=== "From Source"
```sh
mix pleroma.database clean_inlined_replies
```
## Resync data inlined into posts
For legacy and performance reasons some data, e.g. relating to likes and boosts,
is currently copied and inline references post objects. Occasionally this data
may desync from the actual authorative activity and object data stored in the
database which may lead to cosmetic but also functional issues.
For example a particular user may appear unable to like a post.
Run this task to detect and fix such desyncs.
=== "OTP"
```sh
./bin/pleroma_ctl database resync_inlined_caches
```
=== "From Source"
```sh
mix pleroma.database resync_inlined_caches
```
### Options
- `--no-announcements` - Skip fixing announcement counters and lists
- `--no-likes` - Skip fixing like counters and lists
- `--no-reactions` - Skip fixing inlined emoji reaction data
- `-m, --manifest PATH/URL` - path to a custom manifest, it can either be an URL starting with `http`, in that case the manifest will be fetched from that address, or a local path
- `-m, --manifest PATH/URL` - path to a custom manifest, it can either be a URL starting with `http`, in that case the manifest will be fetched from that address, or a local path
## Fetch, verify and install the specified packs from the manifest into `STATIC-DIR/emoji/PACK-NAME`
Frontend can be installed either from local zip file, or automatically downloaded from the web.
Frontend can be installed either from a local ZIP file, or automatically downloaded from the web.
You can give all the options directly on the command line, but missing information will be filled out by looking at the data configured under `frontends.available` in the config files.
You can still install frontends that are not configured, see below.
@ -42,7 +42,7 @@ For a frontend configured under the `available` key, it's enough to install it b
This will download the latest build for the pre-configured `ref` and install it. It can then be configured as the one of the served frontends in the config file (see `primary` or `admin`).
You can override any of the details. To install an Akkoma-FE build from a different URL, you could do this:
You can override any of the details. To install an akkoma-fe build from a different URL, you could do this:
=== "OTP"
@ -56,7 +56,7 @@ You can override any of the details. To install an Akkoma-FE build from a differ
If you don't have a zip file but just want to install a frontend from a local path, you can simply copy the files over a folder of this template: `${instance_static}/frontends/${name}/${ref}`.
If you don't have a ZIP file but just want to install a frontend from a local path, you can simply copy the files over a folder of this template: `${instance_static}/frontends/${name}/${ref}`.
Every command should be ran as the `akkoma` user from it's home directory. For example if you are superuser, you would have to wrap the command in `su akkoma -s $SHELL -lc "$COMMAND"`.
Every command should be ran as the `akkoma` user from its home directory. For example if you are superuser, you would have to wrap the command in `su akkoma -s $SHELL -lc "$COMMAND"`.
??? note "From source note about `MIX_ENV`"
The `mix` command should be prefixed with the name of environment your Akkoma server is running in, usually it's `MIX_ENV=prod`
The `mix` command should be prefixed with the name of the environment your Akkoma server is running in, usually it's `MIX_ENV=prod`
## Disable Multi Factor Authentication (MFA/2FA) for a user
## Disable Multi-Factor Authentication (MFA/2FA) for a user
=== "OTP"
@ -206,7 +196,6 @@
mix pleroma.user reset_mfa <nickname>
```
## Set the value of the given user's settings
=== "OTP"
@ -241,7 +230,6 @@
mix pleroma.user tag <nickname> <tags>
```
## Delete tags from a user
=== "OTP"
@ -256,7 +244,6 @@
mix pleroma.user untag <nickname> <tags>
```
## Toggle confirmation status of the user
=== "OTP"
@ -304,7 +291,7 @@
## Fix following state
Sometimes the system can get into a situation where
it think you're already following someone and won't send a request
it thinks you're already following someone and won't send a request
to the remote instance, or won't let you unfollow someone. This
bug was fixed, but in case you encounter these weird states:
@ -324,4 +311,4 @@ The first argument is the local user's nickname - if you are `myuser@myinstance`
The second is the remote user, consisting of both nickname AND domain.
If you are a weird follow state situation and cannot resolve it with the above, you may need to co-operate with the remote admin to clear the state their side too - they should provide the arguments *backwards*, i.e `fix_follow_state remote local`.
If you are a weird follow state situation and cannot resolve it with the above, you may need to co-operate with the remote admin to clear the state their side too - they should provide the arguments *backwards*, i.e.`fix_follow_state remote local`.
[¹]: We assume the database name is "akkoma". If not, you can find the correct name in your configuration files.
[²]: If you have a from source installation, you need `config/prod.secret.exs` instead of `config/config.exs`. The `config/config.exs` file also exists, but in case of from source installations, it only contains the default values and it is tracked by Git, so you don't need to back it up.
[²]: If you have a from source installation, you need `config/prod.secret.exs` instead of `config/config.exs`. The `config/config.exs` file also exists, but in case of from source installations, it only contains the default values and it is tracked by Git, so you don't need to back it up.
## Restore/Move
1. Optionally reinstall Akkoma (either on the same server or on another server if you want to move servers).
2. Stop the Akkoma service.
3. Go to the working directory of Akkoma (default is `/opt/akkoma`)
4. Copy the abovementioned files back to their original position.
4. Copy the above-mentioned files back to their original position.
5. Drop the existing database and user[¹]. `sudo -Hu postgres psql -c 'DROP DATABASE akkoma;';``sudo -Hu postgres psql -c 'DROP USER akkoma;'`
6. Restore the database schema and akkoma role[¹] (replace the password with the one you find in the configuration file), `sudo -Hu postgres psql -c "CREATE USER akkoma WITH ENCRYPTED PASSWORD '<database-password-wich-you-can-find-in-your-configuration-file>';"` `sudo -Hu postgres psql -c "CREATE DATABASE akkoma OWNER akkoma;"`.
6. Restore the database schema and akkoma role[¹] (replace the password with the one you find in the configuration file), `sudo -Hu postgres psql -c "CREATE USER akkoma WITH ENCRYPTED PASSWORD '<database-password-wich-you-can-find-in-your-configuration-file>';";` `sudo -Hu postgres psql -c "CREATE DATABASE akkoma OWNER akkoma;"`.
7. Now restore the Akkoma instance's data into the empty database schema[¹]: `sudo -Hu postgres pg_restore -d akkoma -v -1 </path/to/backup_location/akkoma.pgdump>`
8. If you installed a newer Akkoma version, you should run the database migrations `./bin/pleroma_ctl migrate`[²].
9. Restart the Akkoma service.
10. Run `sudo -Hu postgres vacuumdb --all --analyze-in-stages`. This will quickly generate the statistics so that postgres can properly plan queries.
11. If setting up on a new server, configure Nginx by using the `installation/nginx/akkoma.nginx` configuration sample or reference the Akkoma installation guide which contains the Nginx configuration instructions.
11. If setting up on a new server, configure nginx by using the `installation/nginx/akkoma.nginx` configuration sample or reference the Akkoma installation guide which contains the nginx configuration instructions.
[¹]: We assume the database name and user are both "akkoma". If not, you can find the correct name in your configuration files.
[²]: If you have a from source installation, the command is `MIX_ENV=prod mix ecto.migrate`. Note that we prefix with `MIX_ENV=prod` to use the `config/prod.secret.exs` configuration file.
## Remove
1. Optionally you can remove the users of your instance. This will trigger delete requests for their accounts and posts. Note that this is 'best effort' and doesn't mean that all traces of your instance will be gone from the fediverse.
* You can do this from the admin-FE where you can select all local users and delete the accounts using the *Moderate multiple users* dropdown.
* You can also list local users and delete them individually using the CLI tasks for [Managing users](./CLI_tasks/user.md).
1. Optionally, you can remove the users of your instance. This will trigger delete requests for their accounts and posts. Note that this is 'best effort' and doesn't mean that all traces of your instance will be gone from the fediverse.
* You can do this from the admin-fe where you can select all local users and delete the accounts using the *Moderate multiple users* dropdown.
* You can also list local users and delete them individually using the CLI tasks for [managing users](./CLI_tasks/user.md).
2. Stop the Akkoma service `systemctl stop akkoma`
3. Disable Akkoma from systemd `systemctl disable akkoma`
4. Remove the files and folders you created during installation (see installation guide). This includes the akkoma, nginx and systemd files and folders.
You may use the eponymous [Prometheus](https://prometheus.io/)
or anything compatible with it like e.g. [VictoriaMetrics](https://victoriametrics.com/).
The latter claims better performance and storage efficiency.
However, at the moment our reference dashboard only works with VictoriaMetrics,
thus if you wish to use use the reference as an easy dropin template you must
use VictoriaMetrics.
Patches to allow the dashboard to work with plain Prometheus are welcome though.
Both of them can usually be easily installed via distro-packages or docker.
Depending on your distro or installation method the preferred way to change the CLI arguments and the location of config files may differ; consult the documentation of your chosen method to find out.
Of special interest is the location of the prometheus scraping config file
and perhaps the maximal data retention period setting,
to manage used disk space and make sure you keep records long enough for your purposes.
It might also be a good idea to set up a minimal buffer of free disk space if you’re tight on that;
with VictoriaMetrics this can be done via the `-storage.minFreeDiskSpaceBytes 1GiB` CLI flag.
Ideally the scraper runs on a different machine than Akkoma to be able to
distinguish Akkoma downtime from scraper downtime, but this is not strictly necessary.
Once you’ve installed one of them, it’s time to add a job for scraping Akkoma.
For Prometheus the `scrape_configs` section will usually be added to the main config file,
for VictoriaMetrics this will be in the file passed via `-promscrape.config file_path`.
In either case a `scrape_configs` with just one job for a single Akkoma instance will look like this:
```yaml
scrape_configs:
- job_name: 'akkoma_scrape_job'
scheme: https
metrics_path: /api/v1/akkoma/metrics
static_configs:
- targets: ['myinstance.example']
# reminder: no Bearer prefix here!
bearer_token: '$ACCESS_TOKEN'
# One minute should be frequent enough, but you can choose any value, or rely on the global default.
# However, for later use in Grafana you want to match this exactly, thus make note.
scrape_interval: '1m'
```
Now (re)start the scraper service, wait for a multiple of the scrape interval and check logs
to make sure no errors occur.
### Step 3: visualise the collected data
At last it’s time to actually get a look at the collected data.
There are many options working with Prometheus-compatible backends
and even software which can act as both the scraper _and_ visualiser in one service.
Here we’ll just deal with Grafana, since we ship a reference Grafana dashboard you can just import.
There are again multiple options for [installing Grafana](https://grafana.com/docs/grafana/latest/setup-grafana/)
and detailing all of them is out of scope here, but it’s nothing too complicated if you already set up Akkoma.
Once you’ve got it running and are logged into Grafana,
you first need to tell it about the scraper which acts a the “data source”.
For this go to the “Connections” category and select “Data Sources”.
Here click the shiny button for adding a new data source,
select the “Prometheus” type and fill in the details
matching how you set up the scraper itself.
In particular, **use the same `Scrape Interval` value!**
Now you’re ready to go to the “Dashboards” page.
Click the “New” button, select “Import” and upload or copy the contents of
the reference dashboard `installation/grafana_dashboard.json` from Akkoma’s source tree.
It will now ask you to select the data source you just configured,
as well as for the name of the job in your scraper config
and your instance domain+port identifier.
For the example settings from step 2 above
the latter two are `akkoma_scrape_job` and `myinstance.example:443`.
*(`443` is the default port for HTTPS)*
That’s it, you’ve got a fancy dashboard with long-term, 24/7 metrics now!
Updating the dashboard can be done by just repeating the import process.
Here’s an example taken from a healthy, small instance where
nobody was logged in except for a few minutes
resulting in an (expected) spike in incoming requests:
!!! note
By default the dashboard does not count downtime of the data source, e.g. the scraper,
towards instance downtime, but a commented out alternative query is provided in the
panel edit menu. If you host the scraper on the same machine as Akkoma you likely want to swap this out.
### Remarks on interpreting the data
What’s kind of load and even error ratio is normal or irregular can depend on
the instance size, chosen configuration and federation partners.
*(E.g. when following relays, much more activities will be received and the received activities will in turn kick off more internal work and also external fetches raising the overall load)*
Here the 24/7 nature of the metric history helps out, since we can just
look at "known-good" time spans to get a feeling for what’s normal and good.
If issues without an immediately clear origin crop up,
we can look for deviations from this known-good pattern.
Still there are some things to be aware of and some common guidelines.
#### Panel-specific time ranges
A select few panels, are set to use a custom time range
independent from what you chose for the dashboard as a whole.
This is indicated with blue text next to the panel title.
Those custom times only take precedence over _relative_ global time ranges.
If you choose fixed start and end dates in the global setting
*(for example to look at a long-term trend after a specific change)*
this will take precedence over custom panel times and everything follows the date range.
In the image above e.g. the uptime percent gauge thus considers the entire last week
while most other panels only display data for the last 6 hours.
#### Long-term trends
The lower section of the dashboard with 24h and 48h averages is particularly useful for observing long-term trends.
E.g. how a patch, version upgrade or database `VACCUUM`/`REINDEX` affects performance.
For small time ranges you can still look at them to make sure the values are at a reasonable level,
but the upper part is probably more interesting.
#### Processing times total
The actions counted by various “time per second” or “time per minute” stats are partially overlapping.
E.g. the time to conclude a HTTP response includes the time it took to run whatever
query was needed to fetch the necessary information from the database.
However not all database queries originate from HTTP requests.
But also, not all of the recorded time might have actually consumed CPU cycles.
Some jobs, e.g. `RemoteFetcherWorker`, will need to fetch data over the network
and often most of the time from job start to completion is just spent waiting
for a reply from the remote server to arrive.
Even a few HTTP endpoints will need to fetch remote data before completing;
e.g. `/inbox` needs to verify the signature of the submission, but if the signing key
wasn’t encountered before it first needs to be fetched.
Getting deliveries from such unknown users happens more often than you might initially assume
due to e.g. Mastodon federating actor deletions to _every server it knows about_
regardless of whether there was ever any contact with the deleted user.
*(Meaning in the end the key lookup will just result in a `410 Gone` response and us dropping the superfluous `Delete`)*
Thus if you just add up all timing stats you’ll count some actions multiple times
and may end up consistently with more processing time being done than time elapsing on the wall clock
even though your server is neither overloaded nor subject to relative time dilation.
For keeping track of CPU and elixir-side(!) IO bottlenecks,
the corresponding BEAM VM gauges are much better indicators.
They should be zero most of the time and never exceed zero by much.
!!! note
The BEAM VM (running our elixir code) cannot know about
the database’s IO activity or CPU cycle consumption,
thus this gauge is no indicator for database bottlenecks.
#### Job failures and federation
Most jobs are automatically retried and may fail (“exception”) due to no fault of your own instance
e.g. network issues or a remote server temporarily being overloaded.
Thus seeing some failures here is normal and nothing to be concerned about;
usually it will just resolve itself on the next retry.
Consistent and/or a relatively high success-to-failure ratio though
is worth looking into using logs.
Of particular importance are Publisher jobs;
they handle delivering your local content to its intended remote recipients.
Again some PublisherWorker exceptions are no cause for concern,
but if all retries for a delivery fail, this means a remote recipient never
received something they should’ve seen.
Due to its particular importance, such final delivery failures are
recorded again in a separate metric.
The reference dashboard shows it in the “AP delivery failures” panel.
Everything listed there exhausted all retries without success.
Ideally this will always be empty and for small instances this should be the
case most of the time.
However, whenever a remote instance which once interacted with
your local instance in the past is decommissioned, delivery failures will likely
eventually show up in your metrics. For example:
- a local user might be followed by an user from the dead instance
- a local posts was in the past fetched by the dead instance and this post is now deleted;
Akkoma will attempt to deliver the `Delete` to the dead instance even if there’s no follow relationship
Delivery failures for such dead instances will typically list a reason like
`http_410`, `http_502`, `http_520`-`http_530` (cloudflare’d instances), `econnrefused`, `nxdomain` or just `timeout`.
If all deliveries to a given remote instance consistently fail for a longer time,
Akkoma will mark it as permanently unreachable and stop even attempting to deliver
to it meaning the errors should go away after a while.
*(If Akkoma sees activity from the presumed dead instance again it will resume deliveries for future content, but anything in the past will remain lost)*
Large instances with many users are more likely to have (had) some relationship to
such a recently decommissioned instances and thus might see failures here more often
even if nothing is wrong with the local Akkoma instance.
If this makes too much noise, consider filtering out telltale delivery failures.
On the opposite side of things, a `http_401` error for example is always worth looking into!
## Built-in Dashboard
Administrators can access a live dashboard under `/phoenix/live_dashboard`
giving an overview of uptime, software versions, database stats and more.
The dashboard also includes a variation of the prometheus metrics, however
they do not exactly match due to respective limitations of the dashboard
and the prometheus exporter.
Even more important, the dashboard collects metrics locally in the browser
only while the page is open and cannot give a view on their past history.
For proper monitoring it is recommended to set up prometheus.
This dashboard can also show a limited subset of Prometheus metrics,
however besides being limited it only starts collecting data when opening
the corresponding page in the browser and the history only exists in ephemeral browser memory.
When navigating away from the page, all history is gone.
However, this is not this dashboards main purpose anyway.
The usefulness of this built-in dashboard are the insights into the current state of
the BEAM VM running Akkoma’s code and statistics about the database and its performance
as well as database diagnostics.
BEAM VM stats include detailed memory consumption breakdowns
and a full list of running processes for example.
## Postgres Statements Statistics
The built-in dashboard can list the queries your instances spends the
most accumulative time on giving insight into potential bottlenecks
and what might be worth optimising.
This is the “Outliers” tab in “Ecto Stats”.
However for this to work you first need to enable a PostgreSQL extension
as follows:
Add the following two lines two your `postgresql.conf` (typically placed in your data dir):
```
shared_preload_libraries = 'pg_stat_statements'
pg_stat_statements.track = all
```
Now restart PostgreSQL. Then connect to your akkoma database using `psql` and run:
```sql
CREATE EXTENSION IF NOT EXISTS pg_stat_statements;
```
Execution time statistics will now start to be gathered.
To get a representative sample of your instances workload you should wait a week or at least a day.
These statistics are never reset automatically, but with new Akkoma releases and
changes in the servers your instance federates with the workload will evolve.
Thus it’s a good idea to reset this occasionally using:
```sql
-- get user oid: SELECT oid FROM pg_roles WHERE rolname = 'akkoma';
-- get db oid: SELECT oid FROM pg_database WHERE datname = 'akkoma';
SELECT pg_stat_statements_reset('<akkoma user oid>'::regclass::oid, '<akkoma database oid>'::regclass::oid);
-- or alternatively, to just reset stats for all users and databases:
-- SELECT pg_stat_statements_reset();
```
## Oban Web
This too requires administrator rights to access and can be found under `/akkoma/oban` if enabled.
The exposed aggregate info is mostly redundant with job statistics already tracked in Prometheus,
but it additionally also:
- shows full argument and meta details for each job
- allows interactively deleting or manually retrying jobs
*(keep this in mind when granting people administrator rights!)*
However, there are two caveats:
1. Just as with the other built-in dashboard, data is not kept around
(although here a **short** backlog actually exists);
when you notice an issue during use and go here to check it likely is already too late.
Job details and history only exists while the jobs are still in the database;
by default failed and succeeded jobs will disappear after about a minute.
2. This dashboard comes with some seemingly constant-ish overhead.
For large instances this appears to be negligible, but small instances on weaker hardware might suffer.
Thus this dashboard can be disabled in the [config](../cheatsheet.md#oban-web).
This is a cheat sheet for Akkoma configuration file, any setting possible to configure should be listed here.
For OTP installations the configuration is typically stored in `/etc/akkoma/config.exs`.
For OTP installations, the configuration is typically stored in `/etc/akkoma/config.exs`.
For from source installations Akkoma configuration works by first importing the base config `config/config.exs`, then overriding it by the environment config `config/$MIX_ENV.exs` and then overriding it by user config `config/$MIX_ENV.secret.exs`. In from source installations you should always make the changes to the user config and NEVER to the base config to avoid breakages and merge conflicts. So for production you change/add configuration to `config/prod.secret.exs`.
For from source installations, Akkoma configuration works by first importing the base config `config/config.exs`, then overriding it by the environment config `config/$MIX_ENV.exs` and then overriding it by user config `config/$MIX_ENV.secret.exs`. In from source installations you should always make the changes to the user config and NEVER to the base config to avoid breakages and merge conflicts. I.e. for production you change/add configuration to `config/prod.secret.exs`.
To add configuration to your config file, you can copy it from the base config. The latest version of it can be viewed [here](https://akkoma.dev/AkkomaGang/akkoma/src/branch/develop/config/config.exs). You can also use this file if you don't know how an option is supposed to be formatted.
@ -12,7 +12,12 @@ To add configuration to your config file, you can copy it from the base config.
* `name`: The instance’s name.
* `email`: Email used to reach an Administrator/Moderator of the instance.
* `notify_email`: Email used for notifications.
* `description`: The instance’s description, can be seen in nodeinfo and ``/api/v1/instance``.
* `short_description`: A brief, plain-text-only instance description.
Can be seen in nodeinfo and `/api/v1/instance`.
Defaults to `description` if unset
* `description`: The instance’s more detailed description.
This is allowed to use HTML if `short_description` is set.
Can be seen in `api/v1/instance`.
* `limit`: Posts character limit (CW/Subject included in the counter).
* `description_limit`: The character limit for image descriptions.
* `remote_limit`: Hard character limit beyond which remote posts will be dropped.
@ -33,12 +38,12 @@ To add configuration to your config file, you can copy it from the base config.
* `federation_incoming_replies_max_depth`: Max. depth of reply-to activities fetching on incoming federation, to prevent out-of-memory situations while fetching very long threads. If set to `nil`, threads of any depth will be fetched. Lower this value if you experience out-of-memory crashes.
* `federation_reachability_timeout_days`: Timeout (in days) of each external federation target being unreachable prior to pausing federating to it.
* `allow_relay`: Permits remote instances to subscribe to all public posts of your instance. This may increase the visibility of your instance.
* `public`: Allows unauthenticated access to public resources on your instance. This is essentially used as the default value for `:restrict_unauthenticated`.
* `public`: Allows unauthenticated access to public resources on your instance. This is essentially used as the default value for `:restrict_unauthenticated`.
See `restrict_unauthenticated` for more details.
* `quarantined_instances`: *DEPRECATED* ActivityPub instances where activities will not be sent. They can still reach there via other means, we just won't send them.
* `quarantined_instances`: *DEPRECATED* ActivityPub instances where activities will not be sent. They can still see our activities via other means, we just won't send them.
* `allowed_post_formats`: MIME-type list of formats allowed to be posted (transformed into HTML).
* `extended_nickname_format`: Set to `true` to use extended local nicknames format (allows underscores/dashes). This will break federation with
older software for theses nicknames.
* `extended_nickname_format`: Set to `true` to use extended local nicknames format (allows underscores/dashes).
This will break federation with older software for theses nicknames.
* `max_pinned_statuses`: The maximum number of pinned statuses. `0` will disable the feature.
* `autofollowed_nicknames`: Set to nicknames of (local) users that every new user should automatically follow.
* `autofollowing_nicknames`: Set to nicknames of (local) users that automatically follows every newly registered user.
@ -46,10 +51,11 @@ To add configuration to your config file, you can copy it from the base config.
* `max_report_comment_size`: The maximum size of the report comment (Default: `1000`).
* `safe_dm_mentions`: If set to true, only mentions at the beginning of a post will be used to address people in direct messages. This is to prevent accidental mentioning of people when talking about them (e.g. "@friend hey i really don't like @enemy"). Default: `false`.
* `healthcheck`: If set to true, system data will be shown on ``/api/v1/pleroma/healthcheck``.
* `remote_post_retention_days`: The default amount of days to retain remote posts when pruning the database.
* `remote_post_retention_days`: The default number of days to retain remote posts when pruning the database.
* `user_bio_length`: A user bio maximum length (default: `5000`).
* `user_name_length`: A user name maximum length (default: `100`).
* `skip_thread_containment`: Skip filter out broken threads. The default is `false`.
* `skip_thread_containment`: **DEPRECATED**, DO NOT CHANGE THE DEFAULT!
Skip filter out broken threads. The default is `false`.
* `limit_to_local_content`: Limit unauthenticated users to search for local statutes and users only. Possible values: `:unauthenticated`, `:all` and `false`. The default is `:unauthenticated`.
* `max_account_fields`: The maximum number of custom fields in the user profile (default: `10`).
* `max_remote_account_fields`: The maximum number of custom fields in the remote user profile (default: `20`).
@ -61,9 +67,9 @@ To add configuration to your config file, you can copy it from the base config.
* `cleanup_attachments_delay`: How many seconds to wait after post deletion before attempting to deletion; useful for “delete & redraft” functionality (default: `1800`)
* `show_reactions`: Let favourites and emoji reactions be viewed through the API (default: `true`).
* `password_reset_token_validity`: The time after which reset tokens aren't accepted anymore, in seconds (default: one day).
* `local_bubble`: Array of domains representing instances closely related to yours. Used to populate the `bubble` timeline. e.g `["example.com"]`, (default: `[]`)
* `languages`: List of Language Codes used by the instance. This is used to try and set a default language from the frontend. It will try and find the first match between the languages set here and the user's browser languages. It will default to the first language in this setting if there is no match.. (default `["en"]`)
* `export_prometheus_metrics`: Enable prometheus metrics, served at `/api/v1/akkoma/metrics`, requiring the `admin:metrics`oauth scope.
* `local_bubble`: Array of domains representing instances closely related to yours. Used to populate the `bubble` timeline. e.g.`["example.com"]`, (default: `[]`)
* `languages`: List of Language Codes used by the instance. This is used to try and set a default language from the frontend. It will try and find the first match between the languages set here and the user's browser languages. It will default to the first language in this setting if there is no match. (default `["en"]`)
* `export_prometheus_metrics`: Enable prometheus metrics, served at `/api/v1/akkoma/metrics`, requiring the `admin:metrics`OAuth scope.
* `privileged_staff`: Set to `true` to give moderators access to a few higher responsibility actions.
* `federated_timeline_available`: Set to `false` to remove access to the federated timeline for all users.
@ -76,15 +82,15 @@ To add configuration to your config file, you can copy it from the base config.
## Welcome
* `direct_message`: - welcome message sent as a direct message.
* `enabled`: Enables the send a direct message to a newly registered user. Defaults to `false`.
* `enabled`: Enables sending a direct message to a newly registered user. Defaults to `false`.
* `sender_nickname`: The nickname of the local user that sends the welcome message.
* `message`: A message that will be send to a newly registered users as a direct message.
* `message`: A message that will be sent to a newly registered users as a direct message.
* `email`: - welcome message sent as a email.
* `enabled`: Enables the send a welcome email to a newly registered user. Defaults to `false`.
* `enabled`: Enables sending a welcome email to newly registered users. Defaults to `false`.
* `sender`: The email address or tuple with `{nickname, email}` that will use as sender to the welcome email.
* `subject`: A subject of welcome email.
* `html`: A html that will be send to a newly registered users as a email.
* `text`: A text that will be send to a newly registered users as a email.
* `html`: A html that will be sent to newly registered users as an email.
* `text`: A text that will be sent to newly registered users as an email.
Example:
@ -135,7 +141,7 @@ To add configuration to your config file, you can copy it from the base config.
* `Pleroma.Web.ActivityPub.MRF.StealEmojiPolicy`: Steals all eligible emoji encountered in posts from remote instances
(See [`:mrf_steal_emoji`](#mrf_steal_emoji))
* `Pleroma.Web.ActivityPub.MRF.SubchainPolicy`: Selectively runs other MRF policies when messages match (See [`:mrf_subchain`](#mrf_subchain)).
* `Pleroma.Web.ActivityPub.MRF.TagPolicy`: Applies policies to individual users based on tags, which can be set using pleroma-fe/admin-fe/any other app that supports Pleroma Admin API. For example it allows marking posts from individual users nsfw (sensitive).
* `Pleroma.Web.ActivityPub.MRF.TagPolicy`: Applies policies to individual users based on tags, which can be set using pleroma-fe/admin-fe/any other app that supports Pleroma Admin API. For example, it allows marking posts from individual users nsfw (sensitive).
* `Pleroma.Web.ActivityPub.MRF.UserAllowListPolicy`: Drops all posts except from users specified in a list.
(See [`:mrf_user_allowlist`](#mrf_user_allowlist))
* `Pleroma.Web.ActivityPub.MRF.VocabularyPolicy`: Restricts activities to a configured set of vocabulary. (See [`:mrf_vocabulary`](#mrf_vocabulary)).
@ -170,7 +176,7 @@ Additionally the following MRFs will *always* be aplied and cannot be disabled:
* `media_removal`: List of instances to strip media attachments from and the reason for doing so.
* `media_nsfw`: List of instances to tag all media as NSFW (sensitive) from and the reason for doing so.
* `federated_timeline_removal`: List of instances to remove from the Federated Timeline (aka The Whole Known Network) and the reason for doing so.
* `reject`: List of instances to reject activities (except deletes) from and the reason for doing so. Additionally prevents activities from being sent to that instance.
* `reject`: List of instances to reject activities (except deletes) from and the reason for doing so. Additionally, prevents activities from being sent to that instance.
* `accept`: List of instances to only accept activities (except deletes) from and the reason for doing so.
* `followers_only`: Force posts from the given instances to be visible by followers only and the reason for doing so.
* `report_removal`: List of instances to reject reports from and the reason for doing so.
* `reject_threshold`: Number of mentioned users after which the messaged gets rejected. Set to 0 to disable.
#### :mrf_keyword
* `reject`: A list of patterns which result in message being rejected, each pattern can be a string or a [regular expression](https://hexdocs.pm/elixir/Regex.html).
* `federated_timeline_removal`: A list of patterns which result in message being removed from federated timelines (a.k.a unlisted), each pattern can be a string or a [regular expression](https://hexdocs.pm/elixir/Regex.html).
* `reject`: A list of patterns which result in a message being rejected, each pattern can be a string or a [regular expression](https://hexdocs.pm/elixir/Regex.html).
* `federated_timeline_removal`: A list of patterns which result in message being removed from federated timelines (a.k.a. unlisted), each pattern can be a string or a [regular expression](https://hexdocs.pm/elixir/Regex.html).
* `replace`: A list of tuples containing `{pattern, replacement}`, `pattern` can be a string or a [regular expression](https://hexdocs.pm/elixir/Regex.html).
This can be used to configure a keyword list that keeps the configuration data for any kind of frontend. By default, settings for `pleroma_fe` and `masto_fe` are configured. You can find the documentation for `pleroma_fe` configuration into [Akkoma-FE configuration and customization for instance administrators](https://docs-fe.akkoma.dev/stable/CONFIGURATION/#options).
This can be used to configure a keyword list that keeps the configuration data for any kind of frontend. By default, settings for `pleroma_fe` and `masto_fe` are configured. You can find the documentation for `pleroma_fe` configuration into [akkoma-fe configuration and customization for instance administrators](https://docs-fe.akkoma.dev/stable/CONFIGURATION/#options).
Frontends can access these settings at `/api/v1/pleroma/frontend_configurations`
To add your own configuration for Akkoma-FE, use it like this:
To add your own configuration for akkoma-fe, use it like this:
```elixir
config :pleroma, :frontend_configurations,
@ -340,7 +346,7 @@ config :pleroma, :frontends,
* `:primary` - The frontend that will be served at `/`
* `:admin` - The frontend that will be served at `/pleroma/admin`
* `:swagger` - Config for developers to act as an API reference to be served at `/pleroma/swaggerui/` (trailing slash _needed_). Disabled by default.
* `:mastodon` - The mastodon-fe configuration. This shouldn't need to be changed. This is served at `/web` when installed.
* `:mastodon` - The masto-fe configuration. This shouldn't need to be changed. This is served at `/web` when installed.
### :static\_fe
@ -352,20 +358,20 @@ Available options:
### :assets
This section configures assets to be used with various frontends. Currently the only option
relates to mascots on the mastodon frontend
This section configures assets to be used with various frontends.
Currently, the only option relates to mascots on masto-fe
* `mascots`: KeywordList of mascots, each element __MUST__ contain both a `url` and a
`mime_type` key.
* `default_mascot`: An element from `mascots` - This will be used as the default mascot
on MastoFE (default: `:pleroma_fox_tan`).
on masto-fe (default: `:pleroma_fox_tan`).
### :manifest
This section describe PWA manifest instance-specific values. Currently this option relate only for MastoFE.
This section describes PWA manifest instance-specific values. Currently this option relate only for masto-fe.
* `icons`: Describe the icons of the app, this a list of maps describing icons in the same way as the
[spec](https://www.w3.org/TR/appmanifest/#imageresource-and-its-members) describes it.
* `icons`: Describe the icons of the app, this is a list of maps describing icons as
detailed in its [spec](https://www.w3.org/TR/appmanifest/#imageresource-and-its-members).
Example:
@ -393,7 +399,7 @@ This section describe PWA manifest instance-specific values. Currently this opti
* `shortcode_globs`: Location of custom emoji files. `*` can be used as a wildcard. Example `["/emoji/custom/**/*.png"]`
* `pack_extensions`: A list of file extensions for emojis, when no emoji.txt for a pack is present. Example `[".png", ".gif"]`
* `groups`: Emojis are ordered in groups (tags). This is an array of key-value pairs where the key is the groupname and the value the location or array of locations. `*` can be used as a wildcard. Example `[Custom: ["/emoji/*.png", "/emoji/custom/*.png"]]`
* `groups`: Emojis are ordered in groups (tags). This is an array of key-value pairs where the key is the groupname and the value the location or array of locations. `*` can be used as a wildcard. Example `[Custom: ["/emoji/*.png", "/emoji/custom/*.png"]]`
* `default_manifest`: Location of the JSON-manifest. This manifest contains information about the emoji-packs you can download. Currently only one manifest can be added (no arrays).
* `shared_pack_cache_seconds_per_file`: When an emoji pack is shared, the archive is created and cached in
memory for this amount of seconds multiplied by the number of files.
@ -404,14 +410,14 @@ This section describe PWA manifest instance-specific values. Currently this opti
* `base_url`: The base URL to access a user-uploaded file.
Using a (sub)domain distinct from the instance endpoint is **strongly** recommended.
* `proxy_opts`: All options defined in `Pleroma.ReverseProxy` documentation, defaults to `[max_body_length: (25*1_048_576)]`.
* `whitelist`: List of hosts with scheme to bypass the mediaproxy (e.g. `https://example.com`)
* `whitelist`: List of hosts with scheme to bypass the mediaproxy (e.g. `https://example.com`)
* `invalidation`: options for remove media from cache after delete object:
* `enabled`: Enables purge cache
* `provider`: Which one of the [purge cache strategy](#purge-cache-strategy) to use.
* `provider`: Which one of the [purge cache strategies](#purge-cache-strategy) to use.
## :media_preview_proxy
* `enabled`: Enables proxying of remote media preview to the instance’s proxy. Requires enabled media proxy (`media_proxy/enabled`).
* `enabled`: Enables proxying of remote media preview to the instance’s proxy. Requires media proxy to be enabled (`media_proxy/enabled`).
* `thumbnail_max_width`: Max width of preview thumbnail for images (video preview always has original dimensions).
* `thumbnail_max_height`: Max height of preview thumbnail for images (video preview always has original dimensions).
* `image_quality`: Quality of the output. Ranges from 0 (min quality) to 100 (max quality).
@ -421,8 +427,8 @@ This section describe PWA manifest instance-specific values. Currently this opti
#### Pleroma.Web.MediaProxy.Invalidation.Script
This strategy allow perform external shell script to purge cache.
Urls of attachments are passed to the script as arguments.
This strategy allows executing an external shell script to purge cache.
URLs of attachments are passed to the script as arguments.
* `script_path`: Path to the external script.
* `url_format`: Set to `:htcacheclean` if using Apache's htcacheclean utility.
`Phoenix` endpoint configuration, all configuration options can be viewed [here](https://hexdocs.pm/phoenix/Phoenix.Endpoint.html#module-dynamic-configuration), only common options are listed here.
* `http` - a list containing http protocol configuration, all configuration options can be viewed [here](https://hexdocs.pm/plug_cowboy/Plug.Cowboy.html#module-options), only common options are listed here. For deployment using docker, you need to set this to `[ip: {0,0,0,0}, port: 4000]` to make akkoma accessible from other containers (such as your nginx server).
* `http` - a list containing HTTP protocol configuration, all configuration options can be viewed [here](https://hexdocs.pm/plug_cowboy/Plug.Cowboy.html#module-options), only common options are listed here. For deployment using Docker, you need to set this to `[ip: {0,0,0,0}, port: 4000]` to make akkoma accessible from other containers (such as your nginx server).
- `ip` - a tuple consisting of 4 integers
- `port`
* `url` - a list containing the configuration for generating urls, accepts
* `url` - a list containing the configuration for generating URLs, accepts
- `host` - the host without the scheme and a post (e.g `example.com`, not `https://example.com:2020`)
- `scheme` - e.g `http`, `https`
- `scheme` - e.g.`http`, `https`
- `port`
- `path`
* `extra_cookie_attrs` - a list of `Key=Value` strings to be added as non-standard cookie attributes. Defaults to `["SameSite=Lax"]`. See the [SameSite article](https://www.owasp.org/index.php/SameSite) on OWASP for more info.
This will make Akkoma listen on `127.0.0.1` port `8080` and generate urls starting with `https://example.com:2020`
This will make Akkoma listen on `127.0.0.1` port `8080` and generate URLs starting with `https://example.com:2020`
### :http_security
* ``enabled``: Whether the managed content security policy is enabled.
* ``sts``: Whether to additionally send a `Strict-Transport-Security` header.
* ``sts_max_age``: The maximum age for the `Strict-Transport-Security` header if sent.
* ``referrer_policy``: The referrer policy to use, either `"same-origin"` or `"no-referrer"`.
* ``report_uri``: Adds the specified url to `report-uri` and `report-to` group in CSP header.
* ``report_uri``: Adds the specified URL to `report-uri` and `report-to` group in CSP header.
### Pleroma.Web.Plugs.RemoteIp
@ -515,11 +521,10 @@ Available options:
* `proxies` - A list of upstream proxy IP subnets in CIDR notation from which we will parse the content of `headers`. Defaults to `[]`. IPv4 entries without a bitmask will be assumed to be /32 and IPv6 /128.
* `reserved` - A list of reserved IP subnets in CIDR notation which should be ignored if found in `headers`. Defaults to `["127.0.0.0/8", "::1/128", "fc00::/7", "10.0.0.0/8", "172.16.0.0/12", "192.168.0.0/16"]`.
### :rate_limit
!!! note
If your instance is behind a reverse proxy ensure [`Pleroma.Web.Plugs.RemoteIp`](#pleroma-plugs-remoteip) is enabled (it is enabled by default).
If your instance is behind a reverse proxy, ensure [`Pleroma.Web.Plugs.RemoteIp`](#pleroma-plugs-remoteip) is enabled (it is enabled by default).
A keyword list of rate limiters where a key is a limiter name and value is the limiter configuration. The basic configuration is a tuple where:
1. In 60 seconds, 15 authentication attempts can be performed from the same IP address.
2. In 1 second, 10 search requests can be performed from the same IP adress by unauthenticated users, while authenticated users can perform 30 search requests per second.
2. In 1 second, 10 search requests can be performed from the same IP address by unauthenticated users, while authenticated users can perform 30 search requests per second.
Supported rate limiters:
* `:search` - Account/Status search.
* `:timeline` - Timeline requests (each timeline has it's own limiter).
* `:timeline` - Timeline requests (each timeline has its own limiter).
* `:app_account_creation` - Account registration from the API.
* `:relations_actions` - Following/Unfollowing in general.
* `:relation_id_action` - Following/Unfollowing for a specific user.
* `:statuses_actions` - Status actions such as: (un)repeating, (un)favouriting, creating, deleting.
* `:status_id_action` - (un)Repeating/(un)Favouriting a particular status.
* `:authentication` - Authentication actions, i.e getting an OAuth token.
* `:authentication` - Authentication actions, i.e. getting an OAuth token.
* `:ap_routes` - Requesting statuses via ActivityPub.
@ -570,6 +575,8 @@ Available caches:
* `receive_timeout`: the amount of time, in ms, to wait for a remote server to respond to a request. (default: `15000`)
* `pool_timeout`: the amount of time, in ms, to wait to check out an HTTP connection from the pool. This likely does not need changing unless your instance is _very_ busy with outbound requests. (default `5000`)
* `protocols`: array of acceptable protocols for outgoing requests; by default both HTTP1 and HTTP2 are supported.
Due to Finch limitations multiplexing cam only be used when this is set to exclusively HTTP2, but this will break federation with HTTP1-only instances.
* `proxy_url`: an upstream proxy to fetch posts and/or media with, (default: `nil`); for example `http://127.0.0.1:3192`. Does not support SOCKS5 proxy, only http(s).
* `send_user_agent`: should we include a user agent with HTTP requests? (default: `true`)
* `user_agent`: what user agent should we use? (default: `:default`), must be string or `:default`
@ -603,7 +610,7 @@ the source code is here: [kocaptcha](https://github.com/koto-bank/kocaptcha). Th
* `uploader`: Which one of the [uploaders](#uploaders) to use.
* `filters`: List of [upload filters](#upload-filters) to use.
* `link_name`: When enabled Akkoma will add a `name` parameter to the url of the upload, for example `https://instance.tld/media/corndog.png?name=corndog.png`. This is needed to provide the correct filename in Content-Disposition headers
* `link_name`: When enabled Akkoma will add a `name` parameter to the URL of the upload, for example `https://instance.tld/media/corndog.png?name=corndog.png`. This is needed to provide the correct filename in Content-Disposition headers
* `base_url`: The base URL to access a user-uploaded file; MUST be configured explicitly.
Using a (sub)domain distinct from the instance endpoint is **strongly** recommended. A good value might be `https://media.myakkoma.instance/media/`.
* `proxy_opts`: Proxy options, see `Pleroma.ReverseProxy` documentation.
@ -616,7 +623,7 @@ the source code is here: [kocaptcha](https://github.com/koto-bank/kocaptcha). Th
#### Pleroma.Uploaders.Local
* `uploads`: Which directory to store the user-uploads in, relative to pleroma’s working directory.
* `uploads`: Which directory to store the user-uploads in, relative to akkoma’s working directory.
#### Pleroma.Uploaders.S3
@ -658,7 +665,7 @@ This filter replaces the declared filename (not the path) of an upload.
#### Pleroma.Upload.Filter.Exiftool.StripMetadata
This filter strips metadata with Exiftool leaving color profiles and orientation intact.
This filter strips metadata with ExifTool leaving color profiles and orientation intact.
* `purge`: List of Exiftool tag names or tag group names to purge
* `preserve`: List of Exiftool tag names or tag group names to preserve even if they occur in the purge list
@ -678,16 +685,16 @@ No specific configuration.
#### Pleroma.Upload.Filter.Mogrify
* `args`: List of actions for the `mogrify` command like `"strip"` or `["strip", "auto-orient", {"implode", "1"}]`.
* `args`: List of actions for the `mogrify` command, like `"strip"` or `["strip", "auto-orient", {"implode", "1"}]`.
## Email
### Pleroma.Emails.Mailer
* `adapter`: one of the mail adapters listed in [Swoosh readme](https://github.com/swoosh/swoosh#adapters), or `Swoosh.Adapters.Local` for in-memory mailbox.
* `api_key` / `password` and / or other adapter-specific settings, per the above documentation.
@ -871,7 +878,7 @@ This will probably take a long time.
### :admin_token
Allows to set a token that can be used to authenticate with the admin api without using an actual user by giving it as the `admin_token` parameter or `x-admin-token` HTTP header. Example:
Allows to set a token that can be used to authenticate with the admin API without using an actual user by giving it as the `admin_token` parameter or `x-admin-token` HTTP header. Example:
Warning: it's discouraged to use this feature because of the associated security risk: static / rarely changed instance-wide token is much weaker compared to email-password pair of a real admin user; consider using HTTP Basic Auth or OAuth-based authentication instead.
!!! warning
It's discouraged to use this feature because of the associated security risk: static / rarely changed instance-wide token is much weaker compared to email-password pair of a real admin user; consider using HTTP Basic Auth or OAuth-based authentication instead.
* `GET /api/v1/accounts/verify_credentials` (with proper `Authorization` header or `access_token` URI param) returns user info on requester (with `acct` field containing local nickname and `fqn` field containing fully-qualified nickname which could generally be used as email stub for OAuth software that demands email field in identity endpoint response, like Peertube).
* `GET /api/v1/accounts/verify_credentials` (with proper `Authorization` header or `access_token` URI parameter) returns user info on requester (with `acct` field containing local nickname and `fqn` field containing fully-qualified nickname which could generally be used as email stub for OAuth software that demands email field in identity endpoint response, like Peertube).
### OAuth consumer mode
OAuth consumer mode allows sign in / sign up via external OAuth providers (e.g. Twitter, Facebook, Google, Microsoft, etc.).
Implementation is based on Ueberauth; see the list of [available strategies](https://github.com/ueberauth/ueberauth/wiki/List-of-Strategies).
Implementation is based on Überauth; see the list of [available strategies](https://github.com/ueberauth/ueberauth/wiki/List-of-Strategies).
!!! note
Each strategy is shipped as a separate dependency; in order to get the strategies, run `OAUTH_CONSUMER_STRATEGIES="..." mix deps.get`, e.g. `OAUTH_CONSUMER_STRATEGIES="twitter facebook google microsoft" mix deps.get`. The server should also be started with `OAUTH_CONSUMER_STRATEGIES="..." mix phx.server` in case you enable any strategies.
* `truncate` - Set to a number to truncate URLs longer then the number. Truncated URLs will end in `...` (default: `false`)
* `truncate` - Set to a number to truncate URLs longer than the number. Truncated URLs will end in `...` (default: `false`)
* `strip_prefix` - Strip the scheme prefix (default: `false`)
* `extra` - link URLs with rarely used schemes (magnet, ipfs, irc, etc.) (default: `true`)
* `validate_tld` - Set to false to disable TLD validation for URLs/emails. Can be set to :no_scheme to validate TLDs only for urls without a scheme (e.g `example.com` will be validated, but `http://example.loki` won't) (default: `:no_scheme`)
* `validate_tld` - Set to false to disable TLD validation for URLs/emails. Can be set to :no_scheme to validate TLDs only for URLs without a scheme (e.g `example.com` will be validated, but `http://example.loki` won't) (default: `:no_scheme`)
Example:
@ -1073,7 +1081,7 @@ git clone <MY MODULE>
## :configurable_from_database
Boolean, enables/disables in-database configuration. Read [Transfering the config to/from the database](../administration/CLI_tasks/config.md) for more information.
Boolean, enables/disables in-database configuration. Read [transferring the config to/from the database](../administration/CLI_tasks/config.md) for more information.
## :database_config_whitelist
@ -1123,12 +1131,13 @@ Turning any of the `:restrict_unauthenticated` options to `true` will restrict a
#### When :instance, :public is `false`
When `:instance, :public` is set to `false`, all of the `:restrict_unauthenticated` options will effectively be set to `true` by default,
When `:instance, :public` is set to `false`, all `:restrict_unauthenticated` options will effectively be set to `true` by default,
meaning that only authenticated users will be able to access the corresponding resources.
If you'd like to allow unauthenticated access to specific resources, you can turn these settings to `false`.
**Note**: setting `restrict_unauthenticated/timelines/local` to `true` has no practical sense if `restrict_unauthenticated/timelines/federated` is set to `false` (since local public activities will still be delivered to unauthenticated users as part of federated timeline).
!!! note
Setting `restrict_unauthenticated/timelines/local` to `true` has no practical sense if `restrict_unauthenticated/timelines/federated` is set to `false` (since local public activities will still be delivered to unauthenticated users as part of federated timeline).
## Pleroma.Web.ApiSpec.CastAndValidate
@ -1145,7 +1154,7 @@ Control favicons for instances.
!!! note
Requires enabled email
* `:purge_after_days` an integer, remove backup achives after N days.
* `:purge_after_days` an integer, remove backup archives after N days.
* `:limit_days` an integer, limit user to export not more often than once per N days.
* `:dir` a string with a path to backup temporary directory or `nil` to let Akkoma choose temporary directory in the following order:
1. the directory named by the TMPDIR environment variable
@ -1157,7 +1166,7 @@ Control favicons for instances.
### Theme settings
Settings to change theme as exposed to the outside world, for software
@ -1182,10 +1191,10 @@ Settings to restrict concurrently running jobs. Jobs which can be configured:
Each job has these settings:
* `:max_running` - max concurrently runnings jobs
* `:max_running` - max concurrently running jobs
* `:max_waiting` - max waiting jobs
### Translation Settings
## Translation Settings
Settings to automatically translate statuses for end users. Currently supported
translation services are DeepL and LibreTranslate. The supported command line tool is [Argos Translate](https://github.com/argosopentech/argos-translate).
@ -1212,6 +1221,15 @@ Translations are available at `/api/v1/statuses/:id/translations/:language`, whe
### `:argos_translate`
- `:command_argos_translate` - command for `argos-translate`. Can be the command if it's in your PATH, or the full path to the file (default: `argos-translate`).
- `:command_argos_translate` - command for `argos-translate`. Can be the command if it's in your PATH, or the full path to the file (default: `argos-translate`).
- `:command_argospm` - command for `argospm`. Can be the command if it's in your PATH, or the full path to the file (default: `argospm`).
- `:strip_html` - Strip html from the post before translating it (default: `true`).
## Oban Web
The built-in Oban Web dashboard grants all administrators access to look at and modify the instance’s job queue.
To enable or disable it the following setting can be set to `true` or `false` respectively:
Frontends in Akkoma are swappable, you can pick which you'd like.
For a basic setup, you can set a frontends for the key `primary` and `admin` and the options of `name` and `ref`. This will then make Akkoma serve the frontend from a folder constructed by concatenating the instance static path, `frontends` and the name and ref.
For a basic setup, you can set a frontend for the key `primary` and `admin` and the options of `name` and `ref`. This will then make Akkoma serve the frontend from a folder constructed by concatenating the instance static path, `frontends` and the name and ref.
The key `primary` refers to the frontend that will be served by default for general requests. The key `admin` refers to the frontend that will be served at the `/pleroma/admin` path.
@ -34,15 +34,15 @@ If you choose not to install a frontend for whatever reason, it is recommended t
You can also replace the default "no frontend" page by placing an `index.html` file under your `instance/static/` directory.
## Mastodon-FE
## masto-fe
Akkoma supports both [glitchsoc](https://github.com/glitch-soc/mastodon)'s more "vanilla" mastodon frontend,
Akkoma supports both [glitch-soc](https://github.com/glitch-soc/mastodon)'s more "vanilla" mastodon frontend,
as well as [fedibird](https://github.com/fedibird/mastodon)'s extended frontend which has near-feature-parity with akkoma (with quoting and reactions).
To enable either one, you must run the `frontend.install` task for either `mastodon-fe` or `fedibird-fe` (both `--ref akkoma`), then make sure
To enable either one, you must run the `frontend.install` task for either `masto-fe` or `fedibird-fe` (both `--ref akkoma`), then make sure
`:pleroma, :frontends, :mastodon` references the one you want.
## Swagger (openAPI) documentation viewer
## Swagger (OpenAPI) documentation viewer
If you're a developer and you'd like a human-readable rendering of the
API documentation, you can enable [Swagger UI](https://github.com/swagger-api/swagger-ui).
@ -81,7 +81,7 @@ Use private `/tmp` and `/var/tmp` folders inside a new file system namespace, wh
> Recommended value: `true`
The `/home`, `/root`, and `/run/user` folders cannot be accessed by this service anymore. If your Akkoma user has its home folder in one of the restricted places, or use one of these folders as its working directory, you have to set this to `false`.
The `/home`, `/root`, and `/run/user` folders cannot be accessed by this service anymore. If your Akkoma user has its home folder in one of the restricted places, or use one of these folders as its working directory, you have to set this to `false`.
@ -11,7 +11,7 @@ Akkoma supports that, but it might be tricky to set up, and any error might prev
It is important to understand that for federation purposes, a user in Akkoma has two unique identifiers associated:
- A webfinger `acct:` URI, used for discovery and as a verifiable global name for the user across Akkoma instances. In our example, our account's acct: URI is `acct:user@example.org`
- A WebFinger `acct:` URI, used for discovery and as a verifiable global name for the user across Akkoma instances. In our example, our account's acct: URI is `acct:user@example.org`
- An author/actor URI, used in every other aspect of federation. This is the way in which users are identified in ActivityPub, the underlying protocol used for federation with other Akkoma instances.
In our case, it is `https://akkoma.example.org/users/user`.
@ -19,7 +19,7 @@ Both account identifiers are unique and required for Akkoma. An important risk i
## WebFinger
As said earlier, each Akkoma user has an `acct`: URI, which is used for discovery and authentication. When you add @user@example.org, a webfinger query is performed. This is done in two steps:
As said earlier, each Akkoma user has an `acct`: URI, which is used for discovery and authentication. When you add @user@example.org, a WebFinger query is performed. This is done in two steps:
1. Querying `https://example.org/.well-known/host-meta` (where the domain of the URL matches the domain part of the `acct`: URI) to get information on how to perform the query.
This file will indeed contain a URL template of the form `https://example.org/.well-known/webfinger?resource={uri}` that will be used in the second step.
@ -27,7 +27,8 @@ This file will indeed contain a URL template of the form `https://example.org/.w
## Configuring your Akkoma instance
**_DO NOT ATTEMPT TO CONFIGURE YOUR INSTANCE THIS WAY IF YOU DID NOT UNDERSTAND THE ABOVE_**
!!! danger
DO NOT ATTEMPT TO CONFIGURE YOUR INSTANCE THIS WAY IF YOU DID NOT UNDERSTAND THE ABOVE
Now, you have Akkoma running at `https://akkoma.example.org` as well as a website at `https://example.org`. If you recall how webfinger queries work, the first step is to query `https://example.org/.well-known/host-meta`, which will contain an URL template.
Now, you have Akkoma running at `https://akkoma.example.org` as well as a website at `https://example.org`. If you recall how WebFinger queries work, the first step is to query `https://example.org/.well-known/host-meta`, which will contain a URL template.
Therefore, the easiest way to configure `example.org` is to redirect `/.well-known/host-meta` to `akkoma.example.org`.
# How to activate Akkoma in-database configuration
## Explanation
The configuration of Akkoma (and Pleroma) has traditionally been managed with a config file, e.g. `config/prod.secret.exs`. This method requires a restart of the application for any configuration changes to take effect. We have made it possible to control most settings in the AdminFE interface after running a migration script.
The configuration of Akkoma (and Pleroma) has traditionally been managed with a config file, e.g. `config/prod.secret.exs`. This method requires a restart of the application for any configuration changes to take effect. We have made it possible to control most settings in the admin-fe interface after running a migration script.
## Migration to database config
1. Run the mix task to migrate to the database.
**Source:**
**Source:**
```
$ mix pleroma.config migrate_to_db
```
```
$ mix pleroma.config migrate_to_db
```
or
or
**OTP:**
**OTP:**
*Note: OTP users need Akkoma to be running for `pleroma_ctl` commands to work*
*Note: OTP users need Akkoma to be running for `pleroma_ctl` commands to work*
```
$ ./bin/pleroma_ctl config migrate_to_db
```
```
$ ./bin/pleroma_ctl config migrate_to_db
```
```
Migrating settings from file: /home/pleroma/config/dev.secret.exs
```
Migrating settings from file: /home/pleroma/config/dev.secret.exs
Settings for key instance migrated.
Settings for group :pleroma migrated.
```
Settings for key instance migrated.
Settings for group :pleroma migrated.
```
2. It is recommended to backup your config file now.
3. Edit your Akkoma config to enable database configuration:
```
config :pleroma, configurable_from_database: true
```
```
config :pleroma, configurable_from_database: true
```
4. ⚠️ **THIS IS NOT REQUIRED** ⚠️
Now you can edit your config file and strip it down to the only settings which are not possible to control in the database. e.g., the Postgres (Repo) and webserver (Endpoint) settings cannot be controlled in the database because the application needs the settings to start up and access the database.
Now you can edit your config file and strip it down to the only settings which are not possible to control in the database. e.g., the Postgres (Repo) and webserver (Endpoint) settings cannot be controlled in the database because the application needs the settings to start up and access the database.
Any settings in the database will override those in the config file, but you may find it less confusing if the setting is only declared in one place.
Any settings in the database will override those in the config file, but you may find it less confusing if the setting is only declared in one place.
A non-exhaustive list of settings that are only possible in the config file include the following:
A non-exhaustive list of settings that are only possible in the config file include the following:
* config :pleroma, Pleroma.Web.Endpoint
* config :pleroma, Pleroma.Repo
* config :pleroma, configurable\_from\_database
* config :pleroma, :database, rum_enabled
* config :pleroma, :connections_pool
* config :pleroma, Pleroma.Web.Endpoint
* config :pleroma, Pleroma.Repo
* config :pleroma, configurable\_from\_database
* config :pleroma, :database, rum_enabled
* config :pleroma, :connections_pool
Here is an example of a server config stripped down after migration:
Here is an example of a server config stripped down after migration:
5. Restart your instance and you can now access the Settings tab in AdminFE.
5. Restart your instance and you can now access the Settings tab in admin-fe.
## Reverting back from database config
1. Run the mix task to migrate back from the database. You'll receive some debugging output and a few messages informing you of what happened.
**Source:**
**Source:**
```
$ mix pleroma.config migrate_from_db
```
```
$ mix pleroma.config migrate_from_db
```
or
or
**OTP:**
**OTP:**
```
$ ./bin/pleroma_ctl config migrate_from_db
```
```
$ ./bin/pleroma_ctl config migrate_from_db
```
```
10:26:30.593 [debug] QUERY OK source="config" db=9.8ms decode=1.2ms queue=26.0ms idle=0.0ms
SELECT c0."id", c0."key", c0."group", c0."value", c0."inserted_at", c0."updated_at" FROM "config" AS c0 []
```
10:26:30.593 [debug] QUERY OK source="config" db=9.8ms decode=1.2ms queue=26.0ms idle=0.0ms
SELECT c0."id", c0."key", c0."group", c0."value", c0."inserted_at", c0."updated_at" FROM "config" AS c0 []
10:26:30.659 [debug] QUERY OK source="config" db=1.1ms idle=80.7ms
SELECT c0."id", c0."key", c0."group", c0."value", c0."inserted_at", c0."updated_at" FROM "config" AS c0 []
Database configuration settings have been saved to config/dev.exported_from_db.secret.exs
```
10:26:30.659 [debug] QUERY OK source="config" db=1.1ms idle=80.7ms
SELECT c0."id", c0."key", c0."group", c0."value", c0."inserted_at", c0."updated_at" FROM "config" AS c0 []
Database configuration settings have been saved to config/dev.exported_from_db.secret.exs
```
2. Remove `config :pleroma, configurable_from_database: true` from your config. The in-database configuration still exists, but it will not be used. Future migrations will erase the database config before importing your config file again.
@ -130,7 +130,7 @@ You can clear the database config with the following command:
Additionally, every time you migrate the configuration to the database the config table is automatically truncated to ensure a clean migration.
### Manually removing a setting
If you encounter a situation where the server cannot run properly because of an invalid setting in the database and this is preventing you from accessing AdminFE, you can manually remove the offending setting if you know which one it is.
If you encounter a situation where the server cannot run properly because of an invalid setting in the database and this is preventing you from accessing admin-fe, you can manually remove the offending setting if you know which one it is.
e.g., here is an example showing a the removal of the `config :pleroma, :instance` settings:
If you want to proxify all http requests (e.g. for TOR) that Akkoma makes to an upstream proxy server, edit your config file (`dev.secret.exs` or `prod.secret.exs`) and add the following:
If you want to proxify all HTTP requests (e.g. for TOR) that Akkoma makes to an upstream proxy server, edit your config file (`dev.secret.exs` or `prod.secret.exs`) and add the following:
@ -15,7 +15,7 @@ In most cases, you would need an extension installed to support parsing CJK text
Once you have the new search config , make sure you test it with the `pleroma` user in PostgreSQL (change `YOUR.CONFIG` to your real configuration name)
@ -47,7 +47,7 @@ Now you'll already be able to select the theme in Pleroma FE from the drop-down.
### Give the theme a name
When you open one of the themes that ship with Akkoma, you'll notice that the json has a `"name"` key. Add a key-value pair to your theme where the key name is `"name"` and the value the name you want to give your theme. After this you can refresh te page in your browser and the name should be visible in the drop-down.
When you open one of the themes that ship with Akkoma, you'll notice that the JSON has a `"name"` key. Add a key-value pair to your theme where the key name is `"name"` and the value the name you want to give your theme. After this you can refresh the page in your browser and the name should be visible in the drop-down.
Example of `my-awesome-theme.json` where we add the name "My Awesome Theme"
If you added it in the back-end configuration file, you'll need to restart your instance for the changes to take effect. If you don't see the changes, it's probably because the browser has cached the previous theme. In that case you'll want to clear browser caches. Alternatively you can use a private/incognito window just to see the changes.
If you added it in the back-end configuration file, you'll need to restart your instance for the changes to take effect. If you don't see the changes, it's probably because the browser has cached the previous theme. In that case you'll want to clear browser caches. Alternatively, you can use a private/incognito window just to see the changes.
This guide is going to focus on the Akkoma federation aspect. The actual installation is neatly explained in the official documentation, and more likely to remain up-to-date.
It might be added to this guide if there will be a need for that.
@ -15,9 +15,10 @@ One using the config, and one using external software (fedproxy). The external s
### Using the Config
**Warning:** So far, everytime I followed this way of federating using I2P, the rest of my federation stopped working. I'm leaving this here in case it will help with making it work.
!!! warning
So far, everytime I followed this way of federating using I2P, the rest of my federation stopped working. I'm leaving this here in case it will help with making it work.
Assuming you're running in prod, cd to your Akkoma folder and append the following to `config/prod.secret.exs`:
Assuming you're running in prod, `cd` to your Akkoma folder and append the following to `config/prod.secret.exs`:
# Configuring Ejabberd (XMPP Server) to use Akkoma for authentication
# Configuring ejabberd (XMPP Server) to use Akkoma for authentication
If you want to give your Akkoma users an XMPP (chat) account, you can configure [Ejabberd](https://github.com/processone/ejabberd) to use your Akkoma server for user authentication, automatically giving every local user an XMPP account.
If you want to give your Akkoma users an XMPP (chat) account, you can configure [ejabberd](https://github.com/processone/ejabberd) to use your Akkoma server for user authentication, automatically giving every local user an XMPP account.
In general, you just have to follow the configuration described at [https://docs.ejabberd.im/admin/configuration/authentication/#external-script](https://docs.ejabberd.im/admin/configuration/authentication/#external-script). Please read this section carefully.
In general, you just have to follow the configuration described at [https://docs.ejabberd.im/admin/configuration/authentication/#external-script](https://docs.ejabberd.im/admin/configuration/authentication/#external-script). Please read this section carefully.
Copy the script below to suitable path on your system and set owner and permissions. Also do not forget adjusting `AKKOMA_HOST` and `AKKOMA_PORT`, if necessary.
Copy the script below to a suitable path on your system and set owner and permissions. Also do not forget adjusting `AKKOMA_HOST` and `AKKOMA_PORT`, if necessary.
@ -5,6 +5,6 @@ If you want to give your Akkoma users an XMPP (chat) account, you can configure
In general, you just have to follow the configuration described at [https://mongooseim.readthedocs.io/en/latest/authentication-backends/HTTP-authentication-module/](https://mongooseim.readthedocs.io/en/latest/authentication-backends/HTTP-authentication-module/) and do these changes to your mongooseim.cfg.
1. Set the auth_method to `{auth_method, http}`.
2. Add the http auth pool like this: `{http, global, auth, [{workers, 50}], [{server, "https://yourakkomainstance.com"}]}`
2. Add the HTTP auth pool like this: `{http, global, auth, [{workers, 50}], [{server, "https://yourakkomainstance.com"}]}`
Restart your MongooseIM server, your users should now be able to connect with their Akkoma credentials.
* sending only public messages to a specific instance
The MRF provides user-configurable policies. The default policy is `NoOpPolicy`, which disables the MRF functionality. Akkoma also includes an easy to use policy called `SimplePolicy` which maps messages matching certain pre-defined criterion to actions built into the policy module.
The MRF provides user-configurable policies. The default policy is `NoOpPolicy`, which disables the MRF functionality. Akkoma also includes an easy-to-use policy called `SimplePolicy` which maps messages matching certain pre-defined criterion to actions built into the policy module.
It is possible to use multiple, active MRF policies at the same time.
@ -29,7 +29,7 @@ config :pleroma, :mrf,
Once `SimplePolicy` is enabled, you can configure various groups in the `:mrf_simple` config object. These groups are:
* `reject`: Servers in this group will have their messages rejected. Also outbound messages will not be sent to these servers.
* `reject`: Servers in this group will have their messages rejected. Also, outbound messages will not be sent to these servers.
* `accept`: If not empty, only messages from these instances will be accepted (whitelist federation).
* `media_nsfw`: Servers in this group will have the #nsfw tag and sensitive setting injected into incoming messages which contain media.
* `media_removal`: Servers in this group will have media stripped from incoming messages.
@ -151,7 +151,7 @@ Please note that the Akkoma developers consider custom MRF policy modules to fal
### MRF policies descriptions
If MRF policy depends on config, it can be added into MRF tab to adminFE by adding `config_description/0` method, which returns a map with a specific structure. See existing MRF's like `lib/pleroma/web/activity_pub/mrf/activity_expiration_policy.ex` for examples. Note that more complex inputs, like tuples or maps, may need extra changes in the adminFE and just adding it to `config_description/0` may not be enough to get these inputs working from the adminFE.
If MRF policy depends on config, it can be added into MRF tab to admin-fe by adding `config_description/0` method, which returns a map with a specific structure. See existing MRF's like `lib/pleroma/web/activity_pub/mrf/activity_expiration_policy.ex` for examples. Note that more complex inputs, like tuples or maps, may need extra changes in the admin-fe and just adding it to `config_description/0` may not be enough to get these inputs working from the admin-fe.
@ -101,7 +101,7 @@ You should now be able to both access your instance using Tor and federate with
### Possible Issues
* In Debian, make sure your hidden service folder `/var/lib/tor/akkoma_hidden_service/` and its contents, has debian-tor as both owner and group by using
* In Debian, make sure your hidden service folder `/var/lib/tor/akkoma_hidden_service/` and its contents, has debian-tor as both owner and group by using
Akkoma is built upon the Erlang/OTP VM known as BEAM. The BEAM VM is highly optimized for latency, but this has drawbacks in environments without dedicated hardware. One of the tricks used by the BEAM VM is [busy waiting](https://en.wikipedia.org/wiki/Busy_waiting). This allows the application to pretend to be busy working so the OS kernel does not pause the application process and switch to another process waiting for the CPU to execute its workload. It does this by spinning for a period of time which inflates the apparent CPU usage of the application so it is immediately ready to execute another task. This can be observed with utilities like **top(1)** which will show consistently high CPU usage for the process. Switching between procesess is a rather expensive operation and also clears CPU caches further affecting latency and performance. The goal of busy waiting is to avoid this penalty.
Akkoma is built upon the Erlang/OTP VM known as BEAM. The BEAM VM is highly optimized for latency, but this has drawbacks in environments without dedicated hardware. One of the tricks used by the BEAM VM is [busy waiting](https://en.wikipedia.org/wiki/Busy_waiting). This allows the application to pretend to be busy working so the OS kernel does not pause the application process and switch to another process waiting for the CPU to execute its workload. It does this by spinning for a period of time which inflates the apparent CPU usage of the application so it is immediately ready to execute another task. This can be observed with utilities like **top(1)** which will show consistently high CPU usage for the process. Switching between processes is a rather expensive operation and also clears CPU caches further affecting latency and performance. The goal of busy waiting is to avoid this penalty.
This strategy is very successful in making a performant and responsive application, but is not desirable on Virtual Machines or hardware with few CPU cores. Akkoma instances are often deployed on the same server as the required PostgreSQL database which can lead to situations where the Akkoma application is holding the CPU in a busy-wait loop and as a result the database cannot process requests in a timely manner. The fewer CPUs available, the more this problem is exacerbated. The latency is further amplified by the OS being installed on a Virtual Machine as the Hypervisor uses CPU time-slicing to pause the entire OS and switch between other tasks.
@ -18,7 +18,6 @@ Please only change these settings if you are experiencing issues or really know
* AWS (known to use burst scheduling)
## Example configurations
Tuning the BEAM requires you provide a config file normally called [vm.args](http://erlang.org/doc/man/erl.html#emulator-flags). If you are using systemd to manage the service you can modify the unit file as such:
@ -10,4 +10,4 @@ It is also recommended to not use "Network Storage" option.
If your server runs other services, you may want to take that into account. E.g. if you have 4G ram, but 1G of it is already used for other services, it may be better to tell PGTune you only have 3G.
In the end, PGTune only provides recomended settings, you can always try to finetune further.
In the end, PGTune only provides recommended settings, you can always try to finetune further.
@ -146,7 +146,7 @@ You then need to set the URL and authentication credentials if relevant.
### Initial indexing
After setting up the configuration, you'll want to index all of your already existsing posts. You'll only have to do it one time, but it might take a while, depending on the amount of posts your instance has seen.
After setting up the configuration, you'll want to index all of your already existsing posts. You'll only have to do it one time, but it might take a while, depending on the amount of posts your instance has seen.
Note the extra `static` folder for the default logo.png location
If you want to give a brand to your instance, You can change the logo of your instance by uploading it to the static directory `$static_dir/static/logo.png`.
If you want to give a brand to your instance, you can change the logo of your instance by uploading it to the static directory `$static_dir/static/logo.png`.
Alternatively, you can specify the path to your logo in [your configuration](../cheatsheet/#frontend_configurations).
@ -91,21 +91,21 @@ Terms of Service will be shown to all users on the registration page. It's the b
## Favicon
The favicon will display on the frontend, and in the browser tab.
The favicon will display on the frontend, and in the browser tab.
Place a PNG file at `$static_dir/favicon.png` to change the favicon. Not that this
is _one level above_ where the logo is placed, it should be on the same level as
the `frontends` directory.
## Styling rendered pages
To overwrite the CSS stylesheet of the OAuth form and other static pages, you can upload your own CSS file to `instance/static/static.css`. This will completely replace the CSS used by those pages, so it might be a good idea to copy the one from `priv/static/instance/static.css` and make your changes.
## Overriding pleroma-fe styles
## Overriding akkoma-fe styles
To overwrite the CSS stylesheet of pleroma-fe, you can put a file at
To overwrite the CSS stylesheet of akkoma-fe, you can put a file at
`$static_dir/static/custom.css` containing your styles. These will be loaded
with the rest of the CSS.
You will probably have to put `!important` on most/all your styles to override the
default ones, due to the specificity precedence of CSS.
default ones, due to the specificity precedence of CSS.
Akkoma does not store remote/federated media by default. The best way to achieve this is to change Nginx to keep its reverse proxy cache
Akkoma does not store remote/federated media by default. The best way to achieve this is to change nginx to keep its reverse proxy cache
for a year and to activate the `MediaProxyWarmingPolicy` MRF policy in Akkoma which will automatically fetch all media through the proxy
as soon as the post is received by your instance.
## Nginx
## nginx
The following are excerpts from the [suggested nginx config](https://akkoma.dev/AkkomaGang/akkoma/src/branch/develop/installation/nginx/akkoma.nginx) that demonstrates the necessary config for the media proxy to work.
