Commit graph

1666 commits

Author SHA1 Message Date
floatingghost 59bfdf2ca4 Merge pull request 'Add limit CLI flags to prune jobs' (#655) from Oneric/akkoma:prune-batch into develop
Some checks are pending
ci/woodpecker/push/build-amd64 Pipeline is pending
ci/woodpecker/push/build-arm64 Pipeline is pending
ci/woodpecker/push/docs Pipeline is pending
ci/woodpecker/push/lint Pipeline is pending
ci/woodpecker/push/test Pipeline is pending
Reviewed-on: #655
2024-06-17 20:47:53 +00:00
Oneric bf8f493ffd Remove proxy_remote vestiges
All checks were successful
ci/woodpecker/pr/lint Pipeline was successful
ci/woodpecker/pr/test Pipeline was successful
ci/woodpecker/pr/build-amd64 Pipeline was successful
ci/woodpecker/pr/build-arm64 Pipeline was successful
ci/woodpecker/pr/docs Pipeline was successful
Ever since 364b6969eb
this setting wasn't used by the backend and a noop.
The stated usecase is better served by setting the base_url
to a local subdomain and using proxying in nginx/Caddy/...
2024-06-16 01:21:52 +02:00
Oneric a3840e7d1f Raise minimum PostgreSQL version to 12
This lets us:
 - avoid issues with broken hash indices for PostgreSQL <10
 - drop runtime checks and legacy codepaths for <11 in db search
 - always enable custom query plans for performance optimisation

PostgreSQL 11 is already EOL since 2023-11-09, so
in theory everyone should already have moved on to 12 anyway.
2024-06-07 16:21:09 +02:00
Norm e2860e5292 Update PGTune recommendations
Some checks are pending
ci/woodpecker/pr/build-amd64 Pipeline is pending
ci/woodpecker/pr/build-arm64 Pipeline is pending
ci/woodpecker/pr/docs Pipeline is pending
ci/woodpecker/pr/lint Pipeline is pending
ci/woodpecker/pr/test Pipeline is pending
From experience, setting DB type to "Online transaction processing
system" seems to give the most optimal configuration in terms of
performance.

I also increased the recomended max connections to 25-30 as that leaves
some room for maintenance tasks to run without running out of
connections.

Finally, I removed the example configs since they're probably out of
date and I think it's better to direct people to use PGTune instead.
2024-06-06 12:18:51 -04:00
Oneric aeaebb566c dbprune: allow splitting array and single activity prunes
The former is typically just a few reports; it doesn't make sense to
rerun it over and over again in batched prunes or if a full prune OOMed.
2024-05-31 17:16:40 +02:00
Oneric 225f87ad62 Also allow limiting the initial prune_object
May sometimes be helpful to get more predictable runtime
than just with an age-based limit.

The subquery for the non-keep-threads path is required
since delte_all does not directly accept limit().

Again most of the diff is just adjusting indentation, best
hide whitespace-only changes with git diff -w or similar.
2024-05-31 17:16:40 +02:00
Oneric e64f031167 Log number of deleted rows in prune_orphaned_activities
This gives feedback when to stop rerunning limited batches.

Most of the diff is just adjusting indentation; best reviewed
with whitespace-only changes hidden, e.g. `git diff -w`.
2024-05-31 17:16:40 +02:00
Oneric fa52093bac Add standalone prune_orphaned_activities CLI task
This part of pruning can be very expensive and bog down the whole
instance to an unusable sate for a long time. It can thus be desireable
to split it from prune_objects and run it on its own in smaller limited batches.

If the batches are smaller enough and spaced out a bit, it may even be possible
to avoid any downtime. If not, the limit can still help to at least make the
downtime duration somewhat more predictable.
2024-05-31 17:16:40 +02:00
Oneric fc7e07f424 meilisearch: enable using search_key
All checks were successful
ci/woodpecker/pr/lint Pipeline was successful
ci/woodpecker/pr/test Pipeline was successful
ci/woodpecker/pr/build-amd64 Pipeline was successful
ci/woodpecker/pr/build-arm64 Pipeline was successful
ci/woodpecker/pr/docs Pipeline was successful
Using only the admin key works as well currently
and Akkoma needs to know the admin key to be able
to add new entries etc. However the Meilisearch
key descriptions suggest the admin key is not
supposed to be used for searches, so let’s not.

For compatibility with existings configs, search_key remains optional.
2024-05-29 23:17:27 +00:00
floatingghost 8afc3bee7a Merge pull request 'Use /var/tmp for media cache path' (#776) from norm/akkoma:nginx-var-tmp into develop
Some checks are pending
ci/woodpecker/push/build-amd64 Pipeline is pending
ci/woodpecker/push/build-arm64 Pipeline is pending
ci/woodpecker/push/docs Pipeline is pending
ci/woodpecker/push/lint Pipeline is pending
ci/woodpecker/push/test Pipeline is pending
Reviewed-on: #776
Reviewed-by: floatingghost <hannah@coffee-and-dreams.uk>
2024-05-28 02:05:17 +00:00
Oneric 05eda169fe Document AP and nodeinfo extensions
Some checks are pending
ci/woodpecker/pr/build-amd64 Pipeline is pending
ci/woodpecker/pr/build-arm64 Pipeline is pending
ci/woodpecker/pr/docs Pipeline is pending
ci/woodpecker/pr/lint Pipeline is pending
ci/woodpecker/pr/test Pipeline is pending
And while add it point to this via a top-level
FEDERATION.md document as standardised by FEP-67ff.

Also add a few missing descriptions to the config cheatsheet
and move the recently removed C2S extension into an appropiate
subsection.
2024-05-26 19:04:06 +02:00
Norm bb29c5bed2 Update tor/i2p guide
Some checks are pending
ci/woodpecker/pr/build-amd64 Pipeline is pending
ci/woodpecker/pr/build-arm64 Pipeline is pending
ci/woodpecker/pr/docs Pipeline is pending
ci/woodpecker/pr/lint Pipeline is pending
ci/woodpecker/pr/test Pipeline is pending
Direct users to add in the appropriate headers and update the listening
port instead of copy/pasting a config that's already outdated and
probably would otherwise have to be synced with the main example nginx
config.
2024-05-16 19:08:02 -04:00
Norm bc46f3da4c Update mediaproxy howto
Since the configuration options on the nginx side already exist in the
sample config, there's no need to tell users to copy-paste those
settings in again.
2024-05-16 19:06:59 -04:00
floatingghost fdeecc7b4c Merge pull request 'Update clients list in docs' (#761) from norm/akkoma:docs-clients-update into develop
Some checks are pending
ci/woodpecker/push/build-amd64 Pipeline is pending
ci/woodpecker/push/build-arm64 Pipeline is pending
ci/woodpecker/push/docs Pipeline is pending
ci/woodpecker/push/lint Pipeline is pending
ci/woodpecker/push/test Pipeline is pending
Reviewed-on: #761
2024-05-06 21:33:26 +00:00
Norm 549d580054 Add Enafore to clients list
Some checks are pending
ci/woodpecker/pr/build-amd64 Pipeline is pending
ci/woodpecker/pr/build-arm64 Pipeline is pending
ci/woodpecker/pr/docs Pipeline is pending
ci/woodpecker/pr/lint Pipeline is pending
ci/woodpecker/pr/test Pipeline is pending
2024-04-26 15:21:58 -04:00
Oneric 5ee0fb18cb exiftool: make stripped tags configurable 2024-04-26 18:57:24 +02:00
Norm 771a306dc1 Update clients list in docs
Some checks are pending
ci/woodpecker/pr/build-amd64 Pipeline is pending
ci/woodpecker/pr/build-arm64 Pipeline is pending
ci/woodpecker/pr/docs Pipeline is pending
ci/woodpecker/pr/lint Pipeline is pending
ci/woodpecker/pr/test Pipeline is pending
- Warn that the apps here are not officially supported
- Update Kaiteki's social profile
- Remove Fedi App
- Add Subway Tooter
2024-04-26 04:17:17 -04:00
Oneric a95af3ee4c exiftool: strip all non-essential tags
Documentation was already clear on this only stripping GPS tags.
But there are more potentially sensitive metadata tags (e.g. author
and possibly description) and the name alone suggests a broader effect.

Thus change the filter to strip all metadata except for colourspace info
and orientation (technically it strips everything and then readds
selected tags).

Explicitly stripping CommonIFD0 is needed since -all does not modify
IFD0 due to TIFF storing some actual image data there. CommonIFD0 then
strips a bunch of commonly used actual metadata tags from IFD0, to my
understanding leaving TIFF image data and custom metadata tags intact.
2024-04-25 23:00:42 +02:00
Oneric 24e608ab5b docs: fix typo 2024-04-25 23:00:42 +02:00
floatingghost b1c6621e66 Merge pull request 'Read image description from EXIF data' (#744) from timorl/akkoma:elseinspe into develop
Some checks are pending
ci/woodpecker/push/build-amd64 Pipeline is pending
ci/woodpecker/push/build-arm64 Pipeline is pending
ci/woodpecker/push/docs Pipeline is pending
ci/woodpecker/push/lint Pipeline is pending
ci/woodpecker/push/test Pipeline is pending
Reviewed-on: #744
2024-04-25 12:52:31 +00:00
Norm 0fa3fbf55e Update OTP install docs to use certbot nginx plugin
Some checks are pending
ci/woodpecker/pr/build-amd64 Pipeline is pending
ci/woodpecker/pr/build-arm64 Pipeline is pending
ci/woodpecker/pr/docs Pipeline is pending
ci/woodpecker/pr/lint Pipeline is pending
ci/woodpecker/pr/test Pipeline is pending
2024-04-23 00:02:54 -04:00
Norm e5f4282cca Update certbot instructions for Alpine Linux 2024-04-23 00:02:54 -04:00
Norm cdde95ad8b Update gentoo install guide to use certbot-nginx 2024-04-23 00:02:54 -04:00
Norm c493769364 Update Nginx setup docs for Fedora and Red Hat OTP 2024-04-23 00:02:15 -04:00
Norm 39b8e73532 Update docs for Arch Linux nginx setup
Alongside moving to certbot's nginx plugin, also use conf.d instead of
recreating the sites-{available,enabled} setup that Debian/Ubuntu uses.

Furthermore, also request a certificate for the media domain at the same
time since that's now required.
2024-04-21 18:19:07 -04:00
Norm 5405828ab1 Update debian install docs to use certbot nginx plugin 2024-04-21 18:19:07 -04:00
timorl cd7af81896
Rename StripLocation to StripMetadata for temporal-proofing reasons
Some checks are pending
ci/woodpecker/pr/build-amd64 Pipeline is pending
ci/woodpecker/pr/build-arm64 Pipeline is pending
ci/woodpecker/pr/docs Pipeline is pending
ci/woodpecker/pr/lint Pipeline is pending
ci/woodpecker/pr/test Pipeline is pending
2024-04-16 20:37:00 +02:00
timorl b144218dce
Merge branch 'develop' into elseinspe
Some checks failed
ci/woodpecker/pr/lint Pipeline failed
ci/woodpecker/pr/test unknown status
ci/woodpecker/pr/build-arm64 unknown status
ci/woodpecker/pr/build-amd64 unknown status
ci/woodpecker/pr/docs unknown status
2024-04-14 20:31:33 +02:00
floatingghost e36c0f96fc Merge pull request 'Add docker override file to docs and gitignore' (#621) from norm/akkoma:docker-compose-override into develop
Some checks are pending
ci/woodpecker/push/build-amd64 Pipeline is pending
ci/woodpecker/push/build-arm64 Pipeline is pending
ci/woodpecker/push/docs Pipeline is pending
ci/woodpecker/push/lint Pipeline is pending
ci/woodpecker/push/test Pipeline is pending
Reviewed-on: #621
2024-04-12 18:50:25 +00:00
Floatingghost d910e8d7d1 Add test suite for elixir1.16
Some checks failed
ci/woodpecker/push/lint Pipeline was successful
ci/woodpecker/push/test Pipeline was successful
ci/woodpecker/push/build-amd64 Pipeline was successful
ci/woodpecker/push/build-arm64 Pipeline was successful
ci/woodpecker/push/docs Pipeline was successful
ci/woodpecker/pr/lint Pipeline was successful
ci/woodpecker/pr/test Pipeline was successful
ci/woodpecker/pr/build-arm64 unknown status
ci/woodpecker/pr/build-amd64 unknown status
ci/woodpecker/pr/docs unknown status
2024-04-12 19:13:33 +01:00
FloatingGhost 61621ebdbc Add tests for extra warnings about media subdomains 2024-04-02 10:54:53 +01:00
Oneric 0648d9ebaa Add mix tasks to detect spoofed posts and users
At least as far as we can
2024-03-26 16:05:20 -01:00
Oneric d441101200 Add mix task to detect uploaded spoof payloads 2024-03-26 16:05:20 -01:00
Oneric d6d838cbe8 StealEmoji: check remote size before downloading
To save on bandwith and avoid OOMs with large files.
Ofc, this relies on the remote server
 (a) sending a content-length header and
 (b) being honest about the size.

Common fedi servers seem to provide the header and (b) at least raises
the required privilege of an malicious actor to a server infrastructure
admin of an explicitly allowed host.

A more complete defense which still works when faced with
a malicious server requires changes in upstream Finch;
see https://github.com/sneako/finch/issues/224
2024-03-18 22:33:10 -01:00
Oneric fb54c47f0b Update example nginx config
To account for our subdomain recommendations
2024-03-18 22:33:10 -01:00
Oneric fc36b04016 Drop media proxy same-domain default for base_url
Even more than with user uploads, a same-domain proxy setup bears
significant security risks due to serving untrusted content under
the main domain space.

A risky setup like that should never be the default.
2024-03-18 22:33:10 -01:00
Oneric 0ec62acb9d Always insert Dedupe upload filter
This actually was already intended before to eradict all future
path-traversal-style exploits and to fix issues with some
characters like akkoma#610 in 0b2ec0ccee. However, Dedupe and
AnonymizeFilename got mixed up. The latter only anonymises the name
in Content-Disposition headers GET parameters (with link_name),
_not_ the upload path.

Even without Dedupe, the upload path is prefixed by an UUID,
so it _should_ already be hard to guess for attackers. But now
we actually can be sure no path shenanigangs occur, uploads
reliably work and save some disk space.

While this makes the final path predictable, this prediction is
not exploitable. Insertion of a back-reference to the upload
itself requires pulling off a successfull preimage attack against
SHA-256, which is deemed infeasible for the foreseeable futures.

Dedupe was already included in the default list in config.exs
since 28cfb2c37a, but this will get overridde by whatever the
config generated by the "pleroma.instance gen" task chose.

Upload+delete tests running in parallel using Dedupe might be flaky, but
this was already true before and needs its own commit to fix eventually.
2024-03-18 22:33:10 -01:00
Oneric fef773ca35 Drop media base_url default and recommend different domain
Same-domain setups enabled now at least two exploits,
so they ought to be discouraged and definitely not be the default.
2024-03-18 22:33:10 -01:00
floatingghost cdf73e0ac8 Merge pull request 'Better document database differences for Pleroma migrations' (#699) from Oneric/akkoma:doc_pleroma-migration-db into develop
Some checks are pending
ci/woodpecker/push/build-amd64 Pipeline is pending
ci/woodpecker/push/build-arm64 Pipeline is pending
ci/woodpecker/push/docs Pipeline is pending
ci/woodpecker/push/lint Pipeline is pending
ci/woodpecker/push/test Pipeline is pending
Reviewed-on: #699
2024-02-24 04:33:43 +00:00
floatingghost 967e6b8ade Merge pull request 'Docs: Add description for mrf_reject_newly_created_account_notes' (#695) from YokaiRick/akkoma:doc_mrf_reject_acc_notes into develop
Some checks are pending
ci/woodpecker/push/build-amd64 Pipeline is pending
ci/woodpecker/push/build-arm64 Pipeline is pending
ci/woodpecker/push/docs Pipeline is pending
ci/woodpecker/push/lint Pipeline is pending
ci/woodpecker/push/test Pipeline is pending
Reviewed-on: #695
2024-02-24 04:31:28 +00:00
Oneric bff2812a93 More prominently document db migrations in migrations from Pleroma
Some checks are pending
ci/woodpecker/pr/build-amd64 Pipeline is pending
ci/woodpecker/pr/build-arm64 Pipeline is pending
ci/woodpecker/pr/docs Pipeline is pending
ci/woodpecker/pr/lint Pipeline is pending
ci/woodpecker/pr/test Pipeline is pending
By now most instance will run a version past 2022-08 but the guide
only documented it for from source installs and Pleroma develop.
2024-02-23 23:54:14 +01:00
Oneric 7964272c98 Document how to avoid data loss on migration from Pleroma 2024-02-23 23:54:09 +01:00
rick c25cfe9b7a fixed spelling
Some checks failed
ci/woodpecker/pr/lint Pipeline was successful
ci/woodpecker/pr/test Pipeline failed
ci/woodpecker/pr/build-arm64 unknown status
ci/woodpecker/pr/build-amd64 unknown status
ci/woodpecker/pr/docs unknown status
2024-02-19 23:25:20 +01:00
Oneric 41dd37d796 doc/cheatsheet: add missing MRFs
Some checks are pending
ci/woodpecker/pr/build-amd64 Pipeline is pending
ci/woodpecker/pr/build-arm64 Pipeline is pending
ci/woodpecker/pr/docs Pipeline is pending
ci/woodpecker/pr/lint Pipeline is pending
ci/woodpecker/pr/test Pipeline is pending
Or mentions of MRFs in the main list
whose options were already documented.
2024-02-19 23:15:47 +01:00
Oneric 9830d54fa1 doc/cheatsheet: sort main MRF list alphabetically
It is too cumbersome to find a specific policy atm
or to check if all are docuemtned yet.
Trivial placeholder policies are excluded from this.
2024-02-19 23:15:30 +01:00
Oneric f254e4f530 doc/cheatsheet: add missing MRF config detail docs
And remove “on by default” text from individual entries.
They are now laready in the “on by default” section.
2024-02-19 23:14:44 +01:00
Oneric da4190c46e doc/cheatsheet: split out always active MRFs
It doesn’t make sense to add/remove them from the policies list
2024-02-19 23:14:24 +01:00
Oneric 7a2d68c3ab doc/cheatsheet: add link to ActivityExpiration config details 2024-02-19 23:14:07 +01:00
Oneric 8e7a89605d doc/cheatsheet: move MRF policies key to end of section
This makes it easier to spot the transparency options
2024-02-19 23:13:48 +01:00
Oneric 1640d19448 doc/cheatsheet: move :activitypub section ahead
Else it is too easy to mistake for another MRF policy.
2024-02-19 23:13:25 +01:00