Merge branch 'develop' into fedi-absturztau-be

This commit is contained in:
Puniko 2023-02-12 09:13:56 +01:00
commit 7bcf64438e
17 changed files with 199 additions and 47 deletions

View file

@ -95,7 +95,7 @@ pipeline:
# Canonical amd64
ubuntu22:
image: hexpm/elixir:1.14.2-erlang-25.1.2-ubuntu-jammy-20220428
image: hexpm/elixir:1.14.3-erlang-25.2.2-ubuntu-jammy-20221130
<<: *on-release
environment:
MIX_ENV: prod
@ -122,7 +122,7 @@ pipeline:
- /bin/sh /entrypoint.sh
debian-bullseye:
image: hexpm/elixir:1.14.2-erlang-25.1.2-debian-bullseye-20221004
image: hexpm/elixir:1.14.3-erlang-25.2.2-debian-bullseye-20230109
<<: *on-release
environment:
MIX_ENV: prod
@ -151,7 +151,7 @@ pipeline:
# Canonical amd64-musl
musl:
image: hexpm/elixir:1.14.2-erlang-25.1.2-alpine-3.16.2
image: hexpm/elixir:1.14.3-erlang-25.2.2-alpine-3.17.0
<<: *on-stable
environment:
MIX_ENV: prod

View file

@ -4,7 +4,7 @@ All notable changes to this project will be documented in this file.
The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/).
## Unreleased
## 2023.02
### Added
- Prometheus metrics exporting from `/api/v1/akkoma/metrics`
@ -37,6 +37,8 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/).
- Simplified HTTP signature processing
- Rich media will now hard-exit after 5 seconds, to prevent timeline hangs
- HTTP Content Security Policy is now far more strict to prevent any potential XSS/CSS leakages
- Follow requests are now paginated, matches mastodon API spec, so use the Link header to paginate.
- `internal.fetch` and `relay` actors are now represented with the actor type `Application`
### Fixed
- /api/v1/accounts/lookup will now respect restrict\_unauthenticated

View file

@ -354,7 +354,7 @@
config :pleroma, :activitypub,
unfollow_blocked: true,
outgoing_blocks: true,
outgoing_blocks: false,
blockers_visible: true,
follow_handshake_timeout: 500,
note_replies_output_limit: 5,

View file

@ -6,6 +6,31 @@ Akkoma performance is largely dependent on performance of the underlying databas
[PgTune](https://pgtune.leopard.in.ua) can be used to get recommended settings. Be sure to set "Number of Connections" to 20, otherwise it might produce settings hurtful to database performance. It is also recommended to not use "Network Storage" option.
If your server runs other services, you may want to take that into account. E.g. if you have 4G ram, but 1G of it is already used for other services, it may be better to tell PGTune you only have 3G. In the end, PGTune only provides recomended settings, you can always try to finetune further.
### Example configurations
Here are some configuration suggestions for PostgreSQL 10+.
#### 1GB RAM, 1 CPU
```
shared_buffers = 256MB
effective_cache_size = 768MB
maintenance_work_mem = 64MB
work_mem = 13107kB
```
#### 2GB RAM, 2 CPU
```
shared_buffers = 512MB
effective_cache_size = 1536MB
maintenance_work_mem = 128MB
work_mem = 26214kB
max_worker_processes = 2
max_parallel_workers_per_gather = 1
max_parallel_workers = 2
```
## Disable generic query plans
When PostgreSQL receives a query, it decides on a strategy for searching the requested data, this is called a query plan. The query planner has two modes: generic and custom. Generic makes a plan for all queries of the same shape, ignoring the parameters, which is then cached and reused. Custom, on the contrary, generates a unique query plan based on query parameters.
@ -23,26 +48,3 @@ config :pleroma, Pleroma.Repo,
```
A more detailed explaination of the issue can be found at <https://blog.soykaf.com/post/postgresql-elixir-troubles/>.
## Example configurations
Here are some configuration suggestions for PostgreSQL 10+.
### 1GB RAM, 1 CPU
```
shared_buffers = 256MB
effective_cache_size = 768MB
maintenance_work_mem = 64MB
work_mem = 13107kB
```
### 2GB RAM, 2 CPU
```
shared_buffers = 512MB
effective_cache_size = 1536MB
maintenance_work_mem = 128MB
work_mem = 26214kB
max_worker_processes = 2
max_parallel_workers_per_gather = 1
max_parallel_workers = 2
```

View file

@ -1 +1,48 @@
This section contains notes and guidelines for developers.
# Contributing to Akkoma
You wish to add a new feature in Akkoma, but don't know how to proceed? This guide takes you through the various steps of the development and contribution process.
If you're looking for stuff to implement or fix, check the [bug-tracker](https://akkoma.dev/AkkomaGang/akkoma/issues) or [forum](https://meta.akkoma.dev/c/requests/5).
Come say hi to us in the [#akkoma-dev chat room](./../#irc)!
## Akkoma Clients
Akkoma is the back-end. Clients have their own repositories and often separate projects. You can check what clients work with Akkoma [on the clients page](../clients/). If you maintain a working client not listed yet, feel free to make a PR [to these docs](./#docs)!
For resources on APIs and such, check the sidebar of this page.
## Docs
The docs are written in Markdown, including certain extensions, and can be found [in the docs folder of the Akkoma repo](https://akkoma.dev/AkkomaGang/akkoma/src/branch/develop/docs/). The content itself is stored in the `docs` subdirectory.
## Technology
Akkoma is written in [Elixir](https://elixir-lang.org/) and uses [Postgresql](https://www.postgresql.org/) for database. We use [Git](https://git-scm.com/) for collaboration and tracking code changes. Furthermore it can typically run on [Unix and Unix-like OS'es](https://en.wikipedia.org/wiki/Unix-like). For development, you should use an OS which [can run Akkoma](../installation/debian_based_en/).
It's good to have at least some basic understanding of at least Git and Elixir. If this is completely new for you, there's some [videos explaining Git](https://git-scm.com/doc) and Codeberg has a nice article explaining the typical [pull requests Git flow](https://docs.codeberg.org/collaborating/pull-requests-and-git-flow/). For Elixir, you can follow Elixir's own [Getting Started guide](https://elixir-lang.org/getting-started/introduction.html).
## Setting up a development environment
The best way to start is getting the software to run from source so you can start poking on it. Check out the [guides for setting up an Akkoma instance for development](setting_up_akkoma_dev/#setting-up-a-akkoma-development-environment).
## General overview
### Modules
Akkoma has several modules. There are modules for [uploading](https://akkoma.dev/AkkomaGang/akkoma/src/branch/develop/lib/pleroma/uploaders), [upload filters](https://akkoma.dev/AkkomaGang/akkoma/src/branch/develop/lib/pleroma/upload/filter), [translators](https://akkoma.dev/AkkomaGang/akkoma/src/branch/develop/lib/pleroma/akkoma/translators)... The most famous ones are without a doubt the [MRF policies](https://akkoma.dev/AkkomaGang/akkoma/src/branch/develop/lib/pleroma/web/activity_pub/mrf). Modules are often self contained and a good way to start with development because you don't have to think about much more than just the module itself. We even have an example on [writing your own MRF policy](/configuration/mrf/#writing-your-own-mrf-policy)!
Another easy entry point is the [mix tasks](https://akkoma.dev/AkkomaGang/akkoma/src/branch/develop/lib/mix/tasks/pleroma). They too are often self contained and don't need you to go through much of the code.
### Activity Streams/Activity Pub
Akkoma uses Activity Streams for both federation, as well as internal representation. It may be interesting to at least go over the specifications of [Activity Pub](https://www.w3.org/TR/activitypub/), [Activity Streams 2.0](https://www.w3.org/TR/activitystreams-core/), and [Activity Streams Vocabulary](https://www.w3.org/TR/activitystreams-vocabulary/). Note that these are not enough to have a full grasp of how everything works, but should at least give you the basics to understand how messages are passed between and inside Akkoma instances.
## Don't forget
When you make changes, you're expected to create [a Pull Request](https://akkoma.dev/AkkomaGang/akkoma/pulls). You don't have to wait until you finish to create the PR, but please do prefix the title of the PR with "WIP: " for as long as you're still working on it. The sooner you create your PR, the sooner people know what you are working on and the sooner you can get feedback and, if needed, help. You can then simply keep working on it until you are finished.
When doing changes, don't forget to add it to the relevant parts of the [CHANGELOG.md](https://akkoma.dev/AkkomaGang/akkoma/src/branch/develop/CHANGELOG.md).
You're expected to write [tests](https://elixirschool.com/en/lessons/testing/basics). While code is generally stored in the `lib` directory, tests are stored in the `test` directory using a similar folder structure. Feel free to peak at other tests to see how they are done. Obviously tests are expected to pass and properly test the functionality you added. If you feel really confident, you could even try to [write a test first and then write the code needed to make it pass](https://en.wikipedia.org/wiki/Test-driven_development)!
Code is formatted using the default formatter that comes with Elixir. You can format a file with e.g. `mix format /path/to/file.ex`. To check if everything is properly formatted, you can run `mix format --check-formatted`.

View file

@ -5,21 +5,36 @@ Akkoma requires some adjustments from the defaults for running the instance loca
## Installing
1. Install Akkoma as explained in [the docs](../installation/debian_based_en.md), with some exceptions:
* You can use your own fork of the repository and add akkoma as a remote `git remote add akkoma 'https://akkoma.dev/AkkomaGang/akkoma.git'`
* You can skip systemd and nginx and all that stuff
* No need to create a dedicated akkoma user, it's easier to just use your own user
* For the DB you can still choose a dedicated user, the mix tasks set it up for you so it's no extra work for you
* You can use your own fork of the repository and add akkoma as a remote `git remote add akkoma 'https://akkoma.dev/AkkomaGang/akkoma.git'`
* For domain you can use `localhost`
* For the DB you can still choose a dedicated user. The mix tasks sets it up, so it's no extra work for you
* instead of creating a `prod.secret.exs`, create `dev.secret.exs`
* No need to prefix with `MIX_ENV=prod`. We're using dev and that's the default MIX_ENV
* You can skip nginx and systemd
* For front-end, you'll probably want to install and use the develop branch instead of the stable branch. There's no guarantee that the stable branch of the FE will always work on the develop branch of the BE.
2. Change the dev.secret.exs
* Change the FE settings to use the installed branch (see also [Frontend Management](/configuration/frontend_management/))
* Change the scheme in `config :pleroma, Pleroma.Web.Endpoint` to http (see examples below)
* If you want to change other settings, you can do that too
3. You can now start the server `mix phx.server`. Once it's build and started, you can access the instance on `http://<host>:<port>` (e.g.http://localhost:4000 ) and should be able to do everything locally you normaly can.
3. You can now start the server with `mix phx.server`. Once it's build and started, you can access the instance on `http://<host>:<port>` (e.g.http://localhost:4000 ) and should be able to do everything locally you normally can.
Example on how to install pleroma-fe and admin-fe using it's develop branch
```sh
mix pleroma.frontend install pleroma-fe --ref develop
mix pleroma.frontend install admin-fe --ref develop
```
Example config to use the pleroma-fe and admin-fe installed from the develop branch
```elixir
config :pleroma, :frontends,
primary: %{"name" => "pleroma-fe", "ref" => "develop"},
admin: %{"name" => "admin-fe", "ref" => "develop"}
```
Example config to change the scheme to http. Change the port if you want to run on another port.
```elixir
config :pleroma, Pleroma.Web.Endpoint,
config :pleroma, Pleroma.Web.Endpoint,
url: [host: "localhost", scheme: "http", port: 4000],
```
@ -94,4 +109,4 @@ Update Akkoma as explained in [the docs](../administration/updating.md). Just ma
## Working on multiple branches
If you develop on a separate branch, it's possible you did migrations that aren't merged into another branch you're working on. If you have multiple things you're working on, it's probably best to set up multiple Akkoma instances each with their own database. If you finished with a branch and want to switch back to develop to start a new branch from there, you can drop the database and recreate the database (e.g. by using `config/setup_db.psql`). The commands to drop and recreate the database can be found in [the docs](../administration/backup.md).
If you develop on a separate branch, it's possible you did migrations that aren't merged into another branch you're working on. In that case, it's probably best to set up multiple Akkoma instances each with their own database. If you finished with a branch and want to switch back to develop to start a new branch from there, you can drop the database and recreate the database (e.g. by using `config/setup_db.psql`). The commands to drop and recreate the database can be found in [the docs](../administration/backup.md).

View file

@ -155,14 +155,13 @@ def following_count(%User{} = user) do
|> Repo.aggregate(:count, :id)
end
def get_follow_requests(%User{id: id}) do
def get_follow_requests_query(%User{id: id}) do
__MODULE__
|> join(:inner, [r], f in assoc(r, :follower))
|> join(:inner, [r], f in assoc(r, :follower), as: :follower)
|> where([r], r.state == ^:follow_pending)
|> where([r], r.following_id == ^id)
|> where([r, f], f.is_active == true)
|> select([r, f], f)
|> Repo.all()
|> where([r, follower: f], f.is_active == true)
|> select([r, follower: f], f)
end
def following?(%User{id: follower_id}, %User{id: followed_id}) do

View file

@ -273,7 +273,13 @@ def cached_muted_users_ap_ids(user) do
defdelegate following(user), to: FollowingRelationship
defdelegate following?(follower, followed), to: FollowingRelationship
defdelegate following_ap_ids(user), to: FollowingRelationship
defdelegate get_follow_requests(user), to: FollowingRelationship
defdelegate get_follow_requests_query(user), to: FollowingRelationship
def get_follow_requests(user) do
get_follow_requests_query(user)
|> Repo.all()
end
defdelegate search(query, opts \\ []), to: User.Search
@doc """
@ -1994,6 +2000,7 @@ defp create_service_actor(uri, nickname) do
%User{
invisible: true,
local: true,
actor_type: "Application",
ap_id: uri,
nickname: nickname,
follower_address: uri <> "/followers"

View file

@ -19,6 +19,7 @@ def index_operation do
summary: "Retrieve follow requests",
security: [%{"oAuth" => ["read:follows", "follow"]}],
operationId: "FollowRequestController.index",
parameters: pagination_params(),
responses: %{
200 =>
Operation.response("Array of Account", "application/json", %Schema{
@ -62,4 +63,22 @@ defp id_param do
required: true
)
end
defp pagination_params do
[
Operation.parameter(:max_id, :query, :string, "Return items older than this ID"),
Operation.parameter(
:since_id,
:query,
:string,
"Return the oldest items newer than this ID"
),
Operation.parameter(
:limit,
:query,
%Schema{type: :integer, default: 20},
"Maximum number of items to return. Will be ignored if it's more than 40"
)
]
end
end

View file

@ -5,9 +5,13 @@
defmodule Pleroma.Web.MastodonAPI.FollowRequestController do
use Pleroma.Web, :controller
import Pleroma.Web.ControllerHelper,
only: [add_link_headers: 2]
alias Pleroma.User
alias Pleroma.Web.CommonAPI
alias Pleroma.Web.Plugs.OAuthScopesPlug
alias Pleroma.Pagination
plug(Pleroma.Web.ApiSpec.CastAndValidate)
plug(:assign_follower when action != :index)
@ -24,10 +28,15 @@ defmodule Pleroma.Web.MastodonAPI.FollowRequestController do
defdelegate open_api_operation(action), to: Pleroma.Web.ApiSpec.FollowRequestOperation
@doc "GET /api/v1/follow_requests"
def index(%{assigns: %{user: followed}} = conn, _params) do
follow_requests = User.get_follow_requests(followed)
def index(%{assigns: %{user: followed}} = conn, params) do
follow_requests =
followed
|> User.get_follow_requests_query()
|> Pagination.fetch_paginated(params, :keyset, :follower)
render(conn, "index.json", for: followed, users: follow_requests, as: :user)
conn
|> add_link_headers(follow_requests)
|> render("index.json", for: followed, users: follow_requests, as: :user)
end
@doc "POST /api/v1/follow_requests/:id/authorize"

View file

@ -334,7 +334,8 @@ defp maybe_put_follow_requests_count(
%User{id: user_id}
) do
count =
User.get_follow_requests(user)
user
|> User.get_follow_requests()
|> length()
data

View file

@ -4,7 +4,7 @@ defmodule Pleroma.Mixfile do
def project do
[
app: :pleroma,
version: version("3.5.0"),
version: version("3.6.0"),
elixir: "~> 1.12",
elixirc_paths: elixirc_paths(Mix.env()),
compilers: [:phoenix] ++ Mix.compilers(),

View file

@ -0,0 +1,21 @@
defmodule Pleroma.Repo.Migrations.InstanceActorsToActorTypeApplication do
use Ecto.Migration
def up do
execute("""
update users
set actor_type = 'Application'
where local
and (ap_id like '%/relay' or ap_id like '%/internal/fetch')
""")
end
def down do
execute("""
update users
set actor_type = 'Person'
where local
and (ap_id like '%/relay' or ap_id like '%/internal/fetch')
""")
end
end

View file

@ -12,6 +12,7 @@ defmodule Pleroma.Web.ActivityPub.ObjectValidators.BlockValidationTest do
describe "blocks" do
setup do
clear_config([:activitypub, :outgoing_blocks], true)
user = insert(:user, local: false)
blocked = insert(:user)

View file

@ -19,6 +19,12 @@ test "gets an actor for the relay" do
assert user.ap_id == "#{Pleroma.Web.Endpoint.url()}/relay"
end
test "relay actor is an application" do
# See <https://www.w3.org/TR/activitystreams-vocabulary/#dfn-application>
user = Relay.get_actor()
assert user.actor_type == "Application"
end
test "relay actor is invisible" do
user = Relay.get_actor()
assert User.invisible?(user)

View file

@ -71,6 +71,7 @@ test "it posts a poll" do
test "it blocks and federates", %{blocker: blocker, blocked: blocked} do
clear_config([:instance, :federating], true)
clear_config([:activitypub, :outgoing_blocks], true)
with_mock Pleroma.Web.Federator,
publish: fn _ -> nil end do

View file

@ -10,6 +10,11 @@ defmodule Pleroma.Web.MastodonAPI.FollowRequestControllerTest do
import Pleroma.Factory
defp extract_next_link_header(header) do
[_, next_link] = Regex.run(~r{<(?<next_link>.*)>; rel="next"}, header)
next_link
end
describe "locked accounts" do
setup do
user = insert(:user, is_locked: true)
@ -31,6 +36,23 @@ test "/api/v1/follow_requests works", %{user: user, conn: conn} do
assert to_string(other_user.id) == relationship["id"]
end
test "/api/v1/follow_requests paginates", %{user: user, conn: conn} do
for _ <- 1..21 do
other_user = insert(:user)
{:ok, _, _, _activity} = CommonAPI.follow(other_user, user)
{:ok, _, _} = User.follow(other_user, user, :follow_pending)
end
conn = get(conn, "/api/v1/follow_requests")
assert length(json_response_and_validate_schema(conn, 200)) == 20
assert [link_header] = get_resp_header(conn, "link")
assert link_header =~ "rel=\"next\""
next_link = extract_next_link_header(link_header)
assert next_link =~ "/api/v1/follow_requests"
conn = get(conn, next_link)
assert length(json_response_and_validate_schema(conn, 200)) == 1
end
test "/api/v1/follow_requests/:id/authorize works", %{user: user, conn: conn} do
other_user = insert(:user)