Merge branch 'develop' into global-status-expiration

CHANGELOG.md

@@ -4,19 +4,28 @@ All notable changes to this project will be documented in this file.
 The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/).
 
 ## [unreleased]
+
+### Changed
+<details>
+  <summary>API Changes</summary>
+- **Breaking:** Emoji API: changed methods and renamed routes.
+</details>
+
 ### Removed
 - **Breaking:** removed `with_move` parameter from notifications timeline.
 
 ### Added
+- Instance: Extend `/api/v1/instance` with Pleroma-specific information.
 - NodeInfo: `pleroma:api/v1/notifications:include_types_filter` to the `features` list.
 - NodeInfo: `pleroma_emoji_reactions` to the `features` list.
 - Configuration: `:restrict_unauthenticated` setting, restrict access for unauthenticated users to timelines (public and federate), user profiles and statuses.
 - New HTTP adapter [gun](https://github.com/ninenines/gun). Gun adapter requires minimum OTP version of 22.2 otherwise Pleroma won’t start. For hackney OTP update is not required.
 - Mix task to create trusted OAuth App.
-- Notifications: Added `follow_request` notification type (configurable, see `[:notifications, :enable_follow_request_notifications]` setting).
+- Notifications: Added `follow_request` notification type.
 - Added `:reject_deletes` group to SimplePolicy
 <details>
   <summary>API Changes</summary>
+- Mastodon API: Extended `/api/v1/instance`.
 - Mastodon API: Support for `include_types` in `/api/v1/notifications`.
 - Mastodon API: Added `/api/v1/notifications/:id/dismiss` endpoint.
 - Mastodon API: Add support for filtering replies in public and home timelines
@@ -27,11 +36,16 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/).
 - Support pagination in conversations API
 - **Breaking**: SimplePolicy `:reject` and `:accept` allow deletions again
 - Fix follower/blocks import when nicknames starts with @
+- Filtering of push notifications on activities from blocked domains
 
 ### Changed
 - MFR policy to set global expiration for all local Create activities
 
 ## [unreleased-patch]
+### Security
+- Disallow re-registration of previously deleted users, which allowed viewing direct messages addressed to them
+- Mastodon API: Fix `POST /api/v1/follow_requests/:id/authorize` allowing to force a follow from a local user even if they didn't request to follow
+
 ### Fixed
 - Logger configuration through AdminFE
 - HTTP Basic Authentication permissions issue
@@ -132,7 +146,7 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/).
 - **Breaking:** Admin API: Return link alongside with token on password reset
 - **Breaking:** Admin API: `PUT /api/pleroma/admin/reports/:id` is now `PATCH /api/pleroma/admin/reports`, see admin_api.md for details
 - **Breaking:** `/api/pleroma/admin/users/invite_token` now uses `POST`, changed accepted params and returns full invite in json instead of only token string.
-- **Breaking** replying to reports is now "report notes", enpoint changed from `POST /api/pleroma/admin/reports/:id/respond` to `POST /api/pleroma/admin/reports/:id/notes`
+- **Breaking** replying to reports is now "report notes", endpoint changed from `POST /api/pleroma/admin/reports/:id/respond` to `POST /api/pleroma/admin/reports/:id/notes`
 - Mastodon API: stopped sanitizing display names, field names and subject fields since they are supposed to be treated as plaintext
 - Admin API: Return `total` when querying for reports
 - Mastodon API: Return `pleroma.direct_conversation_id` when creating a direct message (`POST /api/v1/statuses`)

config/config.exs

@@ -564,8 +564,6 @@
     inactivity_threshold: 7
   }
 
-config :pleroma, :notifications, enable_follow_request_notifications: false
-
 config :pleroma, :oauth2,
   token_expires_in: 600,
   issue_new_refresh_token: true,

config/description.exs

@@ -2262,6 +2262,7 @@
         children: [
           %{
             key: :active,
+            label: "Enabled",
             type: :boolean,
             description: "Globally enable or disable digest emails"
           },
@@ -2288,20 +2289,6 @@
       }
     ]
   },
-  %{
-    group: :pleroma,
-    key: :notifications,
-    type: :group,
-    description: "Notification settings",
-    children: [
-      %{
-        key: :enable_follow_request_notifications,
-        type: :boolean,
-        description:
-          "Enables notifications on new follow requests (causes issues with older PleromaFE versions)."
-      }
-    ]
-  },
   %{
     group: :pleroma,
     key: Pleroma.Emails.UserEmail,

docs/API/differences_in_mastoapi_responses.md

@@ -4,7 +4,7 @@ A Pleroma instance can be identified by "<Mastodon version> (compatible; Pleroma
 
 ## Flake IDs
 
-Pleroma uses 128-bit ids as opposed to Mastodon's 64 bits. However just like Mastodon's ids they are sortable strings
+Pleroma uses 128-bit ids as opposed to Mastodon's 64 bits. However just like Mastodon's ids they are lexically sortable strings
 
 ## Attachment cap
 
@@ -120,6 +120,18 @@ Accepts additional parameters:
 - `exclude_visibilities`: will exclude the notifications for activities with the given visibilities. The parameter accepts an array of visibility types (`public`, `unlisted`, `private`, `direct`). Usage example: `GET /api/v1/notifications?exclude_visibilities[]=direct&exclude_visibilities[]=private`.
 - `include_types`: will include the notifications for activities with the given types. The parameter accepts an array of types (`mention`, `follow`, `reblog`, `favourite`, `move`, `pleroma:emoji_reaction`). Usage example: `GET /api/v1/notifications?include_types[]=mention&include_types[]=reblog`.
 
+## DELETE `/api/v1/notifications/destroy_multiple`
+
+An endpoint to delete multiple statuses by IDs.
+
+Required parameters:
+
+- `ids`: array of activity ids
+
+Usage example: `DELETE /api/v1/notifications/destroy_multiple/?ids[]=1&ids[]=2`.
+
+Returns on success: 200 OK `{}`
+
 ## POST `/api/v1/statuses`
 
 Additional parameters can be added to the JSON body/Form data:
@@ -190,4 +202,19 @@ Has theses additional parameters (which are the same as in Pleroma-API):
 - `bio`: optional
 - `captcha_solution`: optional, contains provider-specific captcha solution,
 - `captcha_token`: optional, contains provider-specific captcha token
+- `captcha_answer_data`: optional, contains provider-specific captcha data
 - `token`: invite token required when the registrations aren't public.
+
+## Instance
+
+`GET /api/v1/instance` has additional fields
+
+- `max_toot_chars`: The maximum characters per post
+- `poll_limits`: The limits of polls
+- `upload_limit`: The maximum upload file size
+- `avatar_upload_limit`: The same for avatars
+- `background_upload_limit`: The same for backgrounds
+- `banner_upload_limit`: The same for banners
+- `pleroma.metadata.features`: A list of supported features
+- `pleroma.metadata.federation`: The federation restrictions of this instance
+- `vapid_public_key`: The public key needed for push messages

docs/API/pleroma_api.md

@@ -323,20 +323,54 @@ The status posting endpoint takes an additional parameter, `in_reply_to_conversa
 * Params: None
 * Response: JSON, returns a list of Mastodon Conversation entities that were marked as read (200 - healthy, 503 unhealthy).
 
-## `GET /api/pleroma/emoji/packs`
+## `GET /api/pleroma/emoji/packs/import`
-### Lists the custom emoji packs on the server
+### Imports packs from filesystem
 * Method `GET`
-* Authentication: not required
+* Authentication: required
 * Params: None
-* Response: JSON, "ok" and 200 status and the JSON hashmap of "pack name" to "pack contents"
+* Response: JSON, returns a list of imported packs.
 
-## `PUT /api/pleroma/emoji/packs/:name`
+## `GET /api/pleroma/emoji/packs/remote`
-### Creates an empty custom emoji pack
+### Make request to another instance for packs list
-* Method `PUT`
+* Method `GET`
+* Authentication: required
+* Params:
+  * `url`: url of the instance to get packs from
+* Response: JSON with the pack list, hashmap with pack name and pack contents
+
+## `POST /api/pleroma/emoji/packs/download`
+### Download pack from another instance
+* Method `POST`
+* Authentication: required
+* Params:
+  * `url`: url of the instance to download from
+  * `name`: pack to download from that instance
+  * `as`: (*optional*) name how to save pack
+* Response: JSON, "ok" with 200 status if the pack was downloaded, or 500 if there were
+  errors downloading the pack
+
+## `POST /api/pleroma/emoji/packs/:name`
+### Creates an empty pack
+* Method `POST`
 * Authentication: required
 * Params: None
 * Response: JSON, "ok" and 200 status or 409 if the pack with that name already exists
 
+## `PATCH /api/pleroma/emoji/packs/:name`
+### Updates (replaces) pack metadata
+* Method `PATCH`
+* Authentication: required
+* Params:
+  * `metadata`: metadata to replace the old one
+    * `license`: Pack license
+    * `homepage`: Pack home page url
+    * `description`: Pack description
+    * `fallback-src`: Fallback url to download pack from
+    * `fallback-src-sha256`: SHA256 encoded for fallback pack archive
+    * `share-files`: is pack allowed for sharing (boolean)
+* Response: JSON, updated "metadata" section of the pack and 200 status or 400 if there was a
+  problem with the new metadata (the error is specified in the "error" part of the response JSON)
+
 ## `DELETE /api/pleroma/emoji/packs/:name`
 ### Delete a custom emoji pack
 * Method `DELETE`
@@ -344,53 +378,51 @@ The status posting endpoint takes an additional parameter, `in_reply_to_conversa
 * Params: None
 * Response: JSON, "ok" and 200 status or 500 if there was an error deleting the pack
 
-## `POST /api/pleroma/emoji/packs/:name/update_file`
+## `POST /api/pleroma/emoji/packs/:name/files`
-### Update a file in a custom emoji pack
+### Add new file to the pack
 * Method `POST`
 * Authentication: required
 * Params:
-    * if the `action` is `add`, adds an emoji named `shortcode` to the pack `pack_name`,
+  * `file`: file needs to be uploaded with the multipart request or link to remote file.
-      that means that the emoji file needs to be uploaded with the request
+  * `shortcode`: (*optional*) shortcode for new emoji, must be uniq for all emoji. If not sended, shortcode will be taken from original filename.
-      (thus requiring it to be a multipart request) and be named `file`.
+  * `filename`: (*optional*) new emoji file name. If not specified will be taken from original filename.
-      There can also be an optional `filename` that will be the new emoji file name
+* Response: JSON, list of files for updated pack (hashmap -> shortcode => filename) with status 200, either error status with error message.
-      (if it's not there, the name will be taken from the uploaded file).
-    * if the `action` is `update`, changes emoji shortcode
-      (from `shortcode` to `new_shortcode` or moves the file (from the current filename to `new_filename`)
-    * if the `action` is `remove`, removes the emoji named `shortcode` and it's associated file
-* Response: JSON, updated "files" section of the pack and 200 status, 409 if the trying to use a shortcode
-  that is already taken, 400 if there was an error with the shortcode, filename or file (additional info
-  in the "error" part of the response JSON)
 
-## `POST /api/pleroma/emoji/packs/:name/update_metadata`
+## `PATCH /api/pleroma/emoji/packs/:name/files`
-### Updates (replaces) pack metadata
+### Update emoji file from pack
-* Method `POST`
+* Method `PATCH`
 * Authentication: required
 * Params:
-  * `new_data`: new metadata to replace the old one
+  * `shortcode`: emoji file shortcode
-* Response: JSON, updated "metadata" section of the pack and 200 status or 400 if there was a
+  * `new_shortcode`: new emoji file shortcode
-  problem with the new metadata (the error is specified in the "error" part of the response JSON)
+  * `new_filename`: new filename for emoji file
+  * `force`: (*optional*) with true value to overwrite existing emoji with new shortcode
+* Response: JSON, list with updated files for updated pack (hashmap -> shortcode => filename) with status 200, either error status with error message.
 
-## `POST /api/pleroma/emoji/packs/download_from`
+## `DELETE /api/pleroma/emoji/packs/:name/files`
-### Requests the instance to download the pack from another instance
+### Delete emoji file from pack
-* Method `POST`
+* Method `DELETE`
 * Authentication: required
 * Params:
-  * `instance_address`: the address of the instance to download from
+  * `shortcode`: emoji file shortcode
-  * `pack_name`: the pack to download from that instance
+* Response: JSON, list with updated files for updated pack (hashmap -> shortcode => filename) with status 200, either error status with error message.
-* Response: JSON, "ok" and 200 status if the pack was downloaded, or 500 if there were
-  errors downloading the pack
 
-## `POST /api/pleroma/emoji/packs/list_from`
+## `GET /api/pleroma/emoji/packs`
-### Requests the instance to list the packs from another instance
+### Lists local custom emoji packs
-* Method `POST`
+* Method `GET`
-* Authentication: required
+* Authentication: not required
-* Params:
+* Params: None
-  * `instance_address`: the address of the instance to download from
+* Response: JSON, "ok" and 200 status and the JSON hashmap of pack name to pack contents
-* Response: JSON with the pack list, same as if the request was made to that instance's
-  list endpoint directly + 200 status
 
-## `GET /api/pleroma/emoji/packs/:name/download_shared`
+## `GET /api/pleroma/emoji/packs/:name`
-### Requests a local pack from the instance
+### Get pack.json for the pack
+* Method `GET`
+* Authentication: not required
+* Params: None
+* Response: JSON, pack json with `files` and `pack` keys with 200 status or 404 if the pack does not exist
+
+## `GET /api/pleroma/emoji/packs/:name/archive`
+### Requests a local pack archive from the instance
 * Method `GET`
 * Authentication: not required
 * Params: None

docs/clients.md

@@ -49,11 +49,11 @@ Feel free to contact us to be added to this list!
 - Platforms: Android
 - Features: Streaming Ready
 
-### Roma
+### Fedi
-- Homepage: <https://www.pleroma.com/#mobileApps>
+- Homepage: <https://www.fediapp.com/>
-- Source Code: [iOS](https://github.com/roma-apps/roma-ios), [Android](https://github.com/roma-apps/roma-android)
+- Source Code: Proprietary, but free
 - Platforms: iOS, Android
-- Features: No Streaming
+- Features: Pleroma-specific features like Reactions
 
 ### Tusky
 - Homepage: <https://tuskyapp.github.io/>

docs/dev.md

@@ -0,0 +1,23 @@
+This document contains notes and guidelines for Pleroma developers.
+
+# Authentication & Authorization
+
+## OAuth token-based authentication & authorization
+
+* Pleroma supports hierarchical OAuth scopes, just like Mastodon but with added granularity of admin scopes. For a reference, see [Mastodon OAuth scopes](https://docs.joinmastodon.org/api/oauth-scopes/).
+
+* It is important to either define OAuth scope restrictions or explicitly mark OAuth scope check as skipped, for every controller action. To define scopes, call `plug(Pleroma.Plugs.OAuthScopesPlug, %{scopes: [...]})`. To explicitly set OAuth scopes check skipped, call `plug(:skip_plug, Pleroma.Plugs.OAuthScopesPlug <when ...>)`.
+
+* In controllers, `use Pleroma.Web, :controller` will result in `action/2` (see `Pleroma.Web.controller/0` for definition) be called prior to actual controller action, and it'll perform security / privacy checks before passing control to actual controller action.
+
+  For routes with `:authenticated_api` pipeline, authentication & authorization are expected, thus `OAuthScopesPlug` will be run unless explicitly skipped (also `EnsureAuthenticatedPlug` will be executed immediately before action even if there was an early run to give an early error, since `OAuthScopesPlug` supports `:proceed_unauthenticated` option, and other plugs may support similar options as well).
+
+  For `:api` pipeline routes, it'll be verified whether `OAuthScopesPlug` was called or explicitly skipped, and if it was not then auth information will be dropped for request. Then `EnsurePublicOrAuthenticatedPlug` will be called to ensure that either the instance is not private or user is authenticated (unless explicitly skipped). Such automated checks help to prevent human errors and result in higher security / privacy for users.
+
+## [HTTP Basic Authentication](https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Authorization)
+
+* With HTTP Basic Auth, OAuth scopes check is _not_ performed for any action (since password is provided during the auth, requester is able to obtain a token with full permissions anyways). `Pleroma.Plugs.AuthenticationPlug` and `Pleroma.Plugs.LegacyAuthenticationPlug` both call `Pleroma.Plugs.OAuthScopesPlug.skip_plug(conn)` when password is provided.
+
+## Auth-related configuration, OAuth consumer mode etc.
+
+See `Authentication` section of [`docs/configuration/cheatsheet.md`](docs/configuration/cheatsheet.md#authentication).

lib/pleroma/application.ex

@@ -73,7 +73,6 @@ def start(_type, _args) do
         Pleroma.Repo,
         Config.TransferTask,
         Pleroma.Emoji,
-        Pleroma.Captcha,
         Pleroma.Plugs.RateLimiter.Supervisor
       ] ++
         cachex_children() ++

lib/pleroma/captcha/captcha.ex

@@ -3,53 +3,22 @@
 # SPDX-License-Identifier: AGPL-3.0-only
 
 defmodule Pleroma.Captcha do
-  import Pleroma.Web.Gettext
-
   alias Calendar.DateTime
   alias Plug.Crypto.KeyGenerator
   alias Plug.Crypto.MessageEncryptor
 
-  use GenServer
-
-  @doc false
-  def start_link(_) do
-    GenServer.start_link(__MODULE__, [], name: __MODULE__)
-  end
-
-  @doc false
-  def init(_) do
-    {:ok, nil}
-  end
-
   @doc """
   Ask the configured captcha service for a new captcha
   """
   def new do
-    GenServer.call(__MODULE__, :new)
+    if not enabled?() do
-  end
+      %{type: :none}
-
-  @doc """
-  Ask the configured captcha service to validate the captcha
-  """
-  def validate(token, captcha, answer_data) do
-    GenServer.call(__MODULE__, {:validate, token, captcha, answer_data})
-  end
-
-  @doc false
-  def handle_call(:new, _from, state) do
-    enabled = Pleroma.Config.get([__MODULE__, :enabled])
-
-    if !enabled do
-      {:reply, %{type: :none}, state}
     else
       new_captcha = method().new()
 
-      secret_key_base = Pleroma.Config.get!([Pleroma.Web.Endpoint, :secret_key_base])
-
       # This make salt a little different for two keys
-      token = new_captcha[:token]
+      {secret, sign_secret} = secret_pair(new_captcha[:token])
-      secret = KeyGenerator.generate(secret_key_base, token <> "_encrypt")
+
-      sign_secret = KeyGenerator.generate(secret_key_base, token <> "_sign")
       # Basically copy what Phoenix.Token does here, add the time to
       # the actual data and make it a binary to then encrypt it
       encrypted_captcha_answer =
@@ -60,55 +29,73 @@ def handle_call(:new, _from, state) do
         |> :erlang.term_to_binary()
         |> MessageEncryptor.encrypt(secret, sign_secret)
 
-      {
-        :reply,
       # Replace the answer with the encrypted answer
-        %{new_captcha | answer_data: encrypted_captcha_answer},
+      %{new_captcha | answer_data: encrypted_captcha_answer}
-        state
-      }
     end
   end
 
-  @doc false
+  @doc """
-  def handle_call({:validate, token, captcha, answer_data}, _from, state) do
+  Ask the configured captcha service to validate the captcha
+  """
+  def validate(token, captcha, answer_data) do
+    with {:ok, %{at: at, answer_data: answer_md5}} <- validate_answer_data(token, answer_data),
+         :ok <- validate_expiration(at),
+         :ok <- validate_usage(token),
+         :ok <- method().validate(token, captcha, answer_md5),
+         {:ok, _} <- mark_captcha_as_used(token) do
+      :ok
+    end
+  end
+
+  def enabled?, do: Pleroma.Config.get([__MODULE__, :enabled], false)
+
+  defp seconds_valid, do: Pleroma.Config.get!([__MODULE__, :seconds_valid])
+
+  defp secret_pair(token) do
     secret_key_base = Pleroma.Config.get!([Pleroma.Web.Endpoint, :secret_key_base])
     secret = KeyGenerator.generate(secret_key_base, token <> "_encrypt")
     sign_secret = KeyGenerator.generate(secret_key_base, token <> "_sign")
 
-    # If the time found is less than (current_time-seconds_valid) then the time has already passed
+    {secret, sign_secret}
-    # Later we check that the time found is more than the presumed invalidatation time, that means
+  end
-    # that the data is still valid and the captcha can be checked
+
-    seconds_valid = Pleroma.Config.get!([Pleroma.Captcha, :seconds_valid])
+  defp validate_answer_data(token, answer_data) do
-    valid_if_after = DateTime.subtract!(DateTime.now_utc(), seconds_valid)
+    {secret, sign_secret} = secret_pair(token)
 
-    result =
     with false <- is_nil(answer_data),
          {:ok, data} <- MessageEncryptor.decrypt(answer_data, secret, sign_secret),
          %{at: at, answer_data: answer_md5} <- :erlang.binary_to_term(data) do
-        try do
+      {:ok, %{at: at, answer_data: answer_md5}}
-          if DateTime.before?(at, valid_if_after),
-            do: throw({:error, dgettext("errors", "CAPTCHA expired")})
-
-          if not is_nil(Cachex.get!(:used_captcha_cache, token)),
-            do: throw({:error, dgettext("errors", "CAPTCHA already used")})
-
-          res = method().validate(token, captcha, answer_md5)
-          # Throw if an error occurs
-          if res != :ok, do: throw(res)
-
-          # Mark this captcha as used
-          {:ok, _} =
-            Cachex.put(:used_captcha_cache, token, true, ttl: :timer.seconds(seconds_valid))
-
-          :ok
-        catch
-          :throw, e -> e
-        end
     else
-        _ -> {:error, dgettext("errors", "Invalid answer data")}
+      _ -> {:error, :invalid_answer_data}
+    end
   end
 
-    {:reply, result, state}
+  defp validate_expiration(created_at) do
+    # If the time found is less than (current_time-seconds_valid) then the time has already passed
+    # Later we check that the time found is more than the presumed invalidatation time, that means
+    # that the data is still valid and the captcha can be checked
+
+    valid_if_after = DateTime.subtract!(DateTime.now_utc(), seconds_valid())
+
+    if DateTime.before?(created_at, valid_if_after) do
+      {:error, :expired}
+    else
+      :ok
+    end
+  end
+
+  defp validate_usage(token) do
+    if is_nil(Cachex.get!(:used_captcha_cache, token)) do
+      :ok
+    else
+      {:error, :already_used}
+    end
+  end
+
+  defp mark_captcha_as_used(token) do
+    ttl = seconds_valid() |> :timer.seconds()
+    Cachex.put(:used_captcha_cache, token, true, ttl: ttl)
   end
 
   defp method, do: Pleroma.Config.get!([__MODULE__, :method])

lib/pleroma/captcha/kocaptcha.ex

@@ -3,7 +3,6 @@
 # SPDX-License-Identifier: AGPL-3.0-only
 
 defmodule Pleroma.Captcha.Kocaptcha do
-  import Pleroma.Web.Gettext
   alias Pleroma.Captcha.Service
   @behaviour Service
 
@@ -13,7 +12,7 @@ def new do
 
     case Tesla.get(endpoint <> "/new") do
       {:error, _} ->
-        %{error: dgettext("errors", "Kocaptcha service unavailable")}
+        %{error: :kocaptcha_service_unavailable}
 
       {:ok, res} ->
         json_resp = Jason.decode!(res.body)
@@ -33,6 +32,6 @@ def validate(_token, captcha, answer_data) do
     if not is_nil(captcha) and
          :crypto.hash(:md5, captcha) |> Base.encode16() == String.upcase(answer_data),
        do: :ok,
-       else: {:error, dgettext("errors", "Invalid CAPTCHA")}
+       else: {:error, :invalid}
   end
 end

lib/pleroma/captcha/native.ex

@@ -3,7 +3,6 @@
 # SPDX-License-Identifier: AGPL-3.0-only
 
 defmodule Pleroma.Captcha.Native do
-  import Pleroma.Web.Gettext
   alias Pleroma.Captcha.Service
   @behaviour Service
 
@@ -11,7 +10,7 @@ defmodule Pleroma.Captcha.Native do
   def new do
     case Captcha.get() do
       :error ->
-        %{error: dgettext("errors", "Captcha error")}
+        %{error: :captcha_error}
 
       {:ok, answer_data, img_binary} ->
         %{
@@ -25,7 +24,7 @@ def new do
 
   @impl Service
   def validate(_token, captcha, captcha) when not is_nil(captcha), do: :ok
-  def validate(_token, _captcha, _answer), do: {:error, dgettext("errors", "Invalid CAPTCHA")}
+  def validate(_token, _captcha, _answer), do: {:error, :invalid}
 
   defp token do
     10

lib/pleroma/constants.ex

@@ -20,4 +20,9 @@ defmodule Pleroma.Constants do
       "deleted_activity_id"
     ]
   )
+
+  const(static_only_files,
+    do:
+      ~w(index.html robots.txt static static-fe finmoji emoji packs sounds images instance sw.js sw-pleroma.js favicon.png schemas doc)
+  )
 end

lib/pleroma/emoji/pack.ex

@@ -0,0 +1,507 @@
+defmodule Pleroma.Emoji.Pack do
+  @derive {Jason.Encoder, only: [:files, :pack]}
+  defstruct files: %{},
+            pack_file: nil,
+            path: nil,
+            pack: %{},
+            name: nil
+
+  @type t() :: %__MODULE__{
+          files: %{String.t() => Path.t()},
+          pack_file: Path.t(),
+          path: Path.t(),
+          pack: map(),
+          name: String.t()
+        }
+
+  alias Pleroma.Emoji
+
+  @spec emoji_path() :: Path.t()
+  def emoji_path do
+    static = Pleroma.Config.get!([:instance, :static_dir])
+    Path.join(static, "emoji")
+  end
+
+  @spec create(String.t()) :: :ok | {:error, File.posix()} | {:error, :empty_values}
+  def create(name) when byte_size(name) > 0 do
+    dir = Path.join(emoji_path(), name)
+
+    with :ok <- File.mkdir(dir) do
+      %__MODULE__{
+        pack_file: Path.join(dir, "pack.json")
+      }
+      |> save_pack()
+    end
+  end
+
+  def create(_), do: {:error, :empty_values}
+
+  @spec show(String.t()) :: {:ok, t()} | {:loaded, nil} | {:error, :empty_values}
+  def show(name) when byte_size(name) > 0 do
+    with {_, %__MODULE__{} = pack} <- {:loaded, load_pack(name)},
+         {_, pack} <- validate_pack(pack) do
+      {:ok, pack}
+    end
+  end
+
+  def show(_), do: {:error, :empty_values}
+
+  @spec delete(String.t()) ::
+          {:ok, [binary()]} | {:error, File.posix(), binary()} | {:error, :empty_values}
+  def delete(name) when byte_size(name) > 0 do
+    emoji_path()
+    |> Path.join(name)
+    |> File.rm_rf()
+  end
+
+  def delete(_), do: {:error, :empty_values}
+
+  @spec add_file(String.t(), String.t(), Path.t(), Plug.Upload.t() | String.t()) ::
+          {:ok, t()} | {:error, File.posix()} | {:error, :empty_values}
+  def add_file(name, shortcode, filename, file)
+      when byte_size(name) > 0 and byte_size(shortcode) > 0 and byte_size(filename) > 0 do
+    with {_, nil} <- {:exists, Emoji.get(shortcode)},
+         {_, %__MODULE__{} = pack} <- {:loaded, load_pack(name)} do
+      file_path = Path.join(pack.path, filename)
+
+      create_subdirs(file_path)
+
+      case file do
+        %Plug.Upload{path: upload_path} ->
+          # Copy the uploaded file from the temporary directory
+          File.copy!(upload_path, file_path)
+
+        url when is_binary(url) ->
+          # Download and write the file
+          file_contents = Tesla.get!(url).body
+          File.write!(file_path, file_contents)
+      end
+
+      files = Map.put(pack.files, shortcode, filename)
+
+      updated_pack = %{pack | files: files}
+
+      case save_pack(updated_pack) do
+        :ok ->
+          Emoji.reload()
+          {:ok, updated_pack}
+
+        e ->
+          e
+      end
+    end
+  end
+
+  def add_file(_, _, _, _), do: {:error, :empty_values}
+
+  defp create_subdirs(file_path) do
+    if String.contains?(file_path, "/") do
+      file_path
+      |> Path.dirname()
+      |> File.mkdir_p!()
+    end
+  end
+
+  @spec delete_file(String.t(), String.t()) ::
+          {:ok, t()} | {:error, File.posix()} | {:error, :empty_values}
+  def delete_file(name, shortcode) when byte_size(name) > 0 and byte_size(shortcode) > 0 do
+    with {_, %__MODULE__{} = pack} <- {:loaded, load_pack(name)},
+         {_, {filename, files}} when not is_nil(filename) <-
+           {:exists, Map.pop(pack.files, shortcode)},
+         emoji <- Path.join(pack.path, filename),
+         {_, true} <- {:exists, File.exists?(emoji)} do
+      emoji_dir = Path.dirname(emoji)
+
+      File.rm!(emoji)
+
+      if String.contains?(filename, "/") and File.ls!(emoji_dir) == [] do
+        File.rmdir!(emoji_dir)
+      end
+
+      updated_pack = %{pack | files: files}
+
+      case save_pack(updated_pack) do
+        :ok ->
+          Emoji.reload()
+          {:ok, updated_pack}
+
+        e ->
+          e
+      end
+    end
+  end
+
+  def delete_file(_, _), do: {:error, :empty_values}
+
+  @spec update_file(String.t(), String.t(), String.t(), String.t(), boolean()) ::
+          {:ok, t()} | {:error, File.posix()} | {:error, :empty_values}
+  def update_file(name, shortcode, new_shortcode, new_filename, force)
+      when byte_size(name) > 0 and byte_size(shortcode) > 0 and byte_size(new_shortcode) > 0 and
+             byte_size(new_filename) > 0 do
+    with {_, %__MODULE__{} = pack} <- {:loaded, load_pack(name)},
+         {_, {filename, files}} when not is_nil(filename) <-
+           {:exists, Map.pop(pack.files, shortcode)},
+         {_, true} <- {:not_used, force or is_nil(Emoji.get(new_shortcode))} do
+      old_path = Path.join(pack.path, filename)
+      old_dir = Path.dirname(old_path)
+      new_path = Path.join(pack.path, new_filename)
+
+      create_subdirs(new_path)
+
+      :ok = File.rename(old_path, new_path)
+
+      if String.contains?(filename, "/") and File.ls!(old_dir) == [] do
+        File.rmdir!(old_dir)
+      end
+
+      files = Map.put(files, new_shortcode, new_filename)
+
+      updated_pack = %{pack | files: files}
+
+      case save_pack(updated_pack) do
+        :ok ->
+          Emoji.reload()
+          {:ok, updated_pack}
+
+        e ->
+          e
+      end
+    end
+  end
+
+  def update_file(_, _, _, _, _), do: {:error, :empty_values}
+
+  @spec import_from_filesystem() :: {:ok, [String.t()]} | {:error, atom()}
+  def import_from_filesystem do
+    emoji_path = emoji_path()
+
+    with {:ok, %{access: :read_write}} <- File.stat(emoji_path),
+         {:ok, results} <- File.ls(emoji_path) do
+      names =
+        results
+        |> Enum.map(&Path.join(emoji_path, &1))
+        |> Enum.reject(fn path ->
+          File.dir?(path) and File.exists?(Path.join(path, "pack.json"))
+        end)
+        |> Enum.map(&write_pack_contents/1)
+        |> Enum.filter(& &1)
+
+      {:ok, names}
+    else
+      {:ok, %{access: _}} -> {:error, :no_read_write}
+      e -> e
+    end
+  end
+
+  defp write_pack_contents(path) do
+    pack = %__MODULE__{
+      files: files_from_path(path),
+      path: path,
+      pack_file: Path.join(path, "pack.json")
+    }
+
+    case save_pack(pack) do
+      :ok -> Path.basename(path)
+      _ -> nil
+    end
+  end
+
+  defp files_from_path(path) do
+    txt_path = Path.join(path, "emoji.txt")
+
+    if File.exists?(txt_path) do
+      # There's an emoji.txt file, it's likely from a pack installed by the pack manager.
+      # Make a pack.json file from the contents of that emoji.txt file
+
+      # FIXME: Copy-pasted from Pleroma.Emoji/load_from_file_stream/2
+
+      # Create a map of shortcodes to filenames from emoji.txt
+      File.read!(txt_path)
+      |> String.split("\n")
+      |> Enum.map(&String.trim/1)
+      |> Enum.map(fn line ->
+        case String.split(line, ~r/,\s*/) do
+          # This matches both strings with and without tags
+          # and we don't care about tags here
+          [name, file | _] ->
+            file_dir_name = Path.dirname(file)
+
+            file =
+              if String.ends_with?(path, file_dir_name) do
+                Path.basename(file)
+              else
+                file
+              end
+
+            {name, file}
+
+          _ ->
+            nil
+        end
+      end)
+      |> Enum.filter(& &1)
+      |> Enum.into(%{})
+    else
+      # If there's no emoji.txt, assume all files
+      # that are of certain extensions from the config are emojis and import them all
+      pack_extensions = Pleroma.Config.get!([:emoji, :pack_extensions])
+      Emoji.Loader.make_shortcode_to_file_map(path, pack_extensions)
+    end
+  end
+
+  @spec list_remote(String.t()) :: {:ok, map()}
+  def list_remote(url) do
+    uri =
+      url
+      |> String.trim()
+      |> URI.parse()
+
+    with {_, true} <- {:shareable, shareable_packs_available?(uri)} do
+      packs =
+        uri
+        |> URI.merge("/api/pleroma/emoji/packs")
+        |> to_string()
+        |> Tesla.get!()
+        |> Map.get(:body)
+        |> Jason.decode!()
+
+      {:ok, packs}
+    end
+  end
+
+  @spec list_local() :: {:ok, map()}
+  def list_local do
+    emoji_path = emoji_path()
+
+    # Create the directory first if it does not exist. This is probably the first request made
+    # with the API so it should be sufficient
+    with {:create_dir, :ok} <- {:create_dir, File.mkdir_p(emoji_path)},
+         {:ls, {:ok, results}} <- {:ls, File.ls(emoji_path)} do
+      packs =
+        results
+        |> Enum.map(&load_pack/1)
+        |> Enum.filter(& &1)
+        |> Enum.map(&validate_pack/1)
+        |> Map.new()
+
+      {:ok, packs}
+    end
+  end
+
+  defp validate_pack(pack) do
+    if downloadable?(pack) do
+      archive = fetch_archive(pack)
+      archive_sha = :crypto.hash(:sha256, archive) |> Base.encode16()
+
+      info =
+        pack.pack
+        |> Map.put("can-download", true)
+        |> Map.put("download-sha256", archive_sha)
+
+      {pack.name, Map.put(pack, :pack, info)}
+    else
+      info = Map.put(pack.pack, "can-download", false)
+      {pack.name, Map.put(pack, :pack, info)}
+    end
+  end
+
+  defp downloadable?(pack) do
+    # If the pack is set as shared, check if it can be downloaded
+    # That means that when asked, the pack can be packed and sent to the remote
+    # Otherwise, they'd have to download it from external-src
+    pack.pack["share-files"] &&
+      Enum.all?(pack.files, fn {_, file} ->
+        File.exists?(Path.join(pack.path, file))
+      end)
+  end
+
+  @spec get_archive(String.t()) :: {:ok, binary()}
+  def get_archive(name) do
+    with {_, %__MODULE__{} = pack} <- {:exists?, load_pack(name)},
+         {_, true} <- {:can_download?, downloadable?(pack)} do
+      {:ok, fetch_archive(pack)}
+    end
+  end
+
+  defp fetch_archive(pack) do
+    hash = :crypto.hash(:md5, File.read!(pack.pack_file))
+
+    case Cachex.get!(:emoji_packs_cache, pack.name) do
+      %{hash: ^hash, pack_data: archive} ->
+        archive
+
+      _ ->
+        create_archive_and_cache(pack, hash)
+    end
+  end
+
+  defp create_archive_and_cache(pack, hash) do
+    files = ['pack.json' | Enum.map(pack.files, fn {_, file} -> to_charlist(file) end)]
+
+    {:ok, {_, result}} =
+      :zip.zip('#{pack.name}.zip', files, [:memory, cwd: to_charlist(pack.path)])
+
+    ttl_per_file = Pleroma.Config.get!([:emoji, :shared_pack_cache_seconds_per_file])
+    overall_ttl = :timer.seconds(ttl_per_file * Enum.count(files))
+
+    Cachex.put!(
+      :emoji_packs_cache,
+      pack.name,
+      # if pack.json MD5 changes, the cache is not valid anymore
+      %{hash: hash, pack_data: result},
+      # Add a minute to cache time for every file in the pack
+      ttl: overall_ttl
+    )
+
+    result
+  end
+
+  @spec download(String.t(), String.t(), String.t()) :: :ok
+  def download(name, url, as) do
+    uri =
+      url
+      |> String.trim()
+      |> URI.parse()
+
+    with {_, true} <- {:shareable, shareable_packs_available?(uri)} do
+      remote_pack =
+        uri
+        |> URI.merge("/api/pleroma/emoji/packs/#{name}")
+        |> to_string()
+        |> Tesla.get!()
+        |> Map.get(:body)
+        |> Jason.decode!()
+
+      result =
+        case remote_pack["pack"] do
+          %{"share-files" => true, "can-download" => true, "download-sha256" => sha} ->
+            {:ok,
+             %{
+               sha: sha,
+               url: URI.merge(uri, "/api/pleroma/emoji/packs/#{name}/archive") |> to_string()
+             }}
+
+          %{"fallback-src" => src, "fallback-src-sha256" => sha} when is_binary(src) ->
+            {:ok,
+             %{
+               sha: sha,
+               url: src,
+               fallback: true
+             }}
+
+          _ ->
+            {:error,
+             "The pack was not set as shared and there is no fallback src to download from"}
+        end
+
+      with {:ok, %{sha: sha, url: url} = pinfo} <- result,
+           %{body: archive} <- Tesla.get!(url),
+           {_, true} <- {:checksum, Base.decode16!(sha) == :crypto.hash(:sha256, archive)} do
+        local_name = as || name
+
+        path = Path.join(emoji_path(), local_name)
+
+        pack = %__MODULE__{
+          name: local_name,
+          path: path,
+          files: remote_pack["files"],
+          pack_file: Path.join(path, "pack.json")
+        }
+
+        File.mkdir_p!(pack.path)
+
+        files = Enum.map(remote_pack["files"], fn {_, path} -> to_charlist(path) end)
+        # Fallback cannot contain a pack.json file
+        files = if pinfo[:fallback], do: files, else: ['pack.json' | files]
+
+        {:ok, _} = :zip.unzip(archive, cwd: to_charlist(pack.path), file_list: files)
+
+        # Fallback can't contain a pack.json file, since that would cause the fallback-src-sha256
+        # in it to depend on itself
+        if pinfo[:fallback] do
+          save_pack(pack)
+        end
+
+        :ok
+      end
+    end
+  end
+
+  defp save_pack(pack), do: File.write(pack.pack_file, Jason.encode!(pack, pretty: true))
+
+  @spec save_metadata(map(), t()) :: {:ok, t()} | {:error, File.posix()}
+  def save_metadata(metadata, %__MODULE__{} = pack) do
+    pack = Map.put(pack, :pack, metadata)
+
+    with :ok <- save_pack(pack) do
+      {:ok, pack}
+    end
+  end
+
+  @spec update_metadata(String.t(), map()) :: {:ok, t()} | {:error, File.posix()}
+  def update_metadata(name, data) do
+    pack = load_pack(name)
+
+    fb_sha_changed? =
+      not is_nil(data["fallback-src"]) and data["fallback-src"] != pack.pack["fallback-src"]
+
+    with {_, true} <- {:update?, fb_sha_changed?},
+         {:ok, %{body: zip}} <- Tesla.get(data["fallback-src"]),
+         {:ok, f_list} <- :zip.unzip(zip, [:memory]),
+         {_, true} <- {:has_all_files?, has_all_files?(pack.files, f_list)} do
+      fallback_sha = :crypto.hash(:sha256, zip) |> Base.encode16()
+
+      data
+      |> Map.put("fallback-src-sha256", fallback_sha)
+      |> save_metadata(pack)
+    else
+      {:update?, _} -> save_metadata(data, pack)
+      e -> e
+    end
+  end
+
+  # Check if all files from the pack.json are in the archive
+  defp has_all_files?(files, f_list) do
+    Enum.all?(files, fn {_, from_manifest} ->
+      List.keyfind(f_list, to_charlist(from_manifest), 0)
+    end)
+  end
+
+  @spec load_pack(String.t()) :: t() | nil
+  def load_pack(name) do
+    pack_file = Path.join([emoji_path(), name, "pack.json"])
+
+    if File.exists?(pack_file) do
+      pack_file
+      |> File.read!()
+      |> from_json()
+      |> Map.put(:pack_file, pack_file)
+      |> Map.put(:path, Path.dirname(pack_file))
+      |> Map.put(:name, name)
+    end
+  end
+
+  defp from_json(json) do
+    map = Jason.decode!(json)
+
+    struct(__MODULE__, %{files: map["files"], pack: map["pack"]})
+  end
+
+  defp shareable_packs_available?(uri) do
+    uri
+    |> URI.merge("/.well-known/nodeinfo")
+    |> to_string()
+    |> Tesla.get!()
+    |> Map.get(:body)
+    |> Jason.decode!()
+    |> Map.get("links")
+    |> List.last()
+    |> Map.get("href")
+    # Get the actual nodeinfo address and fetch it
+    |> Tesla.get!()
+    |> Map.get(:body)
+    |> Jason.decode!()
+    |> get_in(["metadata", "features"])
+    |> Enum.member?("shareable_emoji_packs")
+  end
+end

lib/pleroma/following_relationship.ex

@@ -10,11 +10,12 @@ defmodule Pleroma.FollowingRelationship do
 
   alias Ecto.Changeset
   alias FlakeId.Ecto.CompatType
+  alias Pleroma.FollowingRelationship.State
   alias Pleroma.Repo
   alias Pleroma.User
 
   schema "following_relationships" do
-    field(:state, Pleroma.FollowingRelationship.State, default: :follow_pending)
+    field(:state, State, default: :follow_pending)
 
     belongs_to(:follower, User, type: CompatType)
     belongs_to(:following, User, type: CompatType)
@@ -22,6 +23,11 @@ defmodule Pleroma.FollowingRelationship do
     timestamps()
   end
 
+  @doc "Returns underlying integer code for state atom"
+  def state_int_code(state_atom), do: State.__enum_map__() |> Keyword.fetch!(state_atom)
+
+  def accept_state_code, do: state_int_code(:follow_accept)
+
   def changeset(%__MODULE__{} = following_relationship, attrs) do
     following_relationship
     |> cast(attrs, [:state])
@@ -82,6 +88,29 @@ def follower_count(%User{} = user) do
     |> Repo.aggregate(:count, :id)
   end
 
+  def followers_query(%User{} = user) do
+    __MODULE__
+    |> join(:inner, [r], u in User, on: r.follower_id == u.id)
+    |> where([r], r.following_id == ^user.id)
+    |> where([r], r.state == ^:follow_accept)
+  end
+
+  def followers_ap_ids(%User{} = user, from_ap_ids \\ nil) do
+    query =
+      user
+      |> followers_query()
+      |> select([r, u], u.ap_id)
+
+    query =
+      if from_ap_ids do
+        where(query, [r, u], u.ap_id in ^from_ap_ids)
+      else
+        query
+      end
+
+    Repo.all(query)
+  end
+
   def following_count(%User{id: nil}), do: 0
 
   def following_count(%User{} = user) do
@@ -105,12 +134,16 @@ def following?(%User{id: follower_id}, %User{id: followed_id}) do
     |> Repo.exists?()
   end
 
-  def following(%User{} = user) do
+  def following_query(%User{} = user) do
-    following =
     __MODULE__
     |> join(:inner, [r], u in User, on: r.following_id == u.id)
     |> where([r], r.follower_id == ^user.id)
     |> where([r], r.state == ^:follow_accept)
+  end
+
+  def following(%User{} = user) do
+    following =
+      following_query(user)
       |> select([r, u], u.follower_address)
       |> Repo.all()
 
@@ -171,6 +204,30 @@ def find(following_relationships, follower, following) do
     end)
   end
 
+  @doc """
+  For a query with joined activity,
+  keeps rows where activity's actor is followed by user -or- is NOT domain-blocked by user.
+  """
+  def keep_following_or_not_domain_blocked(query, user) do
+    where(
+      query,
+      [_, activity],
+      fragment(
+        # "(actor's domain NOT in domain_blocks) OR (actor IS in followed AP IDs)"
+        """
+        NOT (substring(? from '.*://([^/]*)') = ANY(?)) OR
+          ? = ANY(SELECT ap_id FROM users AS u INNER JOIN following_relationships AS fr
+            ON u.id = fr.following_id WHERE fr.follower_id = ? AND fr.state = ?)
+        """,
+        activity.actor,
+        ^user.domain_blocks,
+        activity.actor,
+        ^User.binary_id(user.id),
+        ^accept_state_code()
+      )
+    )
+  end
+
   defp validate_not_self_relationship(%Changeset{} = changeset) do
     changeset
     |> validate_follower_id_following_id_inequality()

lib/pleroma/notification.ex

@@ -6,6 +6,7 @@ defmodule Pleroma.Notification do
   use Ecto.Schema
 
   alias Pleroma.Activity
+  alias Pleroma.FollowingRelationship
   alias Pleroma.Notification
   alias Pleroma.Object
   alias Pleroma.Pagination
@@ -81,15 +82,13 @@ def for_user_query(user, opts \\ %{}) do
     |> exclude_visibility(opts)
   end
 
+  # Excludes blocked users and non-followed domain-blocked users
   defp exclude_blocked(query, user, opts) do
     blocked_ap_ids = opts[:blocked_users_ap_ids] || User.blocked_users_ap_ids(user)
 
     query
     |> where([n, a], a.actor not in ^blocked_ap_ids)
-    |> where(
+    |> FollowingRelationship.keep_following_or_not_domain_blocked(user)
-      [n, a],
-      fragment("substring(? from '.*://([^/]*)')", a.actor) not in ^user.domain_blocks
-    )
   end
 
   defp exclude_notification_muted(query, _, %{@include_muted_option => true}) do
@@ -262,6 +261,16 @@ def destroy_multiple(%{id: user_id} = _user, ids) do
     |> Repo.delete_all()
   end
 
+  def dismiss(%Pleroma.Activity{} = activity) do
+    Notification
+    |> where([n], n.activity_id == ^activity.id)
+    |> Repo.delete_all()
+    |> case do
+      {_, notifications} -> {:ok, notifications}
+      _ -> {:error, "Cannot dismiss notification"}
+    end
+  end
+
   def dismiss(%{id: user_id} = _user, id) do
     notification = Repo.get(Notification, id)
 
@@ -284,17 +293,8 @@ def create_notifications(%Activity{data: %{"to" => _, "type" => "Create"}} = act
     end
   end
 
-  def create_notifications(%Activity{data: %{"type" => "Follow"}} = activity) do
-    if Pleroma.Config.get([:notifications, :enable_follow_request_notifications]) ||
-         Activity.follow_accepted?(activity) do
-      do_create_notifications(activity)
-    else
-      {:ok, []}
-    end
-  end
-
   def create_notifications(%Activity{data: %{"type" => type}} = activity)
-      when type in ["Like", "Announce", "Move", "EmojiReact"] do
+      when type in ["Follow", "Like", "Announce", "Move", "EmojiReact"] do
     do_create_notifications(activity)
   end
 
@@ -330,10 +330,11 @@ def create_notification(%Activity{} = activity, %User{} = user, do_send \\ true)
 
   @doc """
   Returns a tuple with 2 elements:
-    {enabled notification receivers, currently disabled receivers (blocking / [thread] muting)}
+    {notification-enabled receivers, currently disabled receivers (blocking / [thread] muting)}
 
   NOTE: might be called for FAKE Activities, see ActivityPub.Utils.get_notified_from_object/1
   """
+  @spec get_notified_from_activity(Activity.t(), boolean()) :: {list(User.t()), list(User.t())}
   def get_notified_from_activity(activity, local_only \\ true)
 
   def get_notified_from_activity(%Activity{data: %{"type" => type}} = activity, local_only)
@@ -346,17 +347,14 @@ def get_notified_from_activity(%Activity{data: %{"type" => type}} = activity, lo
       |> Utils.maybe_notify_followers(activity)
       |> Enum.uniq()
 
-    # Since even subscribers and followers can mute / thread-mute, filtering all above AP IDs
+    potential_receivers = User.get_users_from_set(potential_receiver_ap_ids, local_only)
+
     notification_enabled_ap_ids =
       potential_receiver_ap_ids
+      |> exclude_domain_blocker_ap_ids(activity, potential_receivers)
       |> exclude_relationship_restricted_ap_ids(activity)
       |> exclude_thread_muter_ap_ids(activity)
 
-    potential_receivers =
-      potential_receiver_ap_ids
-      |> Enum.uniq()
-      |> User.get_users_from_set(local_only)
-
     notification_enabled_users =
       Enum.filter(potential_receivers, fn u -> u.ap_id in notification_enabled_ap_ids end)
 
@@ -365,6 +363,38 @@ def get_notified_from_activity(%Activity{data: %{"type" => type}} = activity, lo
 
   def get_notified_from_activity(_, _local_only), do: {[], []}
 
+  @doc "Filters out AP IDs domain-blocking and not following the activity's actor"
+  def exclude_domain_blocker_ap_ids(ap_ids, activity, preloaded_users \\ [])
+
+  def exclude_domain_blocker_ap_ids([], _activity, _preloaded_users), do: []
+
+  def exclude_domain_blocker_ap_ids(ap_ids, %Activity{} = activity, preloaded_users) do
+    activity_actor_domain = activity.actor && URI.parse(activity.actor).host
+
+    users =
+      ap_ids
+      |> Enum.map(fn ap_id ->
+        Enum.find(preloaded_users, &(&1.ap_id == ap_id)) ||
+          User.get_cached_by_ap_id(ap_id)
+      end)
+      |> Enum.filter(& &1)
+
+    domain_blocker_ap_ids = for u <- users, activity_actor_domain in u.domain_blocks, do: u.ap_id
+
+    domain_blocker_follower_ap_ids =
+      if Enum.any?(domain_blocker_ap_ids) do
+        activity
+        |> Activity.user_actor()
+        |> FollowingRelationship.followers_ap_ids(domain_blocker_ap_ids)
+      else
+        []
+      end
+
+    ap_ids
+    |> Kernel.--(domain_blocker_ap_ids)
+    |> Kernel.++(domain_blocker_follower_ap_ids)
+  end
+
   @doc "Filters out AP IDs of users basing on their relationships with activity actor user"
   def exclude_relationship_restricted_ap_ids([], _activity), do: []
 

lib/pleroma/plugs/auth_expected_plug.ex

@@ -1,17 +0,0 @@
-# Pleroma: A lightweight social networking server
-# Copyright © 2017-2020 Pleroma Authors <https://pleroma.social/>
-# SPDX-License-Identifier: AGPL-3.0-only
-
-defmodule Pleroma.Plugs.AuthExpectedPlug do
-  import Plug.Conn
-
-  def init(options), do: options
-
-  def call(conn, _) do
-    put_private(conn, :auth_expected, true)
-  end
-
-  def auth_expected?(conn) do
-    conn.private[:auth_expected]
-  end
-end

lib/pleroma/plugs/ensure_authenticated_plug.ex

@@ -5,17 +5,21 @@
 defmodule Pleroma.Plugs.EnsureAuthenticatedPlug do
   import Plug.Conn
   import Pleroma.Web.TranslationHelpers
+
   alias Pleroma.User
 
+  use Pleroma.Web, :plug
+
   def init(options) do
     options
   end
 
-  def call(%{assigns: %{user: %User{}}} = conn, _) do
+  @impl true
+  def perform(%{assigns: %{user: %User{}}} = conn, _) do
     conn
   end
 
-  def call(conn, options) do
+  def perform(conn, options) do
     perform =
       cond do
         options[:if_func] -> options[:if_func].()

lib/pleroma/plugs/ensure_public_or_authenticated_plug.ex

@@ -5,14 +5,18 @@
 defmodule Pleroma.Plugs.EnsurePublicOrAuthenticatedPlug do
   import Pleroma.Web.TranslationHelpers
   import Plug.Conn
+
   alias Pleroma.Config
   alias Pleroma.User
 
+  use Pleroma.Web, :plug
+
   def init(options) do
     options
   end
 
-  def call(conn, _) do
+  @impl true
+  def perform(conn, _) do
     public? = Config.get!([:instance, :public])
 
     case {public?, conn} do

lib/pleroma/plugs/expect_authenticated_check_plug.ex

@@ -0,0 +1,20 @@
+# Pleroma: A lightweight social networking server
+# Copyright © 2017-2020 Pleroma Authors <https://pleroma.social/>
+# SPDX-License-Identifier: AGPL-3.0-only
+
+defmodule Pleroma.Plugs.ExpectAuthenticatedCheckPlug do
+  @moduledoc """
+  Marks `Pleroma.Plugs.EnsureAuthenticatedPlug` as expected to be executed later in plug chain.
+
+  No-op plug which affects `Pleroma.Web` operation (is checked with `PlugHelper.plug_called?/2`).
+  """
+
+  use Pleroma.Web, :plug
+
+  def init(options), do: options
+
+  @impl true
+  def perform(conn, _) do
+    conn
+  end
+end

lib/pleroma/plugs/expect_public_or_authenticated_check_plug.ex

@@ -0,0 +1,21 @@
+# Pleroma: A lightweight social networking server
+# Copyright © 2017-2020 Pleroma Authors <https://pleroma.social/>
+# SPDX-License-Identifier: AGPL-3.0-only
+
+defmodule Pleroma.Plugs.ExpectPublicOrAuthenticatedCheckPlug do
+  @moduledoc """
+  Marks `Pleroma.Plugs.EnsurePublicOrAuthenticatedPlug` as expected to be executed later in plug
+  chain.
+
+  No-op plug which affects `Pleroma.Web` operation (is checked with `PlugHelper.plug_called?/2`).
+  """
+
+  use Pleroma.Web, :plug
+
+  def init(options), do: options
+
+  @impl true
+  def perform(conn, _) do
+    conn
+  end
+end

lib/pleroma/plugs/instance_static.ex

@@ -3,6 +3,8 @@
 # SPDX-License-Identifier: AGPL-3.0-only
 
 defmodule Pleroma.Plugs.InstanceStatic do
+  require Pleroma.Constants
+
   @moduledoc """
   This is a shim to call `Plug.Static` but with runtime `from` configuration.
 
@@ -21,9 +23,6 @@ def file_path(path) do
     end
   end
 
-  @only ~w(index.html robots.txt static emoji packs sounds images instance favicon.png sw.js
-  sw-pleroma.js)
-
   def init(opts) do
     opts
     |> Keyword.put(:from, "__unconfigured_instance_static_plug")
@@ -31,7 +30,7 @@ def init(opts) do
     |> Plug.Static.init()
   end
 
-  for only <- @only do
+  for only <- Pleroma.Constants.static_only_files() do
     at = Plug.Router.Utils.split("/")
 
     def call(%{request_path: "/" <> unquote(only) <> _} = conn, opts) do

lib/pleroma/plugs/mapped_signature_to_identity_plug.ex

@@ -13,8 +13,9 @@ defmodule Pleroma.Web.Plugs.MappedSignatureToIdentityPlug do
   def init(options), do: options
 
   defp key_id_from_conn(conn) do
-    with %{"keyId" => key_id} <- HTTPSignatures.signature_for_conn(conn) do
+    with %{"keyId" => key_id} <- HTTPSignatures.signature_for_conn(conn),
-      Signature.key_id_to_actor_id(key_id)
+         {:ok, ap_id} <- Signature.key_id_to_actor_id(key_id) do
+      ap_id
     else
       _ ->
         nil

lib/pleroma/plugs/oauth_scopes_plug.ex

@@ -7,15 +7,12 @@ defmodule Pleroma.Plugs.OAuthScopesPlug do
   import Pleroma.Web.Gettext
 
   alias Pleroma.Config
-  alias Pleroma.Plugs.EnsurePublicOrAuthenticatedPlug
-  alias Pleroma.Plugs.PlugHelper
 
   use Pleroma.Web, :plug
 
-  @behaviour Plug
-
   def init(%{scopes: _} = options), do: options
 
+  @impl true
   def perform(%Plug.Conn{assigns: assigns} = conn, %{scopes: scopes} = options) do
     op = options[:op] || :|
     token = assigns[:token]
@@ -31,10 +28,7 @@ def perform(%Plug.Conn{assigns: assigns} = conn, %{scopes: scopes} = options) do
         conn
 
       options[:fallback] == :proceed_unauthenticated ->
-        conn
+        drop_auth_info(conn)
-        |> assign(:user, nil)
-        |> assign(:token, nil)
-        |> maybe_perform_instance_privacy_check(options)
 
       true ->
         missing_scopes = scopes -- matched_scopes
@@ -50,6 +44,15 @@ def perform(%Plug.Conn{assigns: assigns} = conn, %{scopes: scopes} = options) do
     end
   end
 
+  @doc "Drops authentication info from connection"
+  def drop_auth_info(conn) do
+    # To simplify debugging, setting a private variable on `conn` if auth info is dropped
+    conn
+    |> put_private(:authentication_ignored, true)
+    |> assign(:user, nil)
+    |> assign(:token, nil)
+  end
+
   @doc "Filters descendants of supported scopes"
   def filter_descendants(scopes, supported_scopes) do
     Enum.filter(
@@ -71,12 +74,4 @@ def transform_scopes(scopes, options) do
       scopes
     end
   end
-
-  defp maybe_perform_instance_privacy_check(%Plug.Conn{} = conn, options) do
-    if options[:skip_instance_privacy_check] do
-      conn
-    else
-      EnsurePublicOrAuthenticatedPlug.call(conn, [])
-    end
-  end
 end

lib/pleroma/signature.ex

@@ -8,6 +8,7 @@ defmodule Pleroma.Signature do
   alias Pleroma.Keys
   alias Pleroma.User
   alias Pleroma.Web.ActivityPub.ActivityPub
+  alias Pleroma.Web.ActivityPub.ObjectValidators.Types
 
   def key_id_to_actor_id(key_id) do
     uri =
@@ -21,12 +22,23 @@ def key_id_to_actor_id(key_id) do
         uri
       end
 
-    URI.to_string(uri)
+    maybe_ap_id = URI.to_string(uri)
+
+    case Types.ObjectID.cast(maybe_ap_id) do
+      {:ok, ap_id} ->
+        {:ok, ap_id}
+
+      _ ->
+        case Pleroma.Web.WebFinger.finger(maybe_ap_id) do
+          %{"ap_id" => ap_id} -> {:ok, ap_id}
+          _ -> {:error, maybe_ap_id}
+        end
+    end
   end
 
   def fetch_public_key(conn) do
     with %{"keyId" => kid} <- HTTPSignatures.signature_for_conn(conn),
-         actor_id <- key_id_to_actor_id(kid),
+         {:ok, actor_id} <- key_id_to_actor_id(kid),
          {:ok, public_key} <- User.get_public_key_for_ap_id(actor_id) do
       {:ok, public_key}
     else
@@ -37,7 +49,7 @@ def fetch_public_key(conn) do
 
   def refetch_public_key(conn) do
     with %{"keyId" => kid} <- HTTPSignatures.signature_for_conn(conn),
-         actor_id <- key_id_to_actor_id(kid),
+         {:ok, actor_id} <- key_id_to_actor_id(kid),
          {:ok, _user} <- ActivityPub.make_user_from_ap_id(actor_id),
          {:ok, public_key} <- User.get_public_key_for_ap_id(actor_id) do
       {:ok, public_key}

lib/pleroma/tests/auth_test_controller.ex

@@ -0,0 +1,93 @@
+# Pleroma: A lightweight social networking server
+# Copyright © 2017-2020 Pleroma Authors <https://pleroma.social/>
+# SPDX-License-Identifier: AGPL-3.0-only
+
+# A test controller reachable only in :test env.
+defmodule Pleroma.Tests.AuthTestController do
+  @moduledoc false
+
+  use Pleroma.Web, :controller
+
+  alias Pleroma.Plugs.EnsurePublicOrAuthenticatedPlug
+  alias Pleroma.Plugs.OAuthScopesPlug
+  alias Pleroma.User
+
+  # Serves only with proper OAuth token (:api and :authenticated_api)
+  # Skipping EnsurePublicOrAuthenticatedPlug has no effect in this case
+  #
+  # Suggested use case: all :authenticated_api endpoints (makes no sense for :api endpoints)
+  plug(OAuthScopesPlug, %{scopes: ["read"]} when action == :do_oauth_check)
+
+  # Via :api, keeps :user if token has requested scopes (if :user is dropped, serves if public)
+  # Via :authenticated_api, serves if token is present and has requested scopes
+  #
+  # Suggested use case: vast majority of :api endpoints (no sense for :authenticated_api ones)
+  plug(
+    OAuthScopesPlug,
+    %{scopes: ["read"], fallback: :proceed_unauthenticated}
+    when action == :fallback_oauth_check
+  )
+
+  # Keeps :user if present, executes regardless of token / token scopes
+  # Fails with no :user for :authenticated_api / no user for :api on private instance
+  # Note: EnsurePublicOrAuthenticatedPlug is not skipped (private instance fails on no :user)
+  # Note: Basic Auth processing results in :skip_plug call for OAuthScopesPlug
+  #
+  # Suggested use: suppressing OAuth checks for other auth mechanisms (like Basic Auth)
+  # For controller-level use, see :skip_oauth_skip_publicity_check instead
+  plug(
+    :skip_plug,
+    OAuthScopesPlug when action == :skip_oauth_check
+  )
+
+  # (Shouldn't be executed since the plug is skipped)
+  plug(OAuthScopesPlug, %{scopes: ["admin"]} when action == :skip_oauth_check)
+
+  # Via :api, keeps :user if token has requested scopes, and continues with nil :user otherwise
+  # Via :authenticated_api, serves if token is present and has requested scopes
+  #
+  # Suggested use: as :fallback_oauth_check but open with nil :user for :api on private instances
+  plug(
+    :skip_plug,
+    EnsurePublicOrAuthenticatedPlug when action == :fallback_oauth_skip_publicity_check
+  )
+
+  plug(
+    OAuthScopesPlug,
+    %{scopes: ["read"], fallback: :proceed_unauthenticated}
+    when action == :fallback_oauth_skip_publicity_check
+  )
+
+  # Via :api, keeps :user if present, serves regardless of token presence / scopes / :user presence
+  # Via :authenticated_api, serves if :user is set (regardless of token presence and its scopes)
+  #
+  # Suggested use: making an :api endpoint always accessible (e.g. email confirmation endpoint)
+  plug(
+    :skip_plug,
+    [OAuthScopesPlug, EnsurePublicOrAuthenticatedPlug]
+    when action == :skip_oauth_skip_publicity_check
+  )
+
+  # Via :authenticated_api, always fails with 403 (endpoint is insecure)
+  # Via :api, drops :user if present and serves if public (private instance rejects on no user)
+  #
+  # Suggested use: none; please define OAuth rules for all :api / :authenticated_api endpoints
+  plug(:skip_plug, [] when action == :missing_oauth_check_definition)
+
+  def do_oauth_check(conn, _params), do: conn_state(conn)
+
+  def fallback_oauth_check(conn, _params), do: conn_state(conn)
+
+  def skip_oauth_check(conn, _params), do: conn_state(conn)
+
+  def fallback_oauth_skip_publicity_check(conn, _params), do: conn_state(conn)
+
+  def skip_oauth_skip_publicity_check(conn, _params), do: conn_state(conn)
+
+  def missing_oauth_check_definition(conn, _params), do: conn_state(conn)
+
+  defp conn_state(%{assigns: %{user: %User{} = user}} = conn),
+    do: json(conn, %{user_id: user.id})
+
+  defp conn_state(conn), do: json(conn, %{user_id: nil})
+end

lib/pleroma/tests/oauth_test_controller.ex

@@ -1,31 +0,0 @@
-# Pleroma: A lightweight social networking server
-# Copyright © 2017-2020 Pleroma Authors <https://pleroma.social/>
-# SPDX-License-Identifier: AGPL-3.0-only
-
-# A test controller reachable only in :test env.
-# Serves to test OAuth scopes check skipping / enforcement.
-defmodule Pleroma.Tests.OAuthTestController do
-  @moduledoc false
-
-  use Pleroma.Web, :controller
-
-  alias Pleroma.Plugs.OAuthScopesPlug
-
-  plug(:skip_plug, OAuthScopesPlug when action == :skipped_oauth)
-
-  plug(OAuthScopesPlug, %{scopes: ["read"]} when action != :missed_oauth)
-
-  def skipped_oauth(conn, _params) do
-    noop(conn)
-  end
-
-  def performed_oauth(conn, _params) do
-    noop(conn)
-  end
-
-  def missed_oauth(conn, _params) do
-    noop(conn)
-  end
-
-  defp noop(conn), do: json(conn, %{})
-end

lib/pleroma/user.ex

@@ -1445,9 +1445,16 @@ def perform(:delete, %User{} = user) do
     end)
 
     delete_user_activities(user)
+
+    if user.local do
+      user
+      |> change(%{deactivated: true, email: nil})
+      |> update_and_set_cache()
+    else
       invalidate_cache(user)
       Repo.delete(user)
     end
+  end
 
   def perform(:deactivate_async, user, status), do: deactivate(user, status)
 

lib/pleroma/web/activity_pub/activity_pub.ex

@@ -836,7 +836,7 @@ defp exclude_visibility(query, %{"exclude_visibilities" => visibility})
   end
 
   defp exclude_visibility(query, %{"exclude_visibilities" => visibility})
-       when visibility not in @valid_visibilities do
+       when visibility not in [nil | @valid_visibilities] do
     Logger.error("Could not exclude visibility to #{visibility}")
     query
   end
@@ -1043,7 +1043,7 @@ defp restrict_media(_query, %{"only_media" => _val, "skip_preload" => true}) do
     raise "Can't use the child object without preloading!"
   end
 
-  defp restrict_media(query, %{"only_media" => val}) when val == "true" or val == "1" do
+  defp restrict_media(query, %{"only_media" => val}) when val in [true, "true", "1"] do
     from(
       [_activity, object] in query,
       where: fragment("not (?)->'attachment' = (?)", object.data, ^[])
@@ -1052,7 +1052,7 @@ defp restrict_media(query, %{"only_media" => val}) when val == "true" or val ==
 
   defp restrict_media(query, _), do: query
 
-  defp restrict_replies(query, %{"exclude_replies" => val}) when val == "true" or val == "1" do
+  defp restrict_replies(query, %{"exclude_replies" => val}) when val in [true, "true", "1"] do
     from(
       [_activity, object] in query,
       where: fragment("?->>'inReplyTo' is null", object.data)
@@ -1096,7 +1096,7 @@ defp restrict_replies(query, %{
 
   defp restrict_replies(query, _), do: query
 
-  defp restrict_reblogs(query, %{"exclude_reblogs" => val}) when val == "true" or val == "1" do
+  defp restrict_reblogs(query, %{"exclude_reblogs" => val}) when val in [true, "true", "1"] do
     from(activity in query, where: fragment("?->>'type' != 'Announce'", activity.data))
   end
 
@@ -1175,7 +1175,12 @@ defp restrict_unlisted(query) do
     )
   end
 
-  defp restrict_pinned(query, %{"pinned" => "true", "pinned_activity_ids" => ids}) do
+  # TODO: when all endpoints migrated to OpenAPI compare `pinned` with `true` (boolean) only,
+  # the same for `restrict_media/2`, `restrict_replies/2`, 'restrict_reblogs/2'
+  # and `restrict_muted/2`
+
+  defp restrict_pinned(query, %{"pinned" => pinned, "pinned_activity_ids" => ids})
+       when pinned in [true, "true", "1"] do
     from(activity in query, where: activity.id in ^ids)
   end
 
@@ -1536,21 +1541,34 @@ def fetch_follow_information_for_user(user) do
   defp normalize_counter(counter) when is_integer(counter), do: counter
   defp normalize_counter(_), do: 0
 
-  defp maybe_update_follow_information(data) do
+  def maybe_update_follow_information(user_data) do
     with {:enabled, true} <- {:enabled, Config.get([:instance, :external_user_synchronization])},
-         {:ok, info} <- fetch_follow_information_for_user(data) do
+         {_, true} <- {:user_type_check, user_data[:type] in ["Person", "Service"]},
-      info = Map.merge(data[:info] || %{}, info)
+         {_, true} <-
-      Map.put(data, :info, info)
+           {:collections_available,
+            !!(user_data[:following_address] && user_data[:follower_address])},
+         {:ok, info} <-
+           fetch_follow_information_for_user(user_data) do
+      info = Map.merge(user_data[:info] || %{}, info)
+
+      user_data
+      |> Map.put(:info, info)
     else
+      {:user_type_check, false} ->
+        user_data
+
+      {:collections_available, false} ->
+        user_data
+
       {:enabled, false} ->
-        data
+        user_data
 
       e ->
         Logger.error(
-          "Follower/Following counter update for #{data.ap_id} failed.\n" <> inspect(e)
+          "Follower/Following counter update for #{user_data.ap_id} failed.\n" <> inspect(e)
         )
 
-        data
+        user_data
     end
   end
 

lib/pleroma/web/activity_pub/activity_pub_controller.ex

@@ -37,9 +37,10 @@ defmodule Pleroma.Web.ActivityPub.ActivityPubController do
     [unless_func: &FederatingPlug.federating?/0] when action not in @federating_only_actions
   )
 
+  # Note: :following and :followers must be served even without authentication (as via :api)
   plug(
     EnsureAuthenticatedPlug
-    when action in [:read_inbox, :update_outbox, :whoami, :upload_media, :following, :followers]
+    when action in [:read_inbox, :update_outbox, :whoami, :upload_media]
   )
 
   plug(

lib/pleroma/web/activity_pub/object_validators/like_validator.ex

@@ -5,6 +5,7 @@
 defmodule Pleroma.Web.ActivityPub.ObjectValidators.LikeValidator do
   use Ecto.Schema
 
+  alias Pleroma.Object
   alias Pleroma.Web.ActivityPub.ObjectValidators.Types
   alias Pleroma.Web.ActivityPub.Utils
 
@@ -19,8 +20,8 @@ defmodule Pleroma.Web.ActivityPub.ObjectValidators.LikeValidator do
     field(:object, Types.ObjectID)
     field(:actor, Types.ObjectID)
     field(:context, :string)
-    field(:to, {:array, :string})
+    field(:to, {:array, :string}, default: [])
-    field(:cc, {:array, :string})
+    field(:cc, {:array, :string}, default: [])
   end
 
   def cast_and_validate(data) do
@@ -31,7 +32,48 @@ def cast_and_validate(data) do
 
   def cast_data(data) do
     %__MODULE__{}
-    |> cast(data, [:id, :type, :object, :actor, :context, :to, :cc])
+    |> changeset(data)
+  end
+
+  def changeset(struct, data) do
+    struct
+    |> cast(data, __schema__(:fields))
+    |> fix_after_cast()
+  end
+
+  def fix_after_cast(cng) do
+    cng
+    |> fix_recipients()
+    |> fix_context()
+  end
+
+  def fix_context(cng) do
+    object = get_field(cng, :object)
+
+    with nil <- get_field(cng, :context),
+         %Object{data: %{"context" => context}} <- Object.get_cached_by_ap_id(object) do
+      cng
+      |> put_change(:context, context)
+    else
+      _ ->
+        cng
+    end
+  end
+
+  def fix_recipients(cng) do
+    to = get_field(cng, :to)
+    cc = get_field(cng, :cc)
+    object = get_field(cng, :object)
+
+    with {[], []} <- {to, cc},
+         %Object{data: %{"actor" => actor}} <- Object.get_cached_by_ap_id(object),
+         {:ok, actor} <- Types.ObjectID.cast(actor) do
+      cng
+      |> put_change(:to, [actor])
+    else
+      _ ->
+        cng
+    end
   end
 
   def validate_data(data_cng) do

lib/pleroma/web/activity_pub/pipeline.ex

@@ -4,20 +4,33 @@
 
 defmodule Pleroma.Web.ActivityPub.Pipeline do
   alias Pleroma.Activity
+  alias Pleroma.Object
+  alias Pleroma.Repo
   alias Pleroma.Web.ActivityPub.ActivityPub
   alias Pleroma.Web.ActivityPub.MRF
   alias Pleroma.Web.ActivityPub.ObjectValidator
   alias Pleroma.Web.ActivityPub.SideEffects
   alias Pleroma.Web.Federator
 
-  @spec common_pipeline(map(), keyword()) :: {:ok, Activity.t(), keyword()} | {:error, any()}
+  @spec common_pipeline(map(), keyword()) ::
+          {:ok, Activity.t() | Object.t(), keyword()} | {:error, any()}
   def common_pipeline(object, meta) do
+    case Repo.transaction(fn -> do_common_pipeline(object, meta) end) do
+      {:ok, value} ->
+        value
+
+      {:error, e} ->
+        {:error, e}
+    end
+  end
+
+  def do_common_pipeline(object, meta) do
     with {_, {:ok, validated_object, meta}} <-
            {:validate_object, ObjectValidator.validate(object, meta)},
          {_, {:ok, mrfd_object}} <- {:mrf_object, MRF.filter(validated_object)},
-         {_, {:ok, %Activity{} = activity, meta}} <-
+         {_, {:ok, activity, meta}} <-
            {:persist_object, ActivityPub.persist(mrfd_object, meta)},
-         {_, {:ok, %Activity{} = activity, meta}} <-
+         {_, {:ok, activity, meta}} <-
            {:execute_side_effects, SideEffects.handle(activity, meta)},
          {_, {:ok, _}} <- {:federation, maybe_federate(activity, meta)} do
       {:ok, activity, meta}
@@ -27,7 +40,9 @@ def common_pipeline(object, meta) do
     end
   end
 
-  defp maybe_federate(activity, meta) do
+  defp maybe_federate(%Object{}, _), do: {:ok, :not_federated}
+
+  defp maybe_federate(%Activity{} = activity, meta) do
     with {:ok, local} <- Keyword.fetch(meta, :local) do
       if local do
         Federator.publish(activity)

lib/pleroma/web/activity_pub/side_effects.ex

@@ -15,17 +15,12 @@ def handle(object, meta \\ [])
   # - Add like to object
   # - Set up notification
   def handle(%{data: %{"type" => "Like"}} = object, meta) do
-    {:ok, result} =
-      Pleroma.Repo.transaction(fn ->
     liked_object = Object.get_by_ap_id(object.data["object"])
     Utils.add_like_to_object(object, liked_object)
 
     Notification.create_notifications(object)
 
     {:ok, object, meta}
-      end)
-
-    result
   end
 
   # Nothing to do

lib/pleroma/web/activity_pub/transmogrifier.ex

@@ -7,6 +7,7 @@ defmodule Pleroma.Web.ActivityPub.Transmogrifier do
   A module to handle coding from internal to wire ActivityPub and back.
   """
   alias Pleroma.Activity
+  alias Pleroma.EarmarkRenderer
   alias Pleroma.FollowingRelationship
   alias Pleroma.Object
   alias Pleroma.Object.Containment
@@ -14,7 +15,6 @@ defmodule Pleroma.Web.ActivityPub.Transmogrifier do
   alias Pleroma.User
   alias Pleroma.Web.ActivityPub.ActivityPub
   alias Pleroma.Web.ActivityPub.ObjectValidator
-  alias Pleroma.Web.ActivityPub.ObjectValidators.LikeValidator
   alias Pleroma.Web.ActivityPub.Pipeline
   alias Pleroma.Web.ActivityPub.Utils
   alias Pleroma.Web.ActivityPub.Visibility
@@ -43,6 +43,7 @@ def fix_object(object, options \\ []) do
     |> fix_addressing
     |> fix_summary
     |> fix_type(options)
+    |> fix_content
   end
 
   def fix_summary(%{"summary" => nil} = object) do
@@ -357,6 +358,18 @@ def fix_type(%{"inReplyTo" => reply_id, "name" => _} = object, options)
 
   def fix_type(object, _), do: object
 
+  defp fix_content(%{"mediaType" => "text/markdown", "content" => content} = object)
+       when is_binary(content) do
+    html_content =
+      content
+      |> Earmark.as_html!(%Earmark.Options{renderer: EarmarkRenderer})
+      |> Pleroma.HTML.filter_tags()
+
+    Map.merge(object, %{"content" => html_content, "mediaType" => "text/html"})
+  end
+
+  defp fix_content(object), do: object
+
   defp mastodon_follow_hack(%{"id" => id, "actor" => follower_id}, followed) do
     with true <- id =~ "follows",
          %User{local: true} = follower <- User.get_cached_by_ap_id(follower_id),
@@ -644,16 +657,9 @@ def handle_incoming(
   end
 
   def handle_incoming(%{"type" => "Like"} = data, _options) do
-    with {_, {:ok, cast_data_sym}} <-
+    with :ok <- ObjectValidator.fetch_actor_and_object(data),
-           {:casting_data,
+         {:ok, activity, _meta} <-
-            data |> LikeValidator.cast_data() |> Ecto.Changeset.apply_action(:insert)},
+           Pipeline.common_pipeline(data, local: false) do
-         cast_data = ObjectValidator.stringify_keys(Map.from_struct(cast_data_sym)),
-         :ok <- ObjectValidator.fetch_actor_and_object(cast_data),
-         {_, {:ok, cast_data}} <- {:ensure_context_presence, ensure_context_presence(cast_data)},
-         {_, {:ok, cast_data}} <-
-           {:ensure_recipients_presence, ensure_recipients_presence(cast_data)},
-         {_, {:ok, activity, _meta}} <-
-           {:common_pipeline, Pipeline.common_pipeline(cast_data, local: false)} do
       {:ok, activity}
     else
       e -> {:error, e}
@@ -1207,18 +1213,24 @@ def add_attributed_to(object) do
 
   def prepare_attachments(object) do
     attachments =
-      (object["attachment"] || [])
+      object
+      |> Map.get("attachment", [])
       |> Enum.map(fn data ->
         [%{"mediaType" => media_type, "href" => href} | _] = data["url"]
-        %{"url" => href, "mediaType" => media_type, "name" => data["name"], "type" => "Document"}
+
+        %{
+          "url" => href,
+          "mediaType" => media_type,
+          "name" => data["name"],
+          "type" => "Document"
+        }
       end)
 
     Map.put(object, "attachment", attachments)
   end
 
   def strip_internal_fields(object) do
-    object
+    Map.drop(object, Pleroma.Constants.object_internal_fields())
-    |> Map.drop(Pleroma.Constants.object_internal_fields())
   end
 
   defp strip_internal_tags(%{"tag" => tags} = object) do
@@ -1276,45 +1288,4 @@ def maybe_fix_user_url(%{"url" => url} = data) when is_map(url) do
   def maybe_fix_user_url(data), do: data
 
   def maybe_fix_user_object(data), do: maybe_fix_user_url(data)
-
-  defp ensure_context_presence(%{"context" => context} = data) when is_binary(context),
-    do: {:ok, data}
-
-  defp ensure_context_presence(%{"object" => object} = data) when is_binary(object) do
-    with %{data: %{"context" => context}} when is_binary(context) <- Object.normalize(object) do
-      {:ok, Map.put(data, "context", context)}
-    else
-      _ ->
-        {:error, :no_context}
-    end
-  end
-
-  defp ensure_context_presence(_) do
-    {:error, :no_context}
-  end
-
-  defp ensure_recipients_presence(%{"to" => [_ | _], "cc" => [_ | _]} = data),
-    do: {:ok, data}
-
-  defp ensure_recipients_presence(%{"object" => object} = data) do
-    case Object.normalize(object) do
-      %{data: %{"actor" => actor}} ->
-        data =
-          data
-          |> Map.put("to", [actor])
-          |> Map.put("cc", data["cc"] || [])
-
-        {:ok, data}
-
-      nil ->
-        {:error, :no_object}
-
-      _ ->
-        {:error, :no_actor}
-    end
-  end
-
-  defp ensure_recipients_presence(_) do
-    {:error, :no_object}
-  end
 end

lib/pleroma/web/admin_api/admin_api_controller.ex

@@ -48,6 +48,7 @@ defmodule Pleroma.Web.AdminAPI.AdminAPIController do
     %{scopes: ["write:accounts"], admin: true}
     when action in [
            :get_password_reset,
+           :force_password_reset,
            :user_delete,
            :users_create,
            :user_toggle_activation,
@@ -56,7 +57,9 @@ defmodule Pleroma.Web.AdminAPI.AdminAPIController do
            :tag_users,
            :untag_users,
            :right_add,
+           :right_add_multiple,
            :right_delete,
+           :right_delete_multiple,
            :update_user_credentials
          ]
   )
@@ -84,13 +87,13 @@ defmodule Pleroma.Web.AdminAPI.AdminAPIController do
   plug(
     OAuthScopesPlug,
     %{scopes: ["write:reports"], admin: true}
-    when action in [:reports_update]
+    when action in [:reports_update, :report_notes_create, :report_notes_delete]
   )
 
   plug(
     OAuthScopesPlug,
     %{scopes: ["read:statuses"], admin: true}
-    when action == :list_user_statuses
+    when action in [:list_statuses, :list_user_statuses, :list_instance_statuses]
   )
 
   plug(
@@ -102,13 +105,30 @@ defmodule Pleroma.Web.AdminAPI.AdminAPIController do
   plug(
     OAuthScopesPlug,
     %{scopes: ["read"], admin: true}
-    when action in [:config_show, :list_log, :stats]
+    when action in [
+           :config_show,
+           :list_log,
+           :stats,
+           :relay_list,
+           :config_descriptions,
+           :need_reboot
+         ]
   )
 
   plug(
     OAuthScopesPlug,
     %{scopes: ["write"], admin: true}
-    when action == :config_update
+    when action in [
+           :restart,
+           :config_update,
+           :resend_confirmation_email,
+           :confirm_email,
+           :oauth_app_create,
+           :oauth_app_list,
+           :oauth_app_update,
+           :oauth_app_delete,
+           :reload_emoji
+         ]
   )
 
   action_fallback(:errors)
@@ -1103,25 +1123,25 @@ def stats(conn, _) do
     |> json(%{"status_visibility" => count})
   end
 
-  def errors(conn, {:error, :not_found}) do
+  defp errors(conn, {:error, :not_found}) do
     conn
     |> put_status(:not_found)
     |> json(dgettext("errors", "Not found"))
   end
 
-  def errors(conn, {:error, reason}) do
+  defp errors(conn, {:error, reason}) do
     conn
     |> put_status(:bad_request)
     |> json(reason)
   end
 
-  def errors(conn, {:param_cast, _}) do
+  defp errors(conn, {:param_cast, _}) do
     conn
     |> put_status(:bad_request)
     |> json(dgettext("errors", "Invalid parameters"))
   end
 
-  def errors(conn, _) do
+  defp errors(conn, _) do
     conn
     |> put_status(:internal_server_error)
     |> json(dgettext("errors", "Something went wrong"))

lib/pleroma/web/api_spec.ex

@@ -4,6 +4,7 @@
 
 defmodule Pleroma.Web.ApiSpec do
   alias OpenApiSpex.OpenApi
+  alias OpenApiSpex.Operation
   alias Pleroma.Web.Endpoint
   alias Pleroma.Web.Router
 
@@ -24,6 +25,13 @@ def spec do
       # populate the paths from a phoenix router
       paths: OpenApiSpex.Paths.from_router(Router),
       components: %OpenApiSpex.Components{
+        parameters: %{
+          "accountIdOrNickname" =>
+            Operation.parameter(:id, :path, :string, "Account ID or nickname",
+              example: "123",
+              required: true
+            )
+        },
         securitySchemes: %{
           "oAuth" => %OpenApiSpex.SecurityScheme{
             type: "oauth2",

lib/pleroma/web/api_spec/helpers.ex

@@ -3,6 +3,9 @@
 # SPDX-License-Identifier: AGPL-3.0-only
 
 defmodule Pleroma.Web.ApiSpec.Helpers do
+  alias OpenApiSpex.Operation
+  alias OpenApiSpex.Schema
+
   def request_body(description, schema_ref, opts \\ []) do
     media_types = ["application/json", "multipart/form-data", "application/x-www-form-urlencoded"]
 
@@ -24,4 +27,31 @@ def request_body(description, schema_ref, opts \\ []) do
       required: opts[:required] || false
     }
   end
+
+  def pagination_params do
+    [
+      Operation.parameter(:max_id, :query, :string, "Return items older than this ID"),
+      Operation.parameter(:min_id, :query, :string, "Return the oldest items newer than this ID"),
+      Operation.parameter(
+        :since_id,
+        :query,
+        :string,
+        "Return the newest items newer than this ID"
+      ),
+      Operation.parameter(
+        :limit,
+        :query,
+        %Schema{type: :integer, default: 20},
+        "Maximum number of items to return. Will be ignored if it's more than 40"
+      )
+    ]
+  end
+
+  def empty_object_response do
+    Operation.response("Empty object", "application/json", %Schema{type: :object, example: %{}})
+  end
+
+  def empty_array_response do
+    Operation.response("Empty array", "application/json", %Schema{type: :array, example: []})
+  end
 end

lib/pleroma/web/api_spec/operations/account_operation.ex

@@ -0,0 +1,702 @@
+# Pleroma: A lightweight social networking server
+# Copyright © 2017-2020 Pleroma Authors <https://pleroma.social/>
+# SPDX-License-Identifier: AGPL-3.0-only
+
+defmodule Pleroma.Web.ApiSpec.AccountOperation do
+  alias OpenApiSpex.Operation
+  alias OpenApiSpex.Reference
+  alias OpenApiSpex.Schema
+  alias Pleroma.Web.ApiSpec.Schemas.Account
+  alias Pleroma.Web.ApiSpec.Schemas.AccountRelationship
+  alias Pleroma.Web.ApiSpec.Schemas.ActorType
+  alias Pleroma.Web.ApiSpec.Schemas.ApiError
+  alias Pleroma.Web.ApiSpec.Schemas.BooleanLike
+  alias Pleroma.Web.ApiSpec.Schemas.Status
+  alias Pleroma.Web.ApiSpec.Schemas.VisibilityScope
+
+  import Pleroma.Web.ApiSpec.Helpers
+
+  @spec open_api_operation(atom) :: Operation.t()
+  def open_api_operation(action) do
+    operation = String.to_existing_atom("#{action}_operation")
+    apply(__MODULE__, operation, [])
+  end
+
+  @spec create_operation() :: Operation.t()
+  def create_operation do
+    %Operation{
+      tags: ["accounts"],
+      summary: "Register an account",
+      description:
+        "Creates a user and account records. Returns an account access token for the app that initiated the request. The app should save this token for later, and should wait for the user to confirm their account by clicking a link in their email inbox.",
+      operationId: "AccountController.create",
+      requestBody: request_body("Parameters", create_request(), required: true),
+      responses: %{
+        200 => Operation.response("Account", "application/json", create_response()),
+        400 => Operation.response("Error", "application/json", ApiError),
+        403 => Operation.response("Error", "application/json", ApiError),
+        429 => Operation.response("Error", "application/json", ApiError)
+      }
+    }
+  end
+
+  def verify_credentials_operation do
+    %Operation{
+      tags: ["accounts"],
+      description: "Test to make sure that the user token works.",
+      summary: "Verify account credentials",
+      operationId: "AccountController.verify_credentials",
+      security: [%{"oAuth" => ["read:accounts"]}],
+      responses: %{
+        200 => Operation.response("Account", "application/json", Account)
+      }
+    }
+  end
+
+  def update_credentials_operation do
+    %Operation{
+      tags: ["accounts"],
+      summary: "Update account credentials",
+      description: "Update the user's display and preferences.",
+      operationId: "AccountController.update_credentials",
+      security: [%{"oAuth" => ["write:accounts"]}],
+      requestBody: request_body("Parameters", update_creadentials_request(), required: true),
+      responses: %{
+        200 => Operation.response("Account", "application/json", Account),
+        403 => Operation.response("Error", "application/json", ApiError)
+      }
+    }
+  end
+
+  def relationships_operation do
+    %Operation{
+      tags: ["accounts"],
+      summary: "Check relationships to other accounts",
+      operationId: "AccountController.relationships",
+      description: "Find out whether a given account is followed, blocked, muted, etc.",
+      security: [%{"oAuth" => ["read:follows"]}],
+      parameters: [
+        Operation.parameter(
+          :id,
+          :query,
+          %Schema{
+            oneOf: [%Schema{type: :array, items: %Schema{type: :string}}, %Schema{type: :string}]
+          },
+          "Account IDs",
+          example: "123"
+        )
+      ],
+      responses: %{
+        200 => Operation.response("Account", "application/json", array_of_relationships())
+      }
+    }
+  end
+
+  def show_operation do
+    %Operation{
+      tags: ["accounts"],
+      summary: "Account",
+      operationId: "AccountController.show",
+      description: "View information about a profile.",
+      parameters: [%Reference{"$ref": "#/components/parameters/accountIdOrNickname"}],
+      responses: %{
+        200 => Operation.response("Account", "application/json", Account),
+        404 => Operation.response("Error", "application/json", ApiError)
+      }
+    }
+  end
+
+  def statuses_operation do
+    %Operation{
+      tags: ["accounts"],
+      summary: "Statuses",
+      operationId: "AccountController.statuses",
+      description:
+        "Statuses posted to the given account. Public (for public statuses only), or user token + `read:statuses` (for private statuses the user is authorized to see)",
+      parameters:
+        [
+          %Reference{"$ref": "#/components/parameters/accountIdOrNickname"},
+          Operation.parameter(:pinned, :query, BooleanLike, "Include only pinned statuses"),
+          Operation.parameter(:tagged, :query, :string, "With tag"),
+          Operation.parameter(
+            :only_media,
+            :query,
+            BooleanLike,
+            "Include only statuses with media attached"
+          ),
+          Operation.parameter(
+            :with_muted,
+            :query,
+            BooleanLike,
+            "Include statuses from muted acccounts."
+          ),
+          Operation.parameter(:exclude_reblogs, :query, BooleanLike, "Exclude reblogs"),
+          Operation.parameter(:exclude_replies, :query, BooleanLike, "Exclude replies"),
+          Operation.parameter(
+            :exclude_visibilities,
+            :query,
+            %Schema{type: :array, items: VisibilityScope},
+            "Exclude visibilities"
+          )
+        ] ++ pagination_params(),
+      responses: %{
+        200 => Operation.response("Statuses", "application/json", array_of_statuses()),
+        404 => Operation.response("Error", "application/json", ApiError)
+      }
+    }
+  end
+
+  def followers_operation do
+    %Operation{
+      tags: ["accounts"],
+      summary: "Followers",
+      operationId: "AccountController.followers",
+      security: [%{"oAuth" => ["read:accounts"]}],
+      description:
+        "Accounts which follow the given account, if network is not hidden by the account owner.",
+      parameters:
+        [%Reference{"$ref": "#/components/parameters/accountIdOrNickname"}] ++ pagination_params(),
+      responses: %{
+        200 => Operation.response("Accounts", "application/json", array_of_accounts())
+      }
+    }
+  end
+
+  def following_operation do
+    %Operation{
+      tags: ["accounts"],
+      summary: "Following",
+      operationId: "AccountController.following",
+      security: [%{"oAuth" => ["read:accounts"]}],
+      description:
+        "Accounts which the given account is following, if network is not hidden by the account owner.",
+      parameters:
+        [%Reference{"$ref": "#/components/parameters/accountIdOrNickname"}] ++ pagination_params(),
+      responses: %{200 => Operation.response("Accounts", "application/json", array_of_accounts())}
+    }
+  end
+
+  def lists_operation do
+    %Operation{
+      tags: ["accounts"],
+      summary: "Lists containing this account",
+      operationId: "AccountController.lists",
+      security: [%{"oAuth" => ["read:lists"]}],
+      description: "User lists that you have added this account to.",
+      parameters: [%Reference{"$ref": "#/components/parameters/accountIdOrNickname"}],
+      responses: %{200 => Operation.response("Lists", "application/json", array_of_lists())}
+    }
+  end
+
+  def follow_operation do
+    %Operation{
+      tags: ["accounts"],
+      summary: "Follow",
+      operationId: "AccountController.follow",
+      security: [%{"oAuth" => ["follow", "write:follows"]}],
+      description: "Follow the given account",
+      parameters: [
+        %Reference{"$ref": "#/components/parameters/accountIdOrNickname"},
+        Operation.parameter(
+          :reblogs,
+          :query,
+          BooleanLike,
+          "Receive this account's reblogs in home timeline? Defaults to true."
+        )
+      ],
+      responses: %{
+        200 => Operation.response("Relationship", "application/json", AccountRelationship),
+        400 => Operation.response("Error", "application/json", ApiError),
+        404 => Operation.response("Error", "application/json", ApiError)
+      }
+    }
+  end
+
+  def unfollow_operation do
+    %Operation{
+      tags: ["accounts"],
+      summary: "Unfollow",
+      operationId: "AccountController.unfollow",
+      security: [%{"oAuth" => ["follow", "write:follows"]}],
+      description: "Unfollow the given account",
+      parameters: [%Reference{"$ref": "#/components/parameters/accountIdOrNickname"}],
+      responses: %{
+        200 => Operation.response("Relationship", "application/json", AccountRelationship),
+        400 => Operation.response("Error", "application/json", ApiError),
+        404 => Operation.response("Error", "application/json", ApiError)
+      }
+    }
+  end
+
+  def mute_operation do
+    %Operation{
+      tags: ["accounts"],
+      summary: "Mute",
+      operationId: "AccountController.mute",
+      security: [%{"oAuth" => ["follow", "write:mutes"]}],
+      requestBody: request_body("Parameters", mute_request()),
+      description:
+        "Mute the given account. Clients should filter statuses and notifications from this account, if received (e.g. due to a boost in the Home timeline).",
+      parameters: [
+        %Reference{"$ref": "#/components/parameters/accountIdOrNickname"},
+        Operation.parameter(
+          :notifications,
+          :query,
+          %Schema{allOf: [BooleanLike], default: true},
+          "Mute notifications in addition to statuses? Defaults to `true`."
+        )
+      ],
+      responses: %{
+        200 => Operation.response("Relationship", "application/json", AccountRelationship)
+      }
+    }
+  end
+
+  def unmute_operation do
+    %Operation{
+      tags: ["accounts"],
+      summary: "Unmute",
+      operationId: "AccountController.unmute",
+      security: [%{"oAuth" => ["follow", "write:mutes"]}],
+      description: "Unmute the given account.",
+      parameters: [%Reference{"$ref": "#/components/parameters/accountIdOrNickname"}],
+      responses: %{
+        200 => Operation.response("Relationship", "application/json", AccountRelationship)
+      }
+    }
+  end
+
+  def block_operation do
+    %Operation{
+      tags: ["accounts"],
+      summary: "Block",
+      operationId: "AccountController.block",
+      security: [%{"oAuth" => ["follow", "write:blocks"]}],
+      description:
+        "Block the given account. Clients should filter statuses from this account if received (e.g. due to a boost in the Home timeline)",
+      parameters: [%Reference{"$ref": "#/components/parameters/accountIdOrNickname"}],
+      responses: %{
+        200 => Operation.response("Relationship", "application/json", AccountRelationship)
+      }
+    }
+  end
+
+  def unblock_operation do
+    %Operation{
+      tags: ["accounts"],
+      summary: "Unblock",
+      operationId: "AccountController.unblock",
+      security: [%{"oAuth" => ["follow", "write:blocks"]}],
+      description: "Unblock the given account.",
+      parameters: [%Reference{"$ref": "#/components/parameters/accountIdOrNickname"}],
+      responses: %{
+        200 => Operation.response("Relationship", "application/json", AccountRelationship)
+      }
+    }
+  end
+
+  def follow_by_uri_operation do
+    %Operation{
+      tags: ["accounts"],
+      summary: "Follow by URI",
+      operationId: "AccountController.follows",
+      security: [%{"oAuth" => ["follow", "write:follows"]}],
+      requestBody: request_body("Parameters", follow_by_uri_request(), required: true),
+      responses: %{
+        200 => Operation.response("Account", "application/json", AccountRelationship),
+        400 => Operation.response("Error", "application/json", ApiError),
+        404 => Operation.response("Error", "application/json", ApiError)
+      }
+    }
+  end
+
+  def mutes_operation do
+    %Operation{
+      tags: ["accounts"],
+      summary: "Muted accounts",
+      operationId: "AccountController.mutes",
+      description: "Accounts the user has muted.",
+      security: [%{"oAuth" => ["follow", "read:mutes"]}],
+      responses: %{
+        200 => Operation.response("Accounts", "application/json", array_of_accounts())
+      }
+    }
+  end
+
+  def blocks_operation do
+    %Operation{
+      tags: ["accounts"],
+      summary: "Blocked users",
+      operationId: "AccountController.blocks",
+      description: "View your blocks. See also accounts/:id/{block,unblock}",
+      security: [%{"oAuth" => ["read:blocks"]}],
+      responses: %{
+        200 => Operation.response("Accounts", "application/json", array_of_accounts())
+      }
+    }
+  end
+
+  def endorsements_operation do
+    %Operation{
+      tags: ["accounts"],
+      summary: "Endorsements",
+      operationId: "AccountController.endorsements",
+      description: "Not implemented",
+      security: [%{"oAuth" => ["read:accounts"]}],
+      responses: %{
+        200 => empty_array_response()
+      }
+    }
+  end
+
+  def identity_proofs_operation do
+    %Operation{
+      tags: ["accounts"],
+      summary: "Identity proofs",
+      operationId: "AccountController.identity_proofs",
+      description: "Not implemented",
+      responses: %{
+        200 => empty_array_response()
+      }
+    }
+  end
+
+  defp create_request do
+    %Schema{
+      title: "AccountCreateRequest",
+      description: "POST body for creating an account",
+      type: :object,
+      properties: %{
+        reason: %Schema{
+          type: :string,
+          description:
+            "Text that will be reviewed by moderators if registrations require manual approval"
+        },
+        username: %Schema{type: :string, description: "The desired username for the account"},
+        email: %Schema{
+          type: :string,
+          description:
+            "The email address to be used for login. Required when `account_activation_required` is enabled.",
+          format: :email
+        },
+        password: %Schema{
+          type: :string,
+          description: "The password to be used for login",
+          format: :password
+        },
+        agreement: %Schema{
+          type: :boolean,
+          description:
+            "Whether the user agrees to the local rules, terms, and policies. These should be presented to the user in order to allow them to consent before setting this parameter to TRUE."
+        },
+        locale: %Schema{
+          type: :string,
+          description: "The language of the confirmation email that will be sent"
+        },
+        # Pleroma-specific properties:
+        fullname: %Schema{type: :string, description: "Full name"},
+        bio: %Schema{type: :string, description: "Bio", default: ""},
+        captcha_solution: %Schema{
+          type: :string,
+          description: "Provider-specific captcha solution"
+        },
+        captcha_token: %Schema{type: :string, description: "Provider-specific captcha token"},
+        captcha_answer_data: %Schema{type: :string, description: "Provider-specific captcha data"},
+        token: %Schema{
+          type: :string,
+          description: "Invite token required when the registrations aren't public"
+        }
+      },
+      required: [:username, :password, :agreement],
+      example: %{
+        "username" => "cofe",
+        "email" => "cofe@example.com",
+        "password" => "secret",
+        "agreement" => "true",
+        "bio" => "☕️"
+      }
+    }
+  end
+
+  defp create_response do
+    %Schema{
+      title: "AccountCreateResponse",
+      description: "Response schema for an account",
+      type: :object,
+      properties: %{
+        token_type: %Schema{type: :string},
+        access_token: %Schema{type: :string},
+        scope: %Schema{type: :array, items: %Schema{type: :string}},
+        created_at: %Schema{type: :integer, format: :"date-time"}
+      },
+      example: %{
+        "access_token" => "i9hAVVzGld86Pl5JtLtizKoXVvtTlSCJvwaugCxvZzk",
+        "created_at" => 1_585_918_714,
+        "scope" => ["read", "write", "follow", "push"],
+        "token_type" => "Bearer"
+      }
+    }
+  end
+
+  defp update_creadentials_request do
+    %Schema{
+      title: "AccountUpdateCredentialsRequest",
+      description: "POST body for creating an account",
+      type: :object,
+      properties: %{
+        bot: %Schema{
+          type: :boolean,
+          description: "Whether the account has a bot flag."
+        },
+        display_name: %Schema{
+          type: :string,
+          description: "The display name to use for the profile."
+        },
+        note: %Schema{type: :string, description: "The account bio."},
+        avatar: %Schema{
+          type: :string,
+          description: "Avatar image encoded using multipart/form-data",
+          format: :binary
+        },
+        header: %Schema{
+          type: :string,
+          description: "Header image encoded using multipart/form-data",
+          format: :binary
+        },
+        locked: %Schema{
+          type: :boolean,
+          description: "Whether manual approval of follow requests is required."
+        },
+        fields_attributes: %Schema{
+          oneOf: [
+            %Schema{type: :array, items: attribute_field()},
+            %Schema{type: :object, additionalProperties: %Schema{type: attribute_field()}}
+          ]
+        },
+        # NOTE: `source` field is not supported
+        #
+        # source: %Schema{
+        #   type: :object,
+        #   properties: %{
+        #     privacy: %Schema{type: :string},
+        #     sensitive: %Schema{type: :boolean},
+        #     language: %Schema{type: :string}
+        #   }
+        # },
+
+        # Pleroma-specific fields
+        no_rich_text: %Schema{
+          type: :boolean,
+          description: "html tags are stripped from all statuses requested from the API"
+        },
+        hide_followers: %Schema{type: :boolean, description: "user's followers will be hidden"},
+        hide_follows: %Schema{type: :boolean, description: "user's follows will be hidden"},
+        hide_followers_count: %Schema{
+          type: :boolean,
+          description: "user's follower count will be hidden"
+        },
+        hide_follows_count: %Schema{
+          type: :boolean,
+          description: "user's follow count will be hidden"
+        },
+        hide_favorites: %Schema{
+          type: :boolean,
+          description: "user's favorites timeline will be hidden"
+        },
+        show_role: %Schema{
+          type: :boolean,
+          description: "user's role (e.g admin, moderator) will be exposed to anyone in the
+        API"
+        },
+        default_scope: VisibilityScope,
+        pleroma_settings_store: %Schema{
+          type: :object,
+          description: "Opaque user settings to be saved on the backend."
+        },
+        skip_thread_containment: %Schema{
+          type: :boolean,
+          description: "Skip filtering out broken threads"
+        },
+        allow_following_move: %Schema{
+          type: :boolean,
+          description: "Allows automatically follow moved following accounts"
+        },
+        pleroma_background_image: %Schema{
+          type: :string,
+          description: "Sets the background image of the user.",
+          format: :binary
+        },
+        discoverable: %Schema{
+          type: :boolean,
+          description:
+            "Discovery of this account in search results and other services is allowed."
+        },
+        actor_type: ActorType
+      },
+      example: %{
+        bot: false,
+        display_name: "cofe",
+        note: "foobar",
+        fields_attributes: [%{name: "foo", value: "bar"}],
+        no_rich_text: false,
+        hide_followers: true,
+        hide_follows: false,
+        hide_followers_count: false,
+        hide_follows_count: false,
+        hide_favorites: false,
+        show_role: false,
+        default_scope: "private",
+        pleroma_settings_store: %{"pleroma-fe" => %{"key" => "val"}},
+        skip_thread_containment: false,
+        allow_following_move: false,
+        discoverable: false,
+        actor_type: "Person"
+      }
+    }
+  end
+
+  defp array_of_accounts do
+    %Schema{
+      title: "ArrayOfAccounts",
+      type: :array,
+      items: Account
+    }
+  end
+
+  defp array_of_relationships do
+    %Schema{
+      title: "ArrayOfRelationships",
+      description: "Response schema for account relationships",
+      type: :array,
+      items: AccountRelationship,
+      example: [
+        %{
+          "id" => "1",
+          "following" => true,
+          "showing_reblogs" => true,
+          "followed_by" => true,
+          "blocking" => false,
+          "blocked_by" => true,
+          "muting" => false,
+          "muting_notifications" => false,
+          "requested" => false,
+          "domain_blocking" => false,
+          "subscribing" => false,
+          "endorsed" => true
+        },
+        %{
+          "id" => "2",
+          "following" => true,
+          "showing_reblogs" => true,
+          "followed_by" => true,
+          "blocking" => false,
+          "blocked_by" => true,
+          "muting" => true,
+          "muting_notifications" => false,
+          "requested" => true,
+          "domain_blocking" => false,
+          "subscribing" => false,
+          "endorsed" => false
+        },
+        %{
+          "id" => "3",
+          "following" => true,
+          "showing_reblogs" => true,
+          "followed_by" => true,
+          "blocking" => true,
+          "blocked_by" => false,
+          "muting" => true,
+          "muting_notifications" => false,
+          "requested" => false,
+          "domain_blocking" => true,
+          "subscribing" => true,
+          "endorsed" => false
+        }
+      ]
+    }
+  end
+
+  defp follow_by_uri_request do
+    %Schema{
+      title: "AccountFollowsRequest",
+      description: "POST body for muting an account",
+      type: :object,
+      properties: %{
+        uri: %Schema{type: :string, format: :uri}
+      },
+      required: [:uri]
+    }
+  end
+
+  defp mute_request do
+    %Schema{
+      title: "AccountMuteRequest",
+      description: "POST body for muting an account",
+      type: :object,
+      properties: %{
+        notifications: %Schema{
+          type: :boolean,
+          description: "Mute notifications in addition to statuses? Defaults to true.",
+          default: true
+        }
+      },
+      example: %{
+        "notifications" => true
+      }
+    }
+  end
+
+  defp list do
+    %Schema{
+      title: "List",
+      description: "Response schema for a list",
+      type: :object,
+      properties: %{
+        id: %Schema{type: :string},
+        title: %Schema{type: :string}
+      },
+      example: %{
+        "id" => "123",
+        "title" => "my list"
+      }
+    }
+  end
+
+  defp array_of_lists do
+    %Schema{
+      title: "ArrayOfLists",
+      description: "Response schema for lists",
+      type: :array,
+      items: list(),
+      example: [
+        %{"id" => "123", "title" => "my list"},
+        %{"id" => "1337", "title" => "anotehr list"}
+      ]
+    }
+  end
+
+  defp array_of_statuses do
+    %Schema{
+      title: "ArrayOfStatuses",
+      type: :array,
+      items: Status
+    }
+  end
+
+  defp attribute_field do
+    %Schema{
+      title: "AccountAttributeField",
+      description: "Request schema for account custom fields",
+      type: :object,
+      properties: %{
+        name: %Schema{type: :string},
+        value: %Schema{type: :string}
+      },
+      required: [:name, :value],
+      example: %{
+        "name" => "Website",
+        "value" => "https://pleroma.com"
+      }
+    }
+  end
+end

lib/pleroma/web/api_spec/operations/app_operation.ex

@@ -49,11 +49,7 @@ def verify_credentials_operation do
       summary: "Verify your app works",
       description: "Confirm that the app's OAuth2 credentials work.",
       operationId: "AppController.verify_credentials",
-      security: [
+      security: [%{"oAuth" => ["read"]}],
-        %{
-          "oAuth" => ["read"]
-        }
-      ],
       responses: %{
         200 =>
           Operation.response("App", "application/json", %Schema{

lib/pleroma/web/api_spec/operations/custom_emoji_operation.ex

@@ -5,7 +5,7 @@
 defmodule Pleroma.Web.ApiSpec.CustomEmojiOperation do
   alias OpenApiSpex.Operation
   alias OpenApiSpex.Schema
-  alias Pleroma.Web.ApiSpec.Schemas.CustomEmoji
+  alias Pleroma.Web.ApiSpec.Schemas.Emoji
 
   def open_api_operation(action) do
     operation = String.to_existing_atom("#{action}_operation")
@@ -19,17 +19,17 @@ def index_operation do
       description: "Returns custom emojis that are available on the server.",
       operationId: "CustomEmojiController.index",
       responses: %{
-        200 => Operation.response("Custom Emojis", "application/json", custom_emojis_resposnse())
+        200 => Operation.response("Custom Emojis", "application/json", resposnse())
       }
     }
   end
 
-  defp custom_emojis_resposnse do
+  defp resposnse do
     %Schema{
       title: "CustomEmojisResponse",
       description: "Response schema for custom emojis",
       type: :array,
-      items: CustomEmoji,
+      items: custom_emoji(),
       example: [
         %{
           "category" => "Fun",
@@ -58,4 +58,31 @@ defp custom_emojis_resposnse do
       ]
     }
   end
+
+  defp custom_emoji do
+    %Schema{
+      title: "CustomEmoji",
+      description: "Schema for a CustomEmoji",
+      allOf: [
+        Emoji,
+        %Schema{
+          type: :object,
+          properties: %{
+            category: %Schema{type: :string},
+            tags: %Schema{type: :array}
+          }
+        }
+      ],
+      example: %{
+        "category" => "Fun",
+        "shortcode" => "aaaa",
+        "url" =>
+          "https://files.mastodon.social/custom_emojis/images/000/007/118/original/aaaa.png",
+        "static_url" =>
+          "https://files.mastodon.social/custom_emojis/images/000/007/118/static/aaaa.png",
+        "visible_in_picker" => true,
+        "tags" => ["Gif", "Fun"]
+      }
+    }
+  end
 end

lib/pleroma/web/api_spec/operations/domain_block_operation.ex

@@ -5,7 +5,7 @@
 defmodule Pleroma.Web.ApiSpec.DomainBlockOperation do
   alias OpenApiSpex.Operation
   alias OpenApiSpex.Schema
-  alias Pleroma.Web.ApiSpec.Helpers
+  import Pleroma.Web.ApiSpec.Helpers
 
   def open_api_operation(action) do
     operation = String.to_existing_atom("#{action}_operation")
@@ -46,9 +46,7 @@ def create_operation do
       operationId: "DomainBlockController.create",
       requestBody: domain_block_request(),
       security: [%{"oAuth" => ["follow", "write:blocks"]}],
-      responses: %{
+      responses: %{200 => empty_object_response()}
-        200 => Operation.response("Empty object", "application/json", %Schema{type: :object})
-      }
     }
   end
 
@@ -67,7 +65,7 @@ def delete_operation do
   end
 
   defp domain_block_request do
-    Helpers.request_body(
+    request_body(
       "Parameters",
       %Schema{
         type: :object,

lib/pleroma/web/api_spec/operations/notification_operation.ex

@@ -0,0 +1,211 @@
+# Pleroma: A lightweight social networking server
+# Copyright © 2017-2020 Pleroma Authors <https://pleroma.social/>
+# SPDX-License-Identifier: AGPL-3.0-only
+
+defmodule Pleroma.Web.ApiSpec.NotificationOperation do
+  alias OpenApiSpex.Operation
+  alias OpenApiSpex.Operation
+  alias OpenApiSpex.Schema
+  alias Pleroma.Web.ApiSpec.Schemas.Account
+  alias Pleroma.Web.ApiSpec.Schemas.ApiError
+  alias Pleroma.Web.ApiSpec.Schemas.BooleanLike
+  alias Pleroma.Web.ApiSpec.Schemas.Status
+  alias Pleroma.Web.ApiSpec.Schemas.VisibilityScope
+
+  import Pleroma.Web.ApiSpec.Helpers
+
+  def open_api_operation(action) do
+    operation = String.to_existing_atom("#{action}_operation")
+    apply(__MODULE__, operation, [])
+  end
+
+  def index_operation do
+    %Operation{
+      tags: ["Notifications"],
+      summary: "Get all notifications",
+      description:
+        "Notifications concerning the user. This API returns Link headers containing links to the next/previous page. However, the links can also be constructed dynamically using query params and `id` values.",
+      operationId: "NotificationController.index",
+      security: [%{"oAuth" => ["read:notifications"]}],
+      parameters:
+        [
+          Operation.parameter(
+            :exclude_types,
+            :query,
+            %Schema{type: :array, items: notification_type()},
+            "Array of types to exclude"
+          ),
+          Operation.parameter(
+            :account_id,
+            :query,
+            %Schema{type: :string},
+            "Return only notifications received from this account"
+          ),
+          Operation.parameter(
+            :exclude_visibilities,
+            :query,
+            %Schema{type: :array, items: VisibilityScope},
+            "Exclude the notifications for activities with the given visibilities"
+          ),
+          Operation.parameter(
+            :include_types,
+            :query,
+            %Schema{type: :array, items: notification_type()},
+            "Include the notifications for activities with the given types"
+          ),
+          Operation.parameter(
+            :with_muted,
+            :query,
+            BooleanLike,
+            "Include the notifications from muted users"
+          )
+        ] ++ pagination_params(),
+      responses: %{
+        200 =>
+          Operation.response("Array of notifications", "application/json", %Schema{
+            type: :array,
+            items: notification()
+          }),
+        404 => Operation.response("Error", "application/json", ApiError)
+      }
+    }
+  end
+
+  def show_operation do
+    %Operation{
+      tags: ["Notifications"],
+      summary: "Get a single notification",
+      description: "View information about a notification with a given ID.",
+      operationId: "NotificationController.show",
+      security: [%{"oAuth" => ["read:notifications"]}],
+      parameters: [id_param()],
+      responses: %{
+        200 => Operation.response("Notification", "application/json", notification())
+      }
+    }
+  end
+
+  def clear_operation do
+    %Operation{
+      tags: ["Notifications"],
+      summary: "Dismiss all notifications",
+      description: "Clear all notifications from the server.",
+      operationId: "NotificationController.clear",
+      security: [%{"oAuth" => ["write:notifications"]}],
+      responses: %{200 => empty_object_response()}
+    }
+  end
+
+  def dismiss_operation do
+    %Operation{
+      tags: ["Notifications"],
+      summary: "Dismiss a single notification",
+      description: "Clear a single notification from the server.",
+      operationId: "NotificationController.dismiss",
+      parameters: [id_param()],
+      security: [%{"oAuth" => ["write:notifications"]}],
+      responses: %{200 => empty_object_response()}
+    }
+  end
+
+  def dismiss_via_body_operation do
+    %Operation{
+      tags: ["Notifications"],
+      summary: "Dismiss a single notification",
+      deprecated: true,
+      description: "Clear a single notification from the server.",
+      operationId: "NotificationController.dismiss_via_body",
+      requestBody:
+        request_body(
+          "Parameters",
+          %Schema{type: :object, properties: %{id: %Schema{type: :string}}},
+          required: true
+        ),
+      security: [%{"oAuth" => ["write:notifications"]}],
+      responses: %{200 => empty_object_response()}
+    }
+  end
+
+  def destroy_multiple_operation do
+    %Operation{
+      tags: ["Notifications"],
+      summary: "Dismiss multiple notifications",
+      operationId: "NotificationController.destroy_multiple",
+      security: [%{"oAuth" => ["write:notifications"]}],
+      parameters: [
+        Operation.parameter(
+          :ids,
+          :query,
+          %Schema{type: :array, items: %Schema{type: :string}},
+          "Array of notification IDs to dismiss",
+          required: true
+        )
+      ],
+      responses: %{200 => empty_object_response()}
+    }
+  end
+
+  defp notification do
+    %Schema{
+      title: "Notification",
+      description: "Response schema for a notification",
+      type: :object,
+      properties: %{
+        id: %Schema{type: :string},
+        type: notification_type(),
+        created_at: %Schema{type: :string, format: :"date-time"},
+        account: %Schema{
+          allOf: [Account],
+          description: "The account that performed the action that generated the notification."
+        },
+        status: %Schema{
+          allOf: [Status],
+          description:
+            "Status that was the object of the notification, e.g. in mentions, reblogs, favourites, or polls.",
+          nullable: true
+        }
+      },
+      example: %{
+        "id" => "34975861",
+        "type" => "mention",
+        "created_at" => "2019-11-23T07:49:02.064Z",
+        "account" => Account.schema().example,
+        "status" => Status.schema().example
+      }
+    }
+  end
+
+  defp notification_type do
+    %Schema{
+      type: :string,
+      enum: [
+        "follow",
+        "favourite",
+        "reblog",
+        "mention",
+        "poll",
+        "pleroma:emoji_reaction",
+        "move",
+        "follow_request"
+      ],
+      description: """
+      The type of event that resulted in the notification.
+
+      - `follow` - Someone followed you
+      - `mention` - Someone mentioned you in their status
+      - `reblog` - Someone boosted one of your statuses
+      - `favourite` - Someone favourited one of your statuses
+      - `poll` - A poll you have voted in or created has ended
+      - `move` - Someone moved their account
+      - `pleroma:emoji_reaction` - Someone reacted with emoji to your status
+      """
+    }
+  end
+
+  defp id_param do
+    Operation.parameter(:id, :path, :string, "Notification ID",
+      example: "123",
+      required: true
+    )
+  end
+end

lib/pleroma/web/api_spec/operations/report_operation.ex

@@ -0,0 +1,78 @@
+# Pleroma: A lightweight social networking server
+# Copyright © 2017-2020 Pleroma Authors <https://pleroma.social/>
+# SPDX-License-Identifier: AGPL-3.0-only
+
+defmodule Pleroma.Web.ApiSpec.ReportOperation do
+  alias OpenApiSpex.Operation
+  alias OpenApiSpex.Schema
+  alias Pleroma.Web.ApiSpec.Helpers
+  alias Pleroma.Web.ApiSpec.Schemas.ApiError
+
+  def open_api_operation(action) do
+    operation = String.to_existing_atom("#{action}_operation")
+    apply(__MODULE__, operation, [])
+  end
+
+  def create_operation do
+    %Operation{
+      tags: ["reports"],
+      summary: "File a report",
+      description: "Report problematic users to your moderators",
+      operationId: "ReportController.create",
+      security: [%{"oAuth" => ["follow", "write:reports"]}],
+      requestBody: Helpers.request_body("Parameters", create_request(), required: true),
+      responses: %{
+        200 => Operation.response("Report", "application/json", create_response()),
+        400 => Operation.response("Report", "application/json", ApiError)
+      }
+    }
+  end
+
+  defp create_request do
+    %Schema{
+      title: "ReportCreateRequest",
+      description: "POST body for creating a report",
+      type: :object,
+      properties: %{
+        account_id: %Schema{type: :string, description: "ID of the account to report"},
+        status_ids: %Schema{
+          type: :array,
+          items: %Schema{type: :string},
+          description: "Array of Statuses to attach to the report, for context"
+        },
+        comment: %Schema{
+          type: :string,
+          description: "Reason for the report"
+        },
+        forward: %Schema{
+          type: :boolean,
+          default: false,
+          description:
+            "If the account is remote, should the report be forwarded to the remote admin?"
+        }
+      },
+      required: [:account_id],
+      example: %{
+        "account_id" => "123",
+        "status_ids" => ["1337"],
+        "comment" => "bad status!",
+        "forward" => "false"
+      }
+    }
+  end
+
+  defp create_response do
+    %Schema{
+      title: "ReportResponse",
+      type: :object,
+      properties: %{
+        id: %Schema{type: :string, description: "Report ID"},
+        action_taken: %Schema{type: :boolean, description: "Is action taken?"}
+      },
+      example: %{
+        "id" => "123",
+        "action_taken" => false
+      }
+    }
+  end
+end

lib/pleroma/web/api_spec/render_error.ex

@@ -0,0 +1,231 @@
+# Pleroma: A lightweight social networking server
+# Copyright © 2017-2020 Pleroma Authors <https://pleroma.social/>
+# SPDX-License-Identifier: AGPL-3.0-only
+
+defmodule Pleroma.Web.ApiSpec.RenderError do
+  @behaviour Plug
+
+  import Plug.Conn, only: [put_status: 2]
+  import Phoenix.Controller, only: [json: 2]
+  import Pleroma.Web.Gettext
+
+  @impl Plug
+  def init(opts), do: opts
+
+  @impl Plug
+
+  def call(conn, errors) do
+    errors =
+      Enum.map(errors, fn
+        %{name: nil} = err ->
+          %OpenApiSpex.Cast.Error{err | name: List.last(err.path)}
+
+        err ->
+          err
+      end)
+
+    conn
+    |> put_status(:bad_request)
+    |> json(%{
+      error: errors |> Enum.map(&message/1) |> Enum.join(" "),
+      errors: errors |> Enum.map(&render_error/1)
+    })
+  end
+
+  defp render_error(error) do
+    pointer = OpenApiSpex.path_to_string(error)
+
+    %{
+      title: "Invalid value",
+      source: %{
+        pointer: pointer
+      },
+      message: OpenApiSpex.Cast.Error.message(error)
+    }
+  end
+
+  defp message(%{reason: :invalid_schema_type, type: type, name: name}) do
+    gettext("%{name} - Invalid schema.type. Got: %{type}.",
+      name: name,
+      type: inspect(type)
+    )
+  end
+
+  defp message(%{reason: :null_value, name: name} = error) do
+    case error.type do
+      nil ->
+        gettext("%{name} - null value.", name: name)
+
+      type ->
+        gettext("%{name} - null value where %{type} expected.",
+          name: name,
+          type: type
+        )
+    end
+  end
+
+  defp message(%{reason: :all_of, meta: %{invalid_schema: invalid_schema}}) do
+    gettext(
+      "Failed to cast value as %{invalid_schema}. Value must be castable using `allOf` schemas listed.",
+      invalid_schema: invalid_schema
+    )
+  end
+
+  defp message(%{reason: :any_of, meta: %{failed_schemas: failed_schemas}}) do
+    gettext("Failed to cast value using any of: %{failed_schemas}.",
+      failed_schemas: failed_schemas
+    )
+  end
+
+  defp message(%{reason: :one_of, meta: %{failed_schemas: failed_schemas}}) do
+    gettext("Failed to cast value to one of: %{failed_schemas}.", failed_schemas: failed_schemas)
+  end
+
+  defp message(%{reason: :min_length, length: length, name: name}) do
+    gettext("%{name} - String length is smaller than minLength: %{length}.",
+      name: name,
+      length: length
+    )
+  end
+
+  defp message(%{reason: :max_length, length: length, name: name}) do
+    gettext("%{name} - String length is larger than maxLength: %{length}.",
+      name: name,
+      length: length
+    )
+  end
+
+  defp message(%{reason: :unique_items, name: name}) do
+    gettext("%{name} - Array items must be unique.", name: name)
+  end
+
+  defp message(%{reason: :min_items, length: min, value: array, name: name}) do
+    gettext("%{name} - Array length %{length} is smaller than minItems: %{min}.",
+      name: name,
+      length: length(array),
+      min: min
+    )
+  end
+
+  defp message(%{reason: :max_items, length: max, value: array, name: name}) do
+    gettext("%{name} - Array length %{length} is larger than maxItems: %{}.",
+      name: name,
+      length: length(array),
+      max: max
+    )
+  end
+
+  defp message(%{reason: :multiple_of, length: multiple, value: count, name: name}) do
+    gettext("%{name} - %{count} is not a multiple of %{multiple}.",
+      name: name,
+      count: count,
+      multiple: multiple
+    )
+  end
+
+  defp message(%{reason: :exclusive_max, length: max, value: value, name: name})
+       when value >= max do
+    gettext("%{name} - %{value} is larger than exclusive maximum %{max}.",
+      name: name,
+      value: value,
+      max: max
+    )
+  end
+
+  defp message(%{reason: :maximum, length: max, value: value, name: name})
+       when value > max do
+    gettext("%{name} - %{value} is larger than inclusive maximum %{max}.",
+      name: name,
+      value: value,
+      max: max
+    )
+  end
+
+  defp message(%{reason: :exclusive_multiple, length: min, value: value, name: name})
+       when value <= min do
+    gettext("%{name} - %{value} is smaller than exclusive minimum %{min}.",
+      name: name,
+      value: value,
+      min: min
+    )
+  end
+
+  defp message(%{reason: :minimum, length: min, value: value, name: name})
+       when value < min do
+    gettext("%{name} - %{value} is smaller than inclusive minimum %{min}.",
+      name: name,
+      value: value,
+      min: min
+    )
+  end
+
+  defp message(%{reason: :invalid_type, type: type, value: value, name: name}) do
+    gettext("%{name} - Invalid %{type}. Got: %{value}.",
+      name: name,
+      value: OpenApiSpex.TermType.type(value),
+      type: type
+    )
+  end
+
+  defp message(%{reason: :invalid_format, format: format, name: name}) do
+    gettext("%{name} - Invalid format. Expected %{format}.", name: name, format: inspect(format))
+  end
+
+  defp message(%{reason: :invalid_enum, name: name}) do
+    gettext("%{name} - Invalid value for enum.", name: name)
+  end
+
+  defp message(%{reason: :polymorphic_failed, type: polymorphic_type}) do
+    gettext("Failed to cast to any schema in %{polymorphic_type}",
+      polymorphic_type: polymorphic_type
+    )
+  end
+
+  defp message(%{reason: :unexpected_field, name: name}) do
+    gettext("Unexpected field: %{name}.", name: safe_string(name))
+  end
+
+  defp message(%{reason: :no_value_for_discriminator, name: field}) do
+    gettext("Value used as discriminator for `%{field}` matches no schemas.", name: field)
+  end
+
+  defp message(%{reason: :invalid_discriminator_value, name: field}) do
+    gettext("No value provided for required discriminator `%{field}`.", name: field)
+  end
+
+  defp message(%{reason: :unknown_schema, name: name}) do
+    gettext("Unknown schema: %{name}.", name: name)
+  end
+
+  defp message(%{reason: :missing_field, name: name}) do
+    gettext("Missing field: %{name}.", name: name)
+  end
+
+  defp message(%{reason: :missing_header, name: name}) do
+    gettext("Missing header: %{name}.", name: name)
+  end
+
+  defp message(%{reason: :invalid_header, name: name}) do
+    gettext("Invalid value for header: %{name}.", name: name)
+  end
+
+  defp message(%{reason: :max_properties, meta: meta}) do
+    gettext(
+      "Object property count %{property_count} is greater than maxProperties: %{max_properties}.",
+      property_count: meta.property_count,
+      max_properties: meta.max_properties
+    )
+  end
+
+  defp message(%{reason: :min_properties, meta: meta}) do
+    gettext(
+      "Object property count %{property_count} is less than minProperties: %{min_properties}",
+      property_count: meta.property_count,
+      min_properties: meta.min_properties
+    )
+  end
+
+  defp safe_string(string) do
+    to_string(string) |> String.slice(0..39)
+  end
+end

lib/pleroma/web/api_spec/schemas/account.ex

@@ -0,0 +1,167 @@
+# Pleroma: A lightweight social networking server
+# Copyright © 2017-2020 Pleroma Authors <https://pleroma.social/>
+# SPDX-License-Identifier: AGPL-3.0-only
+
+defmodule Pleroma.Web.ApiSpec.Schemas.Account do
+  alias OpenApiSpex.Schema
+  alias Pleroma.Web.ApiSpec.Schemas.AccountField
+  alias Pleroma.Web.ApiSpec.Schemas.AccountRelationship
+  alias Pleroma.Web.ApiSpec.Schemas.ActorType
+  alias Pleroma.Web.ApiSpec.Schemas.Emoji
+  alias Pleroma.Web.ApiSpec.Schemas.FlakeID
+  alias Pleroma.Web.ApiSpec.Schemas.VisibilityScope
+
+  require OpenApiSpex
+
+  OpenApiSpex.schema(%{
+    title: "Account",
+    description: "Response schema for an account",
+    type: :object,
+    properties: %{
+      acct: %Schema{type: :string},
+      avatar_static: %Schema{type: :string, format: :uri},
+      avatar: %Schema{type: :string, format: :uri},
+      bot: %Schema{type: :boolean},
+      created_at: %Schema{type: :string, format: "date-time"},
+      display_name: %Schema{type: :string},
+      emojis: %Schema{type: :array, items: Emoji},
+      fields: %Schema{type: :array, items: AccountField},
+      follow_requests_count: %Schema{type: :integer},
+      followers_count: %Schema{type: :integer},
+      following_count: %Schema{type: :integer},
+      header_static: %Schema{type: :string, format: :uri},
+      header: %Schema{type: :string, format: :uri},
+      id: FlakeID,
+      locked: %Schema{type: :boolean},
+      note: %Schema{type: :string, format: :html},
+      statuses_count: %Schema{type: :integer},
+      url: %Schema{type: :string, format: :uri},
+      username: %Schema{type: :string},
+      pleroma: %Schema{
+        type: :object,
+        properties: %{
+          allow_following_move: %Schema{type: :boolean},
+          background_image: %Schema{type: :string, nullable: true},
+          chat_token: %Schema{type: :string},
+          confirmation_pending: %Schema{type: :boolean},
+          hide_favorites: %Schema{type: :boolean},
+          hide_followers_count: %Schema{type: :boolean},
+          hide_followers: %Schema{type: :boolean},
+          hide_follows_count: %Schema{type: :boolean},
+          hide_follows: %Schema{type: :boolean},
+          is_admin: %Schema{type: :boolean},
+          is_moderator: %Schema{type: :boolean},
+          skip_thread_containment: %Schema{type: :boolean},
+          tags: %Schema{type: :array, items: %Schema{type: :string}},
+          unread_conversation_count: %Schema{type: :integer},
+          notification_settings: %Schema{
+            type: :object,
+            properties: %{
+              followers: %Schema{type: :boolean},
+              follows: %Schema{type: :boolean},
+              non_followers: %Schema{type: :boolean},
+              non_follows: %Schema{type: :boolean},
+              privacy_option: %Schema{type: :boolean}
+            }
+          },
+          relationship: AccountRelationship,
+          settings_store: %Schema{
+            type: :object
+          }
+        }
+      },
+      source: %Schema{
+        type: :object,
+        properties: %{
+          fields: %Schema{type: :array, items: AccountField},
+          note: %Schema{type: :string},
+          privacy: VisibilityScope,
+          sensitive: %Schema{type: :boolean},
+          pleroma: %Schema{
+            type: :object,
+            properties: %{
+              actor_type: ActorType,
+              discoverable: %Schema{type: :boolean},
+              no_rich_text: %Schema{type: :boolean},
+              show_role: %Schema{type: :boolean}
+            }
+          }
+        }
+      }
+    },
+    example: %{
+      "acct" => "foobar",
+      "avatar" => "https://mypleroma.com/images/avi.png",
+      "avatar_static" => "https://mypleroma.com/images/avi.png",
+      "bot" => false,
+      "created_at" => "2020-03-24T13:05:58.000Z",
+      "display_name" => "foobar",
+      "emojis" => [],
+      "fields" => [],
+      "follow_requests_count" => 0,
+      "followers_count" => 0,
+      "following_count" => 1,
+      "header" => "https://mypleroma.com/images/banner.png",
+      "header_static" => "https://mypleroma.com/images/banner.png",
+      "id" => "9tKi3esbG7OQgZ2920",
+      "locked" => false,
+      "note" => "cofe",
+      "pleroma" => %{
+        "allow_following_move" => true,
+        "background_image" => nil,
+        "confirmation_pending" => true,
+        "hide_favorites" => true,
+        "hide_followers" => false,
+        "hide_followers_count" => false,
+        "hide_follows" => false,
+        "hide_follows_count" => false,
+        "is_admin" => false,
+        "is_moderator" => false,
+        "skip_thread_containment" => false,
+        "chat_token" =>
+          "SFMyNTY.g3QAAAACZAAEZGF0YW0AAAASOXRLaTNlc2JHN09RZ1oyOTIwZAAGc2lnbmVkbgYARNplS3EB.Mb_Iaqew2bN1I1o79B_iP7encmVCpTKC4OtHZRxdjKc",
+        "unread_conversation_count" => 0,
+        "tags" => [],
+        "notification_settings" => %{
+          "followers" => true,
+          "follows" => true,
+          "non_followers" => true,
+          "non_follows" => true,
+          "privacy_option" => false
+        },
+        "relationship" => %{
+          "blocked_by" => false,
+          "blocking" => false,
+          "domain_blocking" => false,
+          "endorsed" => false,
+          "followed_by" => false,
+          "following" => false,
+          "id" => "9tKi3esbG7OQgZ2920",
+          "muting" => false,
+          "muting_notifications" => false,
+          "requested" => false,
+          "showing_reblogs" => true,
+          "subscribing" => false
+        },
+        "settings_store" => %{
+          "pleroma-fe" => %{}
+        }
+      },
+      "source" => %{
+        "fields" => [],
+        "note" => "foobar",
+        "pleroma" => %{
+          "actor_type" => "Person",
+          "discoverable" => false,
+          "no_rich_text" => false,
+          "show_role" => true
+        },
+        "privacy" => "public",
+        "sensitive" => false
+      },
+      "statuses_count" => 0,
+      "url" => "https://mypleroma.com/users/foobar",
+      "username" => "foobar"
+    }
+  })
+end

lib/pleroma/web/api_spec/schemas/account_field.ex

@@ -0,0 +1,26 @@
+# Pleroma: A lightweight social networking server
+# Copyright © 2017-2020 Pleroma Authors <https://pleroma.social/>
+# SPDX-License-Identifier: AGPL-3.0-only
+
+defmodule Pleroma.Web.ApiSpec.Schemas.AccountField do
+  alias OpenApiSpex.Schema
+
+  require OpenApiSpex
+
+  OpenApiSpex.schema(%{
+    title: "AccountField",
+    description: "Response schema for account custom fields",
+    type: :object,
+    properties: %{
+      name: %Schema{type: :string},
+      value: %Schema{type: :string, format: :html},
+      verified_at: %Schema{type: :string, format: :"date-time", nullable: true}
+    },
+    example: %{
+      "name" => "Website",
+      "value" =>
+        "<a href=\"https://pleroma.com\" rel=\"me nofollow noopener noreferrer\" target=\"_blank\"><span class=\"invisible\">https://</span><span class=\"\">pleroma.com</span><span class=\"invisible\"></span></a>",
+      "verified_at" => "2019-08-29T04:14:55.571+00:00"
+    }
+  })
+end

lib/pleroma/web/api_spec/schemas/account_relationship.ex

@@ -0,0 +1,44 @@
+# Pleroma: A lightweight social networking server
+# Copyright © 2017-2020 Pleroma Authors <https://pleroma.social/>
+# SPDX-License-Identifier: AGPL-3.0-only
+
+defmodule Pleroma.Web.ApiSpec.Schemas.AccountRelationship do
+  alias OpenApiSpex.Schema
+  alias Pleroma.Web.ApiSpec.Schemas.FlakeID
+
+  require OpenApiSpex
+
+  OpenApiSpex.schema(%{
+    title: "AccountRelationship",
+    description: "Response schema for relationship",
+    type: :object,
+    properties: %{
+      blocked_by: %Schema{type: :boolean},
+      blocking: %Schema{type: :boolean},
+      domain_blocking: %Schema{type: :boolean},
+      endorsed: %Schema{type: :boolean},
+      followed_by: %Schema{type: :boolean},
+      following: %Schema{type: :boolean},
+      id: FlakeID,
+      muting: %Schema{type: :boolean},
+      muting_notifications: %Schema{type: :boolean},
+      requested: %Schema{type: :boolean},
+      showing_reblogs: %Schema{type: :boolean},
+      subscribing: %Schema{type: :boolean}
+    },
+    example: %{
+      "blocked_by" => false,
+      "blocking" => false,
+      "domain_blocking" => false,
+      "endorsed" => false,
+      "followed_by" => false,
+      "following" => false,
+      "id" => "9tKi3esbG7OQgZ2920",
+      "muting" => false,
+      "muting_notifications" => false,
+      "requested" => false,
+      "showing_reblogs" => true,
+      "subscribing" => false
+    }
+  })
+end

lib/pleroma/web/api_spec/schemas/actor_type.ex

@@ -0,0 +1,13 @@
+# Pleroma: A lightweight social networking server
+# Copyright © 2017-2020 Pleroma Authors <https://pleroma.social/>
+# SPDX-License-Identifier: AGPL-3.0-only
+
+defmodule Pleroma.Web.ApiSpec.Schemas.ActorType do
+  require OpenApiSpex
+
+  OpenApiSpex.schema(%{
+    title: "ActorType",
+    type: :string,
+    enum: ["Application", "Group", "Organization", "Person", "Service"]
+  })
+end

lib/pleroma/web/api_spec/schemas/api_error.ex

@@ -0,0 +1,19 @@
+# Pleroma: A lightweight social networking server
+# Copyright © 2017-2020 Pleroma Authors <https://pleroma.social/>
+# SPDX-License-Identifier: AGPL-3.0-only
+
+defmodule Pleroma.Web.ApiSpec.Schemas.ApiError do
+  alias OpenApiSpex.Schema
+
+  require OpenApiSpex
+
+  OpenApiSpex.schema(%{
+    title: "ApiError",
+    description: "Response schema for API error",
+    type: :object,
+    properties: %{error: %Schema{type: :string}},
+    example: %{
+      "error" => "Something went wrong"
+    }
+  })
+end

lib/pleroma/web/api_spec/schemas/boolean_like.ex

@@ -0,0 +1,36 @@
+# Pleroma: A lightweight social networking server
+# Copyright © 2017-2020 Pleroma Authors <https://pleroma.social/>
+# SPDX-License-Identifier: AGPL-3.0-only
+
+defmodule Pleroma.Web.ApiSpec.Schemas.BooleanLike do
+  alias OpenApiSpex.Schema
+
+  require OpenApiSpex
+
+  OpenApiSpex.schema(%{
+    title: "BooleanLike",
+    description: """
+    The following values will be treated as `false`:
+      - false
+      - 0
+      - "0",
+      - "f",
+      - "F",
+      - "false",
+      - "FALSE",
+      - "off",
+      - "OFF"
+
+    All other non-null values will be treated as `true`
+    """,
+    anyOf: [
+      %Schema{type: :boolean},
+      %Schema{type: :string},
+      %Schema{type: :integer}
+    ]
+  })
+
+  def after_cast(value, _schmea) do
+    {:ok, Pleroma.Web.ControllerHelper.truthy_param?(value)}
+  end
+end

lib/pleroma/web/api_spec/schemas/custom_emoji.ex

@@ -1,30 +0,0 @@
-# Pleroma: A lightweight social networking server
-# Copyright © 2017-2020 Pleroma Authors <https://pleroma.social/>
-# SPDX-License-Identifier: AGPL-3.0-only
-
-defmodule Pleroma.Web.ApiSpec.Schemas.CustomEmoji do
-  alias OpenApiSpex.Schema
-
-  require OpenApiSpex
-
-  OpenApiSpex.schema(%{
-    title: "CustomEmoji",
-    description: "Response schema for an CustomEmoji",
-    type: :object,
-    properties: %{
-      shortcode: %Schema{type: :string},
-      url: %Schema{type: :string},
-      static_url: %Schema{type: :string},
-      visible_in_picker: %Schema{type: :boolean},
-      category: %Schema{type: :string},
-      tags: %Schema{type: :array}
-    },
-    example: %{
-      "shortcode" => "aaaa",
-      "url" => "https://files.mastodon.social/custom_emojis/images/000/007/118/original/aaaa.png",
-      "static_url" =>
-        "https://files.mastodon.social/custom_emojis/images/000/007/118/static/aaaa.png",
-      "visible_in_picker" => true
-    }
-  })
-end

lib/pleroma/web/api_spec/schemas/emoji.ex

@@ -0,0 +1,29 @@
+# Pleroma: A lightweight social networking server
+# Copyright © 2017-2020 Pleroma Authors <https://pleroma.social/>
+# SPDX-License-Identifier: AGPL-3.0-only
+
+defmodule Pleroma.Web.ApiSpec.Schemas.Emoji do
+  alias OpenApiSpex.Schema
+
+  require OpenApiSpex
+
+  OpenApiSpex.schema(%{
+    title: "Emoji",
+    description: "Response schema for an emoji",
+    type: :object,
+    properties: %{
+      shortcode: %Schema{type: :string},
+      url: %Schema{type: :string, format: :uri},
+      static_url: %Schema{type: :string, format: :uri},
+      visible_in_picker: %Schema{type: :boolean}
+    },
+    example: %{
+      "shortcode" => "fatyoshi",
+      "url" =>
+        "https://files.mastodon.social/custom_emojis/images/000/023/920/original/e57ecb623faa0dc9.png",
+      "static_url" =>
+        "https://files.mastodon.social/custom_emojis/images/000/023/920/static/e57ecb623faa0dc9.png",
+      "visible_in_picker" => true
+    }
+  })
+end

lib/pleroma/web/api_spec/schemas/flake_id.ex

@@ -0,0 +1,14 @@
+# Pleroma: A lightweight social networking server
+# Copyright © 2017-2020 Pleroma Authors <https://pleroma.social/>
+# SPDX-License-Identifier: AGPL-3.0-only
+
+defmodule Pleroma.Web.ApiSpec.Schemas.FlakeID do
+  require OpenApiSpex
+
+  OpenApiSpex.schema(%{
+    title: "FlakeID",
+    description:
+      "Pleroma uses 128-bit ids as opposed to Mastodon's 64 bits. However just like Mastodon's ids they are lexically sortable strings",
+    type: :string
+  })
+end

lib/pleroma/web/api_spec/schemas/poll.ex

@@ -0,0 +1,36 @@
+# Pleroma: A lightweight social networking server
+# Copyright © 2017-2020 Pleroma Authors <https://pleroma.social/>
+# SPDX-License-Identifier: AGPL-3.0-only
+
+defmodule Pleroma.Web.ApiSpec.Schemas.Poll do
+  alias OpenApiSpex.Schema
+  alias Pleroma.Web.ApiSpec.Schemas.Emoji
+  alias Pleroma.Web.ApiSpec.Schemas.FlakeID
+
+  require OpenApiSpex
+
+  OpenApiSpex.schema(%{
+    title: "Poll",
+    description: "Response schema for account custom fields",
+    type: :object,
+    properties: %{
+      id: FlakeID,
+      expires_at: %Schema{type: :string, format: "date-time"},
+      expired: %Schema{type: :boolean},
+      multiple: %Schema{type: :boolean},
+      votes_count: %Schema{type: :integer},
+      voted: %Schema{type: :boolean},
+      emojis: %Schema{type: :array, items: Emoji},
+      options: %Schema{
+        type: :array,
+        items: %Schema{
+          type: :object,
+          properties: %{
+            title: %Schema{type: :string},
+            votes_count: %Schema{type: :integer}
+          }
+        }
+      }
+    }
+  })
+end

lib/pleroma/web/api_spec/schemas/status.ex

@@ -0,0 +1,226 @@
+# Pleroma: A lightweight social networking server
+# Copyright © 2017-2020 Pleroma Authors <https://pleroma.social/>
+# SPDX-License-Identifier: AGPL-3.0-only
+
+defmodule Pleroma.Web.ApiSpec.Schemas.Status do
+  alias OpenApiSpex.Schema
+  alias Pleroma.Web.ApiSpec.Schemas.Account
+  alias Pleroma.Web.ApiSpec.Schemas.Emoji
+  alias Pleroma.Web.ApiSpec.Schemas.FlakeID
+  alias Pleroma.Web.ApiSpec.Schemas.Poll
+  alias Pleroma.Web.ApiSpec.Schemas.VisibilityScope
+
+  require OpenApiSpex
+
+  OpenApiSpex.schema(%{
+    title: "Status",
+    description: "Response schema for a status",
+    type: :object,
+    properties: %{
+      account: Account,
+      application: %Schema{
+        type: :object,
+        properties: %{
+          name: %Schema{type: :string},
+          website: %Schema{type: :string, nullable: true, format: :uri}
+        }
+      },
+      bookmarked: %Schema{type: :boolean},
+      card: %Schema{
+        type: :object,
+        nullable: true,
+        properties: %{
+          type: %Schema{type: :string, enum: ["link", "photo", "video", "rich"]},
+          provider_name: %Schema{type: :string, nullable: true},
+          provider_url: %Schema{type: :string, format: :uri},
+          url: %Schema{type: :string, format: :uri},
+          image: %Schema{type: :string, nullable: true, format: :uri},
+          title: %Schema{type: :string},
+          description: %Schema{type: :string}
+        }
+      },
+      content: %Schema{type: :string, format: :html},
+      created_at: %Schema{type: :string, format: "date-time"},
+      emojis: %Schema{type: :array, items: Emoji},
+      favourited: %Schema{type: :boolean},
+      favourites_count: %Schema{type: :integer},
+      id: FlakeID,
+      in_reply_to_account_id: %Schema{type: :string, nullable: true},
+      in_reply_to_id: %Schema{type: :string, nullable: true},
+      language: %Schema{type: :string, nullable: true},
+      media_attachments: %Schema{
+        type: :array,
+        items: %Schema{
+          type: :object,
+          properties: %{
+            id: %Schema{type: :string},
+            url: %Schema{type: :string, format: :uri},
+            remote_url: %Schema{type: :string, format: :uri},
+            preview_url: %Schema{type: :string, format: :uri},
+            text_url: %Schema{type: :string, format: :uri},
+            description: %Schema{type: :string},
+            type: %Schema{type: :string, enum: ["image", "video", "audio", "unknown"]},
+            pleroma: %Schema{
+              type: :object,
+              properties: %{mime_type: %Schema{type: :string}}
+            }
+          }
+        }
+      },
+      mentions: %Schema{
+        type: :array,
+        items: %Schema{
+          type: :object,
+          properties: %{
+            id: %Schema{type: :string},
+            acct: %Schema{type: :string},
+            username: %Schema{type: :string},
+            url: %Schema{type: :string, format: :uri}
+          }
+        }
+      },
+      muted: %Schema{type: :boolean},
+      pinned: %Schema{type: :boolean},
+      pleroma: %Schema{
+        type: :object,
+        properties: %{
+          content: %Schema{type: :object, additionalProperties: %Schema{type: :string}},
+          conversation_id: %Schema{type: :integer},
+          direct_conversation_id: %Schema{type: :string, nullable: true},
+          emoji_reactions: %Schema{
+            type: :array,
+            items: %Schema{
+              type: :object,
+              properties: %{
+                name: %Schema{type: :string},
+                count: %Schema{type: :integer},
+                me: %Schema{type: :boolean}
+              }
+            }
+          },
+          expires_at: %Schema{type: :string, format: "date-time", nullable: true},
+          in_reply_to_account_acct: %Schema{type: :string, nullable: true},
+          local: %Schema{type: :boolean},
+          spoiler_text: %Schema{type: :object, additionalProperties: %Schema{type: :string}},
+          thread_muted: %Schema{type: :boolean}
+        }
+      },
+      poll: %Schema{type: Poll, nullable: true},
+      reblog: %Schema{
+        allOf: [%OpenApiSpex.Reference{"$ref": "#/components/schemas/Status"}],
+        nullable: true
+      },
+      reblogged: %Schema{type: :boolean},
+      reblogs_count: %Schema{type: :integer},
+      replies_count: %Schema{type: :integer},
+      sensitive: %Schema{type: :boolean},
+      spoiler_text: %Schema{type: :string},
+      tags: %Schema{
+        type: :array,
+        items: %Schema{
+          type: :object,
+          properties: %{
+            name: %Schema{type: :string},
+            url: %Schema{type: :string, format: :uri}
+          }
+        }
+      },
+      uri: %Schema{type: :string, format: :uri},
+      url: %Schema{type: :string, nullable: true, format: :uri},
+      visibility: VisibilityScope
+    },
+    example: %{
+      "account" => %{
+        "acct" => "nick6",
+        "avatar" => "http://localhost:4001/images/avi.png",
+        "avatar_static" => "http://localhost:4001/images/avi.png",
+        "bot" => false,
+        "created_at" => "2020-04-07T19:48:51.000Z",
+        "display_name" => "Test テスト User 6",
+        "emojis" => [],
+        "fields" => [],
+        "followers_count" => 1,
+        "following_count" => 0,
+        "header" => "http://localhost:4001/images/banner.png",
+        "header_static" => "http://localhost:4001/images/banner.png",
+        "id" => "9toJCsKN7SmSf3aj5c",
+        "locked" => false,
+        "note" => "Tester Number 6",
+        "pleroma" => %{
+          "background_image" => nil,
+          "confirmation_pending" => false,
+          "hide_favorites" => true,
+          "hide_followers" => false,
+          "hide_followers_count" => false,
+          "hide_follows" => false,
+          "hide_follows_count" => false,
+          "is_admin" => false,
+          "is_moderator" => false,
+          "relationship" => %{
+            "blocked_by" => false,
+            "blocking" => false,
+            "domain_blocking" => false,
+            "endorsed" => false,
+            "followed_by" => false,
+            "following" => true,
+            "id" => "9toJCsKN7SmSf3aj5c",
+            "muting" => false,
+            "muting_notifications" => false,
+            "requested" => false,
+            "showing_reblogs" => true,
+            "subscribing" => false
+          },
+          "skip_thread_containment" => false,
+          "tags" => []
+        },
+        "source" => %{
+          "fields" => [],
+          "note" => "Tester Number 6",
+          "pleroma" => %{"actor_type" => "Person", "discoverable" => false},
+          "sensitive" => false
+        },
+        "statuses_count" => 1,
+        "url" => "http://localhost:4001/users/nick6",
+        "username" => "nick6"
+      },
+      "application" => %{"name" => "Web", "website" => nil},
+      "bookmarked" => false,
+      "card" => nil,
+      "content" => "foobar",
+      "created_at" => "2020-04-07T19:48:51.000Z",
+      "emojis" => [],
+      "favourited" => false,
+      "favourites_count" => 0,
+      "id" => "9toJCu5YZW7O7gfvH6",
+      "in_reply_to_account_id" => nil,
+      "in_reply_to_id" => nil,
+      "language" => nil,
+      "media_attachments" => [],
+      "mentions" => [],
+      "muted" => false,
+      "pinned" => false,
+      "pleroma" => %{
+        "content" => %{"text/plain" => "foobar"},
+        "conversation_id" => 345_972,
+        "direct_conversation_id" => nil,
+        "emoji_reactions" => [],
+        "expires_at" => nil,
+        "in_reply_to_account_acct" => nil,
+        "local" => true,
+        "spoiler_text" => %{"text/plain" => ""},
+        "thread_muted" => false
+      },
+      "poll" => nil,
+      "reblog" => nil,
+      "reblogged" => false,
+      "reblogs_count" => 0,
+      "replies_count" => 0,
+      "sensitive" => false,
+      "spoiler_text" => "",
+      "tags" => [],
+      "uri" => "http://localhost:4001/objects/0f5dad44-0e9e-4610-b377-a2631e499190",
+      "url" => "http://localhost:4001/notice/9toJCu5YZW7O7gfvH6",
+      "visibility" => "private"
+    }
+  })
+end

lib/pleroma/web/api_spec/schemas/visibility_scope.ex

@@ -0,0 +1,14 @@
+# Pleroma: A lightweight social networking server
+# Copyright © 2017-2020 Pleroma Authors <https://pleroma.social/>
+# SPDX-License-Identifier: AGPL-3.0-only
+
+defmodule Pleroma.Web.ApiSpec.Schemas.VisibilityScope do
+  require OpenApiSpex
+
+  OpenApiSpex.schema(%{
+    title: "VisibilityScope",
+    description: "Status visibility",
+    type: :string,
+    enum: ["public", "unlisted", "private", "direct"]
+  })
+end

lib/pleroma/web/common_api/common_api.ex

@@ -7,6 +7,7 @@ defmodule Pleroma.Web.CommonAPI do
   alias Pleroma.ActivityExpiration
   alias Pleroma.Conversation.Participation
   alias Pleroma.FollowingRelationship
+  alias Pleroma.Notification
   alias Pleroma.Object
   alias Pleroma.ThreadMute
   alias Pleroma.User
@@ -42,8 +43,8 @@ def unfollow(follower, unfollowed) do
   end
 
   def accept_follow_request(follower, followed) do
-    with {:ok, follower} <- User.follow(follower, followed),
+    with %Activity{} = follow_activity <- Utils.fetch_latest_follow(follower, followed),
-         %Activity{} = follow_activity <- Utils.fetch_latest_follow(follower, followed),
+         {:ok, follower} <- User.follow(follower, followed),
          {:ok, follow_activity} <- Utils.update_follow_state_for_all(follow_activity, "accept"),
          {:ok, _relationship} <- FollowingRelationship.update(follower, followed, :follow_accept),
          {:ok, _activity} <-
@@ -61,6 +62,7 @@ def reject_follow_request(follower, followed) do
     with %Activity{} = follow_activity <- Utils.fetch_latest_follow(follower, followed),
          {:ok, follow_activity} <- Utils.update_follow_state_for_all(follow_activity, "reject"),
          {:ok, _relationship} <- FollowingRelationship.update(follower, followed, :follow_reject),
+         {:ok, _notifications} <- Notification.dismiss(follow_activity),
          {:ok, _activity} <-
            ActivityPub.reject(%{
              to: [follower.ap_id],
@@ -370,9 +372,9 @@ def thread_muted?(user, activity) do
     ThreadMute.exists?(user.id, activity.data["context"])
   end
 
-  def report(user, %{"account_id" => account_id} = data) do
+  def report(user, data) do
-    with {:ok, account} <- get_reported_account(account_id),
+    with {:ok, account} <- get_reported_account(data.account_id),
-         {:ok, {content_html, _, _}} <- make_report_content_html(data["comment"]),
+         {:ok, {content_html, _, _}} <- make_report_content_html(data[:comment]),
          {:ok, statuses} <- get_report_statuses(account, data) do
       ActivityPub.flag(%{
         context: Utils.generate_context_id(),
@@ -380,13 +382,11 @@ def report(user, %{"account_id" => account_id} = data) do
         account: account,
         statuses: statuses,
         content: content_html,
-        forward: data["forward"] || false
+        forward: Map.get(data, :forward, false)
       })
     end
   end
 
-  def report(_user, _params), do: {:error, dgettext("errors", "Valid `account_id` required")}
-
   defp get_reported_account(account_id) do
     case User.get_cached_by_id(account_id) do
       %User{} = account -> {:ok, account}

lib/pleroma/web/common_api/utils.ex

@@ -504,7 +504,8 @@ def make_report_content_html(comment) do
     end
   end
 
-  def get_report_statuses(%User{ap_id: actor}, %{"status_ids" => status_ids}) do
+  def get_report_statuses(%User{ap_id: actor}, %{status_ids: status_ids})
+      when is_list(status_ids) do
     {:ok, Activity.all_by_actor_and_id(actor, status_ids)}
   end
 

lib/pleroma/web/controller_helper.ex

@@ -82,8 +82,9 @@ def add_link_headers(conn, activities, extra_params) do
     end
   end
 
-  def assign_account_by_id(%{params: %{"id" => id}} = conn, _) do
+  def assign_account_by_id(conn, _) do
-    case Pleroma.User.get_cached_by_id(id) do
+    # TODO: use `conn.params[:id]` only after moving to OpenAPI
+    case Pleroma.User.get_cached_by_id(conn.params[:id] || conn.params["id"]) do
       %Pleroma.User{} = account -> assign(conn, :account, account)
       nil -> Pleroma.Web.MastodonAPI.FallbackController.call(conn, {:error, :not_found}) |> halt()
     end

lib/pleroma/web/endpoint.ex

@@ -5,6 +5,8 @@
 defmodule Pleroma.Web.Endpoint do
   use Phoenix.Endpoint, otp_app: :pleroma
 
+  require Pleroma.Constants
+
   socket("/socket", Pleroma.Web.UserSocket)
 
   plug(Pleroma.Plugs.SetLocalePlug)
@@ -34,8 +36,7 @@ defmodule Pleroma.Web.Endpoint do
     Plug.Static,
     at: "/",
     from: :pleroma,
-    only:
+    only: Pleroma.Constants.static_only_files(),
-      ~w(index.html robots.txt static finmoji emoji packs sounds images instance sw.js sw-pleroma.js favicon.png schemas doc),
     # credo:disable-for-previous-line Credo.Check.Readability.MaxLineLength
     gzip: true,
     cache_control_for_etags: @static_cache_control,

lib/pleroma/web/fallback_redirect_controller.ex

@@ -4,7 +4,9 @@
 
 defmodule Fallback.RedirectController do
   use Pleroma.Web, :controller
+
   require Logger
+
   alias Pleroma.User
   alias Pleroma.Web.Metadata
 

lib/pleroma/web/masto_fe_controller.ex

@@ -5,19 +5,25 @@
 defmodule Pleroma.Web.MastoFEController do
   use Pleroma.Web, :controller
 
+  alias Pleroma.Plugs.EnsurePublicOrAuthenticatedPlug
   alias Pleroma.Plugs.OAuthScopesPlug
   alias Pleroma.User
 
   plug(OAuthScopesPlug, %{scopes: ["write:accounts"]} when action == :put_settings)
 
   # Note: :index action handles attempt of unauthenticated access to private instance with redirect
+  plug(:skip_plug, EnsurePublicOrAuthenticatedPlug when action == :index)
+
   plug(
     OAuthScopesPlug,
-    %{scopes: ["read"], fallback: :proceed_unauthenticated, skip_instance_privacy_check: true}
+    %{scopes: ["read"], fallback: :proceed_unauthenticated}
     when action == :index
   )
 
-  plug(Pleroma.Plugs.EnsurePublicOrAuthenticatedPlug when action not in [:index, :manifest])
+  plug(
+    :skip_plug,
+    [OAuthScopesPlug, EnsurePublicOrAuthenticatedPlug] when action == :manifest
+  )
 
   @doc "GET /web/*path"
   def index(%{assigns: %{user: user, token: token}} = conn, _params)

lib/pleroma/web/mastodon_api/controllers/account_controller.ex

@@ -14,6 +14,7 @@ defmodule Pleroma.Web.MastodonAPI.AccountController do
       skip_relationships?: 1
     ]
 
+  alias Pleroma.Plugs.EnsurePublicOrAuthenticatedPlug
   alias Pleroma.Plugs.OAuthScopesPlug
   alias Pleroma.Plugs.RateLimiter
   alias Pleroma.User
@@ -26,18 +27,28 @@ defmodule Pleroma.Web.MastodonAPI.AccountController do
   alias Pleroma.Web.OAuth.Token
   alias Pleroma.Web.TwitterAPI.TwitterAPI
 
-  plug(:skip_plug, OAuthScopesPlug when action == :identity_proofs)
+  plug(OpenApiSpex.Plug.CastAndValidate, render_error: Pleroma.Web.ApiSpec.RenderError)
+
+  plug(:skip_plug, [OAuthScopesPlug, EnsurePublicOrAuthenticatedPlug] when action == :create)
+
+  plug(:skip_plug, EnsurePublicOrAuthenticatedPlug when action in [:show, :statuses])
 
   plug(
     OAuthScopesPlug,
     %{fallback: :proceed_unauthenticated, scopes: ["read:accounts"]}
-    when action == :show
+    when action in [:show, :followers, :following]
+  )
+
+  plug(
+    OAuthScopesPlug,
+    %{fallback: :proceed_unauthenticated, scopes: ["read:statuses"]}
+    when action == :statuses
   )
 
   plug(
     OAuthScopesPlug,
     %{scopes: ["read:accounts"]}
-    when action in [:endorsements, :verify_credentials, :followers, :following]
+    when action in [:verify_credentials, :endorsements, :identity_proofs]
   )
 
   plug(OAuthScopesPlug, %{scopes: ["write:accounts"]} when action == :update_credentials)
@@ -56,21 +67,15 @@ defmodule Pleroma.Web.MastodonAPI.AccountController do
 
   plug(OAuthScopesPlug, %{scopes: ["read:follows"]} when action == :relationships)
 
-  # Note: :follows (POST /api/v1/follows) is the same as :follow, consider removing :follows
   plug(
     OAuthScopesPlug,
-    %{scopes: ["follow", "write:follows"]} when action in [:follows, :follow, :unfollow]
+    %{scopes: ["follow", "write:follows"]} when action in [:follow_by_uri, :follow, :unfollow]
   )
 
   plug(OAuthScopesPlug, %{scopes: ["follow", "read:mutes"]} when action == :mutes)
 
   plug(OAuthScopesPlug, %{scopes: ["follow", "write:mutes"]} when action in [:mute, :unmute])
 
-  plug(
-    Pleroma.Plugs.EnsurePublicOrAuthenticatedPlug
-    when action not in [:create, :show, :statuses]
-  )
-
   @relationship_actions [:follow, :unfollow]
   @needs_account ~W(followers following lists follow unfollow mute unmute block unblock)a
 
@@ -85,28 +90,12 @@ defmodule Pleroma.Web.MastodonAPI.AccountController do
 
   action_fallback(Pleroma.Web.MastodonAPI.FallbackController)
 
-  @doc "POST /api/v1/accounts"
+  defdelegate open_api_operation(action), to: Pleroma.Web.ApiSpec.AccountOperation
-  def create(
-        %{assigns: %{app: app}} = conn,
-        %{"username" => nickname, "password" => _, "agreement" => true} = params
-      ) do
-    params =
-      params
-      |> Map.take([
-        "email",
-        "captcha_solution",
-        "captcha_token",
-        "captcha_answer_data",
-        "token",
-        "password"
-      ])
-      |> Map.put("nickname", nickname)
-      |> Map.put("fullname", params["fullname"] || nickname)
-      |> Map.put("bio", params["bio"] || "")
-      |> Map.put("confirm", params["password"])
-      |> Map.put("trusted_app", app.trusted)
 
+  @doc "POST /api/v1/accounts"
+  def create(%{assigns: %{app: app}, body_params: params} = conn, _params) do
     with :ok <- validate_email_param(params),
+         :ok <- TwitterAPI.validate_captcha(app, params),
          {:ok, user} <- TwitterAPI.register_user(params, need_confirmation: true),
          {:ok, token} <- Token.create_token(app, user, %{scopes: app.scopes}) do
       json(conn, %{
@@ -116,7 +105,7 @@ def create(
         created_at: Token.Utils.format_created_at(token)
       })
     else
-      {:error, errors} -> json_response(conn, :bad_request, errors)
+      {:error, error} -> json_response(conn, :bad_request, %{error: error})
     end
   end
 
@@ -128,11 +117,11 @@ def create(conn, _) do
     render_error(conn, :forbidden, "Invalid credentials")
   end
 
-  defp validate_email_param(%{"email" => _}), do: :ok
+  defp validate_email_param(%{email: email}) when not is_nil(email), do: :ok
 
   defp validate_email_param(_) do
     case Pleroma.Config.get([:instance, :account_activation_required]) do
-      true -> {:error, %{"error" => "Missing parameters"}}
+      true -> {:error, dgettext("errors", "Missing parameter: %{name}", name: "email")}
       _ -> :ok
     end
   end
@@ -150,7 +139,14 @@ def verify_credentials(%{assigns: %{user: user}} = conn, _) do
   end
 
   @doc "PATCH /api/v1/accounts/update_credentials"
-  def update_credentials(%{assigns: %{user: user}} = conn, params) do
+  def update_credentials(%{assigns: %{user: original_user}, body_params: params} = conn, _params) do
+    user = original_user
+
+    params =
+      params
+      |> Enum.filter(fn {_, value} -> not is_nil(value) end)
+      |> Enum.into(%{})
+
     user_params =
       [
         :no_rich_text,
@@ -166,22 +162,22 @@ def update_credentials(%{assigns: %{user: user}} = conn, params) do
         :discoverable
       ]
       |> Enum.reduce(%{}, fn key, acc ->
-        add_if_present(acc, params, to_string(key), key, &{:ok, truthy_param?(&1)})
+        add_if_present(acc, params, key, key, &{:ok, truthy_param?(&1)})
       end)
-      |> add_if_present(params, "display_name", :name)
+      |> add_if_present(params, :display_name, :name)
-      |> add_if_present(params, "note", :bio)
+      |> add_if_present(params, :note, :bio)
-      |> add_if_present(params, "avatar", :avatar)
+      |> add_if_present(params, :avatar, :avatar)
-      |> add_if_present(params, "header", :banner)
+      |> add_if_present(params, :header, :banner)
-      |> add_if_present(params, "pleroma_background_image", :background)
+      |> add_if_present(params, :pleroma_background_image, :background)
       |> add_if_present(
         params,
-        "fields_attributes",
+        :fields_attributes,
         :raw_fields,
         &{:ok, normalize_fields_attributes(&1)}
       )
-      |> add_if_present(params, "pleroma_settings_store", :pleroma_settings_store)
+      |> add_if_present(params, :pleroma_settings_store, :pleroma_settings_store)
-      |> add_if_present(params, "default_scope", :default_scope)
+      |> add_if_present(params, :default_scope, :default_scope)
-      |> add_if_present(params, "actor_type", :actor_type)
+      |> add_if_present(params, :actor_type, :actor_type)
 
     changeset = User.update_changeset(user, user_params)
 
@@ -194,7 +190,7 @@ def update_credentials(%{assigns: %{user: user}} = conn, params) do
 
   defp add_if_present(map, params, params_field, map_field, value_function \\ &{:ok, &1}) do
     with true <- Map.has_key?(params, params_field),
-         {:ok, new_value} <- value_function.(params[params_field]) do
+         {:ok, new_value} <- value_function.(Map.get(params, params_field)) do
       Map.put(map, map_field, new_value)
     else
       _ -> map
@@ -205,12 +201,15 @@ defp normalize_fields_attributes(fields) do
     if Enum.all?(fields, &is_tuple/1) do
       Enum.map(fields, fn {_, v} -> v end)
     else
-      fields
+      Enum.map(fields, fn
+        %{} = field -> %{"name" => field.name, "value" => field.value}
+        field -> field
+      end)
     end
   end
 
   @doc "GET /api/v1/accounts/relationships"
-  def relationships(%{assigns: %{user: user}} = conn, %{"id" => id}) do
+  def relationships(%{assigns: %{user: user}} = conn, %{id: id}) do
     targets = User.get_all_by_ids(List.wrap(id))
 
     render(conn, "relationships.json", user: user, targets: targets)
@@ -220,7 +219,7 @@ def relationships(%{assigns: %{user: user}} = conn, %{"id" => id}) do
   def relationships(%{assigns: %{user: _user}} = conn, _), do: json(conn, [])
 
   @doc "GET /api/v1/accounts/:id"
-  def show(%{assigns: %{user: for_user}} = conn, %{"id" => nickname_or_id}) do
+  def show(%{assigns: %{user: for_user}} = conn, %{id: nickname_or_id}) do
     with %User{} = user <- User.get_cached_by_nickname_or_id(nickname_or_id, for: for_user),
          true <- User.visible_for?(user, for_user) do
       render(conn, "show.json", user: user, for: for_user)
@@ -231,12 +230,14 @@ def show(%{assigns: %{user: for_user}} = conn, %{"id" => nickname_or_id}) do
 
   @doc "GET /api/v1/accounts/:id/statuses"
   def statuses(%{assigns: %{user: reading_user}} = conn, params) do
-    with %User{} = user <- User.get_cached_by_nickname_or_id(params["id"], for: reading_user),
+    with %User{} = user <- User.get_cached_by_nickname_or_id(params.id, for: reading_user),
          true <- User.visible_for?(user, reading_user) do
       params =
         params
-        |> Map.put("tag", params["tagged"])
+        |> Map.delete(:tagged)
-        |> Map.delete("godmode")
+        |> Enum.filter(&(not is_nil(&1)))
+        |> Map.new(fn {key, value} -> {to_string(key), value} end)
+        |> Map.put("tag", params[:tagged])
 
       activities = ActivityPub.fetch_user_activities(user, reading_user, params)
 
@@ -256,6 +257,11 @@ def statuses(%{assigns: %{user: reading_user}} = conn, params) do
 
   @doc "GET /api/v1/accounts/:id/followers"
   def followers(%{assigns: %{user: for_user, account: user}} = conn, params) do
+    params =
+      params
+      |> Enum.map(fn {key, value} -> {to_string(key), value} end)
+      |> Enum.into(%{})
+
     followers =
       cond do
         for_user && user.id == for_user.id -> MastodonAPI.get_followers(user, params)
@@ -270,6 +276,11 @@ def followers(%{assigns: %{user: for_user, account: user}} = conn, params) do
 
   @doc "GET /api/v1/accounts/:id/following"
   def following(%{assigns: %{user: for_user, account: user}} = conn, params) do
+    params =
+      params
+      |> Enum.map(fn {key, value} -> {to_string(key), value} end)
+      |> Enum.into(%{})
+
     followers =
       cond do
         for_user && user.id == for_user.id -> MastodonAPI.get_friends(user, params)
@@ -296,8 +307,8 @@ def follow(%{assigns: %{user: %{id: id}, account: %{id: id}}}, _params) do
     {:error, "Can not follow yourself"}
   end
 
-  def follow(%{assigns: %{user: follower, account: followed}} = conn, _params) do
+  def follow(%{assigns: %{user: follower, account: followed}} = conn, params) do
-    with {:ok, follower} <- MastodonAPI.follow(follower, followed, conn.params) do
+    with {:ok, follower} <- MastodonAPI.follow(follower, followed, params) do
       render(conn, "relationship.json", user: follower, target: followed)
     else
       {:error, message} -> json_response(conn, :forbidden, %{error: message})
@@ -316,10 +327,8 @@ def unfollow(%{assigns: %{user: follower, account: followed}} = conn, _params) d
   end
 
   @doc "POST /api/v1/accounts/:id/mute"
-  def mute(%{assigns: %{user: muter, account: muted}} = conn, params) do
+  def mute(%{assigns: %{user: muter, account: muted}, body_params: params} = conn, _params) do
-    notifications? = params |> Map.get("notifications", true) |> truthy_param?()
+    with {:ok, _user_relationships} <- User.mute(muter, muted, params.notifications) do
-
-    with {:ok, _user_relationships} <- User.mute(muter, muted, notifications?) do
       render(conn, "relationship.json", user: muter, target: muted)
     else
       {:error, message} -> json_response(conn, :forbidden, %{error: message})
@@ -356,7 +365,7 @@ def unblock(%{assigns: %{user: blocker, account: blocked}} = conn, _params) do
   end
 
   @doc "POST /api/v1/follows"
-  def follows(conn, %{"uri" => uri}) do
+  def follow_by_uri(%{body_params: %{uri: uri}} = conn, _) do
     case User.get_cached_by_nickname(uri) do
       %User{} = user ->
         conn

lib/pleroma/web/mastodon_api/controllers/app_controller.ex

@@ -5,6 +5,7 @@
 defmodule Pleroma.Web.MastodonAPI.AppController do
   use Pleroma.Web, :controller
 
+  alias Pleroma.Plugs.EnsurePublicOrAuthenticatedPlug
   alias Pleroma.Plugs.OAuthScopesPlug
   alias Pleroma.Repo
   alias Pleroma.Web.OAuth.App
@@ -13,7 +14,14 @@ defmodule Pleroma.Web.MastodonAPI.AppController do
 
   action_fallback(Pleroma.Web.MastodonAPI.FallbackController)
 
+  plug(
+    :skip_plug,
+    [OAuthScopesPlug, EnsurePublicOrAuthenticatedPlug]
+    when action == :create
+  )
+
   plug(OAuthScopesPlug, %{scopes: ["read"]} when action == :verify_credentials)
+
   plug(OpenApiSpex.Plug.CastAndValidate)
 
   @local_mastodon_name "Mastodon-Local"

lib/pleroma/web/mastodon_api/controllers/auth_controller.ex

@@ -13,10 +13,10 @@ defmodule Pleroma.Web.MastodonAPI.AuthController do
 
   action_fallback(Pleroma.Web.MastodonAPI.FallbackController)
 
-  @local_mastodon_name "Mastodon-Local"
-
   plug(Pleroma.Plugs.RateLimiter, [name: :password_reset] when action == :password_reset)
 
+  @local_mastodon_name "Mastodon-Local"
+
   @doc "GET /web/login"
   def login(%{assigns: %{user: %User{}}} = conn, _params) do
     redirect(conn, to: local_mastodon_root_path(conn))

lib/pleroma/web/mastodon_api/controllers/conversation_controller.ex

@@ -14,9 +14,7 @@ defmodule Pleroma.Web.MastodonAPI.ConversationController do
   action_fallback(Pleroma.Web.MastodonAPI.FallbackController)
 
   plug(OAuthScopesPlug, %{scopes: ["read:statuses"]} when action == :index)
-  plug(OAuthScopesPlug, %{scopes: ["write:conversations"]} when action == :read)
+  plug(OAuthScopesPlug, %{scopes: ["write:conversations"]} when action != :index)
-
-  plug(Pleroma.Plugs.EnsurePublicOrAuthenticatedPlug)
 
   @doc "GET /api/v1/conversations"
   def index(%{assigns: %{user: user}} = conn, params) do
@@ -28,7 +26,7 @@ def index(%{assigns: %{user: user}} = conn, params) do
   end
 
   @doc "POST /api/v1/conversations/:id/read"
-  def read(%{assigns: %{user: user}} = conn, %{"id" => participation_id}) do
+  def mark_as_read(%{assigns: %{user: user}} = conn, %{"id" => participation_id}) do
     with %Participation{} = participation <-
            Repo.get_by(Participation, id: participation_id, user_id: user.id),
          {:ok, participation} <- Participation.mark_as_read(participation) do

lib/pleroma/web/mastodon_api/controllers/custom_emoji_controller.ex

@@ -7,6 +7,12 @@ defmodule Pleroma.Web.MastodonAPI.CustomEmojiController do
 
   plug(OpenApiSpex.Plug.CastAndValidate)
 
+  plug(
+    :skip_plug,
+    [Pleroma.Plugs.OAuthScopesPlug, Pleroma.Plugs.EnsurePublicOrAuthenticatedPlug]
+    when action == :index
+  )
+
   defdelegate open_api_operation(action), to: Pleroma.Web.ApiSpec.CustomEmojiOperation
 
   def index(conn, _params) do

lib/pleroma/web/mastodon_api/controllers/domain_block_controller.ex

@@ -21,8 +21,6 @@ defmodule Pleroma.Web.MastodonAPI.DomainBlockController do
     %{scopes: ["follow", "write:blocks"]} when action != :index
   )
 
-  plug(Pleroma.Plugs.EnsurePublicOrAuthenticatedPlug)
-
   @doc "GET /api/v1/domain_blocks"
   def index(%{assigns: %{user: user}} = conn, _) do
     json(conn, Map.get(user, :domain_blocks, []))

lib/pleroma/web/mastodon_api/controllers/filter_controller.ex

@@ -17,8 +17,6 @@ defmodule Pleroma.Web.MastodonAPI.FilterController do
     %{scopes: ["write:filters"]} when action not in @oauth_read_actions
   )
 
-  plug(Pleroma.Plugs.EnsurePublicOrAuthenticatedPlug)
-
   @doc "GET /api/v1/filters"
   def index(%{assigns: %{user: user}} = conn, _) do
     filters = Filter.get_filters(user)

lib/pleroma/web/mastodon_api/controllers/follow_request_controller.ex

@@ -21,8 +21,6 @@ defmodule Pleroma.Web.MastodonAPI.FollowRequestController do
     %{scopes: ["follow", "write:follows"]} when action != :index
   )
 
-  plug(Pleroma.Plugs.EnsurePublicOrAuthenticatedPlug)
-
   @doc "GET /api/v1/follow_requests"
   def index(%{assigns: %{user: followed}} = conn, _params) do
     follow_requests = User.get_follow_requests(followed)

lib/pleroma/web/mastodon_api/controllers/instance_controller.ex

@@ -5,6 +5,12 @@
 defmodule Pleroma.Web.MastodonAPI.InstanceController do
   use Pleroma.Web, :controller
 
+  plug(
+    :skip_plug,
+    [Pleroma.Plugs.OAuthScopesPlug, Pleroma.Plugs.EnsurePublicOrAuthenticatedPlug]
+    when action in [:show, :peers]
+  )
+
   @doc "GET /api/v1/instance"
   def show(conn, _params) do
     render(conn, "show.json")

lib/pleroma/web/mastodon_api/controllers/list_controller.ex

@@ -11,16 +11,16 @@ defmodule Pleroma.Web.MastodonAPI.ListController do
 
   plug(:list_by_id_and_user when action not in [:index, :create])
 
-  plug(OAuthScopesPlug, %{scopes: ["read:lists"]} when action in [:index, :show, :list_accounts])
+  @oauth_read_actions [:index, :show, :list_accounts]
+
+  plug(OAuthScopesPlug, %{scopes: ["read:lists"]} when action in @oauth_read_actions)
 
   plug(
     OAuthScopesPlug,
     %{scopes: ["write:lists"]}
-    when action in [:create, :update, :delete, :add_to_list, :remove_from_list]
+    when action not in @oauth_read_actions
   )
 
-  plug(Pleroma.Plugs.EnsurePublicOrAuthenticatedPlug)
-
   action_fallback(Pleroma.Web.MastodonAPI.FallbackController)
 
   # GET /api/v1/lists

lib/pleroma/web/mastodon_api/controllers/marker_controller.ex

@@ -13,7 +13,7 @@ defmodule Pleroma.Web.MastodonAPI.MarkerController do
   )
 
   plug(OAuthScopesPlug, %{scopes: ["write:statuses"]} when action == :upsert)
-  plug(Pleroma.Plugs.EnsurePublicOrAuthenticatedPlug)
+
   action_fallback(Pleroma.Web.MastodonAPI.FallbackController)
 
   # GET /api/v1/markers

lib/pleroma/web/mastodon_api/controllers/mastodon_api_controller.ex

@@ -15,9 +15,11 @@ defmodule Pleroma.Web.MastodonAPI.MastodonAPIController do
 
   require Logger
 
-  plug(:skip_plug, Pleroma.Plugs.OAuthScopesPlug when action in [:empty_array, :empty_object])
+  plug(
-
+    :skip_plug,
-  plug(Pleroma.Plugs.EnsurePublicOrAuthenticatedPlug)
+    [Pleroma.Plugs.OAuthScopesPlug, Pleroma.Plugs.EnsurePublicOrAuthenticatedPlug]
+    when action in [:empty_array, :empty_object]
+  )
 
   action_fallback(Pleroma.Web.MastodonAPI.FallbackController)
 

lib/pleroma/web/mastodon_api/controllers/media_controller.ex

@@ -15,8 +15,6 @@ defmodule Pleroma.Web.MastodonAPI.MediaController do
 
   plug(OAuthScopesPlug, %{scopes: ["write:media"]})
 
-  plug(Pleroma.Plugs.EnsurePublicOrAuthenticatedPlug)
-
   @doc "POST /api/v1/media"
   def create(%{assigns: %{user: user}} = conn, %{"file" => file} = data) do
     with {:ok, object} <-

lib/pleroma/web/mastodon_api/controllers/notification_controller.ex

@@ -13,6 +13,8 @@ defmodule Pleroma.Web.MastodonAPI.NotificationController do
 
   @oauth_read_actions [:show, :index]
 
+  plug(OpenApiSpex.Plug.CastAndValidate, render_error: Pleroma.Web.ApiSpec.RenderError)
+
   plug(
     OAuthScopesPlug,
     %{scopes: ["read:notifications"]} when action in @oauth_read_actions
@@ -20,16 +22,16 @@ defmodule Pleroma.Web.MastodonAPI.NotificationController do
 
   plug(OAuthScopesPlug, %{scopes: ["write:notifications"]} when action not in @oauth_read_actions)
 
-  plug(Pleroma.Plugs.EnsurePublicOrAuthenticatedPlug)
+  defdelegate open_api_operation(action), to: Pleroma.Web.ApiSpec.NotificationOperation
 
   # GET /api/v1/notifications
-  def index(conn, %{"account_id" => account_id} = params) do
+  def index(conn, %{account_id: account_id} = params) do
     case Pleroma.User.get_cached_by_id(account_id) do
       %{ap_id: account_ap_id} ->
         params =
           params
-          |> Map.delete("account_id")
+          |> Map.delete(:account_id)
-          |> Map.put("account_ap_id", account_ap_id)
+          |> Map.put(:account_ap_id, account_ap_id)
 
         index(conn, params)
 
@@ -41,6 +43,7 @@ def index(conn, %{"account_id" => account_id} = params) do
   end
 
   def index(%{assigns: %{user: user}} = conn, params) do
+    params = Map.new(params, fn {k, v} -> {to_string(k), v} end)
     notifications = MastodonAPI.get_notifications(user, params)
 
     conn
@@ -53,7 +56,7 @@ def index(%{assigns: %{user: user}} = conn, params) do
   end
 
   # GET /api/v1/notifications/:id
-  def show(%{assigns: %{user: user}} = conn, %{"id" => id}) do
+  def show(%{assigns: %{user: user}} = conn, %{id: id}) do
     with {:ok, notification} <- Notification.get(user, id) do
       render(conn, "show.json", notification: notification, for: user)
     else
@@ -71,8 +74,8 @@ def clear(%{assigns: %{user: user}} = conn, _params) do
   end
 
   # POST /api/v1/notifications/:id/dismiss
-  # POST /api/v1/notifications/dismiss (deprecated)
+
-  def dismiss(%{assigns: %{user: user}} = conn, %{"id" => id} = _params) do
+  def dismiss(%{assigns: %{user: user}} = conn, %{id: id} = _params) do
     with {:ok, _notif} <- Notification.dismiss(user, id) do
       json(conn, %{})
     else
@@ -83,8 +86,13 @@ def dismiss(%{assigns: %{user: user}} = conn, %{"id" => id} = _params) do
     end
   end
 
+  # POST /api/v1/notifications/dismiss (deprecated)
+  def dismiss_via_body(%{body_params: params} = conn, _) do
+    dismiss(conn, params)
+  end
+
   # DELETE /api/v1/notifications/destroy_multiple
-  def destroy_multiple(%{assigns: %{user: user}} = conn, %{"ids" => ids} = _params) do
+  def destroy_multiple(%{assigns: %{user: user}} = conn, %{ids: ids} = _params) do
     Notification.destroy_multiple(user, ids)
     json(conn, %{})
   end

lib/pleroma/web/mastodon_api/controllers/poll_controller.ex

@@ -22,8 +22,6 @@ defmodule Pleroma.Web.MastodonAPI.PollController do
 
   plug(OAuthScopesPlug, %{scopes: ["write:statuses"]} when action == :vote)
 
-  plug(Pleroma.Plugs.EnsurePublicOrAuthenticatedPlug)
-
   @doc "GET /api/v1/polls/:id"
   def show(%{assigns: %{user: user}} = conn, %{"id" => id}) do
     with %Object{} = object <- Object.get_by_id_and_maybe_refetch(id, interval: 60),

lib/pleroma/web/mastodon_api/controllers/report_controller.ex

@@ -9,12 +9,13 @@ defmodule Pleroma.Web.MastodonAPI.ReportController do
 
   action_fallback(Pleroma.Web.MastodonAPI.FallbackController)
 
+  plug(OpenApiSpex.Plug.CastAndValidate, render_error: Pleroma.Web.ApiSpec.RenderError)
   plug(OAuthScopesPlug, %{scopes: ["write:reports"]} when action == :create)
 
-  plug(Pleroma.Plugs.EnsurePublicOrAuthenticatedPlug)
+  defdelegate open_api_operation(action), to: Pleroma.Web.ApiSpec.ReportOperation
 
   @doc "POST /api/v1/reports"
-  def create(%{assigns: %{user: user}} = conn, params) do
+  def create(%{assigns: %{user: user}, body_params: params} = conn, _) do
     with {:ok, activity} <- Pleroma.Web.CommonAPI.report(user, params) do
       render(conn, "show.json", activity: activity)
     end

lib/pleroma/web/mastodon_api/controllers/scheduled_activity_controller.ex

@@ -18,8 +18,6 @@ defmodule Pleroma.Web.MastodonAPI.ScheduledActivityController do
   plug(OAuthScopesPlug, %{scopes: ["read:statuses"]} when action in @oauth_read_actions)
   plug(OAuthScopesPlug, %{scopes: ["write:statuses"]} when action not in @oauth_read_actions)
 
-  plug(Pleroma.Plugs.EnsurePublicOrAuthenticatedPlug)
-
   action_fallback(Pleroma.Web.MastodonAPI.FallbackController)
 
   @doc "GET /api/v1/scheduled_statuses"

lib/pleroma/web/mastodon_api/controllers/search_controller.ex

@@ -21,7 +21,7 @@ defmodule Pleroma.Web.MastodonAPI.SearchController do
   # Note: Mastodon doesn't allow unauthenticated access (requires read:accounts / read:search)
   plug(OAuthScopesPlug, %{scopes: ["read:search"], fallback: :proceed_unauthenticated})
 
-  plug(Pleroma.Plugs.EnsurePublicOrAuthenticatedPlug)
+  # Note: on private instances auth is required (EnsurePublicOrAuthenticatedPlug is not skipped)
 
   plug(RateLimiter, [name: :search] when action in [:search, :search2, :account_search])
 

lib/pleroma/web/mastodon_api/controllers/status_controller.ex

@@ -24,6 +24,8 @@ defmodule Pleroma.Web.MastodonAPI.StatusController do
   alias Pleroma.Web.MastodonAPI.AccountView
   alias Pleroma.Web.MastodonAPI.ScheduledActivityView
 
+  plug(:skip_plug, Pleroma.Plugs.EnsurePublicOrAuthenticatedPlug when action in [:index, :show])
+
   @unauthenticated_access %{fallback: :proceed_unauthenticated, scopes: []}
 
   plug(
@@ -77,8 +79,6 @@ defmodule Pleroma.Web.MastodonAPI.StatusController do
     %{scopes: ["write:bookmarks"]} when action in [:bookmark, :unbookmark]
   )
 
-  plug(Pleroma.Plugs.EnsurePublicOrAuthenticatedPlug when action not in [:index, :show])
-
   @rate_limited_status_actions ~w(reblog unreblog favourite unfavourite create delete)a
 
   plug(
@@ -358,7 +358,7 @@ def context(%{assigns: %{user: user}} = conn, %{"id" => id}) do
   end
 
   @doc "GET /api/v1/favourites"
-  def favourites(%{assigns: %{user: user}} = conn, params) do
+  def favourites(%{assigns: %{user: %User{} = user}} = conn, params) do
     activities =
       ActivityPub.fetch_favourites(
         user,

lib/pleroma/web/mastodon_api/controllers/subscription_controller.ex

@@ -12,7 +12,7 @@ defmodule Pleroma.Web.MastodonAPI.SubscriptionController do
   action_fallback(:errors)
 
   plug(Pleroma.Plugs.OAuthScopesPlug, %{scopes: ["push"]})
-  plug(Pleroma.Plugs.EnsurePublicOrAuthenticatedPlug)
+
   plug(:restrict_push_enabled)
 
   # Creates PushSubscription

lib/pleroma/web/mastodon_api/controllers/timeline_controller.ex

@@ -9,11 +9,14 @@ defmodule Pleroma.Web.MastodonAPI.TimelineController do
     only: [add_link_headers: 2, add_link_headers: 3, truthy_param?: 1, skip_relationships?: 1]
 
   alias Pleroma.Pagination
+  alias Pleroma.Plugs.EnsurePublicOrAuthenticatedPlug
   alias Pleroma.Plugs.OAuthScopesPlug
   alias Pleroma.Plugs.RateLimiter
   alias Pleroma.User
   alias Pleroma.Web.ActivityPub.ActivityPub
 
+  plug(:skip_plug, EnsurePublicOrAuthenticatedPlug when action in [:public, :hashtag])
+
   # TODO: Replace with a macro when there is a Phoenix release with the following commit in it:
   # https://github.com/phoenixframework/phoenix/commit/2e8c63c01fec4dde5467dbbbf9705ff9e780735e
 
@@ -26,7 +29,11 @@ defmodule Pleroma.Web.MastodonAPI.TimelineController do
   plug(OAuthScopesPlug, %{scopes: ["read:statuses"]} when action in [:home, :direct])
   plug(OAuthScopesPlug, %{scopes: ["read:lists"]} when action == :list)
 
-  plug(Pleroma.Plugs.EnsurePublicOrAuthenticatedPlug when action != :public)
+  plug(
+    OAuthScopesPlug,
+    %{scopes: ["read:statuses"], fallback: :proceed_unauthenticated}
+    when action in [:public, :hashtag]
+  )
 
   plug(:put_view, Pleroma.Web.MastodonAPI.StatusView)
 
@@ -94,7 +101,9 @@ def public(%{assigns: %{user: user}} = conn, params) do
 
     restrict? = Pleroma.Config.get([:restrict_unauthenticated, :timelines, cfg_key])
 
-    if not (restrict? and is_nil(user)) do
+    if restrict? and is_nil(user) do
+      render_error(conn, :unauthorized, "authorization required for timeline view")
+    else
       activities =
         params
         |> Map.put("type", ["Create", "Announce"])
@@ -112,12 +121,10 @@ def public(%{assigns: %{user: user}} = conn, params) do
         as: :activity,
         skip_relationships: skip_relationships?(params)
       )
-    else
-      render_error(conn, :unauthorized, "authorization required for timeline view")
     end
   end
 
-  def hashtag_fetching(params, user, local_only) do
+  defp hashtag_fetching(params, user, local_only) do
     tags =
       [params["tag"], params["any"]]
       |> List.flatten()

lib/pleroma/web/mastodon_api/views/instance_view.ex

@@ -5,10 +5,13 @@
 defmodule Pleroma.Web.MastodonAPI.InstanceView do
   use Pleroma.Web, :view
 
+  alias Pleroma.Config
+  alias Pleroma.Web.ActivityPub.MRF
+
   @mastodon_api_level "2.7.2"
 
   def render("show.json", _) do
-    instance = Pleroma.Config.get(:instance)
+    instance = Config.get(:instance)
 
     %{
       uri: Pleroma.Web.base_url(),
@@ -29,7 +32,58 @@ def render("show.json", _) do
       upload_limit: Keyword.get(instance, :upload_limit),
       avatar_upload_limit: Keyword.get(instance, :avatar_upload_limit),
       background_upload_limit: Keyword.get(instance, :background_upload_limit),
-      banner_upload_limit: Keyword.get(instance, :banner_upload_limit)
+      banner_upload_limit: Keyword.get(instance, :banner_upload_limit),
+      pleroma: %{
+        metadata: %{
+          features: features(),
+          federation: federation()
+        },
+        vapid_public_key: Keyword.get(Pleroma.Web.Push.vapid_config(), :public_key)
+      }
     }
   end
+
+  def features do
+    [
+      "pleroma_api",
+      "mastodon_api",
+      "mastodon_api_streaming",
+      "polls",
+      "pleroma_explicit_addressing",
+      "shareable_emoji_packs",
+      "multifetch",
+      "pleroma:api/v1/notifications:include_types_filter",
+      if Config.get([:media_proxy, :enabled]) do
+        "media_proxy"
+      end,
+      if Config.get([:gopher, :enabled]) do
+        "gopher"
+      end,
+      if Config.get([:chat, :enabled]) do
+        "chat"
+      end,
+      if Config.get([:instance, :allow_relay]) do
+        "relay"
+      end,
+      if Config.get([:instance, :safe_dm_mentions]) do
+        "safe_dm_mentions"
+      end,
+      "pleroma_emoji_reactions"
+    ]
+    |> Enum.filter(& &1)
+  end
+
+  def federation do
+    quarantined = Config.get([:instance, :quarantined_instances], [])
+
+    if Config.get([:instance, :mrf_transparency]) do
+      {:ok, data} = MRF.describe()
+
+      data
+      |> Map.merge(%{quarantined_instances: quarantined})
+    else
+      %{}
+    end
+    |> Map.put(:enabled, Config.get([:instance, :federating]))
+  end
 end

lib/pleroma/web/mastodon_api/views/status_view.ex

@@ -529,11 +529,9 @@ def render_content(object), do: object.data["content"] || ""
   """
   @spec build_tags(list(any())) :: list(map())
   def build_tags(object_tags) when is_list(object_tags) do
-    object_tags = for tag when is_binary(tag) <- object_tags, do: tag
+    object_tags
-
+    |> Enum.filter(&is_binary/1)
-    Enum.reduce(object_tags, [], fn tag, tags ->
+    |> Enum.map(&%{name: &1, url: "/tag/#{URI.encode(&1)}"})
-      tags ++ [%{name: tag, url: "/tag/#{URI.encode(tag)}"}]
-    end)
   end
 
   def build_tags(_), do: []

lib/pleroma/web/media_proxy/media_proxy_controller.ex

@@ -4,6 +4,7 @@
 
 defmodule Pleroma.Web.MediaProxy.MediaProxyController do
   use Pleroma.Web, :controller
+
   alias Pleroma.ReverseProxy
   alias Pleroma.Web.MediaProxy
 

lib/pleroma/web/nodeinfo/nodeinfo_controller.ex

@@ -9,8 +9,8 @@ defmodule Pleroma.Web.Nodeinfo.NodeinfoController do
   alias Pleroma.Stats
   alias Pleroma.User
   alias Pleroma.Web
-  alias Pleroma.Web.ActivityPub.MRF
   alias Pleroma.Web.Federator.Publisher
+  alias Pleroma.Web.MastodonAPI.InstanceView
 
   def schemas(conn, _params) do
     response = %{
@@ -34,51 +34,12 @@ def schemas(conn, _params) do
   def raw_nodeinfo do
     stats = Stats.get_stats()
 
-    quarantined = Config.get([:instance, :quarantined_instances], [])
-
     staff_accounts =
       User.all_superusers()
       |> Enum.map(fn u -> u.ap_id end)
 
-    federation_response =
+    features = InstanceView.features()
-      if Config.get([:instance, :mrf_transparency]) do
+    federation = InstanceView.federation()
-        {:ok, data} = MRF.describe()
-
-        data
-        |> Map.merge(%{quarantined_instances: quarantined})
-      else
-        %{}
-      end
-      |> Map.put(:enabled, Config.get([:instance, :federating]))
-
-    features =
-      [
-        "pleroma_api",
-        "mastodon_api",
-        "mastodon_api_streaming",
-        "polls",
-        "pleroma_explicit_addressing",
-        "shareable_emoji_packs",
-        "multifetch",
-        "pleroma:api/v1/notifications:include_types_filter",
-        if Config.get([:media_proxy, :enabled]) do
-          "media_proxy"
-        end,
-        if Config.get([:gopher, :enabled]) do
-          "gopher"
-        end,
-        if Config.get([:chat, :enabled]) do
-          "chat"
-        end,
-        if Config.get([:instance, :allow_relay]) do
-          "relay"
-        end,
-        if Config.get([:instance, :safe_dm_mentions]) do
-          "safe_dm_mentions"
-        end,
-        "pleroma_emoji_reactions"
-      ]
-      |> Enum.filter(& &1)
 
     %{
       version: "2.0",
@@ -106,7 +67,7 @@ def raw_nodeinfo do
           enabled: false
         },
         staffAccounts: staff_accounts,
-        federation: federation_response,
+        federation: federation,
         pollLimits: Config.get([:instance, :poll_limits]),
         postFormats: Config.get([:instance, :allowed_post_formats]),
         uploadLimits: %{

lib/pleroma/web/oauth/oauth_controller.ex

@@ -25,9 +25,10 @@ defmodule Pleroma.Web.OAuth.OAuthController do
 
   plug(:fetch_session)
   plug(:fetch_flash)
-  plug(RateLimiter, [name: :authentication] when action == :create_authorization)
 
-  plug(:skip_plug, Pleroma.Plugs.OAuthScopesPlug)
+  plug(:skip_plug, [Pleroma.Plugs.OAuthScopesPlug, Pleroma.Plugs.EnsurePublicOrAuthenticatedPlug])
+
+  plug(RateLimiter, [name: :authentication] when action == :create_authorization)
 
   action_fallback(Pleroma.Web.OAuth.FallbackController)
 

lib/pleroma/web/pleroma_api/controllers/account_controller.ex

@@ -9,6 +9,7 @@ defmodule Pleroma.Web.PleromaAPI.AccountController do
     only: [json_response: 3, add_link_headers: 2, assign_account_by_id: 2, skip_relationships?: 1]
 
   alias Ecto.Changeset
+  alias Pleroma.Plugs.EnsurePublicOrAuthenticatedPlug
   alias Pleroma.Plugs.OAuthScopesPlug
   alias Pleroma.Plugs.RateLimiter
   alias Pleroma.User
@@ -17,6 +18,11 @@ defmodule Pleroma.Web.PleromaAPI.AccountController do
 
   require Pleroma.Constants
 
+  plug(
+    :skip_plug,
+    [OAuthScopesPlug, EnsurePublicOrAuthenticatedPlug] when action == :confirmation_resend
+  )
+
   plug(
     OAuthScopesPlug,
     %{scopes: ["follow", "write:follows"]} when action in [:subscribe, :unsubscribe]
@@ -33,15 +39,13 @@ defmodule Pleroma.Web.PleromaAPI.AccountController do
          ]
   )
 
-  plug(OAuthScopesPlug, %{scopes: ["read:favourites"]} when action == :favourites)
-
-  # An extra safety measure for possible actions not guarded by OAuth permissions specification
   plug(
-    Pleroma.Plugs.EnsurePublicOrAuthenticatedPlug
+    OAuthScopesPlug,
-    when action != :confirmation_resend
+    %{scopes: ["read:favourites"], fallback: :proceed_unauthenticated} when action == :favourites
   )
 
   plug(RateLimiter, [name: :account_confirmation_resend] when action == :confirmation_resend)
+
   plug(:assign_account_by_id when action in [:favourites, :subscribe, :unsubscribe])
   plug(:put_view, Pleroma.Web.MastodonAPI.AccountView)
 

lib/pleroma/web/pleroma_api/controllers/emoji_api_controller.ex

@@ -1,191 +1,93 @@
 defmodule Pleroma.Web.PleromaAPI.EmojiAPIController do
   use Pleroma.Web, :controller
 
-  alias Pleroma.Plugs.OAuthScopesPlug
+  alias Pleroma.Emoji.Pack
-
-  require Logger
 
   plug(
-    OAuthScopesPlug,
+    Pleroma.Plugs.OAuthScopesPlug,
     %{scopes: ["write"], admin: true}
     when action in [
+           :import_from_filesystem,
+           :remote,
+           :download,
            :create,
+           :update,
            :delete,
-           :download_from,
+           :add_file,
-           :list_from,
-           :import_from_fs,
            :update_file,
-           :update_metadata
+           :delete_file
          ]
   )
 
-  plug(Pleroma.Plugs.EnsurePublicOrAuthenticatedPlug)
+  plug(
-
+    :skip_plug,
-  def emoji_dir_path do
+    [Pleroma.Plugs.OAuthScopesPlug, Pleroma.Plugs.ExpectPublicOrAuthenticatedCheckPlug]
-    Path.join(
+    when action in [:archive, :show, :list]
-      Pleroma.Config.get!([:instance, :static_dir]),
-      "emoji"
   )
-  end
 
-  @doc """
+  def remote(conn, %{"url" => url}) do
-  Lists packs from the remote instance.
+    with {:ok, packs} <- Pack.list_remote(url) do
-
+      json(conn, packs)
-  Since JS cannot ask remote instances for their packs due to CPS, it has to
-  be done by the server
-  """
-  def list_from(conn, %{"instance_address" => address}) do
-    address = String.trim(address)
-
-    if shareable_packs_available(address) do
-      list_resp =
-        "#{address}/api/pleroma/emoji/packs" |> Tesla.get!() |> Map.get(:body) |> Jason.decode!()
-
-      json(conn, list_resp)
     else
+      {:shareable, _} ->
         conn
         |> put_status(:internal_server_error)
         |> json(%{error: "The requested instance does not support sharing emoji packs"})
     end
   end
 
-  @doc """
+  def list(conn, _params) do
-  Lists the packs available on the instance as JSON.
+    emoji_path =
+      Path.join(
+        Pleroma.Config.get!([:instance, :static_dir]),
+        "emoji"
+      )
 
-  The information is public and does not require authentication. The format is
+    with {:ok, packs} <- Pack.list_local() do
-  a map of "pack directory name" to pack.json contents.
+      json(conn, packs)
-  """
-  def list_packs(conn, _params) do
-    # Create the directory first if it does not exist. This is probably the first request made
-    # with the API so it should be sufficient
-    with {:create_dir, :ok} <- {:create_dir, File.mkdir_p(emoji_dir_path())},
-         {:ls, {:ok, results}} <- {:ls, File.ls(emoji_dir_path())} do
-      pack_infos =
-        results
-        |> Enum.filter(&has_pack_json?/1)
-        |> Enum.map(&load_pack/1)
-        # Check if all the files are in place and can be sent
-        |> Enum.map(&validate_pack/1)
-        # Transform into a map of pack-name => pack-data
-        |> Enum.into(%{})
-
-      json(conn, pack_infos)
     else
       {:create_dir, {:error, e}} ->
         conn
         |> put_status(:internal_server_error)
-        |> json(%{error: "Failed to create the emoji pack directory at #{emoji_dir_path()}: #{e}"})
+        |> json(%{error: "Failed to create the emoji pack directory at #{emoji_path}: #{e}"})
 
       {:ls, {:error, e}} ->
         conn
         |> put_status(:internal_server_error)
         |> json(%{
-          error:
+          error: "Failed to get the contents of the emoji pack directory at #{emoji_path}: #{e}"
-            "Failed to get the contents of the emoji pack directory at #{emoji_dir_path()}: #{e}"
         })
     end
   end
 
-  defp has_pack_json?(file) do
+  def show(conn, %{"name" => name}) do
-    dir_path = Path.join(emoji_dir_path(), file)
+    name = String.trim(name)
-    # Filter to only use the pack.json packs
-    File.dir?(dir_path) and File.exists?(Path.join(dir_path, "pack.json"))
-  end
 
-  defp load_pack(pack_name) do
+    with {:ok, pack} <- Pack.show(name) do
-    pack_path = Path.join(emoji_dir_path(), pack_name)
+      json(conn, pack)
-    pack_file = Path.join(pack_path, "pack.json")
-
-    {pack_name, Jason.decode!(File.read!(pack_file))}
-  end
-
-  defp validate_pack({name, pack}) do
-    pack_path = Path.join(emoji_dir_path(), name)
-
-    if can_download?(pack, pack_path) do
-      archive_for_sha = make_archive(name, pack, pack_path)
-      archive_sha = :crypto.hash(:sha256, archive_for_sha) |> Base.encode16()
-
-      pack =
-        pack
-        |> put_in(["pack", "can-download"], true)
-        |> put_in(["pack", "download-sha256"], archive_sha)
-
-      {name, pack}
     else
-      {name, put_in(pack, ["pack", "can-download"], false)}
+      {:loaded, _} ->
+        conn
+        |> put_status(:not_found)
+        |> json(%{error: "Pack #{name} does not exist"})
+
+      {:error, :empty_values} ->
+        conn
+        |> put_status(:bad_request)
+        |> json(%{error: "pack name cannot be empty"})
     end
   end
 
-  defp can_download?(pack, pack_path) do
+  def archive(conn, %{"name" => name}) do
-    # If the pack is set as shared, check if it can be downloaded
+    with {:ok, archive} <- Pack.get_archive(name) do
-    # That means that when asked, the pack can be packed and sent to the remote
+      send_download(conn, {:binary, archive}, filename: "#{name}.zip")
-    # Otherwise, they'd have to download it from external-src
-    pack["pack"]["share-files"] &&
-      Enum.all?(pack["files"], fn {_, path} ->
-        File.exists?(Path.join(pack_path, path))
-      end)
-  end
-
-  defp create_archive_and_cache(name, pack, pack_dir, md5) do
-    files =
-      ['pack.json'] ++
-        (pack["files"] |> Enum.map(fn {_, path} -> to_charlist(path) end))
-
-    {:ok, {_, zip_result}} = :zip.zip('#{name}.zip', files, [:memory, cwd: to_charlist(pack_dir)])
-
-    cache_seconds_per_file = Pleroma.Config.get!([:emoji, :shared_pack_cache_seconds_per_file])
-    cache_ms = :timer.seconds(cache_seconds_per_file * Enum.count(files))
-
-    Cachex.put!(
-      :emoji_packs_cache,
-      name,
-      # if pack.json MD5 changes, the cache is not valid anymore
-      %{pack_json_md5: md5, pack_data: zip_result},
-      # Add a minute to cache time for every file in the pack
-      ttl: cache_ms
-    )
-
-    Logger.debug("Created an archive for the '#{name}' emoji pack, \
-keeping it in cache for #{div(cache_ms, 1000)}s")
-
-    zip_result
-  end
-
-  defp make_archive(name, pack, pack_dir) do
-    # Having a different pack.json md5 invalidates cache
-    pack_file_md5 = :crypto.hash(:md5, File.read!(Path.join(pack_dir, "pack.json")))
-
-    case Cachex.get!(:emoji_packs_cache, name) do
-      %{pack_file_md5: ^pack_file_md5, pack_data: zip_result} ->
-        Logger.debug("Using cache for the '#{name}' shared emoji pack")
-        zip_result
-
-      _ ->
-        create_archive_and_cache(name, pack, pack_dir, pack_file_md5)
-    end
-  end
-
-  @doc """
-  An endpoint for other instances (via admin UI) or users (via browser)
-  to download packs that the instance shares.
-  """
-  def download_shared(conn, %{"name" => name}) do
-    pack_dir = Path.join(emoji_dir_path(), name)
-    pack_file = Path.join(pack_dir, "pack.json")
-
-    with {_, true} <- {:exists?, File.exists?(pack_file)},
-         pack = Jason.decode!(File.read!(pack_file)),
-         {_, true} <- {:can_download?, can_download?(pack, pack_dir)} do
-      zip_result = make_archive(name, pack, pack_dir)
-      send_download(conn, {:binary, zip_result}, filename: "#{name}.zip")
     else
       {:can_download?, _} ->
         conn
         |> put_status(:forbidden)
         |> json(%{
-          error: "Pack #{name} cannot be downloaded from this instance, either pack sharing\
+          error:
-           was disabled for this pack or some files are missing"
+            "Pack #{name} cannot be downloaded from this instance, either pack sharing was disabled for this pack or some files are missing"
         })
 
       {:exists?, _} ->
@@ -195,133 +97,67 @@ def download_shared(conn, %{"name" => name}) do
     end
   end
 
-  defp shareable_packs_available(address) do
+  def download(conn, %{"url" => url, "name" => name} = params) do
-    "#{address}/.well-known/nodeinfo"
+    with :ok <- Pack.download(name, url, params["as"]) do
-    |> Tesla.get!()
-    |> Map.get(:body)
-    |> Jason.decode!()
-    |> Map.get("links")
-    |> List.last()
-    |> Map.get("href")
-    # Get the actual nodeinfo address and fetch it
-    |> Tesla.get!()
-    |> Map.get(:body)
-    |> Jason.decode!()
-    |> get_in(["metadata", "features"])
-    |> Enum.member?("shareable_emoji_packs")
-  end
-
-  @doc """
-  An admin endpoint to request downloading a pack named `pack_name` from the instance
-  `instance_address`.
-
-  If the requested instance's admin chose to share the pack, it will be downloaded
-  from that instance, otherwise it will be downloaded from the fallback source, if there is one.
-  """
-  def download_from(conn, %{"instance_address" => address, "pack_name" => name} = data) do
-    address = String.trim(address)
-
-    if shareable_packs_available(address) do
-      full_pack =
-        "#{address}/api/pleroma/emoji/packs/list"
-        |> Tesla.get!()
-        |> Map.get(:body)
-        |> Jason.decode!()
-        |> Map.get(name)
-
-      pack_info_res =
-        case full_pack["pack"] do
-          %{"share-files" => true, "can-download" => true, "download-sha256" => sha} ->
-            {:ok,
-             %{
-               sha: sha,
-               uri: "#{address}/api/pleroma/emoji/packs/download_shared/#{name}"
-             }}
-
-          %{"fallback-src" => src, "fallback-src-sha256" => sha} when is_binary(src) ->
-            {:ok,
-             %{
-               sha: sha,
-               uri: src,
-               fallback: true
-             }}
-
-          _ ->
-            {:error,
-             "The pack was not set as shared and there is no fallback src to download from"}
-        end
-
-      with {:ok, %{sha: sha, uri: uri} = pinfo} <- pack_info_res,
-           %{body: emoji_archive} <- Tesla.get!(uri),
-           {_, true} <- {:checksum, Base.decode16!(sha) == :crypto.hash(:sha256, emoji_archive)} do
-        local_name = data["as"] || name
-        pack_dir = Path.join(emoji_dir_path(), local_name)
-        File.mkdir_p!(pack_dir)
-
-        files = Enum.map(full_pack["files"], fn {_, path} -> to_charlist(path) end)
-        # Fallback cannot contain a pack.json file
-        files = if pinfo[:fallback], do: files, else: ['pack.json'] ++ files
-
-        {:ok, _} = :zip.unzip(emoji_archive, cwd: to_charlist(pack_dir), file_list: files)
-
-        # Fallback can't contain a pack.json file, since that would cause the fallback-src-sha256
-        # in it to depend on itself
-        if pinfo[:fallback] do
-          pack_file_path = Path.join(pack_dir, "pack.json")
-
-          File.write!(pack_file_path, Jason.encode!(full_pack, pretty: true))
-        end
-
       json(conn, "ok")
     else
-        {:error, e} ->
+      {:shareable, _} ->
-          conn |> put_status(:internal_server_error) |> json(%{error: e})
+        conn
+        |> put_status(:internal_server_error)
+        |> json(%{error: "The requested instance does not support sharing emoji packs"})
 
       {:checksum, _} ->
         conn
         |> put_status(:internal_server_error)
         |> json(%{error: "SHA256 for the pack doesn't match the one sent by the server"})
-      end
+
-    else
+      {:error, e} ->
         conn
         |> put_status(:internal_server_error)
-      |> json(%{error: "The requested instance does not support sharing emoji packs"})
+        |> json(%{error: e})
     end
   end
 
-  @doc """
-  Creates an empty pack named `name` which then can be updated via the admin UI.
-  """
   def create(conn, %{"name" => name}) do
-    pack_dir = Path.join(emoji_dir_path(), name)
+    name = String.trim(name)
 
-    if not File.exists?(pack_dir) do
+    with :ok <- Pack.create(name) do
-      File.mkdir_p!(pack_dir)
+      json(conn, "ok")
-
-      pack_file_p = Path.join(pack_dir, "pack.json")
-
-      File.write!(
-        pack_file_p,
-        Jason.encode!(%{pack: %{}, files: %{}}, pretty: true)
-      )
-
-      conn |> json("ok")
     else
+      {:error, :eexist} ->
         conn
         |> put_status(:conflict)
         |> json(%{error: "A pack named \"#{name}\" already exists"})
+
+      {:error, :empty_values} ->
+        conn
+        |> put_status(:bad_request)
+        |> json(%{error: "pack name cannot be empty"})
+
+      {:error, _} ->
+        render_error(
+          conn,
+          :internal_server_error,
+          "Unexpected error occurred while creating pack."
+        )
     end
   end
 
-  @doc """
-  Deletes the pack `name` and all it's files.
-  """
   def delete(conn, %{"name" => name}) do
-    pack_dir = Path.join(emoji_dir_path(), name)
+    name = String.trim(name)
 
-    case File.rm_rf(pack_dir) do
+    with {:ok, deleted} when deleted != [] <- Pack.delete(name) do
-      {:ok, _} ->
+      json(conn, "ok")
-        conn |> json("ok")
+    else
+      {:ok, []} ->
+        conn
+        |> put_status(:not_found)
+        |> json(%{error: "Pack #{name} does not exist"})
+
+      {:error, :empty_values} ->
+        conn
+        |> put_status(:bad_request)
+        |> json(%{error: "pack name cannot be empty"})
 
       {:error, _, _} ->
         conn
@@ -330,265 +166,128 @@ def delete(conn, %{"name" => name}) do
     end
   end
 
-  @doc """
+  def update(conn, %{"name" => name, "metadata" => metadata}) do
-  An endpoint to update `pack_names`'s metadata.
+    with {:ok, pack} <- Pack.update_metadata(name, metadata) do
-
+      json(conn, pack.pack)
-  `new_data` is the new metadata for the pack, that will replace the old metadata.
-  """
-  def update_metadata(conn, %{"pack_name" => name, "new_data" => new_data}) do
-    pack_file_p = Path.join([emoji_dir_path(), name, "pack.json"])
-
-    full_pack = Jason.decode!(File.read!(pack_file_p))
-
-    # The new fallback-src is in the new data and it's not the same as it was in the old data
-    should_update_fb_sha =
-      not is_nil(new_data["fallback-src"]) and
-        new_data["fallback-src"] != full_pack["pack"]["fallback-src"]
-
-    with {_, true} <- {:should_update?, should_update_fb_sha},
-         %{body: pack_arch} <- Tesla.get!(new_data["fallback-src"]),
-         {:ok, flist} <- :zip.unzip(pack_arch, [:memory]),
-         {_, true} <- {:has_all_files?, has_all_files?(full_pack, flist)} do
-      fallback_sha = :crypto.hash(:sha256, pack_arch) |> Base.encode16()
-
-      new_data = Map.put(new_data, "fallback-src-sha256", fallback_sha)
-      update_metadata_and_send(conn, full_pack, new_data, pack_file_p)
     else
-      {:should_update?, _} ->
-        update_metadata_and_send(conn, full_pack, new_data, pack_file_p)
-
       {:has_all_files?, _} ->
         conn
         |> put_status(:bad_request)
         |> json(%{error: "The fallback archive does not have all files specified in pack.json"})
-    end
-  end
 
-  # Check if all files from the pack.json are in the archive
+      {:error, _} ->
-  defp has_all_files?(%{"files" => files}, flist) do
+        render_error(
-    Enum.all?(files, fn {_, from_manifest} ->
-      Enum.find(flist, fn {from_archive, _} ->
-        to_string(from_archive) == from_manifest
-      end)
-    end)
-  end
-
-  defp update_metadata_and_send(conn, full_pack, new_data, pack_file_p) do
-    full_pack = Map.put(full_pack, "pack", new_data)
-    File.write!(pack_file_p, Jason.encode!(full_pack, pretty: true))
-
-    # Send new data back with fallback sha filled
-    json(conn, new_data)
-  end
-
-  defp get_filename(%{"filename" => filename}), do: filename
-
-  defp get_filename(%{"file" => file}) do
-    case file do
-      %Plug.Upload{filename: filename} -> filename
-      url when is_binary(url) -> Path.basename(url)
-    end
-  end
-
-  defp empty?(str), do: String.trim(str) == ""
-
-  defp update_file_and_send(conn, updated_full_pack, pack_file_p) do
-    # Write the emoji pack file
-    File.write!(pack_file_p, Jason.encode!(updated_full_pack, pretty: true))
-
-    # Return the modified file list
-    json(conn, updated_full_pack["files"])
-  end
-
-  @doc """
-  Updates a file in a pack.
-
-  Updating can mean three things:
-
-  - `add` adds an emoji named `shortcode` to the pack `pack_name`,
-    that means that the emoji file needs to be uploaded with the request
-    (thus requiring it to be a multipart request) and be named `file`.
-    There can also be an optional `filename` that will be the new emoji file name
-    (if it's not there, the name will be taken from the uploaded file).
-  - `update` changes emoji shortcode (from `shortcode` to `new_shortcode` or moves the file
-    (from the current filename to `new_filename`)
-  - `remove` removes the emoji named `shortcode` and it's associated file
-  """
-
-  # Add
-  def update_file(
           conn,
-        %{"pack_name" => pack_name, "action" => "add", "shortcode" => shortcode} = params
+          :internal_server_error,
-      ) do
+          "Unexpected error occurred while updating pack metadata."
-    pack_dir = Path.join(emoji_dir_path(), pack_name)
+        )
-    pack_file_p = Path.join(pack_dir, "pack.json")
+    end
-
-    full_pack = Jason.decode!(File.read!(pack_file_p))
-
-    with {_, false} <- {:has_shortcode, Map.has_key?(full_pack["files"], shortcode)},
-         filename <- get_filename(params),
-         false <- empty?(shortcode),
-         false <- empty?(filename) do
-      file_path = Path.join(pack_dir, filename)
-
-      # If the name contains directories, create them
-      if String.contains?(file_path, "/") do
-        File.mkdir_p!(Path.dirname(file_path))
   end
 
-      case params["file"] do
+  def add_file(conn, %{"name" => name} = params) do
-        %Plug.Upload{path: upload_path} ->
+    filename = params["filename"] || get_filename(params["file"])
-          # Copy the uploaded file from the temporary directory
+    shortcode = params["shortcode"] || Path.basename(filename, Path.extname(filename))
-          File.copy!(upload_path, file_path)
 
-        url when is_binary(url) ->
+    with {:ok, pack} <- Pack.add_file(name, shortcode, filename, params["file"]) do
-          # Download and write the file
+      json(conn, pack.files)
-          file_contents = Tesla.get!(url).body
-          File.write!(file_path, file_contents)
-      end
-
-      updated_full_pack = put_in(full_pack, ["files", shortcode], filename)
-      update_file_and_send(conn, updated_full_pack, pack_file_p)
     else
-      {:has_shortcode, _} ->
+      {:exists, _} ->
         conn
         |> put_status(:conflict)
         |> json(%{error: "An emoji with the \"#{shortcode}\" shortcode already exists"})
 
-      true ->
+      {:loaded, _} ->
         conn
         |> put_status(:bad_request)
-        |> json(%{error: "shortcode or filename cannot be empty"})
+        |> json(%{error: "pack \"#{name}\" is not found"})
-    end
-  end
 
-  # Remove
+      {:error, :empty_values} ->
-  def update_file(conn, %{
-        "pack_name" => pack_name,
-        "action" => "remove",
-        "shortcode" => shortcode
-      }) do
-    pack_dir = Path.join(emoji_dir_path(), pack_name)
-    pack_file_p = Path.join(pack_dir, "pack.json")
-
-    full_pack = Jason.decode!(File.read!(pack_file_p))
-
-    if Map.has_key?(full_pack["files"], shortcode) do
-      {emoji_file_path, updated_full_pack} = pop_in(full_pack, ["files", shortcode])
-
-      emoji_file_path = Path.join(pack_dir, emoji_file_path)
-
-      # Delete the emoji file
-      File.rm!(emoji_file_path)
-
-      # If the old directory has no more files, remove it
-      if String.contains?(emoji_file_path, "/") do
-        dir = Path.dirname(emoji_file_path)
-
-        if Enum.empty?(File.ls!(dir)) do
-          File.rmdir!(dir)
-        end
-      end
-
-      update_file_and_send(conn, updated_full_pack, pack_file_p)
-    else
         conn
         |> put_status(:bad_request)
-      |> json(%{error: "Emoji \"#{shortcode}\" does not exist"})
+        |> json(%{error: "pack name, shortcode or filename cannot be empty"})
-    end
-  end
 
-  # Update
+      {:error, _} ->
-  def update_file(
+        render_error(
           conn,
-        %{"pack_name" => pack_name, "action" => "update", "shortcode" => shortcode} = params
+          :internal_server_error,
-      ) do
+          "Unexpected error occurred while adding file to pack."
-    pack_dir = Path.join(emoji_dir_path(), pack_name)
+        )
-    pack_file_p = Path.join(pack_dir, "pack.json")
-
-    full_pack = Jason.decode!(File.read!(pack_file_p))
-
-    with {_, true} <- {:has_shortcode, Map.has_key?(full_pack["files"], shortcode)},
-         %{"new_shortcode" => new_shortcode, "new_filename" => new_filename} <- params,
-         false <- empty?(new_shortcode),
-         false <- empty?(new_filename) do
-      # First, remove the old shortcode, saving the old path
-      {old_emoji_file_path, updated_full_pack} = pop_in(full_pack, ["files", shortcode])
-      old_emoji_file_path = Path.join(pack_dir, old_emoji_file_path)
-      new_emoji_file_path = Path.join(pack_dir, new_filename)
-
-      # If the name contains directories, create them
-      if String.contains?(new_emoji_file_path, "/") do
-        File.mkdir_p!(Path.dirname(new_emoji_file_path))
-      end
-
-      # Move/Rename the old filename to a new filename
-      # These are probably on the same filesystem, so just rename should work
-      :ok = File.rename(old_emoji_file_path, new_emoji_file_path)
-
-      # If the old directory has no more files, remove it
-      if String.contains?(old_emoji_file_path, "/") do
-        dir = Path.dirname(old_emoji_file_path)
-
-        if Enum.empty?(File.ls!(dir)) do
-          File.rmdir!(dir)
     end
   end
 
-      # Then, put in the new shortcode with the new path
+  def update_file(conn, %{"name" => name, "shortcode" => shortcode} = params) do
-      updated_full_pack = put_in(updated_full_pack, ["files", new_shortcode], new_filename)
+    new_shortcode = params["new_shortcode"]
-      update_file_and_send(conn, updated_full_pack, pack_file_p)
+    new_filename = params["new_filename"]
+    force = params["force"] == true
+
+    with {:ok, pack} <- Pack.update_file(name, shortcode, new_shortcode, new_filename, force) do
+      json(conn, pack.files)
     else
-      {:has_shortcode, _} ->
+      {:exists, _} ->
         conn
         |> put_status(:bad_request)
         |> json(%{error: "Emoji \"#{shortcode}\" does not exist"})
 
-      true ->
+      {:not_used, _} ->
+        conn
+        |> put_status(:conflict)
+        |> json(%{
+          error:
+            "New shortcode \"#{new_shortcode}\" is already used. If you want to override emoji use 'force' option"
+        })
+
+      {:loaded, _} ->
+        conn
+        |> put_status(:bad_request)
+        |> json(%{error: "pack \"#{name}\" is not found"})
+
+      {:error, :empty_values} ->
         conn
         |> put_status(:bad_request)
         |> json(%{error: "new_shortcode or new_filename cannot be empty"})
 
-      _ ->
+      {:error, _} ->
-        conn
+        render_error(
-        |> put_status(:bad_request)
+          conn,
-        |> json(%{error: "new_shortcode or new_file were not specified"})
+          :internal_server_error,
+          "Unexpected error occurred while updating file in pack."
+        )
     end
   end
 
-  def update_file(conn, %{"action" => action}) do
+  def delete_file(conn, %{"name" => name, "shortcode" => shortcode}) do
-    conn
+    with {:ok, pack} <- Pack.delete_file(name, shortcode) do
-    |> put_status(:bad_request)
+      json(conn, pack.files)
-    |> json(%{error: "Unknown action: #{action}"})
-  end
-
-  @doc """
-  Imports emoji from the filesystem.
-
-  Importing means checking all the directories in the
-  `$instance_static/emoji/` for directories which do not have
-  `pack.json`. If one has an emoji.txt file, that file will be used
-  to create a `pack.json` file with it's contents. If the directory has
-  neither, all the files with specific configured extenstions will be
-  assumed to be emojis and stored in the new `pack.json` file.
-  """
-  def import_from_fs(conn, _params) do
-    emoji_path = emoji_dir_path()
-
-    with {:ok, %{access: :read_write}} <- File.stat(emoji_path),
-         {:ok, results} <- File.ls(emoji_path) do
-      imported_pack_names =
-        results
-        |> Enum.filter(fn file ->
-          dir_path = Path.join(emoji_path, file)
-          # Find the directories that do NOT have pack.json
-          File.dir?(dir_path) and not File.exists?(Path.join(dir_path, "pack.json"))
-        end)
-        |> Enum.map(&write_pack_json_contents/1)
-
-      json(conn, imported_pack_names)
     else
-      {:ok, %{access: _}} ->
+      {:exists, _} ->
+        conn
+        |> put_status(:bad_request)
+        |> json(%{error: "Emoji \"#{shortcode}\" does not exist"})
+
+      {:loaded, _} ->
+        conn
+        |> put_status(:bad_request)
+        |> json(%{error: "pack \"#{name}\" is not found"})
+
+      {:error, :empty_values} ->
+        conn
+        |> put_status(:bad_request)
+        |> json(%{error: "pack name or shortcode cannot be empty"})
+
+      {:error, _} ->
+        render_error(
+          conn,
+          :internal_server_error,
+          "Unexpected error occurred while removing file from pack."
+        )
+    end
+  end
+
+  def import_from_filesystem(conn, _params) do
+    with {:ok, names} <- Pack.import_from_filesystem() do
+      json(conn, names)
+    else
+      {:error, :no_read_write} ->
         conn
         |> put_status(:internal_server_error)
         |> json(%{error: "Error: emoji pack directory must be writable"})
@@ -600,44 +299,6 @@ def import_from_fs(conn, _params) do
     end
   end
 
-  defp write_pack_json_contents(dir) do
+  defp get_filename(%Plug.Upload{filename: filename}), do: filename
-    dir_path = Path.join(emoji_dir_path(), dir)
+  defp get_filename(url) when is_binary(url), do: Path.basename(url)
-    emoji_txt_path = Path.join(dir_path, "emoji.txt")
-
-    files_for_pack = files_for_pack(emoji_txt_path, dir_path)
-    pack_json_contents = Jason.encode!(%{pack: %{}, files: files_for_pack})
-
-    File.write!(Path.join(dir_path, "pack.json"), pack_json_contents)
-
-    dir
-  end
-
-  defp files_for_pack(emoji_txt_path, dir_path) do
-    if File.exists?(emoji_txt_path) do
-      # There's an emoji.txt file, it's likely from a pack installed by the pack manager.
-      # Make a pack.json file from the contents of that emoji.txt fileh
-
-      # FIXME: Copy-pasted from Pleroma.Emoji/load_from_file_stream/2
-
-      # Create a map of shortcodes to filenames from emoji.txt
-      File.read!(emoji_txt_path)
-      |> String.split("\n")
-      |> Enum.map(&String.trim/1)
-      |> Enum.map(fn line ->
-        case String.split(line, ~r/,\s*/) do
-          # This matches both strings with and without tags
-          # and we don't care about tags here
-          [name, file | _] -> {name, file}
-          _ -> nil
-        end
-      end)
-      |> Enum.filter(fn x -> not is_nil(x) end)
-      |> Enum.into(%{})
-    else
-      # If there's no emoji.txt, assume all files
-      # that are of certain extensions from the config are emojis and import them all
-      pack_extensions = Pleroma.Config.get!([:emoji, :pack_extensions])
-      Pleroma.Emoji.Loader.make_shortcode_to_file_map(dir_path, pack_extensions)
-    end
-  end
 end

lib/pleroma/web/pleroma_api/controllers/mascot_controller.ex

@@ -12,8 +12,6 @@ defmodule Pleroma.Web.PleromaAPI.MascotController do
   plug(OAuthScopesPlug, %{scopes: ["read:accounts"]} when action == :show)
   plug(OAuthScopesPlug, %{scopes: ["write:accounts"]} when action != :show)
 
-  plug(Pleroma.Plugs.EnsurePublicOrAuthenticatedPlug)
-
   @doc "GET /api/v1/pleroma/mascot"
   def show(%{assigns: %{user: user}} = conn, _params) do
     json(conn, User.get_mascot(user))

lib/pleroma/web/pleroma_api/controllers/pleroma_api_controller.ex

@@ -26,6 +26,12 @@ defmodule Pleroma.Web.PleromaAPI.PleromaAPIController do
     when action in [:conversation, :conversation_statuses]
   )
 
+  plug(
+    OAuthScopesPlug,
+    %{scopes: ["read:statuses"], fallback: :proceed_unauthenticated}
+    when action == :emoji_reactions_by
+  )
+
   plug(
     OAuthScopesPlug,
     %{scopes: ["write:statuses"]}
@@ -34,12 +40,14 @@ defmodule Pleroma.Web.PleromaAPI.PleromaAPIController do
 
   plug(
     OAuthScopesPlug,
-    %{scopes: ["write:conversations"]} when action in [:update_conversation, :read_conversations]
+    %{scopes: ["write:conversations"]}
+    when action in [:update_conversation, :mark_conversations_as_read]
   )
 
-  plug(OAuthScopesPlug, %{scopes: ["write:notifications"]} when action == :read_notification)
+  plug(
-
+    OAuthScopesPlug,
-  plug(Pleroma.Plugs.EnsurePublicOrAuthenticatedPlug)
+    %{scopes: ["write:notifications"]} when action == :mark_notifications_as_read
+  )
 
   def emoji_reactions_by(%{assigns: %{user: user}} = conn, %{"id" => activity_id} = params) do
     with %Activity{} = activity <- Activity.get_by_id_with_object(activity_id),
@@ -53,7 +61,10 @@ def emoji_reactions_by(%{assigns: %{user: user}} = conn, %{"id" => activity_id}
           else
             users =
               Enum.map(user_ap_ids, &User.get_cached_by_ap_id/1)
-              |> Enum.filter(& &1)
+              |> Enum.filter(fn
+                %{deactivated: false} -> true
+                _ -> false
+              end)
 
             %{
               name: emoji,
@@ -167,7 +178,7 @@ def update_conversation(
     end
   end
 
-  def read_conversations(%{assigns: %{user: user}} = conn, _params) do
+  def mark_conversations_as_read(%{assigns: %{user: user}} = conn, _params) do
     with {:ok, _, participations} <- Participation.mark_all_as_read(user) do
       conn
       |> add_link_headers(participations)
@@ -176,7 +187,7 @@ def read_conversations(%{assigns: %{user: user}} = conn, _params) do
     end
   end
 
-  def read_notification(%{assigns: %{user: user}} = conn, %{"id" => notification_id}) do
+  def mark_notifications_as_read(%{assigns: %{user: user}} = conn, %{"id" => notification_id}) do
     with {:ok, notification} <- Notification.read_one(user, notification_id) do
       conn
       |> put_view(NotificationView)
@@ -189,7 +200,7 @@ def read_notification(%{assigns: %{user: user}} = conn, %{"id" => notification_i
     end
   end
 
-  def read_notification(%{assigns: %{user: user}} = conn, %{"max_id" => max_id} = params) do
+  def mark_notifications_as_read(%{assigns: %{user: user}} = conn, %{"max_id" => max_id} = params) do
     with notifications <- Notification.set_read_up_to(user, max_id) do
       notifications = Enum.take(notifications, 80)
 

lib/pleroma/web/pleroma_api/controllers/scrobble_controller.ex

@@ -13,10 +13,12 @@ defmodule Pleroma.Web.PleromaAPI.ScrobbleController do
   alias Pleroma.Web.CommonAPI
   alias Pleroma.Web.MastodonAPI.StatusView
 
-  plug(OAuthScopesPlug, %{scopes: ["read"]} when action == :user_scrobbles)
+  plug(
-  plug(OAuthScopesPlug, %{scopes: ["write"]} when action != :user_scrobbles)
+    OAuthScopesPlug,
+    %{scopes: ["read"], fallback: :proceed_unauthenticated} when action == :user_scrobbles
+  )
 
-  plug(Pleroma.Plugs.EnsurePublicOrAuthenticatedPlug)
+  plug(OAuthScopesPlug, %{scopes: ["write"]} when action != :user_scrobbles)
 
   def new_scrobble(%{assigns: %{user: user}} = conn, %{"title" => _} = params) do
     params =

lib/pleroma/web/router.ex

@@ -16,6 +16,14 @@ defmodule Pleroma.Web.Router do
     plug(Pleroma.Plugs.UserEnabledPlug)
   end
 
+  pipeline :expect_authentication do
+    plug(Pleroma.Plugs.ExpectAuthenticatedCheckPlug)
+  end
+
+  pipeline :expect_public_instance_or_authentication do
+    plug(Pleroma.Plugs.ExpectPublicOrAuthenticatedCheckPlug)
+  end
+
   pipeline :authenticate do
     plug(Pleroma.Plugs.OAuthPlug)
     plug(Pleroma.Plugs.BasicAuthDecoderPlug)
@@ -39,20 +47,22 @@ defmodule Pleroma.Web.Router do
   end
 
   pipeline :api do
+    plug(:expect_public_instance_or_authentication)
     plug(:base_api)
     plug(:after_auth)
     plug(Pleroma.Plugs.IdempotencyPlug)
   end
 
   pipeline :authenticated_api do
+    plug(:expect_authentication)
     plug(:base_api)
-    plug(Pleroma.Plugs.AuthExpectedPlug)
     plug(:after_auth)
     plug(Pleroma.Plugs.EnsureAuthenticatedPlug)
     plug(Pleroma.Plugs.IdempotencyPlug)
   end
 
   pipeline :admin_api do
+    plug(:expect_authentication)
     plug(:base_api)
     plug(Pleroma.Plugs.AdminSecretAuthenticationPlug)
     plug(:after_auth)
@@ -200,24 +210,28 @@ defmodule Pleroma.Web.Router do
   end
 
   scope "/api/pleroma/emoji", Pleroma.Web.PleromaAPI do
-    scope "/packs" do
     # Modifying packs
+    scope "/packs" do
       pipe_through(:admin_api)
 
-      post("/import_from_fs", EmojiAPIController, :import_from_fs)
+      get("/import", EmojiAPIController, :import_from_filesystem)
+      get("/remote", EmojiAPIController, :remote)
+      post("/download", EmojiAPIController, :download)
 
-      post("/:pack_name/update_file", EmojiAPIController, :update_file)
+      post("/:name", EmojiAPIController, :create)
-      post("/:pack_name/update_metadata", EmojiAPIController, :update_metadata)
+      patch("/:name", EmojiAPIController, :update)
-      put("/:name", EmojiAPIController, :create)
       delete("/:name", EmojiAPIController, :delete)
-      post("/download_from", EmojiAPIController, :download_from)
+
-      post("/list_from", EmojiAPIController, :list_from)
+      post("/:name/files", EmojiAPIController, :add_file)
+      patch("/:name/files", EmojiAPIController, :update_file)
+      delete("/:name/files", EmojiAPIController, :delete_file)
     end
 
-    scope "/packs" do
     # Pack info / downloading
-      get("/", EmojiAPIController, :list_packs)
+    scope "/packs" do
-      get("/:name/download_shared/", EmojiAPIController, :download_shared)
+      get("/", EmojiAPIController, :list)
+      get("/:name", EmojiAPIController, :show)
+      get("/:name/archive", EmojiAPIController, :archive)
     end
   end
 
@@ -277,7 +291,7 @@ defmodule Pleroma.Web.Router do
 
       get("/conversations/:id/statuses", PleromaAPIController, :conversation_statuses)
       get("/conversations/:id", PleromaAPIController, :conversation)
-      post("/conversations/read", PleromaAPIController, :read_conversations)
+      post("/conversations/read", PleromaAPIController, :mark_conversations_as_read)
     end
 
     scope [] do
@@ -286,7 +300,7 @@ defmodule Pleroma.Web.Router do
       patch("/conversations/:id", PleromaAPIController, :update_conversation)
       put("/statuses/:id/reactions/:emoji", PleromaAPIController, :react_with_emoji)
       delete("/statuses/:id/reactions/:emoji", PleromaAPIController, :unreact_with_emoji)
-      post("/notifications/read", PleromaAPIController, :read_notification)
+      post("/notifications/read", PleromaAPIController, :mark_notifications_as_read)
 
       patch("/accounts/update_avatar", AccountController, :update_avatar)
       patch("/accounts/update_banner", AccountController, :update_banner)
@@ -322,53 +336,84 @@ defmodule Pleroma.Web.Router do
     pipe_through(:authenticated_api)
 
     get("/accounts/verify_credentials", AccountController, :verify_credentials)
+    patch("/accounts/update_credentials", AccountController, :update_credentials)
 
     get("/accounts/relationships", AccountController, :relationships)
-
     get("/accounts/:id/lists", AccountController, :lists)
     get("/accounts/:id/identity_proofs", AccountController, :identity_proofs)
-
+    get("/endorsements", AccountController, :endorsements)
-    get("/follow_requests", FollowRequestController, :index)
     get("/blocks", AccountController, :blocks)
     get("/mutes", AccountController, :mutes)
 
-    get("/timelines/home", TimelineController, :home)
+    post("/follows", AccountController, :follow_by_uri)
-    get("/timelines/direct", TimelineController, :direct)
+    post("/accounts/:id/follow", AccountController, :follow)
+    post("/accounts/:id/unfollow", AccountController, :unfollow)
+    post("/accounts/:id/block", AccountController, :block)
+    post("/accounts/:id/unblock", AccountController, :unblock)
+    post("/accounts/:id/mute", AccountController, :mute)
+    post("/accounts/:id/unmute", AccountController, :unmute)
 
-    get("/favourites", StatusController, :favourites)
+    get("/apps/verify_credentials", AppController, :verify_credentials)
-    get("/bookmarks", StatusController, :bookmarks)
 
-    get("/notifications", NotificationController, :index)
+    get("/conversations", ConversationController, :index)
-    get("/notifications/:id", NotificationController, :show)
+    post("/conversations/:id/read", ConversationController, :mark_as_read)
-    post("/notifications/:id/dismiss", NotificationController, :dismiss)
-    post("/notifications/clear", NotificationController, :clear)
-    delete("/notifications/destroy_multiple", NotificationController, :destroy_multiple)
-    # Deprecated: was removed in Mastodon v3, use `/notifications/:id/dismiss` instead
-    post("/notifications/dismiss", NotificationController, :dismiss)
 
-    get("/scheduled_statuses", ScheduledActivityController, :index)
+    get("/domain_blocks", DomainBlockController, :index)
-    get("/scheduled_statuses/:id", ScheduledActivityController, :show)
+    post("/domain_blocks", DomainBlockController, :create)
+    delete("/domain_blocks", DomainBlockController, :delete)
+
+    get("/filters", FilterController, :index)
+
+    post("/filters", FilterController, :create)
+    get("/filters/:id", FilterController, :show)
+    put("/filters/:id", FilterController, :update)
+    delete("/filters/:id", FilterController, :delete)
+
+    get("/follow_requests", FollowRequestController, :index)
+    post("/follow_requests/:id/authorize", FollowRequestController, :authorize)
+    post("/follow_requests/:id/reject", FollowRequestController, :reject)
 
     get("/lists", ListController, :index)
     get("/lists/:id", ListController, :show)
     get("/lists/:id/accounts", ListController, :list_accounts)
 
-    get("/domain_blocks", DomainBlockController, :index)
+    delete("/lists/:id", ListController, :delete)
+    post("/lists", ListController, :create)
+    put("/lists/:id", ListController, :update)
+    post("/lists/:id/accounts", ListController, :add_to_list)
+    delete("/lists/:id/accounts", ListController, :remove_from_list)
 
-    get("/filters", FilterController, :index)
+    get("/markers", MarkerController, :index)
+    post("/markers", MarkerController, :upsert)
 
-    get("/suggestions", SuggestionController, :index)
+    post("/media", MediaController, :create)
+    put("/media/:id", MediaController, :update)
 
-    get("/conversations", ConversationController, :index)
+    get("/notifications", NotificationController, :index)
-    post("/conversations/:id/read", ConversationController, :read)
+    get("/notifications/:id", NotificationController, :show)
 
-    get("/endorsements", AccountController, :endorsements)
+    post("/notifications/:id/dismiss", NotificationController, :dismiss)
+    post("/notifications/clear", NotificationController, :clear)
+    delete("/notifications/destroy_multiple", NotificationController, :destroy_multiple)
+    # Deprecated: was removed in Mastodon v3, use `/notifications/:id/dismiss` instead
+    post("/notifications/dismiss", NotificationController, :dismiss_via_body)
 
-    patch("/accounts/update_credentials", AccountController, :update_credentials)
+    post("/polls/:id/votes", PollController, :vote)
+
+    post("/reports", ReportController, :create)
+
+    get("/scheduled_statuses", ScheduledActivityController, :index)
+    get("/scheduled_statuses/:id", ScheduledActivityController, :show)
+
+    put("/scheduled_statuses/:id", ScheduledActivityController, :update)
+    delete("/scheduled_statuses/:id", ScheduledActivityController, :delete)
+
+    # Unlike `GET /api/v1/accounts/:id/favourites`, demands authentication
+    get("/favourites", StatusController, :favourites)
+    get("/bookmarks", StatusController, :bookmarks)
 
     post("/statuses", StatusController, :create)
     delete("/statuses/:id", StatusController, :delete)
-
     post("/statuses/:id/reblog", StatusController, :reblog)
     post("/statuses/:id/unreblog", StatusController, :unreblog)
     post("/statuses/:id/favourite", StatusController, :favourite)
@@ -380,49 +425,16 @@ defmodule Pleroma.Web.Router do
     post("/statuses/:id/mute", StatusController, :mute_conversation)
     post("/statuses/:id/unmute", StatusController, :unmute_conversation)
 
-    put("/scheduled_statuses/:id", ScheduledActivityController, :update)
-    delete("/scheduled_statuses/:id", ScheduledActivityController, :delete)
-
-    post("/polls/:id/votes", PollController, :vote)
-
-    post("/media", MediaController, :create)
-    put("/media/:id", MediaController, :update)
-
-    delete("/lists/:id", ListController, :delete)
-    post("/lists", ListController, :create)
-    put("/lists/:id", ListController, :update)
-
-    post("/lists/:id/accounts", ListController, :add_to_list)
-    delete("/lists/:id/accounts", ListController, :remove_from_list)
-
-    post("/filters", FilterController, :create)
-    get("/filters/:id", FilterController, :show)
-    put("/filters/:id", FilterController, :update)
-    delete("/filters/:id", FilterController, :delete)
-
-    post("/reports", ReportController, :create)
-
-    post("/follows", AccountController, :follows)
-    post("/accounts/:id/follow", AccountController, :follow)
-    post("/accounts/:id/unfollow", AccountController, :unfollow)
-    post("/accounts/:id/block", AccountController, :block)
-    post("/accounts/:id/unblock", AccountController, :unblock)
-    post("/accounts/:id/mute", AccountController, :mute)
-    post("/accounts/:id/unmute", AccountController, :unmute)
-
-    post("/follow_requests/:id/authorize", FollowRequestController, :authorize)
-    post("/follow_requests/:id/reject", FollowRequestController, :reject)
-
-    post("/domain_blocks", DomainBlockController, :create)
-    delete("/domain_blocks", DomainBlockController, :delete)
-
     post("/push/subscription", SubscriptionController, :create)
     get("/push/subscription", SubscriptionController, :get)
     put("/push/subscription", SubscriptionController, :update)
     delete("/push/subscription", SubscriptionController, :delete)
 
-    get("/markers", MarkerController, :index)
+    get("/suggestions", SuggestionController, :index)
-    post("/markers", MarkerController, :upsert)
+
+    get("/timelines/home", TimelineController, :home)
+    get("/timelines/direct", TimelineController, :direct)
+    get("/timelines/list/:list_id", TimelineController, :list)
   end
 
   scope "/api/web", Pleroma.Web do
@@ -434,15 +446,24 @@ defmodule Pleroma.Web.Router do
   scope "/api/v1", Pleroma.Web.MastodonAPI do
     pipe_through(:api)
 
-    post("/accounts", AccountController, :create)
     get("/accounts/search", SearchController, :account_search)
+    get("/search", SearchController, :search)
+
+    get("/accounts/:id/statuses", AccountController, :statuses)
+    get("/accounts/:id/followers", AccountController, :followers)
+    get("/accounts/:id/following", AccountController, :following)
+    get("/accounts/:id", AccountController, :show)
+
+    post("/accounts", AccountController, :create)
 
     get("/instance", InstanceController, :show)
     get("/instance/peers", InstanceController, :peers)
 
     post("/apps", AppController, :create)
-    get("/apps/verify_credentials", AppController, :verify_credentials)
 
+    get("/statuses", StatusController, :index)
+    get("/statuses/:id", StatusController, :show)
+    get("/statuses/:id/context", StatusController, :context)
     get("/statuses/:id/card", StatusController, :card)
     get("/statuses/:id/favourited_by", StatusController, :favourited_by)
     get("/statuses/:id/reblogged_by", StatusController, :reblogged_by)
@@ -453,20 +474,8 @@ defmodule Pleroma.Web.Router do
 
     get("/timelines/public", TimelineController, :public)
     get("/timelines/tag/:tag", TimelineController, :hashtag)
-    get("/timelines/list/:list_id", TimelineController, :list)
-
-    get("/statuses", StatusController, :index)
-    get("/statuses/:id", StatusController, :show)
-    get("/statuses/:id/context", StatusController, :context)
 
     get("/polls/:id", PollController, :show)
-
-    get("/accounts/:id/statuses", AccountController, :statuses)
-    get("/accounts/:id/followers", AccountController, :followers)
-    get("/accounts/:id/following", AccountController, :following)
-    get("/accounts/:id", AccountController, :show)
-
-    get("/search", SearchController, :search)
   end
 
   scope "/api/v2", Pleroma.Web.MastodonAPI do
@@ -507,7 +516,11 @@ defmodule Pleroma.Web.Router do
     get("/oauth_tokens", TwitterAPI.Controller, :oauth_tokens)
     delete("/oauth_tokens/:id", TwitterAPI.Controller, :revoke_token)
 
-    post("/qvitter/statuses/notifications/read", TwitterAPI.Controller, :notifications_read)
+    post(
+      "/qvitter/statuses/notifications/read",
+      TwitterAPI.Controller,
+      :mark_notifications_as_read
+    )
   end
 
   pipeline :ostatus do
@@ -572,6 +585,7 @@ defmodule Pleroma.Web.Router do
     post("/users/:nickname/outbox", ActivityPubController, :update_outbox)
     post("/api/ap/upload_media", ActivityPubController, :upload_media)
 
+    # The following two are S2S as well, see `ActivityPub.fetch_follow_information_for_user/1`:
     get("/users/:nickname/followers", ActivityPubController, :followers)
     get("/users/:nickname/following", ActivityPubController, :following)
   end
@@ -647,11 +661,28 @@ defmodule Pleroma.Web.Router do
 
   # Test-only routes needed to test action dispatching and plug chain execution
   if Pleroma.Config.get(:env) == :test do
+    @test_actions [
+      :do_oauth_check,
+      :fallback_oauth_check,
+      :skip_oauth_check,
+      :fallback_oauth_skip_publicity_check,
+      :skip_oauth_skip_publicity_check,
+      :missing_oauth_check_definition
+    ]
+
+    scope "/test/api", Pleroma.Tests do
+      pipe_through(:api)
+
+      for action <- @test_actions do
+        get("/#{action}", AuthTestController, action)
+      end
+    end
+
     scope "/test/authenticated_api", Pleroma.Tests do
       pipe_through(:authenticated_api)
 
-      for action <- [:skipped_oauth, :performed_oauth, :missed_oauth] do
+      for action <- @test_actions do
-        get("/#{action}", OAuthTestController, action)
+        get("/#{action}", AuthTestController, action)
       end
     end
   end

lib/pleroma/web/templates/layout/static_fe.html.eex

@@ -5,7 +5,7 @@
     <meta name="viewport" content="width=device-width,initial-scale=1,minimal-ui" />
     <title><%= Pleroma.Config.get([:instance, :name]) %></title>
     <%= Phoenix.HTML.raw(assigns[:meta] || "") %>
-    <link rel="stylesheet" href="/static/static-fe.css">
+    <link rel="stylesheet" href="/static-fe/static-fe.css">
   </head>
   <body>
     <div class="container">

lib/pleroma/web/twitter_api/controllers/util_controller.ex

@@ -25,13 +25,6 @@ defmodule Pleroma.Web.TwitterAPI.UtilController do
     when action == :follow_import
   )
 
-  # Note: follower can submit the form (with password auth) not being signed in (having no token)
-  plug(
-    OAuthScopesPlug,
-    %{fallback: :proceed_unauthenticated, scopes: ["follow", "write:follows"]}
-    when action == :do_remote_follow
-  )
-
   plug(OAuthScopesPlug, %{scopes: ["follow", "write:blocks"]} when action == :blocks_import)
 
   plug(

lib/pleroma/web/twitter_api/twitter_api.ex

@@ -3,82 +3,39 @@
 # SPDX-License-Identifier: AGPL-3.0-only
 
 defmodule Pleroma.Web.TwitterAPI.TwitterAPI do
+  import Pleroma.Web.Gettext
+
   alias Pleroma.Emails.Mailer
   alias Pleroma.Emails.UserEmail
   alias Pleroma.Repo
   alias Pleroma.User
   alias Pleroma.UserInviteToken
 
-  require Pleroma.Constants
-
   def register_user(params, opts \\ []) do
-    token = params["token"]
+    params =
-    trusted_app? = params["trusted_app"]
+      params
+      |> Map.take([:email, :token, :password])
+      |> Map.put(:bio, params |> Map.get(:bio, "") |> User.parse_bio())
+      |> Map.put(:nickname, params[:username])
+      |> Map.put(:name, Map.get(params, :fullname, params[:username]))
+      |> Map.put(:password_confirmation, params[:password])
 
-    params = %{
+    if Pleroma.Config.get([:instance, :registrations_open]) do
-      nickname: params["nickname"],
-      name: params["fullname"],
-      bio: User.parse_bio(params["bio"]),
-      email: params["email"],
-      password: params["password"],
-      password_confirmation: params["confirm"],
-      captcha_solution: params["captcha_solution"],
-      captcha_token: params["captcha_token"],
-      captcha_answer_data: params["captcha_answer_data"]
-    }
-
-    captcha_enabled = Pleroma.Config.get([Pleroma.Captcha, :enabled])
-    # true if captcha is disabled or enabled and valid, false otherwise
-    captcha_ok =
-      if trusted_app? || not captcha_enabled do
-        :ok
-      else
-        Pleroma.Captcha.validate(
-          params[:captcha_token],
-          params[:captcha_solution],
-          params[:captcha_answer_data]
-        )
-      end
-
-    # Captcha invalid
-    if captcha_ok != :ok do
-      {:error, error} = captcha_ok
-      # I have no idea how this error handling works
-      {:error, %{error: Jason.encode!(%{captcha: [error]})}}
-    else
-      registration_process(
-        params,
-        %{
-          registrations_open: Pleroma.Config.get([:instance, :registrations_open]),
-          token: token
-        },
-        opts
-      )
-    end
-  end
-
-  defp registration_process(params, %{registrations_open: true}, opts) do
       create_user(params, opts)
+    else
+      create_user_with_invite(params, opts)
+    end
   end
 
-  defp registration_process(params, %{token: token}, opts) do
+  defp create_user_with_invite(params, opts) do
-    invite =
+    with %{token: token} when is_binary(token) <- params,
-      unless is_nil(token) do
+         %UserInviteToken{} = invite <- Repo.get_by(UserInviteToken, %{token: token}),
-        Repo.get_by(UserInviteToken, %{token: token})
+         true <- UserInviteToken.valid_invite?(invite) do
-      end
-
-    valid_invite? = invite && UserInviteToken.valid_invite?(invite)
-
-    case invite do
-      nil ->
-        {:error, "Invalid token"}
-
-      invite when valid_invite? ->
       UserInviteToken.update_usage!(invite)
       create_user(params, opts)
-
+    else
-      _ ->
+      nil -> {:error, "Invalid token"}
-        {:error, "Expired token"}
+      _ -> {:error, "Expired token"}
     end
   end
 
@@ -91,16 +48,17 @@ defp create_user(params, opts) do
 
       {:error, changeset} ->
         errors =
-          Ecto.Changeset.traverse_errors(changeset, fn {msg, _opts} -> msg end)
+          changeset
+          |> Ecto.Changeset.traverse_errors(fn {msg, _opts} -> msg end)
           |> Jason.encode!()
 
-        {:error, %{error: errors}}
+        {:error, errors}
     end
   end
 
   def password_reset(nickname_or_email) do
     with true <- is_binary(nickname_or_email),
-         %User{local: true, email: email} = user when not is_nil(email) <-
+         %User{local: true, email: email} = user when is_binary(email) <-
            User.get_by_nickname_or_email(nickname_or_email),
          {:ok, token_record} <- Pleroma.PasswordResetToken.create_token(user) do
       user
@@ -122,4 +80,58 @@ def password_reset(nickname_or_email) do
         {:error, "unknown user"}
     end
   end
+
+  def validate_captcha(app, params) do
+    if app.trusted || not Pleroma.Captcha.enabled?() do
+      :ok
+    else
+      do_validate_captcha(params)
+    end
+  end
+
+  defp do_validate_captcha(params) do
+    with :ok <- validate_captcha_presence(params),
+         :ok <-
+           Pleroma.Captcha.validate(
+             params[:captcha_token],
+             params[:captcha_solution],
+             params[:captcha_answer_data]
+           ) do
+      :ok
+    else
+      {:error, :captcha_error} ->
+        captcha_error(dgettext("errors", "CAPTCHA Error"))
+
+      {:error, :invalid} ->
+        captcha_error(dgettext("errors", "Invalid CAPTCHA"))
+
+      {:error, :kocaptcha_service_unavailable} ->
+        captcha_error(dgettext("errors", "Kocaptcha service unavailable"))
+
+      {:error, :expired} ->
+        captcha_error(dgettext("errors", "CAPTCHA expired"))
+
+      {:error, :already_used} ->
+        captcha_error(dgettext("errors", "CAPTCHA already used"))
+
+      {:error, :invalid_answer_data} ->
+        captcha_error(dgettext("errors", "Invalid answer data"))
+
+      {:error, error} ->
+        captcha_error(error)
+    end
+  end
+
+  defp validate_captcha_presence(params) do
+    [:captcha_solution, :captcha_token, :captcha_answer_data]
+    |> Enum.find_value(:ok, fn key ->
+      unless is_binary(params[key]) do
+        error = dgettext("errors", "Invalid CAPTCHA (Missing parameter: %{name})", name: key)
+        {:error, error}
+      end
+    end)
+  end
+
+  # For some reason FE expects error message to be a serialized JSON
+  defp captcha_error(error), do: {:error, Jason.encode!(%{captcha: [error]})}
 end

lib/pleroma/web/twitter_api/twitter_api_controller.ex

@@ -6,6 +6,7 @@ defmodule Pleroma.Web.TwitterAPI.Controller do
   use Pleroma.Web, :controller
 
   alias Pleroma.Notification
+  alias Pleroma.Plugs.EnsurePublicOrAuthenticatedPlug
   alias Pleroma.Plugs.OAuthScopesPlug
   alias Pleroma.User
   alias Pleroma.Web.OAuth.Token
@@ -13,12 +14,18 @@ defmodule Pleroma.Web.TwitterAPI.Controller do
 
   require Logger
 
-  plug(OAuthScopesPlug, %{scopes: ["write:notifications"]} when action == :notifications_read)
+  plug(
+    OAuthScopesPlug,
+    %{scopes: ["write:notifications"]} when action == :mark_notifications_as_read
+  )
+
+  plug(
+    :skip_plug,
+    [OAuthScopesPlug, EnsurePublicOrAuthenticatedPlug] when action == :confirm_email
+  )
 
   plug(:skip_plug, OAuthScopesPlug when action in [:oauth_tokens, :revoke_token])
 
-  plug(Pleroma.Plugs.EnsurePublicOrAuthenticatedPlug)
-
   action_fallback(:errors)
 
   def confirm_email(conn, %{"user_id" => uid, "token" => token}) do
@@ -46,13 +53,13 @@ def revoke_token(%{assigns: %{user: user}} = conn, %{"id" => id} = _params) do
     json_reply(conn, 201, "")
   end
 
-  def errors(conn, {:param_cast, _}) do
+  defp errors(conn, {:param_cast, _}) do
     conn
     |> put_status(400)
     |> json("Invalid parameters")
   end
 
-  def errors(conn, _) do
+  defp errors(conn, _) do
     conn
     |> put_status(500)
     |> json("Something went wrong")
@@ -64,7 +71,10 @@ defp json_reply(conn, status, json) do
     |> send_resp(status, json)
   end
 
-  def notifications_read(%{assigns: %{user: user}} = conn, %{"latest_id" => latest_id} = params) do
+  def mark_notifications_as_read(
+        %{assigns: %{user: user}} = conn,
+        %{"latest_id" => latest_id} = params
+      ) do
     Notification.set_read_up_to(user, latest_id)
 
     notifications = Notification.for_user(user, params)
@@ -75,7 +85,7 @@ def notifications_read(%{assigns: %{user: user}} = conn, %{"latest_id" => latest
     |> render("index.json", %{notifications: notifications, for: user})
   end
 
-  def notifications_read(%{assigns: %{user: _user}} = conn, _) do
+  def mark_notifications_as_read(%{assigns: %{user: _user}} = conn, _) do
     bad_request_reply(conn, "You need to specify latest_id")
   end
 

lib/pleroma/web/web.ex

@@ -2,6 +2,11 @@
 # Copyright © 2017-2020 Pleroma Authors <https://pleroma.social/>
 # SPDX-License-Identifier: AGPL-3.0-only
 
+defmodule Pleroma.Web.Plug do
+  # Substitute for `call/2` which is defined with `use Pleroma.Web, :plug`
+  @callback perform(Plug.Conn.t(), Plug.opts()) :: Plug.Conn.t()
+end
+
 defmodule Pleroma.Web do
   @moduledoc """
   A module that keeps using definitions for controllers,
@@ -20,44 +25,91 @@ defmodule Pleroma.Web do
   below.
   """
 
+  alias Pleroma.Plugs.EnsureAuthenticatedPlug
+  alias Pleroma.Plugs.EnsurePublicOrAuthenticatedPlug
+  alias Pleroma.Plugs.ExpectAuthenticatedCheckPlug
+  alias Pleroma.Plugs.ExpectPublicOrAuthenticatedCheckPlug
+  alias Pleroma.Plugs.OAuthScopesPlug
+  alias Pleroma.Plugs.PlugHelper
+
   def controller do
     quote do
       use Phoenix.Controller, namespace: Pleroma.Web
 
       import Plug.Conn
+
       import Pleroma.Web.Gettext
       import Pleroma.Web.Router.Helpers
       import Pleroma.Web.TranslationHelpers
 
-      alias Pleroma.Plugs.PlugHelper
-
       plug(:set_put_layout)
 
       defp set_put_layout(conn, _) do
         put_layout(conn, Pleroma.Config.get(:app_layout, "app.html"))
       end
 
-      # Marks a plug intentionally skipped and blocks its execution if it's present in plugs chain
+      # Marks plugs intentionally skipped and blocks their execution if present in plugs chain
-      defp skip_plug(conn, plug_module) do
+      defp skip_plug(conn, plug_modules) do
+        plug_modules
+        |> List.wrap()
+        |> Enum.reduce(
+          conn,
+          fn plug_module, conn ->
             try do
               plug_module.skip_plug(conn)
             rescue
               UndefinedFunctionError ->
-            raise "#{plug_module} is not skippable. Append `use Pleroma.Web, :plug` to its code."
+                raise "`#{plug_module}` is not skippable. Append `use Pleroma.Web, :plug` to its code."
             end
           end
+        )
+      end
 
       # Executed just before actual controller action, invokes before-action hooks (callbacks)
       defp action(conn, params) do
-        with %Plug.Conn{halted: false} <- maybe_halt_on_missing_oauth_scopes_check(conn) do
+        with %{halted: false} = conn <- maybe_drop_authentication_if_oauth_check_ignored(conn),
+             %{halted: false} = conn <- maybe_perform_public_or_authenticated_check(conn),
+             %{halted: false} = conn <- maybe_perform_authenticated_check(conn),
+             %{halted: false} = conn <- maybe_halt_on_missing_oauth_scopes_check(conn) do
           super(conn, params)
         end
       end
 
+      # For non-authenticated API actions, drops auth info if OAuth scopes check was ignored
+      #   (neither performed nor explicitly skipped)
+      defp maybe_drop_authentication_if_oauth_check_ignored(conn) do
+        if PlugHelper.plug_called?(conn, ExpectPublicOrAuthenticatedCheckPlug) and
+             not PlugHelper.plug_called_or_skipped?(conn, OAuthScopesPlug) do
+          OAuthScopesPlug.drop_auth_info(conn)
+        else
+          conn
+        end
+      end
+
+      # Ensures instance is public -or- user is authenticated if such check was scheduled
+      defp maybe_perform_public_or_authenticated_check(conn) do
+        if PlugHelper.plug_called?(conn, ExpectPublicOrAuthenticatedCheckPlug) do
+          EnsurePublicOrAuthenticatedPlug.call(conn, %{})
+        else
+          conn
+        end
+      end
+
+      # Ensures user is authenticated if such check was scheduled
+      # Note: runs prior to action even if it was already executed earlier in plug chain
+      #   (since OAuthScopesPlug has option of proceeding unauthenticated)
+      defp maybe_perform_authenticated_check(conn) do
+        if PlugHelper.plug_called?(conn, ExpectAuthenticatedCheckPlug) do
+          EnsureAuthenticatedPlug.call(conn, %{})
+        else
+          conn
+        end
+      end
+
       # Halts if authenticated API action neither performs nor explicitly skips OAuth scopes check
       defp maybe_halt_on_missing_oauth_scopes_check(conn) do
-        if Pleroma.Plugs.AuthExpectedPlug.auth_expected?(conn) &&
+        if PlugHelper.plug_called?(conn, ExpectAuthenticatedCheckPlug) and
-             not PlugHelper.plug_called_or_skipped?(conn, Pleroma.Plugs.OAuthScopesPlug) do
+             not PlugHelper.plug_called_or_skipped?(conn, OAuthScopesPlug) do
           conn
           |> render_error(
             :forbidden,
@@ -132,7 +184,8 @@ def channel do
 
   def plug do
     quote do
-      alias Pleroma.Plugs.PlugHelper
+      @behaviour Pleroma.Web.Plug
+      @behaviour Plug
 
       @doc """
       Marks a plug intentionally skipped and blocks its execution if it's present in plugs chain.
@@ -146,14 +199,22 @@ def skip_plug(conn) do
       end
 
       @impl Plug
-      @doc "If marked as skipped, returns `conn`, and calls `perform/2` otherwise."
+      @doc """
+      If marked as skipped, returns `conn`, otherwise calls `perform/2`.
+      Note: multiple invocations of the same plug (with different or same options) are allowed.
+      """
       def call(%Plug.Conn{} = conn, options) do
         if PlugHelper.plug_skipped?(conn, __MODULE__) do
           conn
         else
-          conn
+          conn =
-          |> PlugHelper.append_to_private_list(PlugHelper.called_plugs_list_id(), __MODULE__)
+            PlugHelper.append_to_private_list(
-          |> perform(options)
+              conn,
+              PlugHelper.called_plugs_list_id(),
+              __MODULE__
+            )
+
+          apply(__MODULE__, :perform, [conn, options])
         end
       end
     end

mix.exs

@@ -189,7 +189,9 @@ defp deps do
        ref: "e0f16822d578866e186a0974d65ad58cddc1e2ab"},
       {:mox, "~> 0.5", only: :test},
       {:restarter, path: "./restarter"},
-      {:open_api_spex, "~> 3.6"}
+      {:open_api_spex,
+       git: "https://git.pleroma.social/pleroma/elixir-libraries/open_api_spex.git",
+       ref: "b862ebd78de0df95875cf46feb6e9607130dc2a8"}
     ] ++ oauth_deps()
   end
 

mix.lock

@@ -74,7 +74,7 @@
   "nimble_parsec": {:hex, :nimble_parsec, "0.5.3", "def21c10a9ed70ce22754fdeea0810dafd53c2db3219a0cd54cf5526377af1c6", [:mix], [], "hexpm", "589b5af56f4afca65217a1f3eb3fee7e79b09c40c742fddc1c312b3ac0b3399f"},
   "nodex": {:git, "https://git.pleroma.social/pleroma/nodex", "cb6730f943cfc6aad674c92161be23a8411f15d1", [ref: "cb6730f943cfc6aad674c92161be23a8411f15d1"]},
   "oban": {:hex, :oban, "1.2.0", "7cca94d341be43d220571e28f69131c4afc21095b25257397f50973d3fc59b07", [:mix], [{:ecto_sql, "~> 3.1", [hex: :ecto_sql, repo: "hexpm", optional: false]}, {:jason, "~> 1.1", [hex: :jason, repo: "hexpm", optional: false]}, {:postgrex, "~> 0.14", [hex: :postgrex, repo: "hexpm", optional: false]}, {:telemetry, "~> 0.4", [hex: :telemetry, repo: "hexpm", optional: false]}], "hexpm", "ba5f8b3f7d76967b3e23cf8014f6a13e4ccb33431e4808f036709a7f822362ee"},
-  "open_api_spex": {:hex, :open_api_spex, "3.6.0", "64205aba9f2607f71b08fd43e3351b9c5e9898ec5ef49fc0ae35890da502ade9", [:mix], [{:jason, "~> 1.0", [hex: :jason, repo: "hexpm", optional: true]}, {:plug, "~> 1.7", [hex: :plug, repo: "hexpm", optional: false]}, {:poison, "~> 3.1", [hex: :poison, repo: "hexpm", optional: true]}], "hexpm", "126ba3473966277132079cb1d5bf1e3df9e36fe2acd00166e75fd125cecb59c5"},
+  "open_api_spex": {:git, "https://git.pleroma.social/pleroma/elixir-libraries/open_api_spex.git", "b862ebd78de0df95875cf46feb6e9607130dc2a8", [ref: "b862ebd78de0df95875cf46feb6e9607130dc2a8"]},
   "parse_trans": {:hex, :parse_trans, "3.3.0", "09765507a3c7590a784615cfd421d101aec25098d50b89d7aa1d66646bc571c1", [:rebar3], [], "hexpm", "17ef63abde837ad30680ea7f857dd9e7ced9476cdd7b0394432af4bfc241b960"},
   "pbkdf2_elixir": {:hex, :pbkdf2_elixir, "0.12.4", "8dd29ed783f2e12195d7e0a4640effc0a7c37e6537da491f1db01839eee6d053", [:mix], [], "hexpm", "595d09db74cb093b1903381c9de423276a931a2480a46a1a5dc7f932a2a6375b"},
   "phoenix": {:hex, :phoenix, "1.4.13", "67271ad69b51f3719354604f4a3f968f83aa61c19199343656c9caee057ff3b8", [:mix], [{:jason, "~> 1.0", [hex: :jason, repo: "hexpm", optional: true]}, {:phoenix_pubsub, "~> 1.1", [hex: :phoenix_pubsub, repo: "hexpm", optional: false]}, {:plug, "~> 1.8.1 or ~> 1.9", [hex: :plug, repo: "hexpm", optional: false]}, {:plug_cowboy, "~> 1.0 or ~> 2.0", [hex: :plug_cowboy, repo: "hexpm", optional: true]}, {:telemetry, "~> 0.4", [hex: :telemetry, repo: "hexpm", optional: false]}], "hexpm", "ab765a0feddb81fc62e2116c827b5f068df85159c162bee760745276ad7ddc1b"},