Merge branch 'develop' of https://git.pleroma.social/pleroma/pleroma into develop

CHANGELOG.md

@@ -4,35 +4,52 @@ All notable changes to this project will be documented in this file.
 The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/).
 
 ## [unreleased]
+
+### Changed
+<details>
+  <summary>API Changes</summary>
+- **Breaking:** Emoji API: changed methods and renamed routes.
+</details>
+
 ### Removed
 - **Breaking:** removed `with_move` parameter from notifications timeline.
 
 ### Added
+- Instance: Extend `/api/v1/instance` with Pleroma-specific information.
 - NodeInfo: `pleroma:api/v1/notifications:include_types_filter` to the `features` list.
 - NodeInfo: `pleroma_emoji_reactions` to the `features` list.
 - Configuration: `:restrict_unauthenticated` setting, restrict access for unauthenticated users to timelines (public and federate), user profiles and statuses.
 - New HTTP adapter [gun](https://github.com/ninenines/gun). Gun adapter requires minimum OTP version of 22.2 otherwise Pleroma won’t start. For hackney OTP update is not required.
 - Mix task to create trusted OAuth App.
-- Notifications: Added `follow_request` notification type (configurable, see `[:notifications, :enable_follow_request_notifications]` setting).
+- Notifications: Added `follow_request` notification type.
 - Added `:reject_deletes` group to SimplePolicy
 <details>
   <summary>API Changes</summary>
+- Mastodon API: Extended `/api/v1/instance`.
 - Mastodon API: Support for `include_types` in `/api/v1/notifications`.
 - Mastodon API: Added `/api/v1/notifications/:id/dismiss` endpoint.
 - Mastodon API: Add support for filtering replies in public and home timelines
 - Admin API: endpoints for create/update/delete OAuth Apps.
+- Admin API: endpoint for status view.
 </details>
 
 ### Fixed
 - Support pagination in conversations API
 - **Breaking**: SimplePolicy `:reject` and `:accept` allow deletions again
 - Fix follower/blocks import when nicknames starts with @
+- Filtering of push notifications on activities from blocked domains
+- Resolving Peertube accounts with Webfinger
 
 ## [unreleased-patch]
+### Security
+- Disallow re-registration of previously deleted users, which allowed viewing direct messages addressed to them
+- Mastodon API: Fix `POST /api/v1/follow_requests/:id/authorize` allowing to force a follow from a local user even if they didn't request to follow
+
 ### Fixed
 - Logger configuration through AdminFE
 - HTTP Basic Authentication permissions issue
 - ObjectAgePolicy didn't filter out old messages
+- Transmogrifier: Keep object sensitive settings for outgoing representation (AP C2S)
 
 ### Added
 - NodeInfo: ObjectAgePolicy settings to the `federation` list.
@@ -129,7 +146,7 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/).
 - **Breaking:** Admin API: Return link alongside with token on password reset
 - **Breaking:** Admin API: `PUT /api/pleroma/admin/reports/:id` is now `PATCH /api/pleroma/admin/reports`, see admin_api.md for details
 - **Breaking:** `/api/pleroma/admin/users/invite_token` now uses `POST`, changed accepted params and returns full invite in json instead of only token string.
-- **Breaking** replying to reports is now "report notes", enpoint changed from `POST /api/pleroma/admin/reports/:id/respond` to `POST /api/pleroma/admin/reports/:id/notes`
+- **Breaking** replying to reports is now "report notes", endpoint changed from `POST /api/pleroma/admin/reports/:id/respond` to `POST /api/pleroma/admin/reports/:id/notes`
 - Mastodon API: stopped sanitizing display names, field names and subject fields since they are supposed to be treated as plaintext
 - Admin API: Return `total` when querying for reports
 - Mastodon API: Return `pleroma.direct_conversation_id` when creating a direct message (`POST /api/v1/statuses`)
@@ -139,6 +156,7 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/).
 - Mastodon API: `pleroma.thread_muted` to the Status entity
 - Mastodon API: Mark the direct conversation as read for the author when they send a new direct message
 - Mastodon API, streaming: Add `pleroma.direct_conversation_id` to the `conversation` stream event payload.
+- Mastodon API: Add `pleroma.unread_count` to the Marker entity
 - Admin API: Render whole status in grouped reports
 - Mastodon API: User timelines will now respect blocks, unless you are getting the user timeline of somebody you blocked (which would be empty otherwise).
 - Mastodon API: Favoriting / Repeating a post multiple times will now return the identical response every time. Before, executing that action twice would return an error ("already favorited") on the second try.

config/config.exs

@@ -239,7 +239,18 @@
   account_field_value_length: 2048,
   external_user_synchronization: true,
   extended_nickname_format: true,
-  cleanup_attachments: false
+  cleanup_attachments: false,
+  multi_factor_authentication: [
+    totp: [
+      # digits 6 or 8
+      digits: 6,
+      period: 30
+    ],
+    backup_codes: [
+      number: 5,
+      length: 16
+    ]
+  ]
 
 config :pleroma, :extensions, output_relationships_in_statuses_by_default: true
 
@@ -563,8 +574,6 @@
     inactivity_threshold: 7
   }
 
-config :pleroma, :notifications, enable_follow_request_notifications: false
-
 config :pleroma, :oauth2,
   token_expires_in: 600,
   issue_new_refresh_token: true,
@@ -656,6 +665,8 @@
   profiles: %{local: false, remote: false},
   activities: %{local: false, remote: false}
 
+config :pleroma, Pleroma.Web.ApiSpec.CastAndValidate, strict: false
+
 # Import environment specific config. This must remain at the bottom
 # of this file so it overrides the configuration defined above.
 import_config "#{Mix.env()}.exs"

config/description.exs

@@ -919,6 +919,62 @@
         key: :external_user_synchronization,
         type: :boolean,
         description: "Enabling following/followers counters synchronization for external users"
+      },
+      %{
+        key: :multi_factor_authentication,
+        type: :keyword,
+        description: "Multi-factor authentication settings",
+        suggestions: [
+          [
+            totp: [digits: 6, period: 30],
+            backup_codes: [number: 5, length: 16]
+          ]
+        ],
+        children: [
+          %{
+            key: :totp,
+            type: :keyword,
+            description: "TOTP settings",
+            suggestions: [digits: 6, period: 30],
+            children: [
+              %{
+                key: :digits,
+                type: :integer,
+                suggestions: [6],
+                description:
+                  "Determines the length of a one-time pass-code, in characters. Defaults to 6 characters."
+              },
+              %{
+                key: :period,
+                type: :integer,
+                suggestions: [30],
+                description:
+                  "a period for which the TOTP code will be valid, in seconds. Defaults to 30 seconds."
+              }
+            ]
+          },
+          %{
+            key: :backup_codes,
+            type: :keyword,
+            description: "MFA backup codes settings",
+            suggestions: [number: 5, length: 16],
+            children: [
+              %{
+                key: :number,
+                type: :integer,
+                suggestions: [5],
+                description: "number of backup codes to generate."
+              },
+              %{
+                key: :length,
+                type: :integer,
+                suggestions: [16],
+                description:
+                  "Determines the length of backup one-time pass-codes, in characters. Defaults to 16 characters."
+              }
+            ]
+          }
+        ]
       }
     ]
   },
@@ -2247,6 +2303,7 @@
         children: [
           %{
             key: :active,
+            label: "Enabled",
             type: :boolean,
             description: "Globally enable or disable digest emails"
           },
@@ -2273,20 +2330,6 @@
       }
     ]
   },
-  %{
-    group: :pleroma,
-    key: :notifications,
-    type: :group,
-    description: "Notification settings",
-    children: [
-      %{
-        key: :enable_follow_request_notifications,
-        type: :boolean,
-        description:
-          "Enables notifications on new follow requests (causes issues with older PleromaFE versions)."
-      }
-    ]
-  },
   %{
     group: :pleroma,
     key: Pleroma.Emails.UserEmail,
@@ -3208,5 +3251,19 @@
         ]
       }
     ]
+  },
+  %{
+    group: :pleroma,
+    key: Pleroma.Web.ApiSpec.CastAndValidate,
+    type: :group,
+    children: [
+      %{
+        key: :strict,
+        type: :boolean,
+        description:
+          "Enables strict input validation (useful in development, not recommended in production)",
+        suggestions: [false]
+      }
+    ]
   }
 ]

config/dev.exs

@@ -52,6 +52,8 @@
   hostname: "localhost",
   pool_size: 10
 
+config :pleroma, Pleroma.Web.ApiSpec.CastAndValidate, strict: true
+
 if File.exists?("./config/dev.secret.exs") do
   import_config "dev.secret.exs"
 else

config/test.exs

@@ -56,6 +56,19 @@
   ignore_hosts: [],
   ignore_tld: ["local", "localdomain", "lan"]
 
+config :pleroma, :instance,
+  multi_factor_authentication: [
+    totp: [
+      # digits 6 or 8
+      digits: 6,
+      period: 30
+    ],
+    backup_codes: [
+      number: 2,
+      length: 6
+    ]
+  ]
+
 config :web_push_encryption, :vapid_details,
   subject: "mailto:administrator@example.com",
   public_key:
@@ -96,6 +109,8 @@
 
 config :pleroma, Pleroma.Plugs.RemoteIp, enabled: false
 
+config :pleroma, Pleroma.Web.ApiSpec.CastAndValidate, strict: true
+
 if File.exists?("./config/test.secret.exs") do
   import_config "test.secret.exs"
 else

docs/API/admin_api.md

@@ -409,6 +409,7 @@ Note: Available `:permission_group` is currently moderator and admin. 404 is ret
 
 ### Get a password reset token for a given nickname
 
+
 - Params: none
 - Response:
 
@@ -427,6 +428,14 @@ Note: Available `:permission_group` is currently moderator and admin. 404 is ret
   - `nicknames`
 - Response: none (code `204`)
 
+## PUT `/api/pleroma/admin/users/disable_mfa`
+
+### Disable mfa for user's account.
+
+- Params:
+  - `nickname`
+- Response: User’s nickname
+
 ## `GET /api/pleroma/admin/users/:nickname/credentials`
 
 ### Get the user's email, password, display and settings-related fields
@@ -755,6 +764,17 @@ Note: Available `:permission_group` is currently moderator and admin. 404 is ret
     - 400 Bad Request `"Invalid parameters"` when `status` is missing
   - On success: `204`, empty response
 
+## `GET /api/pleroma/admin/statuses/:id`
+
+### Show status by id
+
+- Params:
+  - `id`: required, status id
+- Response:
+  - On failure:
+    - 404 Not Found `"Not Found"`
+  - On success: JSON, Mastodon Status entity
+
 ## `PUT /api/pleroma/admin/statuses/:id`
 
 ### Change the scope of an individual reported status

docs/API/differences_in_mastoapi_responses.md

@@ -61,6 +61,7 @@ Has these additional fields under the `pleroma` object:
 - `deactivated`: boolean, true when the user is deactivated
 - `allow_following_move`: boolean, true when the user allows automatically follow moved following accounts
 - `unread_conversation_count`: The count of unread conversations. Only returned to the account owner.
+- `unread_notifications_count`: The count of unread notifications. Only returned to the account owner.
 
 ### Source
 
@@ -202,4 +203,25 @@ Has theses additional parameters (which are the same as in Pleroma-API):
 - `bio`: optional
 - `captcha_solution`: optional, contains provider-specific captcha solution,
 - `captcha_token`: optional, contains provider-specific captcha token
+- `captcha_answer_data`: optional, contains provider-specific captcha data
 - `token`: invite token required when the registrations aren't public.
+
+## Instance
+
+`GET /api/v1/instance` has additional fields
+
+- `max_toot_chars`: The maximum characters per post
+- `poll_limits`: The limits of polls
+- `upload_limit`: The maximum upload file size
+- `avatar_upload_limit`: The same for avatars
+- `background_upload_limit`: The same for backgrounds
+- `banner_upload_limit`: The same for banners
+- `pleroma.metadata.features`: A list of supported features
+- `pleroma.metadata.federation`: The federation restrictions of this instance
+- `vapid_public_key`: The public key needed for push messages
+
+## Markers
+
+Has these additional fields under the `pleroma` object:
+
+- `unread_count`: contains number unread notifications

docs/API/pleroma_api.md

@@ -70,7 +70,49 @@ Request parameters can be passed via [query strings](https://en.wikipedia.org/wi
 * Response: JSON. Returns `{"status": "success"}` if the account was successfully disabled, `{"error": "[error message]"}` otherwise
 * Example response: `{"error": "Invalid password."}`
 
-## `/api/pleroma/admin/`…
+## `/api/pleroma/accounts/mfa`
+#### Gets current MFA settings
+* method: `GET`
+* Authentication: required
+* OAuth scope: `read:security`
+* Response: JSON. Returns `{"enabled": "false", "totp": false }`
+
+## `/api/pleroma/accounts/mfa/setup/totp`
+#### Pre-setup the MFA/TOTP method
+* method: `GET`
+* Authentication: required
+* OAuth scope: `write:security`
+* Response: JSON. Returns `{"key": [secret_key], "provisioning_uri": "[qr code uri]"  }` when successful, otherwise returns HTTP 422 `{"error": "error_msg"}`
+
+## `/api/pleroma/accounts/mfa/confirm/totp`
+#### Confirms & enables MFA/TOTP support for user account.
+* method: `POST`
+* Authentication: required
+* OAuth scope: `write:security`
+* Params:
+    * `password`: user's password
+    * `code`: token from TOTP App
+* Response: JSON. Returns `{}` if the enable was successful, HTTP 422 `{"error": "[error message]"}` otherwise
+
+
+## `/api/pleroma/accounts/mfa/totp`
+####  Disables MFA/TOTP method for user account.
+* method: `DELETE`
+* Authentication: required
+* OAuth scope: `write:security`
+* Params:
+    * `password`: user's password
+* Response: JSON. Returns `{}` if the disable was successful, HTTP 422 `{"error": "[error message]"}` otherwise
+* Example response: `{"error": "Invalid password."}`
+
+## `/api/pleroma/accounts/mfa/backup_codes`
+####  Generstes backup codes MFA for user account.
+* method: `GET`
+* Authentication: required
+* OAuth scope: `write:security`
+* Response: JSON. Returns `{"codes": codes}`when successful, otherwise HTTP 422 `{"error": "[error message]"}`
+
+## `/api/pleroma/admin/`
 See [Admin-API](admin_api.md)
 
 ## `/api/v1/pleroma/notifications/read`
@@ -323,20 +365,54 @@ The status posting endpoint takes an additional parameter, `in_reply_to_conversa
 * Params: None
 * Response: JSON, returns a list of Mastodon Conversation entities that were marked as read (200 - healthy, 503 unhealthy).
 
-## `GET /api/pleroma/emoji/packs`
+## `GET /api/pleroma/emoji/packs/import`
-### Lists the custom emoji packs on the server
+### Imports packs from filesystem
 * Method `GET`
-* Authentication: not required
+* Authentication: required
 * Params: None
-* Response: JSON, "ok" and 200 status and the JSON hashmap of "pack name" to "pack contents"
+* Response: JSON, returns a list of imported packs.
 
-## `PUT /api/pleroma/emoji/packs/:name`
+## `GET /api/pleroma/emoji/packs/remote`
-### Creates an empty custom emoji pack
+### Make request to another instance for packs list
-* Method `PUT`
+* Method `GET`
+* Authentication: required
+* Params:
+  * `url`: url of the instance to get packs from
+* Response: JSON with the pack list, hashmap with pack name and pack contents
+
+## `POST /api/pleroma/emoji/packs/download`
+### Download pack from another instance
+* Method `POST`
+* Authentication: required
+* Params:
+  * `url`: url of the instance to download from
+  * `name`: pack to download from that instance
+  * `as`: (*optional*) name how to save pack
+* Response: JSON, "ok" with 200 status if the pack was downloaded, or 500 if there were
+  errors downloading the pack
+
+## `POST /api/pleroma/emoji/packs/:name`
+### Creates an empty pack
+* Method `POST`
 * Authentication: required
 * Params: None
 * Response: JSON, "ok" and 200 status or 409 if the pack with that name already exists
 
+## `PATCH /api/pleroma/emoji/packs/:name`
+### Updates (replaces) pack metadata
+* Method `PATCH`
+* Authentication: required
+* Params:
+  * `metadata`: metadata to replace the old one
+    * `license`: Pack license
+    * `homepage`: Pack home page url
+    * `description`: Pack description
+    * `fallback-src`: Fallback url to download pack from
+    * `fallback-src-sha256`: SHA256 encoded for fallback pack archive
+    * `share-files`: is pack allowed for sharing (boolean)
+* Response: JSON, updated "metadata" section of the pack and 200 status or 400 if there was a
+  problem with the new metadata (the error is specified in the "error" part of the response JSON)
+
 ## `DELETE /api/pleroma/emoji/packs/:name`
 ### Delete a custom emoji pack
 * Method `DELETE`
@@ -344,53 +420,51 @@ The status posting endpoint takes an additional parameter, `in_reply_to_conversa
 * Params: None
 * Response: JSON, "ok" and 200 status or 500 if there was an error deleting the pack
 
-## `POST /api/pleroma/emoji/packs/:name/update_file`
+## `POST /api/pleroma/emoji/packs/:name/files`
-### Update a file in a custom emoji pack
+### Add new file to the pack
 * Method `POST`
 * Authentication: required
 * Params:
-    * if the `action` is `add`, adds an emoji named `shortcode` to the pack `pack_name`,
+  * `file`: file needs to be uploaded with the multipart request or link to remote file.
-      that means that the emoji file needs to be uploaded with the request
+  * `shortcode`: (*optional*) shortcode for new emoji, must be uniq for all emoji. If not sended, shortcode will be taken from original filename.
-      (thus requiring it to be a multipart request) and be named `file`.
+  * `filename`: (*optional*) new emoji file name. If not specified will be taken from original filename.
-      There can also be an optional `filename` that will be the new emoji file name
+* Response: JSON, list of files for updated pack (hashmap -> shortcode => filename) with status 200, either error status with error message.
-      (if it's not there, the name will be taken from the uploaded file).
-    * if the `action` is `update`, changes emoji shortcode
-      (from `shortcode` to `new_shortcode` or moves the file (from the current filename to `new_filename`)
-    * if the `action` is `remove`, removes the emoji named `shortcode` and it's associated file
-* Response: JSON, updated "files" section of the pack and 200 status, 409 if the trying to use a shortcode
-  that is already taken, 400 if there was an error with the shortcode, filename or file (additional info
-  in the "error" part of the response JSON)
 
-## `POST /api/pleroma/emoji/packs/:name/update_metadata`
+## `PATCH /api/pleroma/emoji/packs/:name/files`
-### Updates (replaces) pack metadata
+### Update emoji file from pack
-* Method `POST`
+* Method `PATCH`
 * Authentication: required
 * Params:
-  * `new_data`: new metadata to replace the old one
+  * `shortcode`: emoji file shortcode
-* Response: JSON, updated "metadata" section of the pack and 200 status or 400 if there was a
+  * `new_shortcode`: new emoji file shortcode
-  problem with the new metadata (the error is specified in the "error" part of the response JSON)
+  * `new_filename`: new filename for emoji file
+  * `force`: (*optional*) with true value to overwrite existing emoji with new shortcode
+* Response: JSON, list with updated files for updated pack (hashmap -> shortcode => filename) with status 200, either error status with error message.
 
-## `POST /api/pleroma/emoji/packs/download_from`
+## `DELETE /api/pleroma/emoji/packs/:name/files`
-### Requests the instance to download the pack from another instance
+### Delete emoji file from pack
-* Method `POST`
+* Method `DELETE`
 * Authentication: required
 * Params:
-  * `instance_address`: the address of the instance to download from
+  * `shortcode`: emoji file shortcode
-  * `pack_name`: the pack to download from that instance
+* Response: JSON, list with updated files for updated pack (hashmap -> shortcode => filename) with status 200, either error status with error message.
-* Response: JSON, "ok" and 200 status if the pack was downloaded, or 500 if there were
-  errors downloading the pack
 
-## `POST /api/pleroma/emoji/packs/list_from`
+## `GET /api/pleroma/emoji/packs`
-### Requests the instance to list the packs from another instance
+### Lists local custom emoji packs
-* Method `POST`
+* Method `GET`
-* Authentication: required
+* Authentication: not required
-* Params:
+* Params: None
-  * `instance_address`: the address of the instance to download from
+* Response: JSON, "ok" and 200 status and the JSON hashmap of pack name to pack contents
-* Response: JSON with the pack list, same as if the request was made to that instance's
-  list endpoint directly + 200 status
 
-## `GET /api/pleroma/emoji/packs/:name/download_shared`
+## `GET /api/pleroma/emoji/packs/:name`
-### Requests a local pack from the instance
+### Get pack.json for the pack
+* Method `GET`
+* Authentication: not required
+* Params: None
+* Response: JSON, pack json with `files` and `pack` keys with 200 status or 404 if the pack does not exist
+
+## `GET /api/pleroma/emoji/packs/:name/archive`
+### Requests a local pack archive from the instance
 * Method `GET`
 * Authentication: not required
 * Params: None

docs/clients.md

@@ -49,11 +49,11 @@ Feel free to contact us to be added to this list!
 - Platforms: Android
 - Features: Streaming Ready
 
-### Roma
+### Fedi
-- Homepage: <https://www.pleroma.com/#mobileApps>
+- Homepage: <https://www.fediapp.com/>
-- Source Code: [iOS](https://github.com/roma-apps/roma-ios), [Android](https://github.com/roma-apps/roma-android)
+- Source Code: Proprietary, but free
 - Platforms: iOS, Android
-- Features: No Streaming
+- Features: Pleroma-specific features like Reactions
 
 ### Tusky
 - Homepage: <https://tuskyapp.github.io/>

docs/configuration/cheatsheet.md

@@ -8,6 +8,10 @@ For from source installations Pleroma configuration works by first importing the
 
 To add configuration to your config file, you can copy it from the base config. The latest version of it can be viewed [here](https://git.pleroma.social/pleroma/pleroma/blob/develop/config/config.exs). You can also use this file if you don't know how an option is supposed to be formatted.
 
+## :chat
+
+* `enabled` - Enables the backend chat. Defaults to `true`.
+
 ## :instance
 * `name`: The instance’s name.
 * `email`: Email used to reach an Administrator/Moderator of the instance.
@@ -903,12 +907,18 @@ config :auto_linker,
 
 * `runtime_dir`: A path to custom Elixir modules (such as MRF policies).
 
-
 ## :configurable_from_database
 
 Boolean, enables/disables in-database configuration. Read [Transfering the config to/from the database](../administration/CLI_tasks/config.md) for more information.
 
 
+### Multi-factor authentication -  :two_factor_authentication
+* `totp` - a list containing TOTP configuration
+  - `digits` - Determines the length of a one-time pass-code in characters. Defaults to 6 characters.
+  - `period` - a period for which the TOTP code will be valid in seconds. Defaults to 30 seconds.
+* `backup_codes` - a list containing backup codes configuration
+  - `number` - number of backup codes to generate.
+  - `length` - backup code length. Defaults to 16 characters.
 
 ## Restrict entities access for unauthenticated users
 
@@ -924,4 +934,9 @@ Restrict access for unauthenticated users to timelines (public and federate), us
   * `remote`
 * `activities` - statuses
   * `local`
-  * `remote`
+  * `remote`
+
+
+## Pleroma.Web.ApiSpec.CastAndValidate
+
+* `:strict` a boolean, enables strict input validation (useful in development, not recommended in production). Defaults to `false`.

installation/pleroma-apache.conf

@@ -32,9 +32,8 @@ CustomLog ${APACHE_LOG_DIR}/access.log combined
 
 <VirtualHost *:443>
     SSLEngine on
-    SSLCertificateFile      /etc/letsencrypt/live/${servername}/cert.pem
+    SSLCertificateFile      /etc/letsencrypt/live/${servername}/fullchain.pem
     SSLCertificateKeyFile   /etc/letsencrypt/live/${servername}/privkey.pem
-    SSLCertificateChainFile /etc/letsencrypt/live/${servername}/fullchain.pem
 
     # Mozilla modern configuration, tweak to your needs
     SSLProtocol             all -SSLv3 -TLSv1 -TLSv1.1

lib/mix/tasks/pleroma/activity.ex

@@ -16,6 +16,7 @@ defmodule Mix.Tasks.Pleroma.Activity do
   """
   def run(["get", id | _rest]) do
     start_pleroma()
+
     id
     |> Activity.get_by_id()
     |> IO.inspect()

lib/mix/tasks/pleroma/user.ex

@@ -8,6 +8,8 @@ defmodule Mix.Tasks.Pleroma.User do
   alias Ecto.Changeset
   alias Pleroma.User
   alias Pleroma.UserInviteToken
+  alias Pleroma.Web.ActivityPub.Builder
+  alias Pleroma.Web.ActivityPub.Pipeline
 
   @shortdoc "Manages Pleroma users"
   @moduledoc File.read!("docs/administration/CLI_tasks/user.md")
@@ -96,8 +98,9 @@ def run(["new", nickname, email | rest]) do
   def run(["rm", nickname]) do
     start_pleroma()
 
-    with %User{local: true} = user <- User.get_cached_by_nickname(nickname) do
+    with %User{local: true} = user <- User.get_cached_by_nickname(nickname),
-      User.perform(:delete, user)
+         {:ok, delete_data, _} <- Builder.delete(user, user.ap_id),
+         {:ok, _delete, _} <- Pipeline.common_pipeline(delete_data, local: true) do
       shell_info("User #{nickname} deleted.")
     else
       _ -> shell_error("No local user #{nickname}")
@@ -335,32 +338,36 @@ def run(["delete_activities", nickname]) do
 
   def run(["change_email", nickname, email]) do
     start_pleroma()
+
     with %User{} = user <- User.get_cached_by_nickname(nickname) do
-       user
+      user
-       |> User.update_changeset(%{"email" => email})
+      |> User.update_changeset(%{"email" => email})
-       |> User.update_and_set_cache()
+      |> User.update_and_set_cache()
-       shell_info("#{nickname}'s email updated")
+
+      shell_info("#{nickname}'s email updated")
     end
   end
 
   def run(["show", nickname]) do
     start_pleroma()
+
     nickname
     |> User.get_cached_by_nickname()
-    |> IO.inspect
+    |> IO.inspect()
   end
 
   def run(["send_confirmation", nickname]) do
     start_pleroma()
+
     with %User{} = user <- User.get_cached_by_nickname(nickname) do
-       user
+      user
-       |> Pleroma.Emails.UserEmail.account_confirmation_email()
+      |> Pleroma.Emails.UserEmail.account_confirmation_email()
-       |> IO.inspect
+      |> IO.inspect()
-       |> Pleroma.Emails.Mailer.deliver!()
+      |> Pleroma.Emails.Mailer.deliver!()
-       shell_info("#{nickname}'s email sent")
+
+      shell_info("#{nickname}'s email sent")
     end
   end
-  
 
   def run(["toggle_confirmed", nickname]) do
     start_pleroma()

lib/pleroma/application.ex

@@ -73,7 +73,6 @@ def start(_type, _args) do
         Pleroma.Repo,
         Config.TransferTask,
         Pleroma.Emoji,
-        Pleroma.Captcha,
         Pleroma.Plugs.RateLimiter.Supervisor
       ] ++
         cachex_children() ++
@@ -174,7 +173,14 @@ defp chat_enabled?, do: Config.get([:chat, :enabled])
   defp streamer_child(env) when env in [:test, :benchmark], do: []
 
   defp streamer_child(_) do
-    [Pleroma.Web.Streamer.supervisor()]
+    [
+      {Registry,
+       [
+         name: Pleroma.Web.Streamer.registry(),
+         keys: :duplicate,
+         partitions: System.schedulers_online()
+       ]}
+    ]
   end
 
   defp chat_child(_env, true) do

lib/pleroma/captcha/captcha.ex

@@ -3,53 +3,22 @@
 # SPDX-License-Identifier: AGPL-3.0-only
 
 defmodule Pleroma.Captcha do
-  import Pleroma.Web.Gettext
-
   alias Calendar.DateTime
   alias Plug.Crypto.KeyGenerator
   alias Plug.Crypto.MessageEncryptor
 
-  use GenServer
-
-  @doc false
-  def start_link(_) do
-    GenServer.start_link(__MODULE__, [], name: __MODULE__)
-  end
-
-  @doc false
-  def init(_) do
-    {:ok, nil}
-  end
-
   @doc """
   Ask the configured captcha service for a new captcha
   """
   def new do
-    GenServer.call(__MODULE__, :new)
+    if not enabled?() do
-  end
+      %{type: :none}
-
-  @doc """
-  Ask the configured captcha service to validate the captcha
-  """
-  def validate(token, captcha, answer_data) do
-    GenServer.call(__MODULE__, {:validate, token, captcha, answer_data})
-  end
-
-  @doc false
-  def handle_call(:new, _from, state) do
-    enabled = Pleroma.Config.get([__MODULE__, :enabled])
-
-    if !enabled do
-      {:reply, %{type: :none}, state}
     else
       new_captcha = method().new()
 
-      secret_key_base = Pleroma.Config.get!([Pleroma.Web.Endpoint, :secret_key_base])
-
       # This make salt a little different for two keys
-      token = new_captcha[:token]
+      {secret, sign_secret} = secret_pair(new_captcha[:token])
-      secret = KeyGenerator.generate(secret_key_base, token <> "_encrypt")
+
-      sign_secret = KeyGenerator.generate(secret_key_base, token <> "_sign")
       # Basically copy what Phoenix.Token does here, add the time to
       # the actual data and make it a binary to then encrypt it
       encrypted_captcha_answer =
@@ -60,55 +29,73 @@ def handle_call(:new, _from, state) do
         |> :erlang.term_to_binary()
         |> MessageEncryptor.encrypt(secret, sign_secret)
 
-      {
+      # Replace the answer with the encrypted answer
-        :reply,
+      %{new_captcha | answer_data: encrypted_captcha_answer}
-        # Replace the answer with the encrypted answer
-        %{new_captcha | answer_data: encrypted_captcha_answer},
-        state
-      }
     end
   end
 
-  @doc false
+  @doc """
-  def handle_call({:validate, token, captcha, answer_data}, _from, state) do
+  Ask the configured captcha service to validate the captcha
+  """
+  def validate(token, captcha, answer_data) do
+    with {:ok, %{at: at, answer_data: answer_md5}} <- validate_answer_data(token, answer_data),
+         :ok <- validate_expiration(at),
+         :ok <- validate_usage(token),
+         :ok <- method().validate(token, captcha, answer_md5),
+         {:ok, _} <- mark_captcha_as_used(token) do
+      :ok
+    end
+  end
+
+  def enabled?, do: Pleroma.Config.get([__MODULE__, :enabled], false)
+
+  defp seconds_valid, do: Pleroma.Config.get!([__MODULE__, :seconds_valid])
+
+  defp secret_pair(token) do
     secret_key_base = Pleroma.Config.get!([Pleroma.Web.Endpoint, :secret_key_base])
     secret = KeyGenerator.generate(secret_key_base, token <> "_encrypt")
     sign_secret = KeyGenerator.generate(secret_key_base, token <> "_sign")
 
+    {secret, sign_secret}
+  end
+
+  defp validate_answer_data(token, answer_data) do
+    {secret, sign_secret} = secret_pair(token)
+
+    with false <- is_nil(answer_data),
+         {:ok, data} <- MessageEncryptor.decrypt(answer_data, secret, sign_secret),
+         %{at: at, answer_data: answer_md5} <- :erlang.binary_to_term(data) do
+      {:ok, %{at: at, answer_data: answer_md5}}
+    else
+      _ -> {:error, :invalid_answer_data}
+    end
+  end
+
+  defp validate_expiration(created_at) do
     # If the time found is less than (current_time-seconds_valid) then the time has already passed
     # Later we check that the time found is more than the presumed invalidatation time, that means
     # that the data is still valid and the captcha can be checked
-    seconds_valid = Pleroma.Config.get!([Pleroma.Captcha, :seconds_valid])
-    valid_if_after = DateTime.subtract!(DateTime.now_utc(), seconds_valid)
 
-    result =
+    valid_if_after = DateTime.subtract!(DateTime.now_utc(), seconds_valid())
-      with false <- is_nil(answer_data),
-           {:ok, data} <- MessageEncryptor.decrypt(answer_data, secret, sign_secret),
-           %{at: at, answer_data: answer_md5} <- :erlang.binary_to_term(data) do
-        try do
-          if DateTime.before?(at, valid_if_after),
-            do: throw({:error, dgettext("errors", "CAPTCHA expired")})
 
-          if not is_nil(Cachex.get!(:used_captcha_cache, token)),
+    if DateTime.before?(created_at, valid_if_after) do
-            do: throw({:error, dgettext("errors", "CAPTCHA already used")})
+      {:error, :expired}
+    else
+      :ok
+    end
+  end
 
-          res = method().validate(token, captcha, answer_md5)
+  defp validate_usage(token) do
-          # Throw if an error occurs
+    if is_nil(Cachex.get!(:used_captcha_cache, token)) do
-          if res != :ok, do: throw(res)
+      :ok
+    else
+      {:error, :already_used}
+    end
+  end
 
-          # Mark this captcha as used
+  defp mark_captcha_as_used(token) do
-          {:ok, _} =
+    ttl = seconds_valid() |> :timer.seconds()
-            Cachex.put(:used_captcha_cache, token, true, ttl: :timer.seconds(seconds_valid))
+    Cachex.put(:used_captcha_cache, token, true, ttl: ttl)
-
-          :ok
-        catch
-          :throw, e -> e
-        end
-      else
-        _ -> {:error, dgettext("errors", "Invalid answer data")}
-      end
-
-    {:reply, result, state}
   end
 
   defp method, do: Pleroma.Config.get!([__MODULE__, :method])

lib/pleroma/captcha/kocaptcha.ex

@@ -3,7 +3,6 @@
 # SPDX-License-Identifier: AGPL-3.0-only
 
 defmodule Pleroma.Captcha.Kocaptcha do
-  import Pleroma.Web.Gettext
   alias Pleroma.Captcha.Service
   @behaviour Service
 
@@ -13,7 +12,7 @@ def new do
 
     case Tesla.get(endpoint <> "/new") do
       {:error, _} ->
-        %{error: dgettext("errors", "Kocaptcha service unavailable")}
+        %{error: :kocaptcha_service_unavailable}
 
       {:ok, res} ->
         json_resp = Jason.decode!(res.body)
@@ -33,6 +32,6 @@ def validate(_token, captcha, answer_data) do
     if not is_nil(captcha) and
          :crypto.hash(:md5, captcha) |> Base.encode16() == String.upcase(answer_data),
        do: :ok,
-       else: {:error, dgettext("errors", "Invalid CAPTCHA")}
+       else: {:error, :invalid}
   end
 end

lib/pleroma/captcha/native.ex

@@ -3,7 +3,6 @@
 # SPDX-License-Identifier: AGPL-3.0-only
 
 defmodule Pleroma.Captcha.Native do
-  import Pleroma.Web.Gettext
   alias Pleroma.Captcha.Service
   @behaviour Service
 
@@ -11,7 +10,7 @@ defmodule Pleroma.Captcha.Native do
   def new do
     case Captcha.get() do
       :error ->
-        %{error: dgettext("errors", "Captcha error")}
+        %{error: :captcha_error}
 
       {:ok, answer_data, img_binary} ->
         %{
@@ -25,7 +24,7 @@ def new do
 
   @impl Service
   def validate(_token, captcha, captcha) when not is_nil(captcha), do: :ok
-  def validate(_token, _captcha, _answer), do: {:error, dgettext("errors", "Invalid CAPTCHA")}
+  def validate(_token, _captcha, _answer), do: {:error, :invalid}
 
   defp token do
     10

lib/pleroma/constants.ex

@@ -20,4 +20,9 @@ defmodule Pleroma.Constants do
       "deleted_activity_id"
     ]
   )
+
+  const(static_only_files,
+    do:
+      ~w(index.html robots.txt static static-fe finmoji emoji packs sounds images instance sw.js sw-pleroma.js favicon.png schemas doc)
+  )
 end

lib/pleroma/conversation/participation.ex

@@ -128,7 +128,7 @@ def for_user(user, params \\ %{}) do
     |> Pleroma.Pagination.fetch_paginated(params)
   end
 
-  def restrict_recipients(query, user, %{"recipients" => user_ids}) do
+  def restrict_recipients(query, user, %{recipients: user_ids}) do
     user_binary_ids =
       [user.id | user_ids]
       |> Enum.uniq()
@@ -172,7 +172,7 @@ def for_user_with_last_activity_id(user, params \\ %{}) do
         | last_activity_id: activity_id
       }
     end)
-    |> Enum.filter(& &1.last_activity_id)
+    |> Enum.reject(&is_nil(&1.last_activity_id))
   end
 
   def get(_, _ \\ [])

lib/pleroma/emoji/pack.ex

@@ -0,0 +1,507 @@
+defmodule Pleroma.Emoji.Pack do
+  @derive {Jason.Encoder, only: [:files, :pack]}
+  defstruct files: %{},
+            pack_file: nil,
+            path: nil,
+            pack: %{},
+            name: nil
+
+  @type t() :: %__MODULE__{
+          files: %{String.t() => Path.t()},
+          pack_file: Path.t(),
+          path: Path.t(),
+          pack: map(),
+          name: String.t()
+        }
+
+  alias Pleroma.Emoji
+
+  @spec emoji_path() :: Path.t()
+  def emoji_path do
+    static = Pleroma.Config.get!([:instance, :static_dir])
+    Path.join(static, "emoji")
+  end
+
+  @spec create(String.t()) :: :ok | {:error, File.posix()} | {:error, :empty_values}
+  def create(name) when byte_size(name) > 0 do
+    dir = Path.join(emoji_path(), name)
+
+    with :ok <- File.mkdir(dir) do
+      %__MODULE__{
+        pack_file: Path.join(dir, "pack.json")
+      }
+      |> save_pack()
+    end
+  end
+
+  def create(_), do: {:error, :empty_values}
+
+  @spec show(String.t()) :: {:ok, t()} | {:loaded, nil} | {:error, :empty_values}
+  def show(name) when byte_size(name) > 0 do
+    with {_, %__MODULE__{} = pack} <- {:loaded, load_pack(name)},
+         {_, pack} <- validate_pack(pack) do
+      {:ok, pack}
+    end
+  end
+
+  def show(_), do: {:error, :empty_values}
+
+  @spec delete(String.t()) ::
+          {:ok, [binary()]} | {:error, File.posix(), binary()} | {:error, :empty_values}
+  def delete(name) when byte_size(name) > 0 do
+    emoji_path()
+    |> Path.join(name)
+    |> File.rm_rf()
+  end
+
+  def delete(_), do: {:error, :empty_values}
+
+  @spec add_file(String.t(), String.t(), Path.t(), Plug.Upload.t() | String.t()) ::
+          {:ok, t()} | {:error, File.posix()} | {:error, :empty_values}
+  def add_file(name, shortcode, filename, file)
+      when byte_size(name) > 0 and byte_size(shortcode) > 0 and byte_size(filename) > 0 do
+    with {_, nil} <- {:exists, Emoji.get(shortcode)},
+         {_, %__MODULE__{} = pack} <- {:loaded, load_pack(name)} do
+      file_path = Path.join(pack.path, filename)
+
+      create_subdirs(file_path)
+
+      case file do
+        %Plug.Upload{path: upload_path} ->
+          # Copy the uploaded file from the temporary directory
+          File.copy!(upload_path, file_path)
+
+        url when is_binary(url) ->
+          # Download and write the file
+          file_contents = Tesla.get!(url).body
+          File.write!(file_path, file_contents)
+      end
+
+      files = Map.put(pack.files, shortcode, filename)
+
+      updated_pack = %{pack | files: files}
+
+      case save_pack(updated_pack) do
+        :ok ->
+          Emoji.reload()
+          {:ok, updated_pack}
+
+        e ->
+          e
+      end
+    end
+  end
+
+  def add_file(_, _, _, _), do: {:error, :empty_values}
+
+  defp create_subdirs(file_path) do
+    if String.contains?(file_path, "/") do
+      file_path
+      |> Path.dirname()
+      |> File.mkdir_p!()
+    end
+  end
+
+  @spec delete_file(String.t(), String.t()) ::
+          {:ok, t()} | {:error, File.posix()} | {:error, :empty_values}
+  def delete_file(name, shortcode) when byte_size(name) > 0 and byte_size(shortcode) > 0 do
+    with {_, %__MODULE__{} = pack} <- {:loaded, load_pack(name)},
+         {_, {filename, files}} when not is_nil(filename) <-
+           {:exists, Map.pop(pack.files, shortcode)},
+         emoji <- Path.join(pack.path, filename),
+         {_, true} <- {:exists, File.exists?(emoji)} do
+      emoji_dir = Path.dirname(emoji)
+
+      File.rm!(emoji)
+
+      if String.contains?(filename, "/") and File.ls!(emoji_dir) == [] do
+        File.rmdir!(emoji_dir)
+      end
+
+      updated_pack = %{pack | files: files}
+
+      case save_pack(updated_pack) do
+        :ok ->
+          Emoji.reload()
+          {:ok, updated_pack}
+
+        e ->
+          e
+      end
+    end
+  end
+
+  def delete_file(_, _), do: {:error, :empty_values}
+
+  @spec update_file(String.t(), String.t(), String.t(), String.t(), boolean()) ::
+          {:ok, t()} | {:error, File.posix()} | {:error, :empty_values}
+  def update_file(name, shortcode, new_shortcode, new_filename, force)
+      when byte_size(name) > 0 and byte_size(shortcode) > 0 and byte_size(new_shortcode) > 0 and
+             byte_size(new_filename) > 0 do
+    with {_, %__MODULE__{} = pack} <- {:loaded, load_pack(name)},
+         {_, {filename, files}} when not is_nil(filename) <-
+           {:exists, Map.pop(pack.files, shortcode)},
+         {_, true} <- {:not_used, force or is_nil(Emoji.get(new_shortcode))} do
+      old_path = Path.join(pack.path, filename)
+      old_dir = Path.dirname(old_path)
+      new_path = Path.join(pack.path, new_filename)
+
+      create_subdirs(new_path)
+
+      :ok = File.rename(old_path, new_path)
+
+      if String.contains?(filename, "/") and File.ls!(old_dir) == [] do
+        File.rmdir!(old_dir)
+      end
+
+      files = Map.put(files, new_shortcode, new_filename)
+
+      updated_pack = %{pack | files: files}
+
+      case save_pack(updated_pack) do
+        :ok ->
+          Emoji.reload()
+          {:ok, updated_pack}
+
+        e ->
+          e
+      end
+    end
+  end
+
+  def update_file(_, _, _, _, _), do: {:error, :empty_values}
+
+  @spec import_from_filesystem() :: {:ok, [String.t()]} | {:error, atom()}
+  def import_from_filesystem do
+    emoji_path = emoji_path()
+
+    with {:ok, %{access: :read_write}} <- File.stat(emoji_path),
+         {:ok, results} <- File.ls(emoji_path) do
+      names =
+        results
+        |> Enum.map(&Path.join(emoji_path, &1))
+        |> Enum.reject(fn path ->
+          File.dir?(path) and File.exists?(Path.join(path, "pack.json"))
+        end)
+        |> Enum.map(&write_pack_contents/1)
+        |> Enum.filter(& &1)
+
+      {:ok, names}
+    else
+      {:ok, %{access: _}} -> {:error, :no_read_write}
+      e -> e
+    end
+  end
+
+  defp write_pack_contents(path) do
+    pack = %__MODULE__{
+      files: files_from_path(path),
+      path: path,
+      pack_file: Path.join(path, "pack.json")
+    }
+
+    case save_pack(pack) do
+      :ok -> Path.basename(path)
+      _ -> nil
+    end
+  end
+
+  defp files_from_path(path) do
+    txt_path = Path.join(path, "emoji.txt")
+
+    if File.exists?(txt_path) do
+      # There's an emoji.txt file, it's likely from a pack installed by the pack manager.
+      # Make a pack.json file from the contents of that emoji.txt file
+
+      # FIXME: Copy-pasted from Pleroma.Emoji/load_from_file_stream/2
+
+      # Create a map of shortcodes to filenames from emoji.txt
+      File.read!(txt_path)
+      |> String.split("\n")
+      |> Enum.map(&String.trim/1)
+      |> Enum.map(fn line ->
+        case String.split(line, ~r/,\s*/) do
+          # This matches both strings with and without tags
+          # and we don't care about tags here
+          [name, file | _] ->
+            file_dir_name = Path.dirname(file)
+
+            file =
+              if String.ends_with?(path, file_dir_name) do
+                Path.basename(file)
+              else
+                file
+              end
+
+            {name, file}
+
+          _ ->
+            nil
+        end
+      end)
+      |> Enum.filter(& &1)
+      |> Enum.into(%{})
+    else
+      # If there's no emoji.txt, assume all files
+      # that are of certain extensions from the config are emojis and import them all
+      pack_extensions = Pleroma.Config.get!([:emoji, :pack_extensions])
+      Emoji.Loader.make_shortcode_to_file_map(path, pack_extensions)
+    end
+  end
+
+  @spec list_remote(String.t()) :: {:ok, map()}
+  def list_remote(url) do
+    uri =
+      url
+      |> String.trim()
+      |> URI.parse()
+
+    with {_, true} <- {:shareable, shareable_packs_available?(uri)} do
+      packs =
+        uri
+        |> URI.merge("/api/pleroma/emoji/packs")
+        |> to_string()
+        |> Tesla.get!()
+        |> Map.get(:body)
+        |> Jason.decode!()
+
+      {:ok, packs}
+    end
+  end
+
+  @spec list_local() :: {:ok, map()}
+  def list_local do
+    emoji_path = emoji_path()
+
+    # Create the directory first if it does not exist. This is probably the first request made
+    # with the API so it should be sufficient
+    with {:create_dir, :ok} <- {:create_dir, File.mkdir_p(emoji_path)},
+         {:ls, {:ok, results}} <- {:ls, File.ls(emoji_path)} do
+      packs =
+        results
+        |> Enum.map(&load_pack/1)
+        |> Enum.filter(& &1)
+        |> Enum.map(&validate_pack/1)
+        |> Map.new()
+
+      {:ok, packs}
+    end
+  end
+
+  defp validate_pack(pack) do
+    if downloadable?(pack) do
+      archive = fetch_archive(pack)
+      archive_sha = :crypto.hash(:sha256, archive) |> Base.encode16()
+
+      info =
+        pack.pack
+        |> Map.put("can-download", true)
+        |> Map.put("download-sha256", archive_sha)
+
+      {pack.name, Map.put(pack, :pack, info)}
+    else
+      info = Map.put(pack.pack, "can-download", false)
+      {pack.name, Map.put(pack, :pack, info)}
+    end
+  end
+
+  defp downloadable?(pack) do
+    # If the pack is set as shared, check if it can be downloaded
+    # That means that when asked, the pack can be packed and sent to the remote
+    # Otherwise, they'd have to download it from external-src
+    pack.pack["share-files"] &&
+      Enum.all?(pack.files, fn {_, file} ->
+        File.exists?(Path.join(pack.path, file))
+      end)
+  end
+
+  @spec get_archive(String.t()) :: {:ok, binary()}
+  def get_archive(name) do
+    with {_, %__MODULE__{} = pack} <- {:exists?, load_pack(name)},
+         {_, true} <- {:can_download?, downloadable?(pack)} do
+      {:ok, fetch_archive(pack)}
+    end
+  end
+
+  defp fetch_archive(pack) do
+    hash = :crypto.hash(:md5, File.read!(pack.pack_file))
+
+    case Cachex.get!(:emoji_packs_cache, pack.name) do
+      %{hash: ^hash, pack_data: archive} ->
+        archive
+
+      _ ->
+        create_archive_and_cache(pack, hash)
+    end
+  end
+
+  defp create_archive_and_cache(pack, hash) do
+    files = ['pack.json' | Enum.map(pack.files, fn {_, file} -> to_charlist(file) end)]
+
+    {:ok, {_, result}} =
+      :zip.zip('#{pack.name}.zip', files, [:memory, cwd: to_charlist(pack.path)])
+
+    ttl_per_file = Pleroma.Config.get!([:emoji, :shared_pack_cache_seconds_per_file])
+    overall_ttl = :timer.seconds(ttl_per_file * Enum.count(files))
+
+    Cachex.put!(
+      :emoji_packs_cache,
+      pack.name,
+      # if pack.json MD5 changes, the cache is not valid anymore
+      %{hash: hash, pack_data: result},
+      # Add a minute to cache time for every file in the pack
+      ttl: overall_ttl
+    )
+
+    result
+  end
+
+  @spec download(String.t(), String.t(), String.t()) :: :ok
+  def download(name, url, as) do
+    uri =
+      url
+      |> String.trim()
+      |> URI.parse()
+
+    with {_, true} <- {:shareable, shareable_packs_available?(uri)} do
+      remote_pack =
+        uri
+        |> URI.merge("/api/pleroma/emoji/packs/#{name}")
+        |> to_string()
+        |> Tesla.get!()
+        |> Map.get(:body)
+        |> Jason.decode!()
+
+      result =
+        case remote_pack["pack"] do
+          %{"share-files" => true, "can-download" => true, "download-sha256" => sha} ->
+            {:ok,
+             %{
+               sha: sha,
+               url: URI.merge(uri, "/api/pleroma/emoji/packs/#{name}/archive") |> to_string()
+             }}
+
+          %{"fallback-src" => src, "fallback-src-sha256" => sha} when is_binary(src) ->
+            {:ok,
+             %{
+               sha: sha,
+               url: src,
+               fallback: true
+             }}
+
+          _ ->
+            {:error,
+             "The pack was not set as shared and there is no fallback src to download from"}
+        end
+
+      with {:ok, %{sha: sha, url: url} = pinfo} <- result,
+           %{body: archive} <- Tesla.get!(url),
+           {_, true} <- {:checksum, Base.decode16!(sha) == :crypto.hash(:sha256, archive)} do
+        local_name = as || name
+
+        path = Path.join(emoji_path(), local_name)
+
+        pack = %__MODULE__{
+          name: local_name,
+          path: path,
+          files: remote_pack["files"],
+          pack_file: Path.join(path, "pack.json")
+        }
+
+        File.mkdir_p!(pack.path)
+
+        files = Enum.map(remote_pack["files"], fn {_, path} -> to_charlist(path) end)
+        # Fallback cannot contain a pack.json file
+        files = if pinfo[:fallback], do: files, else: ['pack.json' | files]
+
+        {:ok, _} = :zip.unzip(archive, cwd: to_charlist(pack.path), file_list: files)
+
+        # Fallback can't contain a pack.json file, since that would cause the fallback-src-sha256
+        # in it to depend on itself
+        if pinfo[:fallback] do
+          save_pack(pack)
+        end
+
+        :ok
+      end
+    end
+  end
+
+  defp save_pack(pack), do: File.write(pack.pack_file, Jason.encode!(pack, pretty: true))
+
+  @spec save_metadata(map(), t()) :: {:ok, t()} | {:error, File.posix()}
+  def save_metadata(metadata, %__MODULE__{} = pack) do
+    pack = Map.put(pack, :pack, metadata)
+
+    with :ok <- save_pack(pack) do
+      {:ok, pack}
+    end
+  end
+
+  @spec update_metadata(String.t(), map()) :: {:ok, t()} | {:error, File.posix()}
+  def update_metadata(name, data) do
+    pack = load_pack(name)
+
+    fb_sha_changed? =
+      not is_nil(data["fallback-src"]) and data["fallback-src"] != pack.pack["fallback-src"]
+
+    with {_, true} <- {:update?, fb_sha_changed?},
+         {:ok, %{body: zip}} <- Tesla.get(data["fallback-src"]),
+         {:ok, f_list} <- :zip.unzip(zip, [:memory]),
+         {_, true} <- {:has_all_files?, has_all_files?(pack.files, f_list)} do
+      fallback_sha = :crypto.hash(:sha256, zip) |> Base.encode16()
+
+      data
+      |> Map.put("fallback-src-sha256", fallback_sha)
+      |> save_metadata(pack)
+    else
+      {:update?, _} -> save_metadata(data, pack)
+      e -> e
+    end
+  end
+
+  # Check if all files from the pack.json are in the archive
+  defp has_all_files?(files, f_list) do
+    Enum.all?(files, fn {_, from_manifest} ->
+      List.keyfind(f_list, to_charlist(from_manifest), 0)
+    end)
+  end
+
+  @spec load_pack(String.t()) :: t() | nil
+  def load_pack(name) do
+    pack_file = Path.join([emoji_path(), name, "pack.json"])
+
+    if File.exists?(pack_file) do
+      pack_file
+      |> File.read!()
+      |> from_json()
+      |> Map.put(:pack_file, pack_file)
+      |> Map.put(:path, Path.dirname(pack_file))
+      |> Map.put(:name, name)
+    end
+  end
+
+  defp from_json(json) do
+    map = Jason.decode!(json)
+
+    struct(__MODULE__, %{files: map["files"], pack: map["pack"]})
+  end
+
+  defp shareable_packs_available?(uri) do
+    uri
+    |> URI.merge("/.well-known/nodeinfo")
+    |> to_string()
+    |> Tesla.get!()
+    |> Map.get(:body)
+    |> Jason.decode!()
+    |> Map.get("links")
+    |> List.last()
+    |> Map.get("href")
+    # Get the actual nodeinfo address and fetch it
+    |> Tesla.get!()
+    |> Map.get(:body)
+    |> Jason.decode!()
+    |> get_in(["metadata", "features"])
+    |> Enum.member?("shareable_emoji_packs")
+  end
+end

lib/pleroma/filter.ex

@@ -89,11 +89,10 @@ def delete(%Pleroma.Filter{id: filter_key} = filter) when is_nil(filter_key) do
     |> Repo.delete()
   end
 
-  def update(%Pleroma.Filter{} = filter) do
+  def update(%Pleroma.Filter{} = filter, params) do
-    destination = Map.from_struct(filter)
+    filter
-
+    |> cast(params, [:phrase, :context, :hide, :expires_at, :whole_word])
-    Pleroma.Filter.get(filter.filter_id, %{id: filter.user_id})
+    |> validate_required([:phrase, :context])
-    |> cast(destination, [:phrase, :context, :hide, :expires_at, :whole_word])
     |> Repo.update()
   end
 end

lib/pleroma/following_relationship.ex

@@ -10,11 +10,12 @@ defmodule Pleroma.FollowingRelationship do
 
   alias Ecto.Changeset
   alias FlakeId.Ecto.CompatType
+  alias Pleroma.FollowingRelationship.State
   alias Pleroma.Repo
   alias Pleroma.User
 
   schema "following_relationships" do
-    field(:state, Pleroma.FollowingRelationship.State, default: :follow_pending)
+    field(:state, State, default: :follow_pending)
 
     belongs_to(:follower, User, type: CompatType)
     belongs_to(:following, User, type: CompatType)
@@ -22,6 +23,11 @@ defmodule Pleroma.FollowingRelationship do
     timestamps()
   end
 
+  @doc "Returns underlying integer code for state atom"
+  def state_int_code(state_atom), do: State.__enum_map__() |> Keyword.fetch!(state_atom)
+
+  def accept_state_code, do: state_int_code(:follow_accept)
+
   def changeset(%__MODULE__{} = following_relationship, attrs) do
     following_relationship
     |> cast(attrs, [:state])
@@ -82,6 +88,29 @@ def follower_count(%User{} = user) do
     |> Repo.aggregate(:count, :id)
   end
 
+  def followers_query(%User{} = user) do
+    __MODULE__
+    |> join(:inner, [r], u in User, on: r.follower_id == u.id)
+    |> where([r], r.following_id == ^user.id)
+    |> where([r], r.state == ^:follow_accept)
+  end
+
+  def followers_ap_ids(%User{} = user, from_ap_ids \\ nil) do
+    query =
+      user
+      |> followers_query()
+      |> select([r, u], u.ap_id)
+
+    query =
+      if from_ap_ids do
+        where(query, [r, u], u.ap_id in ^from_ap_ids)
+      else
+        query
+      end
+
+    Repo.all(query)
+  end
+
   def following_count(%User{id: nil}), do: 0
 
   def following_count(%User{} = user) do
@@ -105,12 +134,16 @@ def following?(%User{id: follower_id}, %User{id: followed_id}) do
     |> Repo.exists?()
   end
 
+  def following_query(%User{} = user) do
+    __MODULE__
+    |> join(:inner, [r], u in User, on: r.following_id == u.id)
+    |> where([r], r.follower_id == ^user.id)
+    |> where([r], r.state == ^:follow_accept)
+  end
+
   def following(%User{} = user) do
     following =
-      __MODULE__
+      following_query(user)
-      |> join(:inner, [r], u in User, on: r.following_id == u.id)
-      |> where([r], r.follower_id == ^user.id)
-      |> where([r], r.state == ^:follow_accept)
       |> select([r, u], u.follower_address)
       |> Repo.all()
 
@@ -171,6 +204,30 @@ def find(following_relationships, follower, following) do
     end)
   end
 
+  @doc """
+  For a query with joined activity,
+  keeps rows where activity's actor is followed by user -or- is NOT domain-blocked by user.
+  """
+  def keep_following_or_not_domain_blocked(query, user) do
+    where(
+      query,
+      [_, activity],
+      fragment(
+        # "(actor's domain NOT in domain_blocks) OR (actor IS in followed AP IDs)"
+        """
+        NOT (substring(? from '.*://([^/]*)') = ANY(?)) OR
+          ? = ANY(SELECT ap_id FROM users AS u INNER JOIN following_relationships AS fr
+            ON u.id = fr.following_id WHERE fr.follower_id = ? AND fr.state = ?)
+        """,
+        activity.actor,
+        ^user.domain_blocks,
+        activity.actor,
+        ^User.binary_id(user.id),
+        ^accept_state_code()
+      )
+    )
+  end
+
   defp validate_not_self_relationship(%Changeset{} = changeset) do
     changeset
     |> validate_follower_id_following_id_inequality()

lib/pleroma/marker.ex

@@ -9,24 +9,34 @@ defmodule Pleroma.Marker do
   import Ecto.Query
 
   alias Ecto.Multi
+  alias Pleroma.Notification
   alias Pleroma.Repo
   alias Pleroma.User
+  alias __MODULE__
 
   @timelines ["notifications"]
+  @type t :: %__MODULE__{}
 
   schema "markers" do
     field(:last_read_id, :string, default: "")
     field(:timeline, :string, default: "")
     field(:lock_version, :integer, default: 0)
+    field(:unread_count, :integer, default: 0, virtual: true)
 
     belongs_to(:user, User, type: FlakeId.Ecto.CompatType)
     timestamps()
   end
 
+  @doc "Gets markers by user and timeline."
+  @spec get_markers(User.t(), list(String)) :: list(t())
   def get_markers(user, timelines \\ []) do
-    Repo.all(get_query(user, timelines))
+    user
+    |> get_query(timelines)
+    |> unread_count_query()
+    |> Repo.all()
   end
 
+  @spec upsert(User.t(), map()) :: {:ok | :error, any()}
   def upsert(%User{} = user, attrs) do
     attrs
     |> Map.take(@timelines)
@@ -45,6 +55,27 @@ def upsert(%User{} = user, attrs) do
     |> Repo.transaction()
   end
 
+  @spec multi_set_last_read_id(Multi.t(), User.t(), String.t()) :: Multi.t()
+  def multi_set_last_read_id(multi, %User{} = user, "notifications") do
+    multi
+    |> Multi.run(:counters, fn _repo, _changes ->
+      {:ok, %{last_read_id: Repo.one(Notification.last_read_query(user))}}
+    end)
+    |> Multi.insert(
+      :marker,
+      fn %{counters: attrs} ->
+        %Marker{timeline: "notifications", user_id: user.id}
+        |> struct(attrs)
+        |> Ecto.Changeset.change()
+      end,
+      returning: true,
+      on_conflict: {:replace, [:last_read_id]},
+      conflict_target: [:user_id, :timeline]
+    )
+  end
+
+  def multi_set_last_read_id(multi, _, _), do: multi
+
   defp get_marker(user, timeline) do
     case Repo.find_resource(get_query(user, timeline)) do
       {:ok, marker} -> %__MODULE__{marker | user: user}
@@ -71,4 +102,16 @@ defp get_query(user, timelines) do
     |> by_user_id(user.id)
     |> by_timeline(timelines)
   end
+
+  defp unread_count_query(query) do
+    from(
+      q in query,
+      left_join: n in "notifications",
+      on: n.user_id == q.user_id and n.seen == false,
+      group_by: [:id],
+      select_merge: %{
+        unread_count: fragment("count(?)", n.id)
+      }
+    )
+  end
 end

lib/pleroma/mfa.ex

@@ -0,0 +1,156 @@
+# Pleroma: A lightweight social networking server
+# Copyright © 2017-2019 Pleroma Authors <https://pleroma.social/>
+# SPDX-License-Identifier: AGPL-3.0-only
+
+defmodule Pleroma.MFA do
+  @moduledoc """
+  The MFA context.
+  """
+
+  alias Comeonin.Pbkdf2
+  alias Pleroma.User
+
+  alias Pleroma.MFA.BackupCodes
+  alias Pleroma.MFA.Changeset
+  alias Pleroma.MFA.Settings
+  alias Pleroma.MFA.TOTP
+
+  @doc """
+  Returns MFA methods the user has enabled.
+
+  ## Examples
+
+    iex> Pleroma.MFA.supported_method(User)
+    "totp, u2f"
+  """
+  @spec supported_methods(User.t()) :: String.t()
+  def supported_methods(user) do
+    settings = fetch_settings(user)
+
+    Settings.mfa_methods()
+    |> Enum.reduce([], fn m, acc ->
+      if method_enabled?(m, settings) do
+        acc ++ [m]
+      else
+        acc
+      end
+    end)
+    |> Enum.join(",")
+  end
+
+  @doc "Checks that user enabled MFA"
+  def require?(user) do
+    fetch_settings(user).enabled
+  end
+
+  @doc """
+  Display MFA settings of user
+  """
+  def mfa_settings(user) do
+    settings = fetch_settings(user)
+
+    Settings.mfa_methods()
+    |> Enum.map(fn m -> [m, method_enabled?(m, settings)] end)
+    |> Enum.into(%{enabled: settings.enabled}, fn [a, b] -> {a, b} end)
+  end
+
+  @doc false
+  def fetch_settings(%User{} = user) do
+    user.multi_factor_authentication_settings || %Settings{}
+  end
+
+  @doc "clears backup codes"
+  def invalidate_backup_code(%User{} = user, hash_code) do
+    %{backup_codes: codes} = fetch_settings(user)
+
+    user
+    |> Changeset.cast_backup_codes(codes -- [hash_code])
+    |> User.update_and_set_cache()
+  end
+
+  @doc "generates backup codes"
+  @spec generate_backup_codes(User.t()) :: {:ok, list(binary)} | {:error, String.t()}
+  def generate_backup_codes(%User{} = user) do
+    with codes <- BackupCodes.generate(),
+         hashed_codes <- Enum.map(codes, &Pbkdf2.hashpwsalt/1),
+         changeset <- Changeset.cast_backup_codes(user, hashed_codes),
+         {:ok, _} <- User.update_and_set_cache(changeset) do
+      {:ok, codes}
+    else
+      {:error, msg} ->
+        %{error: msg}
+    end
+  end
+
+  @doc """
+  Generates secret key and set delivery_type to 'app' for TOTP method.
+  """
+  @spec setup_totp(User.t()) :: {:ok, User.t()} | {:error, Ecto.Changeset.t()}
+  def setup_totp(user) do
+    user
+    |> Changeset.setup_totp(%{secret: TOTP.generate_secret(), delivery_type: "app"})
+    |> User.update_and_set_cache()
+  end
+
+  @doc """
+  Confirms the TOTP method for user.
+
+  `attrs`:
+    `password` - current user password
+    `code` - TOTP token
+  """
+  @spec confirm_totp(User.t(), map()) :: {:ok, User.t()} | {:error, Ecto.Changeset.t() | atom()}
+  def confirm_totp(%User{} = user, attrs) do
+    with settings <- user.multi_factor_authentication_settings.totp,
+         {:ok, :pass} <- TOTP.validate_token(settings.secret, attrs["code"]) do
+      user
+      |> Changeset.confirm_totp()
+      |> User.update_and_set_cache()
+    end
+  end
+
+  @doc """
+  Disables the TOTP method for user.
+
+  `attrs`:
+    `password` - current user password
+  """
+  @spec disable_totp(User.t()) :: {:ok, User.t()} | {:error, Ecto.Changeset.t()}
+  def disable_totp(%User{} = user) do
+    user
+    |> Changeset.disable_totp()
+    |> Changeset.disable()
+    |> User.update_and_set_cache()
+  end
+
+  @doc """
+  Force disables all MFA methods for user.
+  """
+  @spec disable(User.t()) :: {:ok, User.t()} | {:error, Ecto.Changeset.t()}
+  def disable(%User{} = user) do
+    user
+    |> Changeset.disable_totp()
+    |> Changeset.disable(true)
+    |> User.update_and_set_cache()
+  end
+
+  @doc """
+  Checks if the user has MFA method enabled.
+  """
+  def method_enabled?(method, settings) do
+    with {:ok, %{confirmed: true} = _} <- Map.fetch(settings, method) do
+      true
+    else
+      _ -> false
+    end
+  end
+
+  @doc """
+  Checks if the user has enabled at least one MFA method.
+  """
+  def enabled?(settings) do
+    Settings.mfa_methods()
+    |> Enum.map(fn m -> method_enabled?(m, settings) end)
+    |> Enum.any?()
+  end
+end

lib/pleroma/mfa/backup_codes.ex

@@ -0,0 +1,31 @@
+# Pleroma: A lightweight social networking server
+# Copyright © 2017-2019 Pleroma Authors <https://pleroma.social/>
+# SPDX-License-Identifier: AGPL-3.0-only
+
+defmodule Pleroma.MFA.BackupCodes do
+  @moduledoc """
+  This module contains functions for generating backup codes.
+  """
+  alias Pleroma.Config
+
+  @config_ns [:instance, :multi_factor_authentication, :backup_codes]
+
+  @doc """
+  Generates backup codes.
+  """
+  @spec generate(Keyword.t()) :: list(String.t())
+  def generate(opts \\ []) do
+    number_of_codes = Keyword.get(opts, :number_of_codes, default_backup_codes_number())
+    code_length = Keyword.get(opts, :length, default_backup_codes_code_length())
+
+    Enum.map(1..number_of_codes, fn _ ->
+      :crypto.strong_rand_bytes(div(code_length, 2))
+      |> Base.encode16(case: :lower)
+    end)
+  end
+
+  defp default_backup_codes_number, do: Config.get(@config_ns ++ [:number], 5)
+
+  defp default_backup_codes_code_length,
+    do: Config.get(@config_ns ++ [:length], 16)
+end

lib/pleroma/mfa/changeset.ex

@@ -0,0 +1,64 @@
+# Pleroma: A lightweight social networking server
+# Copyright © 2017-2019 Pleroma Authors <https://pleroma.social/>
+# SPDX-License-Identifier: AGPL-3.0-only
+
+defmodule Pleroma.MFA.Changeset do
+  alias Pleroma.MFA
+  alias Pleroma.MFA.Settings
+  alias Pleroma.User
+
+  def disable(%Ecto.Changeset{} = changeset, force \\ false) do
+    settings =
+      changeset
+      |> Ecto.Changeset.apply_changes()
+      |> MFA.fetch_settings()
+
+    if force || not MFA.enabled?(settings) do
+      put_change(changeset, %Settings{settings | enabled: false})
+    else
+      changeset
+    end
+  end
+
+  def disable_totp(%User{multi_factor_authentication_settings: settings} = user) do
+    user
+    |> put_change(%Settings{settings | totp: %Settings.TOTP{}})
+  end
+
+  def confirm_totp(%User{multi_factor_authentication_settings: settings} = user) do
+    totp_settings = %Settings.TOTP{settings.totp | confirmed: true}
+
+    user
+    |> put_change(%Settings{settings | totp: totp_settings, enabled: true})
+  end
+
+  def setup_totp(%User{} = user, attrs) do
+    mfa_settings = MFA.fetch_settings(user)
+
+    totp_settings =
+      %Settings.TOTP{}
+      |> Ecto.Changeset.cast(attrs, [:secret, :delivery_type])
+
+    user
+    |> put_change(%Settings{mfa_settings | totp: Ecto.Changeset.apply_changes(totp_settings)})
+  end
+
+  def cast_backup_codes(%User{} = user, codes) do
+    user
+    |> put_change(%Settings{
+      user.multi_factor_authentication_settings
+      | backup_codes: codes
+    })
+  end
+
+  defp put_change(%User{} = user, settings) do
+    user
+    |> Ecto.Changeset.change()
+    |> put_change(settings)
+  end
+
+  defp put_change(%Ecto.Changeset{} = changeset, settings) do
+    changeset
+    |> Ecto.Changeset.put_change(:multi_factor_authentication_settings, settings)
+  end
+end

lib/pleroma/mfa/settings.ex

@@ -0,0 +1,24 @@
+# Pleroma: A lightweight social networking server
+# Copyright © 2017-2019 Pleroma Authors <https://pleroma.social/>
+# SPDX-License-Identifier: AGPL-3.0-only
+
+defmodule Pleroma.MFA.Settings do
+  use Ecto.Schema
+
+  @primary_key false
+
+  @mfa_methods [:totp]
+  embedded_schema do
+    field(:enabled, :boolean, default: false)
+    field(:backup_codes, {:array, :string}, default: [])
+
+    embeds_one :totp, TOTP, on_replace: :delete, primary_key: false do
+      field(:secret, :string)
+      # app | sms
+      field(:delivery_type, :string, default: "app")
+      field(:confirmed, :boolean, default: false)
+    end
+  end
+
+  def mfa_methods, do: @mfa_methods
+end

lib/pleroma/mfa/token.ex

@@ -0,0 +1,106 @@
+# Pleroma: A lightweight social networking server
+# Copyright © 2017-2019 Pleroma Authors <https://pleroma.social/>
+# SPDX-License-Identifier: AGPL-3.0-only
+
+defmodule Pleroma.MFA.Token do
+  use Ecto.Schema
+  import Ecto.Query
+  import Ecto.Changeset
+
+  alias Pleroma.Repo
+  alias Pleroma.User
+  alias Pleroma.Web.OAuth.Authorization
+  alias Pleroma.Web.OAuth.Token, as: OAuthToken
+
+  @expires 300
+
+  schema "mfa_tokens" do
+    field(:token, :string)
+    field(:valid_until, :naive_datetime_usec)
+
+    belongs_to(:user, User, type: FlakeId.Ecto.CompatType)
+    belongs_to(:authorization, Authorization)
+
+    timestamps()
+  end
+
+  def get_by_token(token) do
+    from(
+      t in __MODULE__,
+      where: t.token == ^token,
+      preload: [:user, :authorization]
+    )
+    |> Repo.find_resource()
+  end
+
+  def validate(token) do
+    with {:fetch_token, {:ok, token}} <- {:fetch_token, get_by_token(token)},
+         {:expired, false} <- {:expired, is_expired?(token)} do
+      {:ok, token}
+    else
+      {:expired, _} -> {:error, :expired_token}
+      {:fetch_token, _} -> {:error, :not_found}
+      error -> {:error, error}
+    end
+  end
+
+  def create_token(%User{} = user) do
+    %__MODULE__{}
+    |> change
+    |> assign_user(user)
+    |> put_token
+    |> put_valid_until
+    |> Repo.insert()
+  end
+
+  def create_token(user, authorization) do
+    %__MODULE__{}
+    |> change
+    |> assign_user(user)
+    |> assign_authorization(authorization)
+    |> put_token
+    |> put_valid_until
+    |> Repo.insert()
+  end
+
+  defp assign_user(changeset, user) do
+    changeset
+    |> put_assoc(:user, user)
+    |> validate_required([:user])
+  end
+
+  defp assign_authorization(changeset, authorization) do
+    changeset
+    |> put_assoc(:authorization, authorization)
+    |> validate_required([:authorization])
+  end
+
+  defp put_token(changeset) do
+    changeset
+    |> change(%{token: OAuthToken.Utils.generate_token()})
+    |> validate_required([:token])
+    |> unique_constraint(:token)
+  end
+
+  defp put_valid_until(changeset) do
+    expires_in = NaiveDateTime.add(NaiveDateTime.utc_now(), @expires)
+
+    changeset
+    |> change(%{valid_until: expires_in})
+    |> validate_required([:valid_until])
+  end
+
+  def is_expired?(%__MODULE__{valid_until: valid_until}) do
+    NaiveDateTime.diff(NaiveDateTime.utc_now(), valid_until) > 0
+  end
+
+  def is_expired?(_), do: false
+
+  def delete_expired_tokens do
+    from(
+      q in __MODULE__,
+      where: fragment("?", q.valid_until) < ^Timex.now()
+    )
+    |> Repo.delete_all()
+  end
+end

lib/pleroma/mfa/totp.ex

@@ -0,0 +1,86 @@
+# Pleroma: A lightweight social networking server
+# Copyright © 2017-2019 Pleroma Authors <https://pleroma.social/>
+# SPDX-License-Identifier: AGPL-3.0-only
+
+defmodule Pleroma.MFA.TOTP do
+  @moduledoc """
+  This module represents functions to create secrets for
+  TOTP Application as well as validate them with a time based token.
+  """
+  alias Pleroma.Config
+
+  @config_ns [:instance, :multi_factor_authentication, :totp]
+
+  @doc """
+  https://github.com/google/google-authenticator/wiki/Key-Uri-Format
+  """
+  def provisioning_uri(secret, label, opts \\ []) do
+    query =
+      %{
+        secret: secret,
+        issuer: Keyword.get(opts, :issuer, default_issuer()),
+        digits: Keyword.get(opts, :digits, default_digits()),
+        period: Keyword.get(opts, :period, default_period())
+      }
+      |> Enum.filter(fn {_, v} -> not is_nil(v) end)
+      |> Enum.into(%{})
+      |> URI.encode_query()
+
+    %URI{scheme: "otpauth", host: "totp", path: "/" <> label, query: query}
+    |> URI.to_string()
+  end
+
+  defp default_period, do: Config.get(@config_ns ++ [:period])
+  defp default_digits, do: Config.get(@config_ns ++ [:digits])
+
+  defp default_issuer,
+    do: Config.get(@config_ns ++ [:issuer], Config.get([:instance, :name]))
+
+  @doc "Creates a random Base 32 encoded string"
+  def generate_secret do
+    Base.encode32(:crypto.strong_rand_bytes(10))
+  end
+
+  @doc "Generates a valid token based on a secret"
+  def generate_token(secret) do
+    :pot.totp(secret)
+  end
+
+  @doc """
+  Validates a given token based on a secret.
+
+  optional parameters:
+  `token_length` default `6`
+  `interval_length` default `30`
+  `window` default 0
+
+  Returns {:ok, :pass} if the token is valid and
+  {:error, :invalid_token} if it is not.
+  """
+  @spec validate_token(String.t(), String.t()) ::
+          {:ok, :pass} | {:error, :invalid_token | :invalid_secret_and_token}
+  def validate_token(secret, token)
+      when is_binary(secret) and is_binary(token) do
+    opts = [
+      token_length: default_digits(),
+      interval_length: default_period()
+    ]
+
+    validate_token(secret, token, opts)
+  end
+
+  def validate_token(_, _), do: {:error, :invalid_secret_and_token}
+
+  @doc "See `validate_token/2`"
+  @spec validate_token(String.t(), String.t(), Keyword.t()) ::
+          {:ok, :pass} | {:error, :invalid_token | :invalid_secret_and_token}
+  def validate_token(secret, token, options)
+      when is_binary(secret) and is_binary(token) do
+    case :pot.valid_totp(token, secret, options) do
+      true -> {:ok, :pass}
+      false -> {:error, :invalid_token}
+    end
+  end
+
+  def validate_token(_, _, _), do: {:error, :invalid_secret_and_token}
+end

lib/pleroma/notification.ex

@@ -5,7 +5,10 @@
 defmodule Pleroma.Notification do
   use Ecto.Schema
 
+  alias Ecto.Multi
   alias Pleroma.Activity
+  alias Pleroma.FollowingRelationship
+  alias Pleroma.Marker
   alias Pleroma.Notification
   alias Pleroma.Object
   alias Pleroma.Pagination
@@ -33,11 +36,30 @@ defmodule Pleroma.Notification do
     timestamps()
   end
 
+  @spec unread_notifications_count(User.t()) :: integer()
+  def unread_notifications_count(%User{id: user_id}) do
+    from(q in __MODULE__,
+      where: q.user_id == ^user_id and q.seen == false
+    )
+    |> Repo.aggregate(:count, :id)
+  end
+
   def changeset(%Notification{} = notification, attrs) do
     notification
     |> cast(attrs, [:seen])
   end
 
+  @spec last_read_query(User.t()) :: Ecto.Queryable.t()
+  def last_read_query(user) do
+    from(q in Pleroma.Notification,
+      where: q.user_id == ^user.id,
+      where: q.seen == true,
+      select: type(q.id, :string),
+      limit: 1,
+      order_by: [desc: :id]
+    )
+  end
+
   defp for_user_query_ap_id_opts(user, opts) do
     ap_id_relationships =
       [:block] ++
@@ -81,15 +103,13 @@ def for_user_query(user, opts \\ %{}) do
     |> exclude_visibility(opts)
   end
 
+  # Excludes blocked users and non-followed domain-blocked users
   defp exclude_blocked(query, user, opts) do
     blocked_ap_ids = opts[:blocked_users_ap_ids] || User.blocked_users_ap_ids(user)
 
     query
     |> where([n, a], a.actor not in ^blocked_ap_ids)
-    |> where(
+    |> FollowingRelationship.keep_following_or_not_domain_blocked(user)
-      [n, a],
-      fragment("substring(? from '.*://([^/]*)')", a.actor) not in ^user.domain_blocks
-    )
   end
 
   defp exclude_notification_muted(query, _, %{@include_muted_option => true}) do
@@ -186,25 +206,23 @@ def for_user_since(user, date) do
     |> Repo.all()
   end
 
-  def set_read_up_to(%{id: user_id} = _user, id) do
+  def set_read_up_to(%{id: user_id} = user, id) do
     query =
       from(
         n in Notification,
         where: n.user_id == ^user_id,
         where: n.id <= ^id,
         where: n.seen == false,
-        update: [
-          set: [
-            seen: true,
-            updated_at: ^NaiveDateTime.utc_now()
-          ]
-        ],
         # Ideally we would preload object and activities here
         # but Ecto does not support preloads in update_all
         select: n.id
       )
 
-    {_, notification_ids} = Repo.update_all(query, [])
+    {:ok, %{ids: {_, notification_ids}}} =
+      Multi.new()
+      |> Multi.update_all(:ids, query, set: [seen: true, updated_at: NaiveDateTime.utc_now()])
+      |> Marker.multi_set_last_read_id(user, "notifications")
+      |> Repo.transaction()
 
     Notification
     |> where([n], n.id in ^notification_ids)
@@ -221,11 +239,18 @@ def set_read_up_to(%{id: user_id} = _user, id) do
     |> Repo.all()
   end
 
+  @spec read_one(User.t(), String.t()) ::
+          {:ok, Notification.t()} | {:error, Ecto.Changeset.t()} | nil
   def read_one(%User{} = user, notification_id) do
     with {:ok, %Notification{} = notification} <- get(user, notification_id) do
-      notification
+      Multi.new()
-      |> changeset(%{seen: true})
+      |> Multi.update(:update, changeset(notification, %{seen: true}))
-      |> Repo.update()
+      |> Marker.multi_set_last_read_id(user, "notifications")
+      |> Repo.transaction()
+      |> case do
+        {:ok, %{update: notification}} -> {:ok, notification}
+        {:error, :update, changeset, _} -> {:error, changeset}
+      end
     end
   end
 
@@ -262,6 +287,16 @@ def destroy_multiple(%{id: user_id} = _user, ids) do
     |> Repo.delete_all()
   end
 
+  def dismiss(%Pleroma.Activity{} = activity) do
+    Notification
+    |> where([n], n.activity_id == ^activity.id)
+    |> Repo.delete_all()
+    |> case do
+      {_, notifications} -> {:ok, notifications}
+      _ -> {:error, "Cannot dismiss notification"}
+    end
+  end
+
   def dismiss(%{id: user_id} = _user, id) do
     notification = Repo.get(Notification, id)
 
@@ -284,17 +319,8 @@ def create_notifications(%Activity{data: %{"to" => _, "type" => "Create"}} = act
     end
   end
 
-  def create_notifications(%Activity{data: %{"type" => "Follow"}} = activity) do
-    if Pleroma.Config.get([:notifications, :enable_follow_request_notifications]) ||
-         Activity.follow_accepted?(activity) do
-      do_create_notifications(activity)
-    else
-      {:ok, []}
-    end
-  end
-
   def create_notifications(%Activity{data: %{"type" => type}} = activity)
-      when type in ["Like", "Announce", "Move", "EmojiReact"] do
+      when type in ["Follow", "Like", "Announce", "Move", "EmojiReact"] do
     do_create_notifications(activity)
   end
 
@@ -316,8 +342,11 @@ defp do_create_notifications(%Activity{} = activity) do
   # TODO move to sql, too.
   def create_notification(%Activity{} = activity, %User{} = user, do_send \\ true) do
     unless skip?(activity, user) do
-      notification = %Notification{user_id: user.id, activity: activity}
+      {:ok, %{notification: notification}} =
-      {:ok, notification} = Repo.insert(notification)
+        Multi.new()
+        |> Multi.insert(:notification, %Notification{user_id: user.id, activity: activity})
+        |> Marker.multi_set_last_read_id(user, "notifications")
+        |> Repo.transaction()
 
       if do_send do
         Streamer.stream(["user", "user:notification"], notification)
@@ -330,10 +359,11 @@ def create_notification(%Activity{} = activity, %User{} = user, do_send \\ true)
 
   @doc """
   Returns a tuple with 2 elements:
-    {enabled notification receivers, currently disabled receivers (blocking / [thread] muting)}
+    {notification-enabled receivers, currently disabled receivers (blocking / [thread] muting)}
 
   NOTE: might be called for FAKE Activities, see ActivityPub.Utils.get_notified_from_object/1
   """
+  @spec get_notified_from_activity(Activity.t(), boolean()) :: {list(User.t()), list(User.t())}
   def get_notified_from_activity(activity, local_only \\ true)
 
   def get_notified_from_activity(%Activity{data: %{"type" => type}} = activity, local_only)
@@ -346,17 +376,14 @@ def get_notified_from_activity(%Activity{data: %{"type" => type}} = activity, lo
       |> Utils.maybe_notify_followers(activity)
       |> Enum.uniq()
 
-    # Since even subscribers and followers can mute / thread-mute, filtering all above AP IDs
+    potential_receivers = User.get_users_from_set(potential_receiver_ap_ids, local_only)
+
     notification_enabled_ap_ids =
       potential_receiver_ap_ids
+      |> exclude_domain_blocker_ap_ids(activity, potential_receivers)
       |> exclude_relationship_restricted_ap_ids(activity)
       |> exclude_thread_muter_ap_ids(activity)
 
-    potential_receivers =
-      potential_receiver_ap_ids
-      |> Enum.uniq()
-      |> User.get_users_from_set(local_only)
-
     notification_enabled_users =
       Enum.filter(potential_receivers, fn u -> u.ap_id in notification_enabled_ap_ids end)
 
@@ -365,6 +392,38 @@ def get_notified_from_activity(%Activity{data: %{"type" => type}} = activity, lo
 
   def get_notified_from_activity(_, _local_only), do: {[], []}
 
+  @doc "Filters out AP IDs domain-blocking and not following the activity's actor"
+  def exclude_domain_blocker_ap_ids(ap_ids, activity, preloaded_users \\ [])
+
+  def exclude_domain_blocker_ap_ids([], _activity, _preloaded_users), do: []
+
+  def exclude_domain_blocker_ap_ids(ap_ids, %Activity{} = activity, preloaded_users) do
+    activity_actor_domain = activity.actor && URI.parse(activity.actor).host
+
+    users =
+      ap_ids
+      |> Enum.map(fn ap_id ->
+        Enum.find(preloaded_users, &(&1.ap_id == ap_id)) ||
+          User.get_cached_by_ap_id(ap_id)
+      end)
+      |> Enum.filter(& &1)
+
+    domain_blocker_ap_ids = for u <- users, activity_actor_domain in u.domain_blocks, do: u.ap_id
+
+    domain_blocker_follower_ap_ids =
+      if Enum.any?(domain_blocker_ap_ids) do
+        activity
+        |> Activity.user_actor()
+        |> FollowingRelationship.followers_ap_ids(domain_blocker_ap_ids)
+      else
+        []
+      end
+
+    ap_ids
+    |> Kernel.--(domain_blocker_ap_ids)
+    |> Kernel.++(domain_blocker_follower_ap_ids)
+  end
+
   @doc "Filters out AP IDs of users basing on their relationships with activity actor user"
   def exclude_relationship_restricted_ap_ids([], _activity), do: []
 

lib/pleroma/plugs/ensure_authenticated_plug.ex

@@ -15,26 +15,25 @@ def init(options) do
   end
 
   @impl true
+  def perform(
+        %{
+          assigns: %{
+            auth_credentials: %{password: _},
+            user: %User{multi_factor_authentication_settings: %{enabled: true}}
+          }
+        } = conn,
+        _
+      ) do
+    conn
+    |> render_error(:forbidden, "Two-factor authentication enabled, you must use a access token.")
+    |> halt()
+  end
+
   def perform(%{assigns: %{user: %User{}}} = conn, _) do
     conn
   end
 
-  def perform(conn, options) do
+  def perform(conn, _) do
-    perform =
-      cond do
-        options[:if_func] -> options[:if_func].()
-        options[:unless_func] -> !options[:unless_func].()
-        true -> true
-      end
-
-    if perform do
-      fail(conn)
-    else
-      conn
-    end
-  end
-
-  def fail(conn) do
     conn
     |> render_error(:forbidden, "Invalid credentials.")
     |> halt()

lib/pleroma/plugs/federating_plug.ex

@@ -19,6 +19,9 @@ def call(conn, _opts) do
 
   def federating?, do: Pleroma.Config.get([:instance, :federating])
 
+  # Definition for the use in :if_func / :unless_func plug options
+  def federating?(_conn), do: federating?()
+
   defp fail(conn) do
     conn
     |> put_status(404)

lib/pleroma/plugs/instance_static.ex

@@ -3,6 +3,8 @@
 # SPDX-License-Identifier: AGPL-3.0-only
 
 defmodule Pleroma.Plugs.InstanceStatic do
+  require Pleroma.Constants
+
   @moduledoc """
   This is a shim to call `Plug.Static` but with runtime `from` configuration.
 
@@ -21,9 +23,6 @@ def file_path(path) do
     end
   end
 
-  @only ~w(index.html robots.txt static emoji packs sounds images instance favicon.png sw.js
-  sw-pleroma.js)
-
   def init(opts) do
     opts
     |> Keyword.put(:from, "__unconfigured_instance_static_plug")
@@ -31,7 +30,7 @@ def init(opts) do
     |> Plug.Static.init()
   end
 
-  for only <- @only do
+  for only <- Pleroma.Constants.static_only_files() do
     at = Plug.Router.Utils.split("/")
 
     def call(%{request_path: "/" <> unquote(only) <> _} = conn, opts) do

lib/pleroma/plugs/mapped_signature_to_identity_plug.ex

@@ -13,8 +13,9 @@ defmodule Pleroma.Web.Plugs.MappedSignatureToIdentityPlug do
   def init(options), do: options
 
   defp key_id_from_conn(conn) do
-    with %{"keyId" => key_id} <- HTTPSignatures.signature_for_conn(conn) do
+    with %{"keyId" => key_id} <- HTTPSignatures.signature_for_conn(conn),
-      Signature.key_id_to_actor_id(key_id)
+         {:ok, ap_id} <- Signature.key_id_to_actor_id(key_id) do
+      ap_id
     else
       _ ->
         nil

lib/pleroma/signature.ex

@@ -8,6 +8,7 @@ defmodule Pleroma.Signature do
   alias Pleroma.Keys
   alias Pleroma.User
   alias Pleroma.Web.ActivityPub.ActivityPub
+  alias Pleroma.Web.ActivityPub.ObjectValidators.Types
 
   def key_id_to_actor_id(key_id) do
     uri =
@@ -21,12 +22,23 @@ def key_id_to_actor_id(key_id) do
         uri
       end
 
-    URI.to_string(uri)
+    maybe_ap_id = URI.to_string(uri)
+
+    case Types.ObjectID.cast(maybe_ap_id) do
+      {:ok, ap_id} ->
+        {:ok, ap_id}
+
+      _ ->
+        case Pleroma.Web.WebFinger.finger(maybe_ap_id) do
+          %{"ap_id" => ap_id} -> {:ok, ap_id}
+          _ -> {:error, maybe_ap_id}
+        end
+    end
   end
 
   def fetch_public_key(conn) do
     with %{"keyId" => kid} <- HTTPSignatures.signature_for_conn(conn),
-         actor_id <- key_id_to_actor_id(kid),
+         {:ok, actor_id} <- key_id_to_actor_id(kid),
          {:ok, public_key} <- User.get_public_key_for_ap_id(actor_id) do
       {:ok, public_key}
     else
@@ -37,7 +49,7 @@ def fetch_public_key(conn) do
 
   def refetch_public_key(conn) do
     with %{"keyId" => kid} <- HTTPSignatures.signature_for_conn(conn),
-         actor_id <- key_id_to_actor_id(kid),
+         {:ok, actor_id} <- key_id_to_actor_id(kid),
          {:ok, _user} <- ActivityPub.make_user_from_ap_id(actor_id),
          {:ok, public_key} <- User.get_public_key_for_ap_id(actor_id) do
       {:ok, public_key}

lib/pleroma/stats.ex

@@ -91,7 +91,7 @@ def calculate_stat_data do
       peers: peers,
       stats: %{
         domain_count: domain_count,
-        status_count: status_count,
+        status_count: status_count || 0,
         user_count: user_count
       }
     }

lib/pleroma/user.ex

@@ -20,6 +20,7 @@ defmodule Pleroma.User do
   alias Pleroma.Formatter
   alias Pleroma.HTML
   alias Pleroma.Keys
+  alias Pleroma.MFA
   alias Pleroma.Notification
   alias Pleroma.Object
   alias Pleroma.Registration
@@ -29,7 +30,9 @@ defmodule Pleroma.User do
   alias Pleroma.UserRelationship
   alias Pleroma.Web
   alias Pleroma.Web.ActivityPub.ActivityPub
+  alias Pleroma.Web.ActivityPub.Builder
   alias Pleroma.Web.ActivityPub.ObjectValidators.Types
+  alias Pleroma.Web.ActivityPub.Pipeline
   alias Pleroma.Web.ActivityPub.Utils
   alias Pleroma.Web.CommonAPI
   alias Pleroma.Web.CommonAPI.Utils, as: CommonUtils
@@ -113,7 +116,6 @@ defmodule Pleroma.User do
     field(:is_admin, :boolean, default: false)
     field(:show_role, :boolean, default: true)
     field(:settings, :map, default: nil)
-    field(:magic_key, :string, default: nil)
     field(:uri, Types.Uri, default: nil)
     field(:hide_followers_count, :boolean, default: false)
     field(:hide_follows_count, :boolean, default: false)
@@ -189,6 +191,12 @@ defmodule Pleroma.User do
     # `:subscribers` is deprecated (replaced with `subscriber_users` relation)
     field(:subscribers, {:array, :string}, default: [])
 
+    embeds_one(
+      :multi_factor_authentication_settings,
+      MFA.Settings,
+      on_replace: :delete
+    )
+
     timestamps()
   end
 
@@ -387,7 +395,6 @@ def remote_user_changeset(struct \\ %User{local: false}, params) do
         :banner,
         :locked,
         :last_refreshed_at,
-        :magic_key,
         :uri,
         :follower_address,
         :following_address,
@@ -927,6 +934,7 @@ def get_cached_by_nickname_or_id(nickname_or_id, opts \\ []) do
     end
   end
 
+  @spec get_by_nickname(String.t()) :: User.t() | nil
   def get_by_nickname(nickname) do
     Repo.get_by(User, nickname: nickname) ||
       if Regex.match?(~r(@#{Pleroma.Web.Endpoint.host()})i, nickname) do
@@ -1427,8 +1435,6 @@ def perform(:force_password_reset, user), do: force_password_reset(user)
 
   @spec perform(atom(), User.t()) :: {:ok, User.t()}
   def perform(:delete, %User{} = user) do
-    {:ok, _user} = ActivityPub.delete(user)
-
     # Remove all relationships
     user
     |> get_followers()
@@ -1445,8 +1451,15 @@ def perform(:delete, %User{} = user) do
     end)
 
     delete_user_activities(user)
-    invalidate_cache(user)
+
-    Repo.delete(user)
+    if user.local do
+      user
+      |> change(%{deactivated: true, email: nil})
+      |> update_and_set_cache()
+    else
+      invalidate_cache(user)
+      Repo.delete(user)
+    end
   end
 
   def perform(:deactivate_async, user, status), do: deactivate(user, status)
@@ -1531,37 +1544,29 @@ def follow_import(%User{} = follower, followed_identifiers)
     })
   end
 
-  def delete_user_activities(%User{ap_id: ap_id}) do
+  def delete_user_activities(%User{ap_id: ap_id} = user) do
     ap_id
     |> Activity.Queries.by_actor()
     |> RepoStreamer.chunk_stream(50)
-    |> Stream.each(fn activities -> Enum.each(activities, &delete_activity/1) end)
+    |> Stream.each(fn activities ->
+      Enum.each(activities, fn activity -> delete_activity(activity, user) end)
+    end)
     |> Stream.run()
   end
 
-  defp delete_activity(%{data: %{"type" => "Create"}} = activity) do
+  defp delete_activity(%{data: %{"type" => "Create", "object" => object}}, user) do
-    activity
+    {:ok, delete_data, _} = Builder.delete(user, object)
-    |> Object.normalize()
+
-    |> ActivityPub.delete()
+    Pipeline.common_pipeline(delete_data, local: user.local)
   end
 
-  defp delete_activity(%{data: %{"type" => "Like"}} = activity) do
+  defp delete_activity(%{data: %{"type" => type}} = activity, user)
-    object = Object.normalize(activity)
+       when type in ["Like", "Announce"] do
-
+    {:ok, undo, _} = Builder.undo(user, activity)
-    activity.actor
+    Pipeline.common_pipeline(undo, local: user.local)
-    |> get_cached_by_ap_id()
-    |> ActivityPub.unlike(object)
   end
 
-  defp delete_activity(%{data: %{"type" => "Announce"}} = activity) do
+  defp delete_activity(_activity, _user), do: "Doing nothing"
-    object = Object.normalize(activity)
-
-    activity.actor
-    |> get_cached_by_ap_id()
-    |> ActivityPub.unannounce(object)
-  end
-
-  defp delete_activity(_activity), do: "Doing nothing"
 
   def html_filter_policy(%User{no_rich_text: true}) do
     Pleroma.HTML.Scrubber.TwitterText

lib/pleroma/user/query.ex

@@ -45,6 +45,7 @@ defmodule Pleroma.User.Query do
             is_admin: boolean(),
             is_moderator: boolean(),
             super_users: boolean(),
+            exclude_service_users: boolean(),
             followers: User.t(),
             friends: User.t(),
             recipients_from_activity: [String.t()],
@@ -88,6 +89,10 @@ defp compose_query({key, value}, query)
     where(query, [u], ilike(field(u, ^key), ^"%#{value}%"))
   end
 
+  defp compose_query({:exclude_service_users, _}, query) do
+    where(query, [u], not like(u.ap_id, "%/relay") and not like(u.ap_id, "%/internal/fetch"))
+  end
+
   defp compose_query({key, value}, query)
        when key in @equal_criteria and not_empty_string(value) do
     where(query, [u], ^[{key, value}])
@@ -98,7 +103,7 @@ defp compose_query({key, values}, query) when key in @contains_criteria and is_l
   end
 
   defp compose_query({:tags, tags}, query) when is_list(tags) and length(tags) > 0 do
-    Enum.reduce(tags, query, &prepare_tag_criteria/2)
+    where(query, [u], fragment("? && ?", u.tags, ^tags))
   end
 
   defp compose_query({:is_admin, _}, query) do
@@ -192,10 +197,6 @@ defp compose_query({:limit, limit}, query) do
 
   defp compose_query(_unsupported_param, query), do: query
 
-  defp prepare_tag_criteria(tag, query) do
-    or_where(query, [u], fragment("? = any(?)", ^tag, u.tags))
-  end
-
   defp location_query(query, local) do
     where(query, [u], u.local == ^local)
     |> where([u], not is_nil(u.nickname))

lib/pleroma/web/activity_pub/activity_pub.ex

@@ -170,12 +170,6 @@ def insert(map, local \\ true, fake \\ false, bypass_actor_check \\ false) when
 
       BackgroundWorker.enqueue("fetch_data_for_activity", %{"activity_id" => activity.id})
 
-      Notification.create_notifications(activity)
-
-      conversation = create_or_bump_conversation(activity, map["actor"])
-      participations = get_participations(conversation)
-      stream_out(activity)
-      stream_out_participations(participations)
       {:ok, activity}
     else
       %Activity{} = activity ->
@@ -198,6 +192,15 @@ def insert(map, local \\ true, fake \\ false, bypass_actor_check \\ false) when
     end
   end
 
+  def notify_and_stream(activity) do
+    Notification.create_notifications(activity)
+
+    conversation = create_or_bump_conversation(activity, activity.actor)
+    participations = get_participations(conversation)
+    stream_out(activity)
+    stream_out_participations(participations)
+  end
+
   defp create_or_bump_conversation(activity, actor) do
     with {:ok, conversation} <- Conversation.create_or_bump_for(activity),
          %User{} = user <- User.get_cached_by_ap_id(actor),
@@ -274,6 +277,7 @@ defp do_create(%{to: to, actor: actor, context: context, object: object} = param
          _ <- increase_poll_votes_if_vote(create_data),
          {:quick_insert, false, activity} <- {:quick_insert, quick_insert?, activity},
          {:ok, _actor} <- increase_note_count_if_public(actor, activity),
+         _ <- notify_and_stream(activity),
          :ok <- maybe_federate(activity) do
       {:ok, activity}
     else
@@ -301,6 +305,7 @@ def listen(%{to: to, actor: actor, context: context, object: object} = params) d
              additional
            ),
          {:ok, activity} <- insert(listen_data, local),
+         _ <- notify_and_stream(activity),
          :ok <- maybe_federate(activity) do
       {:ok, activity}
     end
@@ -325,6 +330,7 @@ def accept_or_reject(type, %{to: to, actor: actor, object: object} = params) do
            %{"to" => to, "type" => type, "actor" => actor.ap_id, "object" => object}
            |> Utils.maybe_put("id", activity_id),
          {:ok, activity} <- insert(data, local),
+         _ <- notify_and_stream(activity),
          :ok <- maybe_federate(activity) do
       {:ok, activity}
     end
@@ -344,83 +350,12 @@ def update(%{to: to, cc: cc, actor: actor, object: object} = params) do
          },
          data <- Utils.maybe_put(data, "id", activity_id),
          {:ok, activity} <- insert(data, local),
+         _ <- notify_and_stream(activity),
          :ok <- maybe_federate(activity) do
       {:ok, activity}
     end
   end
 
-  @spec react_with_emoji(User.t(), Object.t(), String.t(), keyword()) ::
-          {:ok, Activity.t(), Object.t()} | {:error, any()}
-  def react_with_emoji(user, object, emoji, options \\ []) do
-    with {:ok, result} <-
-           Repo.transaction(fn -> do_react_with_emoji(user, object, emoji, options) end) do
-      result
-    end
-  end
-
-  defp do_react_with_emoji(user, object, emoji, options) do
-    with local <- Keyword.get(options, :local, true),
-         activity_id <- Keyword.get(options, :activity_id, nil),
-         true <- Pleroma.Emoji.is_unicode_emoji?(emoji),
-         reaction_data <- make_emoji_reaction_data(user, object, emoji, activity_id),
-         {:ok, activity} <- insert(reaction_data, local),
-         {:ok, object} <- add_emoji_reaction_to_object(activity, object),
-         :ok <- maybe_federate(activity) do
-      {:ok, activity, object}
-    else
-      false -> {:error, false}
-      {:error, error} -> Repo.rollback(error)
-    end
-  end
-
-  @spec unreact_with_emoji(User.t(), String.t(), keyword()) ::
-          {:ok, Activity.t(), Object.t()} | {:error, any()}
-  def unreact_with_emoji(user, reaction_id, options \\ []) do
-    with {:ok, result} <-
-           Repo.transaction(fn -> do_unreact_with_emoji(user, reaction_id, options) end) do
-      result
-    end
-  end
-
-  defp do_unreact_with_emoji(user, reaction_id, options) do
-    with local <- Keyword.get(options, :local, true),
-         activity_id <- Keyword.get(options, :activity_id, nil),
-         user_ap_id <- user.ap_id,
-         %Activity{actor: ^user_ap_id} = reaction_activity <- Activity.get_by_ap_id(reaction_id),
-         object <- Object.normalize(reaction_activity),
-         unreact_data <- make_undo_data(user, reaction_activity, activity_id),
-         {:ok, activity} <- insert(unreact_data, local),
-         {:ok, object} <- remove_emoji_reaction_from_object(reaction_activity, object),
-         :ok <- maybe_federate(activity) do
-      {:ok, activity, object}
-    else
-      {:error, error} -> Repo.rollback(error)
-    end
-  end
-
-  @spec unlike(User.t(), Object.t(), String.t() | nil, boolean()) ::
-          {:ok, Activity.t(), Activity.t(), Object.t()} | {:ok, Object.t()} | {:error, any()}
-  def unlike(%User{} = actor, %Object{} = object, activity_id \\ nil, local \\ true) do
-    with {:ok, result} <-
-           Repo.transaction(fn -> do_unlike(actor, object, activity_id, local) end) do
-      result
-    end
-  end
-
-  defp do_unlike(actor, object, activity_id, local) do
-    with %Activity{} = like_activity <- get_existing_like(actor.ap_id, object),
-         unlike_data <- make_unlike_data(actor, like_activity, activity_id),
-         {:ok, unlike_activity} <- insert(unlike_data, local),
-         {:ok, _activity} <- Repo.delete(like_activity),
-         {:ok, object} <- remove_like_from_object(like_activity, object),
-         :ok <- maybe_federate(unlike_activity) do
-      {:ok, unlike_activity, like_activity, object}
-    else
-      nil -> {:ok, object}
-      {:error, error} -> Repo.rollback(error)
-    end
-  end
-
   @spec announce(User.t(), Object.t(), String.t() | nil, boolean(), boolean()) ::
           {:ok, Activity.t(), Object.t()} | {:error, any()}
   def announce(
@@ -442,6 +377,7 @@ defp do_announce(user, object, activity_id, local, public) do
          announce_data <- make_announce_data(user, object, activity_id, public),
          {:ok, activity} <- insert(announce_data, local),
          {:ok, object} <- add_announce_to_object(activity, object),
+         _ <- notify_and_stream(activity),
          :ok <- maybe_federate(activity) do
       {:ok, activity, object}
     else
@@ -450,34 +386,6 @@ defp do_announce(user, object, activity_id, local, public) do
     end
   end
 
-  @spec unannounce(User.t(), Object.t(), String.t() | nil, boolean()) ::
-          {:ok, Activity.t(), Object.t()} | {:ok, Object.t()} | {:error, any()}
-  def unannounce(
-        %User{} = actor,
-        %Object{} = object,
-        activity_id \\ nil,
-        local \\ true
-      ) do
-    with {:ok, result} <-
-           Repo.transaction(fn -> do_unannounce(actor, object, activity_id, local) end) do
-      result
-    end
-  end
-
-  defp do_unannounce(actor, object, activity_id, local) do
-    with %Activity{} = announce_activity <- get_existing_announce(actor.ap_id, object),
-         unannounce_data <- make_unannounce_data(actor, announce_activity, activity_id),
-         {:ok, unannounce_activity} <- insert(unannounce_data, local),
-         :ok <- maybe_federate(unannounce_activity),
-         {:ok, _activity} <- Repo.delete(announce_activity),
-         {:ok, object} <- remove_announce_from_object(announce_activity, object) do
-      {:ok, unannounce_activity, object}
-    else
-      nil -> {:ok, object}
-      {:error, error} -> Repo.rollback(error)
-    end
-  end
-
   @spec follow(User.t(), User.t(), String.t() | nil, boolean()) ::
           {:ok, Activity.t()} | {:error, any()}
   def follow(follower, followed, activity_id \\ nil, local \\ true) do
@@ -490,6 +398,7 @@ def follow(follower, followed, activity_id \\ nil, local \\ true) do
   defp do_follow(follower, followed, activity_id, local) do
     with data <- make_follow_data(follower, followed, activity_id),
          {:ok, activity} <- insert(data, local),
+         _ <- notify_and_stream(activity),
          :ok <- maybe_federate(activity) do
       {:ok, activity}
     else
@@ -511,6 +420,7 @@ defp do_unfollow(follower, followed, activity_id, local) do
          {:ok, follow_activity} <- update_follow_state(follow_activity, "cancelled"),
          unfollow_data <- make_unfollow_data(follower, followed, follow_activity, activity_id),
          {:ok, activity} <- insert(unfollow_data, local),
+         _ <- notify_and_stream(activity),
          :ok <- maybe_federate(activity) do
       {:ok, activity}
     else
@@ -519,67 +429,6 @@ defp do_unfollow(follower, followed, activity_id, local) do
     end
   end
 
-  @spec delete(User.t() | Object.t(), keyword()) :: {:ok, User.t() | Object.t()} | {:error, any()}
-  def delete(entity, options \\ []) do
-    with {:ok, result} <- Repo.transaction(fn -> do_delete(entity, options) end) do
-      result
-    end
-  end
-
-  defp do_delete(%User{ap_id: ap_id, follower_address: follower_address} = user, _) do
-    with data <- %{
-           "to" => [follower_address],
-           "type" => "Delete",
-           "actor" => ap_id,
-           "object" => %{"type" => "Person", "id" => ap_id}
-         },
-         {:ok, activity} <- insert(data, true, true, true),
-         :ok <- maybe_federate(activity) do
-      {:ok, user}
-    end
-  end
-
-  defp do_delete(%Object{data: %{"id" => id, "actor" => actor}} = object, options) do
-    local = Keyword.get(options, :local, true)
-    activity_id = Keyword.get(options, :activity_id, nil)
-    actor = Keyword.get(options, :actor, actor)
-
-    user = User.get_cached_by_ap_id(actor)
-    to = (object.data["to"] || []) ++ (object.data["cc"] || [])
-
-    with create_activity <- Activity.get_create_by_object_ap_id(id),
-         data <-
-           %{
-             "type" => "Delete",
-             "actor" => actor,
-             "object" => id,
-             "to" => to,
-             "deleted_activity_id" => create_activity && create_activity.id
-           }
-           |> maybe_put("id", activity_id),
-         {:ok, activity} <- insert(data, local, false),
-         {:ok, object, _create_activity} <- Object.delete(object),
-         stream_out_participations(object, user),
-         _ <- decrease_replies_count_if_reply(object),
-         {:ok, _actor} <- decrease_note_count_if_public(user, object),
-         :ok <- maybe_federate(activity) do
-      {:ok, activity}
-    else
-      {:error, error} ->
-        Repo.rollback(error)
-    end
-  end
-
-  defp do_delete(%Object{data: %{"type" => "Tombstone", "id" => ap_id}}, _) do
-    activity =
-      ap_id
-      |> Activity.Queries.by_object_id()
-      |> Activity.Queries.by_type("Delete")
-      |> Repo.one()
-
-    {:ok, activity}
-  end
-
   @spec block(User.t(), User.t(), String.t() | nil, boolean()) ::
           {:ok, Activity.t()} | {:error, any()}
   def block(blocker, blocked, activity_id \\ nil, local \\ true) do
@@ -601,6 +450,7 @@ defp do_block(blocker, blocked, activity_id, local) do
     with true <- outgoing_blocks,
          block_data <- make_block_data(blocker, blocked, activity_id),
          {:ok, activity} <- insert(block_data, local),
+         _ <- notify_and_stream(activity),
          :ok <- maybe_federate(activity) do
       {:ok, activity}
     else
@@ -608,27 +458,6 @@ defp do_block(blocker, blocked, activity_id, local) do
     end
   end
 
-  @spec unblock(User.t(), User.t(), String.t() | nil, boolean()) ::
-          {:ok, Activity.t()} | {:error, any()} | nil
-  def unblock(blocker, blocked, activity_id \\ nil, local \\ true) do
-    with {:ok, result} <-
-           Repo.transaction(fn -> do_unblock(blocker, blocked, activity_id, local) end) do
-      result
-    end
-  end
-
-  defp do_unblock(blocker, blocked, activity_id, local) do
-    with %Activity{} = block_activity <- fetch_latest_block(blocker, blocked),
-         unblock_data <- make_unblock_data(blocker, blocked, block_activity, activity_id),
-         {:ok, activity} <- insert(unblock_data, local),
-         :ok <- maybe_federate(activity) do
-      {:ok, activity}
-    else
-      nil -> nil
-      {:error, error} -> Repo.rollback(error)
-    end
-  end
-
   @spec flag(map()) :: {:ok, Activity.t()} | {:error, any()}
   def flag(
         %{
@@ -655,6 +484,7 @@ def flag(
     with flag_data <- make_flag_data(params, additional),
          {:ok, activity} <- insert(flag_data, local),
          {:ok, stripped_activity} <- strip_report_status_data(activity),
+         _ <- notify_and_stream(activity),
          :ok <- maybe_federate(stripped_activity) do
       User.all_superusers()
       |> Enum.filter(fn user -> not is_nil(user.email) end)
@@ -678,7 +508,8 @@ def move(%User{} = origin, %User{} = target, local \\ true) do
     }
 
     with true <- origin.ap_id in target.also_known_as,
-         {:ok, activity} <- insert(params, local) do
+         {:ok, activity} <- insert(params, local),
+         _ <- notify_and_stream(activity) do
       maybe_federate(activity)
 
       BackgroundWorker.enqueue("move_following", %{
@@ -1530,21 +1361,34 @@ def fetch_follow_information_for_user(user) do
   defp normalize_counter(counter) when is_integer(counter), do: counter
   defp normalize_counter(_), do: 0
 
-  defp maybe_update_follow_information(data) do
+  def maybe_update_follow_information(user_data) do
     with {:enabled, true} <- {:enabled, Config.get([:instance, :external_user_synchronization])},
-         {:ok, info} <- fetch_follow_information_for_user(data) do
+         {_, true} <- {:user_type_check, user_data[:type] in ["Person", "Service"]},
-      info = Map.merge(data[:info] || %{}, info)
+         {_, true} <-
-      Map.put(data, :info, info)
+           {:collections_available,
+            !!(user_data[:following_address] && user_data[:follower_address])},
+         {:ok, info} <-
+           fetch_follow_information_for_user(user_data) do
+      info = Map.merge(user_data[:info] || %{}, info)
+
+      user_data
+      |> Map.put(:info, info)
     else
+      {:user_type_check, false} ->
+        user_data
+
+      {:collections_available, false} ->
+        user_data
+
       {:enabled, false} ->
-        data
+        user_data
 
       e ->
         Logger.error(
-          "Follower/Following counter update for #{data.ap_id} failed.\n" <> inspect(e)
+          "Follower/Following counter update for #{user_data.ap_id} failed.\n" <> inspect(e)
         )
 
-        data
+        user_data
     end
   end
 

lib/pleroma/web/activity_pub/activity_pub_controller.ex

@@ -34,12 +34,13 @@ defmodule Pleroma.Web.ActivityPub.ActivityPubController do
 
   plug(
     EnsureAuthenticatedPlug,
-    [unless_func: &FederatingPlug.federating?/0] when action not in @federating_only_actions
+    [unless_func: &FederatingPlug.federating?/1] when action not in @federating_only_actions
   )
 
+  # Note: :following and :followers must be served even without authentication (as via :api)
   plug(
     EnsureAuthenticatedPlug
-    when action in [:read_inbox, :update_outbox, :whoami, :upload_media, :following, :followers]
+    when action in [:read_inbox, :update_outbox, :whoami, :upload_media]
   )
 
   plug(
@@ -395,7 +396,10 @@ def read_inbox(%{assigns: %{user: %User{nickname: as_nickname}}} = conn, %{
     |> json(err)
   end
 
-  defp handle_user_activity(%User{} = user, %{"type" => "Create"} = params) do
+  defp handle_user_activity(
+         %User{} = user,
+         %{"type" => "Create", "object" => %{"type" => "Note"}} = params
+       ) do
     object =
       params["object"]
       |> Map.merge(Map.take(params, ["to", "cc"]))
@@ -414,7 +418,8 @@ defp handle_user_activity(%User{} = user, %{"type" => "Create"} = params) do
   defp handle_user_activity(%User{} = user, %{"type" => "Delete"} = params) do
     with %Object{} = object <- Object.normalize(params["object"]),
          true <- user.is_moderator || user.ap_id == object.data["actor"],
-         {:ok, delete} <- ActivityPub.delete(object) do
+         {:ok, delete_data, _} <- Builder.delete(user, object.data["id"]),
+         {:ok, delete, _} <- Pipeline.common_pipeline(delete_data, local: true) do
       {:ok, delete}
     else
       _ -> {:error, dgettext("errors", "Can't delete object")}

lib/pleroma/web/activity_pub/builder.ex

@@ -10,8 +10,71 @@ defmodule Pleroma.Web.ActivityPub.Builder do
   alias Pleroma.Web.ActivityPub.Utils
   alias Pleroma.Web.ActivityPub.Visibility
 
+  @spec emoji_react(User.t(), Object.t(), String.t()) :: {:ok, map(), keyword()}
+  def emoji_react(actor, object, emoji) do
+    with {:ok, data, meta} <- object_action(actor, object) do
+      data =
+        data
+        |> Map.put("content", emoji)
+        |> Map.put("type", "EmojiReact")
+
+      {:ok, data, meta}
+    end
+  end
+
+  @spec undo(User.t(), Activity.t()) :: {:ok, map(), keyword()}
+  def undo(actor, object) do
+    {:ok,
+     %{
+       "id" => Utils.generate_activity_id(),
+       "actor" => actor.ap_id,
+       "type" => "Undo",
+       "object" => object.data["id"],
+       "to" => object.data["to"] || [],
+       "cc" => object.data["cc"] || []
+     }, []}
+  end
+
+  @spec delete(User.t(), String.t()) :: {:ok, map(), keyword()}
+  def delete(actor, object_id) do
+    object = Object.normalize(object_id, false)
+
+    user = !object && User.get_cached_by_ap_id(object_id)
+
+    to =
+      case {object, user} do
+        {%Object{}, _} ->
+          # We are deleting an object, address everyone who was originally mentioned
+          (object.data["to"] || []) ++ (object.data["cc"] || [])
+
+        {_, %User{follower_address: follower_address}} ->
+          # We are deleting a user, address the followers of that user
+          [follower_address]
+      end
+
+    {:ok,
+     %{
+       "id" => Utils.generate_activity_id(),
+       "actor" => actor.ap_id,
+       "object" => object_id,
+       "to" => to,
+       "type" => "Delete"
+     }, []}
+  end
+
   @spec like(User.t(), Object.t()) :: {:ok, map(), keyword()}
   def like(actor, object) do
+    with {:ok, data, meta} <- object_action(actor, object) do
+      data =
+        data
+        |> Map.put("type", "Like")
+
+      {:ok, data, meta}
+    end
+  end
+
+  @spec object_action(User.t(), Object.t()) :: {:ok, map(), keyword()}
+  defp object_action(actor, object) do
     object_actor = User.get_cached_by_ap_id(object.data["actor"])
 
     # Address the actor of the object, and our actor's follower collection if the post is public.
@@ -33,7 +96,6 @@ def like(actor, object) do
      %{
        "id" => Utils.generate_activity_id(),
        "actor" => actor.ap_id,
-       "type" => "Like",
        "object" => object.data["id"],
        "to" => to,
        "cc" => cc,

lib/pleroma/web/activity_pub/object_validator.ex

@@ -11,11 +11,35 @@ defmodule Pleroma.Web.ActivityPub.ObjectValidator do
 
   alias Pleroma.Object
   alias Pleroma.User
+  alias Pleroma.Web.ActivityPub.ObjectValidators.DeleteValidator
+  alias Pleroma.Web.ActivityPub.ObjectValidators.EmojiReactValidator
   alias Pleroma.Web.ActivityPub.ObjectValidators.LikeValidator
+  alias Pleroma.Web.ActivityPub.ObjectValidators.Types
+  alias Pleroma.Web.ActivityPub.ObjectValidators.UndoValidator
 
   @spec validate(map(), keyword()) :: {:ok, map(), keyword()} | {:error, any()}
   def validate(object, meta)
 
+  def validate(%{"type" => "Undo"} = object, meta) do
+    with {:ok, object} <-
+           object
+           |> UndoValidator.cast_and_validate()
+           |> Ecto.Changeset.apply_action(:insert) do
+      object = stringify_keys(object)
+      {:ok, object, meta}
+    end
+  end
+
+  def validate(%{"type" => "Delete"} = object, meta) do
+    with cng <- DeleteValidator.cast_and_validate(object),
+         do_not_federate <- DeleteValidator.do_not_federate?(cng),
+         {:ok, object} <- Ecto.Changeset.apply_action(cng, :insert) do
+      object = stringify_keys(object)
+      meta = Keyword.put(meta, :do_not_federate, do_not_federate)
+      {:ok, object, meta}
+    end
+  end
+
   def validate(%{"type" => "Like"} = object, meta) do
     with {:ok, object} <-
            object |> LikeValidator.cast_and_validate() |> Ecto.Changeset.apply_action(:insert) do
@@ -24,13 +48,35 @@ def validate(%{"type" => "Like"} = object, meta) do
     end
   end
 
+  def validate(%{"type" => "EmojiReact"} = object, meta) do
+    with {:ok, object} <-
+           object
+           |> EmojiReactValidator.cast_and_validate()
+           |> Ecto.Changeset.apply_action(:insert) do
+      object = stringify_keys(object |> Map.from_struct())
+      {:ok, object, meta}
+    end
+  end
+
+  def stringify_keys(%{__struct__: _} = object) do
+    object
+    |> Map.from_struct()
+    |> stringify_keys
+  end
+
   def stringify_keys(object) do
     object
     |> Map.new(fn {key, val} -> {to_string(key), val} end)
   end
 
+  def fetch_actor(object) do
+    with {:ok, actor} <- Types.ObjectID.cast(object["actor"]) do
+      User.get_or_fetch_by_ap_id(actor)
+    end
+  end
+
   def fetch_actor_and_object(object) do
-    User.get_or_fetch_by_ap_id(object["actor"])
+    fetch_actor(object)
     Object.normalize(object["object"])
     :ok
   end

lib/pleroma/web/activity_pub/object_validators/common_validations.ex

@@ -5,10 +5,33 @@
 defmodule Pleroma.Web.ActivityPub.ObjectValidators.CommonValidations do
   import Ecto.Changeset
 
+  alias Pleroma.Activity
   alias Pleroma.Object
   alias Pleroma.User
 
-  def validate_actor_presence(cng, field_name \\ :actor) do
+  def validate_recipients_presence(cng, fields \\ [:to, :cc]) do
+    non_empty =
+      fields
+      |> Enum.map(fn field -> get_field(cng, field) end)
+      |> Enum.any?(fn
+        [] -> false
+        _ -> true
+      end)
+
+    if non_empty do
+      cng
+    else
+      fields
+      |> Enum.reduce(cng, fn field, cng ->
+        cng
+        |> add_error(field, "no recipients in any field")
+      end)
+    end
+  end
+
+  def validate_actor_presence(cng, options \\ []) do
+    field_name = Keyword.get(options, :field_name, :actor)
+
     cng
     |> validate_change(field_name, fn field_name, actor ->
       if User.get_cached_by_ap_id(actor) do
@@ -19,14 +42,39 @@ def validate_actor_presence(cng, field_name \\ :actor) do
     end)
   end
 
-  def validate_object_presence(cng, field_name \\ :object) do
+  def validate_object_presence(cng, options \\ []) do
+    field_name = Keyword.get(options, :field_name, :object)
+    allowed_types = Keyword.get(options, :allowed_types, false)
+
     cng
-    |> validate_change(field_name, fn field_name, object ->
+    |> validate_change(field_name, fn field_name, object_id ->
-      if Object.get_cached_by_ap_id(object) do
+      object = Object.get_cached_by_ap_id(object_id) || Activity.get_by_ap_id(object_id)
-        []
+
-      else
+      cond do
-        [{field_name, "can't find object"}]
+        !object ->
+          [{field_name, "can't find object"}]
+
+        object && allowed_types && object.data["type"] not in allowed_types ->
+          [{field_name, "object not in allowed types"}]
+
+        true ->
+          []
       end
     end)
   end
+
+  def validate_object_or_user_presence(cng, options \\ []) do
+    field_name = Keyword.get(options, :field_name, :object)
+    options = Keyword.put(options, :field_name, field_name)
+
+    actor_cng =
+      cng
+      |> validate_actor_presence(options)
+
+    object_cng =
+      cng
+      |> validate_object_presence(options)
+
+    if actor_cng.valid?, do: actor_cng, else: object_cng
+  end
 end

lib/pleroma/web/activity_pub/object_validators/delete_validator.ex

@@ -0,0 +1,99 @@
+# Pleroma: A lightweight social networking server
+# Copyright © 2017-2020 Pleroma Authors <https://pleroma.social/>
+# SPDX-License-Identifier: AGPL-3.0-only
+
+defmodule Pleroma.Web.ActivityPub.ObjectValidators.DeleteValidator do
+  use Ecto.Schema
+
+  alias Pleroma.Activity
+  alias Pleroma.User
+  alias Pleroma.Web.ActivityPub.ObjectValidators.Types
+
+  import Ecto.Changeset
+  import Pleroma.Web.ActivityPub.ObjectValidators.CommonValidations
+
+  @primary_key false
+
+  embedded_schema do
+    field(:id, Types.ObjectID, primary_key: true)
+    field(:type, :string)
+    field(:actor, Types.ObjectID)
+    field(:to, Types.Recipients, default: [])
+    field(:cc, Types.Recipients, default: [])
+    field(:deleted_activity_id, Types.ObjectID)
+    field(:object, Types.ObjectID)
+  end
+
+  def cast_data(data) do
+    %__MODULE__{}
+    |> cast(data, __schema__(:fields))
+  end
+
+  def add_deleted_activity_id(cng) do
+    object =
+      cng
+      |> get_field(:object)
+
+    with %Activity{id: id} <- Activity.get_create_by_object_ap_id(object) do
+      cng
+      |> put_change(:deleted_activity_id, id)
+    else
+      _ -> cng
+    end
+  end
+
+  @deletable_types ~w{
+    Answer
+    Article
+    Audio
+    Event
+    Note
+    Page
+    Question
+    Video
+  }
+  def validate_data(cng) do
+    cng
+    |> validate_required([:id, :type, :actor, :to, :cc, :object])
+    |> validate_inclusion(:type, ["Delete"])
+    |> validate_actor_presence()
+    |> validate_deletion_rights()
+    |> validate_object_or_user_presence(allowed_types: @deletable_types)
+    |> add_deleted_activity_id()
+  end
+
+  def do_not_federate?(cng) do
+    !same_domain?(cng)
+  end
+
+  defp same_domain?(cng) do
+    actor_uri =
+      cng
+      |> get_field(:actor)
+      |> URI.parse()
+
+    object_uri =
+      cng
+      |> get_field(:object)
+      |> URI.parse()
+
+    object_uri.host == actor_uri.host
+  end
+
+  def validate_deletion_rights(cng) do
+    actor = User.get_cached_by_ap_id(get_field(cng, :actor))
+
+    if User.superuser?(actor) || same_domain?(cng) do
+      cng
+    else
+      cng
+      |> add_error(:actor, "is not allowed to delete object")
+    end
+  end
+
+  def cast_and_validate(data) do
+    data
+    |> cast_data
+    |> validate_data
+  end
+end

lib/pleroma/web/activity_pub/object_validators/emoji_react_validator.ex

@@ -0,0 +1,81 @@
+# Pleroma: A lightweight social networking server
+# Copyright © 2017-2020 Pleroma Authors <https://pleroma.social/>
+# SPDX-License-Identifier: AGPL-3.0-only
+
+defmodule Pleroma.Web.ActivityPub.ObjectValidators.EmojiReactValidator do
+  use Ecto.Schema
+
+  alias Pleroma.Object
+  alias Pleroma.Web.ActivityPub.ObjectValidators.Types
+
+  import Ecto.Changeset
+  import Pleroma.Web.ActivityPub.ObjectValidators.CommonValidations
+
+  @primary_key false
+
+  embedded_schema do
+    field(:id, Types.ObjectID, primary_key: true)
+    field(:type, :string)
+    field(:object, Types.ObjectID)
+    field(:actor, Types.ObjectID)
+    field(:context, :string)
+    field(:content, :string)
+    field(:to, {:array, :string}, default: [])
+    field(:cc, {:array, :string}, default: [])
+  end
+
+  def cast_and_validate(data) do
+    data
+    |> cast_data()
+    |> validate_data()
+  end
+
+  def cast_data(data) do
+    %__MODULE__{}
+    |> changeset(data)
+  end
+
+  def changeset(struct, data) do
+    struct
+    |> cast(data, __schema__(:fields))
+    |> fix_after_cast()
+  end
+
+  def fix_after_cast(cng) do
+    cng
+    |> fix_context()
+  end
+
+  def fix_context(cng) do
+    object = get_field(cng, :object)
+
+    with nil <- get_field(cng, :context),
+         %Object{data: %{"context" => context}} <- Object.get_cached_by_ap_id(object) do
+      cng
+      |> put_change(:context, context)
+    else
+      _ ->
+        cng
+    end
+  end
+
+  def validate_emoji(cng) do
+    content = get_field(cng, :content)
+
+    if Pleroma.Emoji.is_unicode_emoji?(content) do
+      cng
+    else
+      cng
+      |> add_error(:content, "must be a single character emoji")
+    end
+  end
+
+  def validate_data(data_cng) do
+    data_cng
+    |> validate_inclusion(:type, ["EmojiReact"])
+    |> validate_required([:id, :type, :object, :actor, :context, :to, :cc, :content])
+    |> validate_actor_presence()
+    |> validate_object_presence()
+    |> validate_emoji()
+  end
+end

lib/pleroma/web/activity_pub/object_validators/like_validator.ex

@@ -5,6 +5,7 @@
 defmodule Pleroma.Web.ActivityPub.ObjectValidators.LikeValidator do
   use Ecto.Schema
 
+  alias Pleroma.Object
   alias Pleroma.Web.ActivityPub.ObjectValidators.Types
   alias Pleroma.Web.ActivityPub.Utils
 
@@ -19,8 +20,8 @@ defmodule Pleroma.Web.ActivityPub.ObjectValidators.LikeValidator do
     field(:object, Types.ObjectID)
     field(:actor, Types.ObjectID)
     field(:context, :string)
-    field(:to, {:array, :string})
+    field(:to, Types.Recipients, default: [])
-    field(:cc, {:array, :string})
+    field(:cc, Types.Recipients, default: [])
   end
 
   def cast_and_validate(data) do
@@ -31,7 +32,48 @@ def cast_and_validate(data) do
 
   def cast_data(data) do
     %__MODULE__{}
-    |> cast(data, [:id, :type, :object, :actor, :context, :to, :cc])
+    |> changeset(data)
+  end
+
+  def changeset(struct, data) do
+    struct
+    |> cast(data, __schema__(:fields))
+    |> fix_after_cast()
+  end
+
+  def fix_after_cast(cng) do
+    cng
+    |> fix_recipients()
+    |> fix_context()
+  end
+
+  def fix_context(cng) do
+    object = get_field(cng, :object)
+
+    with nil <- get_field(cng, :context),
+         %Object{data: %{"context" => context}} <- Object.get_cached_by_ap_id(object) do
+      cng
+      |> put_change(:context, context)
+    else
+      _ ->
+        cng
+    end
+  end
+
+  def fix_recipients(cng) do
+    to = get_field(cng, :to)
+    cc = get_field(cng, :cc)
+    object = get_field(cng, :object)
+
+    with {[], []} <- {to, cc},
+         %Object{data: %{"actor" => actor}} <- Object.get_cached_by_ap_id(object),
+         {:ok, actor} <- Types.ObjectID.cast(actor) do
+      cng
+      |> put_change(:to, [actor])
+    else
+      _ ->
+        cng
+    end
   end
 
   def validate_data(data_cng) do

lib/pleroma/web/activity_pub/object_validators/types/recipients.ex

@@ -0,0 +1,34 @@
+defmodule Pleroma.Web.ActivityPub.ObjectValidators.Types.Recipients do
+  use Ecto.Type
+
+  alias Pleroma.Web.ActivityPub.ObjectValidators.Types.ObjectID
+
+  def type, do: {:array, ObjectID}
+
+  def cast(object) when is_binary(object) do
+    cast([object])
+  end
+
+  def cast(data) when is_list(data) do
+    data
+    |> Enum.reduce({:ok, []}, fn element, acc ->
+      case {acc, ObjectID.cast(element)} do
+        {:error, _} -> :error
+        {_, :error} -> :error
+        {{:ok, list}, {:ok, id}} -> {:ok, [id | list]}
+      end
+    end)
+  end
+
+  def cast(_) do
+    :error
+  end
+
+  def dump(data) do
+    {:ok, data}
+  end
+
+  def load(data) do
+    {:ok, data}
+  end
+end

lib/pleroma/web/activity_pub/object_validators/undo_validator.ex

@@ -0,0 +1,62 @@
+# Pleroma: A lightweight social networking server
+# Copyright © 2017-2020 Pleroma Authors <https://pleroma.social/>
+# SPDX-License-Identifier: AGPL-3.0-only
+
+defmodule Pleroma.Web.ActivityPub.ObjectValidators.UndoValidator do
+  use Ecto.Schema
+
+  alias Pleroma.Activity
+  alias Pleroma.Web.ActivityPub.ObjectValidators.Types
+
+  import Ecto.Changeset
+  import Pleroma.Web.ActivityPub.ObjectValidators.CommonValidations
+
+  @primary_key false
+
+  embedded_schema do
+    field(:id, Types.ObjectID, primary_key: true)
+    field(:type, :string)
+    field(:object, Types.ObjectID)
+    field(:actor, Types.ObjectID)
+    field(:to, {:array, :string}, default: [])
+    field(:cc, {:array, :string}, default: [])
+  end
+
+  def cast_and_validate(data) do
+    data
+    |> cast_data()
+    |> validate_data()
+  end
+
+  def cast_data(data) do
+    %__MODULE__{}
+    |> changeset(data)
+  end
+
+  def changeset(struct, data) do
+    struct
+    |> cast(data, __schema__(:fields))
+  end
+
+  def validate_data(data_cng) do
+    data_cng
+    |> validate_inclusion(:type, ["Undo"])
+    |> validate_required([:id, :type, :object, :actor, :to, :cc])
+    |> validate_actor_presence()
+    |> validate_object_presence()
+    |> validate_undo_rights()
+  end
+
+  def validate_undo_rights(cng) do
+    actor = get_field(cng, :actor)
+    object = get_field(cng, :object)
+
+    with %Activity{data: %{"actor" => object_actor}} <- Activity.get_by_ap_id(object),
+         true <- object_actor != actor do
+      cng
+      |> add_error(:actor, "not the same as object actor")
+    else
+      _ -> cng
+    end
+  end
+end

lib/pleroma/web/activity_pub/pipeline.ex

@@ -4,20 +4,33 @@
 
 defmodule Pleroma.Web.ActivityPub.Pipeline do
   alias Pleroma.Activity
+  alias Pleroma.Object
+  alias Pleroma.Repo
   alias Pleroma.Web.ActivityPub.ActivityPub
   alias Pleroma.Web.ActivityPub.MRF
   alias Pleroma.Web.ActivityPub.ObjectValidator
   alias Pleroma.Web.ActivityPub.SideEffects
   alias Pleroma.Web.Federator
 
-  @spec common_pipeline(map(), keyword()) :: {:ok, Activity.t(), keyword()} | {:error, any()}
+  @spec common_pipeline(map(), keyword()) ::
+          {:ok, Activity.t() | Object.t(), keyword()} | {:error, any()}
   def common_pipeline(object, meta) do
+    case Repo.transaction(fn -> do_common_pipeline(object, meta) end) do
+      {:ok, value} ->
+        value
+
+      {:error, e} ->
+        {:error, e}
+    end
+  end
+
+  def do_common_pipeline(object, meta) do
     with {_, {:ok, validated_object, meta}} <-
            {:validate_object, ObjectValidator.validate(object, meta)},
          {_, {:ok, mrfd_object}} <- {:mrf_object, MRF.filter(validated_object)},
-         {_, {:ok, %Activity{} = activity, meta}} <-
+         {_, {:ok, activity, meta}} <-
            {:persist_object, ActivityPub.persist(mrfd_object, meta)},
-         {_, {:ok, %Activity{} = activity, meta}} <-
+         {_, {:ok, activity, meta}} <-
            {:execute_side_effects, SideEffects.handle(activity, meta)},
          {_, {:ok, _}} <- {:federation, maybe_federate(activity, meta)} do
       {:ok, activity, meta}
@@ -27,9 +40,13 @@ def common_pipeline(object, meta) do
     end
   end
 
-  defp maybe_federate(activity, meta) do
+  defp maybe_federate(%Object{}, _), do: {:ok, :not_federated}
+
+  defp maybe_federate(%Activity{} = activity, meta) do
     with {:ok, local} <- Keyword.fetch(meta, :local) do
-      if local do
+      do_not_federate = meta[:do_not_federate]
+
+      if !do_not_federate && local do
         Federator.publish(activity)
         {:ok, :federated}
       else

lib/pleroma/web/activity_pub/side_effects.ex

@@ -5,8 +5,12 @@ defmodule Pleroma.Web.ActivityPub.SideEffects do
   liked object, a `Follow` activity will add the user to the follower
   collection, and so on.
   """
+  alias Pleroma.Activity
   alias Pleroma.Notification
   alias Pleroma.Object
+  alias Pleroma.Repo
+  alias Pleroma.User
+  alias Pleroma.Web.ActivityPub.ActivityPub
   alias Pleroma.Web.ActivityPub.Utils
 
   def handle(object, meta \\ [])
@@ -15,21 +19,115 @@ def handle(object, meta \\ [])
   # - Add like to object
   # - Set up notification
   def handle(%{data: %{"type" => "Like"}} = object, meta) do
-    {:ok, result} =
+    liked_object = Object.get_by_ap_id(object.data["object"])
-      Pleroma.Repo.transaction(fn ->
+    Utils.add_like_to_object(object, liked_object)
-        liked_object = Object.get_by_ap_id(object.data["object"])
-        Utils.add_like_to_object(object, liked_object)
 
-        Notification.create_notifications(object)
+    Notification.create_notifications(object)
 
-        {:ok, object, meta}
+    {:ok, object, meta}
-      end)
+  end
 
-    result
+  def handle(%{data: %{"type" => "Undo", "object" => undone_object}} = object, meta) do
+    with undone_object <- Activity.get_by_ap_id(undone_object),
+         :ok <- handle_undoing(undone_object) do
+      {:ok, object, meta}
+    end
+  end
+
+  # Tasks this handles:
+  # - Add reaction to object
+  # - Set up notification
+  def handle(%{data: %{"type" => "EmojiReact"}} = object, meta) do
+    reacted_object = Object.get_by_ap_id(object.data["object"])
+    Utils.add_emoji_reaction_to_object(object, reacted_object)
+
+    Notification.create_notifications(object)
+
+    {:ok, object, meta}
+  end
+
+  # Tasks this handles:
+  # - Delete and unpins the create activity
+  # - Replace object with Tombstone
+  # - Set up notification
+  # - Reduce the user note count
+  # - Reduce the reply count
+  # - Stream out the activity
+  def handle(%{data: %{"type" => "Delete", "object" => deleted_object}} = object, meta) do
+    deleted_object =
+      Object.normalize(deleted_object, false) || User.get_cached_by_ap_id(deleted_object)
+
+    result =
+      case deleted_object do
+        %Object{} ->
+          with {:ok, deleted_object, activity} <- Object.delete(deleted_object),
+               %User{} = user <- User.get_cached_by_ap_id(deleted_object.data["actor"]) do
+            User.remove_pinnned_activity(user, activity)
+
+            {:ok, user} = ActivityPub.decrease_note_count_if_public(user, deleted_object)
+
+            if in_reply_to = deleted_object.data["inReplyTo"] do
+              Object.decrease_replies_count(in_reply_to)
+            end
+
+            ActivityPub.stream_out(object)
+            ActivityPub.stream_out_participations(deleted_object, user)
+            :ok
+          end
+
+        %User{} ->
+          with {:ok, _} <- User.delete(deleted_object) do
+            :ok
+          end
+      end
+
+    if result == :ok do
+      Notification.create_notifications(object)
+      {:ok, object, meta}
+    else
+      {:error, result}
+    end
   end
 
   # Nothing to do
   def handle(object, meta) do
     {:ok, object, meta}
   end
+
+  def handle_undoing(%{data: %{"type" => "Like"}} = object) do
+    with %Object{} = liked_object <- Object.get_by_ap_id(object.data["object"]),
+         {:ok, _} <- Utils.remove_like_from_object(object, liked_object),
+         {:ok, _} <- Repo.delete(object) do
+      :ok
+    end
+  end
+
+  def handle_undoing(%{data: %{"type" => "EmojiReact"}} = object) do
+    with %Object{} = reacted_object <- Object.get_by_ap_id(object.data["object"]),
+         {:ok, _} <- Utils.remove_emoji_reaction_from_object(object, reacted_object),
+         {:ok, _} <- Repo.delete(object) do
+      :ok
+    end
+  end
+
+  def handle_undoing(%{data: %{"type" => "Announce"}} = object) do
+    with %Object{} = liked_object <- Object.get_by_ap_id(object.data["object"]),
+         {:ok, _} <- Utils.remove_announce_from_object(object, liked_object),
+         {:ok, _} <- Repo.delete(object) do
+      :ok
+    end
+  end
+
+  def handle_undoing(
+        %{data: %{"type" => "Block", "actor" => blocker, "object" => blocked}} = object
+      ) do
+    with %User{} = blocker <- User.get_cached_by_ap_id(blocker),
+         %User{} = blocked <- User.get_cached_by_ap_id(blocked),
+         {:ok, _} <- User.unblock(blocker, blocked),
+         {:ok, _} <- Repo.delete(object) do
+      :ok
+    end
+  end
+
+  def handle_undoing(object), do: {:error, ["don't know how to handle", object]}
 end

lib/pleroma/web/activity_pub/transmogrifier.ex

@@ -7,6 +7,7 @@ defmodule Pleroma.Web.ActivityPub.Transmogrifier do
   A module to handle coding from internal to wire ActivityPub and back.
   """
   alias Pleroma.Activity
+  alias Pleroma.EarmarkRenderer
   alias Pleroma.FollowingRelationship
   alias Pleroma.Object
   alias Pleroma.Object.Containment
@@ -14,7 +15,6 @@ defmodule Pleroma.Web.ActivityPub.Transmogrifier do
   alias Pleroma.User
   alias Pleroma.Web.ActivityPub.ActivityPub
   alias Pleroma.Web.ActivityPub.ObjectValidator
-  alias Pleroma.Web.ActivityPub.ObjectValidators.LikeValidator
   alias Pleroma.Web.ActivityPub.Pipeline
   alias Pleroma.Web.ActivityPub.Utils
   alias Pleroma.Web.ActivityPub.Visibility
@@ -43,6 +43,7 @@ def fix_object(object, options \\ []) do
     |> fix_addressing
     |> fix_summary
     |> fix_type(options)
+    |> fix_content
   end
 
   def fix_summary(%{"summary" => nil} = object) do
@@ -357,6 +358,18 @@ def fix_type(%{"inReplyTo" => reply_id, "name" => _} = object, options)
 
   def fix_type(object, _), do: object
 
+  defp fix_content(%{"mediaType" => "text/markdown", "content" => content} = object)
+       when is_binary(content) do
+    html_content =
+      content
+      |> Earmark.as_html!(%Earmark.Options{renderer: EarmarkRenderer})
+      |> Pleroma.HTML.filter_tags()
+
+    Map.merge(object, %{"content" => html_content, "mediaType" => "text/html"})
+  end
+
+  defp fix_content(object), do: object
+
   defp mastodon_follow_hack(%{"id" => id, "actor" => follower_id}, followed) do
     with true <- id =~ "follows",
          %User{local: true} = follower <- User.get_cached_by_ap_id(follower_id),
@@ -643,44 +656,16 @@ def handle_incoming(
     |> handle_incoming(options)
   end
 
-  def handle_incoming(%{"type" => "Like"} = data, _options) do
+  def handle_incoming(%{"type" => type} = data, _options) when type in ["Like", "EmojiReact"] do
-    with {_, {:ok, cast_data_sym}} <-
+    with :ok <- ObjectValidator.fetch_actor_and_object(data),
-           {:casting_data,
+         {:ok, activity, _meta} <-
-            data |> LikeValidator.cast_data() |> Ecto.Changeset.apply_action(:insert)},
+           Pipeline.common_pipeline(data, local: false) do
-         cast_data = ObjectValidator.stringify_keys(Map.from_struct(cast_data_sym)),
-         :ok <- ObjectValidator.fetch_actor_and_object(cast_data),
-         {_, {:ok, cast_data}} <- {:ensure_context_presence, ensure_context_presence(cast_data)},
-         {_, {:ok, cast_data}} <-
-           {:ensure_recipients_presence, ensure_recipients_presence(cast_data)},
-         {_, {:ok, activity, _meta}} <-
-           {:common_pipeline, Pipeline.common_pipeline(cast_data, local: false)} do
       {:ok, activity}
     else
       e -> {:error, e}
     end
   end
 
-  def handle_incoming(
-        %{
-          "type" => "EmojiReact",
-          "object" => object_id,
-          "actor" => _actor,
-          "id" => id,
-          "content" => emoji
-        } = data,
-        _options
-      ) do
-    with actor <- Containment.get_actor(data),
-         {:ok, %User{} = actor} <- User.get_or_fetch_by_ap_id(actor),
-         {:ok, object} <- get_obj_helper(object_id),
-         {:ok, activity, _object} <-
-           ActivityPub.react_with_emoji(actor, object, emoji, activity_id: id, local: false) do
-      {:ok, activity}
-    else
-      _e -> :error
-    end
-  end
-
   def handle_incoming(
         %{"type" => "Announce", "object" => object_id, "actor" => _actor, "id" => id} = data,
         _options
@@ -729,55 +714,12 @@ def handle_incoming(
     end
   end
 
-  # TODO: We presently assume that any actor on the same origin domain as the object being
-  # deleted has the rights to delete that object.  A better way to validate whether or not
-  # the object should be deleted is to refetch the object URI, which should return either
-  # an error or a tombstone.  This would allow us to verify that a deletion actually took
-  # place.
   def handle_incoming(
-        %{"type" => "Delete", "object" => object_id, "actor" => actor, "id" => id} = data,
+        %{"type" => "Delete"} = data,
         _options
       ) do
-    object_id = Utils.get_ap_id(object_id)
+    with {:ok, activity, _} <- Pipeline.common_pipeline(data, local: false) do
-
-    with actor <- Containment.get_actor(data),
-         {:ok, %User{} = actor} <- User.get_or_fetch_by_ap_id(actor),
-         {:ok, object} <- get_obj_helper(object_id),
-         :ok <- Containment.contain_origin(actor.ap_id, object.data),
-         {:ok, activity} <-
-           ActivityPub.delete(object, local: false, activity_id: id, actor: actor.ap_id) do
       {:ok, activity}
-    else
-      nil ->
-        case User.get_cached_by_ap_id(object_id) do
-          %User{ap_id: ^actor} = user ->
-            User.delete(user)
-
-          nil ->
-            :error
-        end
-
-      _e ->
-        :error
-    end
-  end
-
-  def handle_incoming(
-        %{
-          "type" => "Undo",
-          "object" => %{"type" => "Announce", "object" => object_id},
-          "actor" => _actor,
-          "id" => id
-        } = data,
-        _options
-      ) do
-    with actor <- Containment.get_actor(data),
-         {:ok, %User{} = actor} <- User.get_or_fetch_by_ap_id(actor),
-         {:ok, object} <- get_obj_helper(object_id),
-         {:ok, activity, _} <- ActivityPub.unannounce(actor, object, id, false) do
-      {:ok, activity}
-    else
-      _e -> :error
     end
   end
 
@@ -803,75 +745,13 @@ def handle_incoming(
   def handle_incoming(
         %{
           "type" => "Undo",
-          "object" => %{"type" => "EmojiReact", "id" => reaction_activity_id},
+          "object" => %{"type" => type}
-          "actor" => _actor,
-          "id" => id
         } = data,
         _options
-      ) do
+      )
-    with actor <- Containment.get_actor(data),
+      when type in ["Like", "EmojiReact", "Announce", "Block"] do
-         {:ok, %User{} = actor} <- User.get_or_fetch_by_ap_id(actor),
+    with {:ok, activity, _} <- Pipeline.common_pipeline(data, local: false) do
-         {:ok, activity, _} <-
-           ActivityPub.unreact_with_emoji(actor, reaction_activity_id,
-             activity_id: id,
-             local: false
-           ) do
       {:ok, activity}
-    else
-      _e -> :error
-    end
-  end
-
-  def handle_incoming(
-        %{
-          "type" => "Undo",
-          "object" => %{"type" => "Block", "object" => blocked},
-          "actor" => blocker,
-          "id" => id
-        } = _data,
-        _options
-      ) do
-    with %User{local: true} = blocked <- User.get_cached_by_ap_id(blocked),
-         {:ok, %User{} = blocker} <- User.get_or_fetch_by_ap_id(blocker),
-         {:ok, activity} <- ActivityPub.unblock(blocker, blocked, id, false) do
-      User.unblock(blocker, blocked)
-      {:ok, activity}
-    else
-      _e -> :error
-    end
-  end
-
-  def handle_incoming(
-        %{"type" => "Block", "object" => blocked, "actor" => blocker, "id" => id} = _data,
-        _options
-      ) do
-    with %User{local: true} = blocked = User.get_cached_by_ap_id(blocked),
-         {:ok, %User{} = blocker} = User.get_or_fetch_by_ap_id(blocker),
-         {:ok, activity} <- ActivityPub.block(blocker, blocked, id, false) do
-      User.unfollow(blocker, blocked)
-      User.block(blocker, blocked)
-      {:ok, activity}
-    else
-      _e -> :error
-    end
-  end
-
-  def handle_incoming(
-        %{
-          "type" => "Undo",
-          "object" => %{"type" => "Like", "object" => object_id},
-          "actor" => _actor,
-          "id" => id
-        } = data,
-        _options
-      ) do
-    with actor <- Containment.get_actor(data),
-         {:ok, %User{} = actor} <- User.get_or_fetch_by_ap_id(actor),
-         {:ok, object} <- get_obj_helper(object_id),
-         {:ok, activity, _, _} <- ActivityPub.unlike(actor, object, id, false) do
-      {:ok, activity}
-    else
-      _e -> :error
     end
   end
 
@@ -893,6 +773,21 @@ def handle_incoming(
     end
   end
 
+  def handle_incoming(
+        %{"type" => "Block", "object" => blocked, "actor" => blocker, "id" => id} = _data,
+        _options
+      ) do
+    with %User{local: true} = blocked = User.get_cached_by_ap_id(blocked),
+         {:ok, %User{} = blocker} = User.get_or_fetch_by_ap_id(blocker),
+         {:ok, activity} <- ActivityPub.block(blocker, blocked, id, false) do
+      User.unfollow(blocker, blocked)
+      User.block(blocker, blocked)
+      {:ok, activity}
+    else
+      _e -> :error
+    end
+  end
+
   def handle_incoming(
         %{
           "type" => "Move",
@@ -1189,6 +1084,10 @@ def set_conversation(object) do
     Map.put(object, "conversation", object["context"])
   end
 
+  def set_sensitive(%{"sensitive" => true} = object) do
+    object
+  end
+
   def set_sensitive(object) do
     tags = object["tag"] || []
     Map.put(object, "sensitive", "nsfw" in tags)
@@ -1207,18 +1106,24 @@ def add_attributed_to(object) do
 
   def prepare_attachments(object) do
     attachments =
-      (object["attachment"] || [])
+      object
+      |> Map.get("attachment", [])
       |> Enum.map(fn data ->
         [%{"mediaType" => media_type, "href" => href} | _] = data["url"]
-        %{"url" => href, "mediaType" => media_type, "name" => data["name"], "type" => "Document"}
+
+        %{
+          "url" => href,
+          "mediaType" => media_type,
+          "name" => data["name"],
+          "type" => "Document"
+        }
       end)
 
     Map.put(object, "attachment", attachments)
   end
 
   def strip_internal_fields(object) do
-    object
+    Map.drop(object, Pleroma.Constants.object_internal_fields())
-    |> Map.drop(Pleroma.Constants.object_internal_fields())
   end
 
   defp strip_internal_tags(%{"tag" => tags} = object) do
@@ -1276,45 +1181,4 @@ def maybe_fix_user_url(%{"url" => url} = data) when is_map(url) do
   def maybe_fix_user_url(data), do: data
 
   def maybe_fix_user_object(data), do: maybe_fix_user_url(data)
-
-  defp ensure_context_presence(%{"context" => context} = data) when is_binary(context),
-    do: {:ok, data}
-
-  defp ensure_context_presence(%{"object" => object} = data) when is_binary(object) do
-    with %{data: %{"context" => context}} when is_binary(context) <- Object.normalize(object) do
-      {:ok, Map.put(data, "context", context)}
-    else
-      _ ->
-        {:error, :no_context}
-    end
-  end
-
-  defp ensure_context_presence(_) do
-    {:error, :no_context}
-  end
-
-  defp ensure_recipients_presence(%{"to" => [_ | _], "cc" => [_ | _]} = data),
-    do: {:ok, data}
-
-  defp ensure_recipients_presence(%{"object" => object} = data) do
-    case Object.normalize(object) do
-      %{data: %{"actor" => actor}} ->
-        data =
-          data
-          |> Map.put("to", [actor])
-          |> Map.put("cc", data["cc"] || [])
-
-        {:ok, data}
-
-      nil ->
-        {:error, :no_object}
-
-      _ ->
-        {:error, :no_actor}
-    end
-  end
-
-  defp ensure_recipients_presence(_) do
-    {:error, :no_object}
-  end
 end

lib/pleroma/web/activity_pub/utils.ex

@@ -512,7 +512,7 @@ def get_latest_reaction(internal_activity_id, %{ap_id: ap_id}, emoji) do
   #### Announce-related helpers
 
   @doc """
-  Retruns an existing announce activity if the notice has already been announced
+  Returns an existing announce activity if the notice has already been announced
   """
   @spec get_existing_announce(String.t(), map()) :: Activity.t() | nil
   def get_existing_announce(actor, %{data: %{"id" => ap_id}}) do
@@ -562,45 +562,6 @@ def make_announce_data(
     |> maybe_put("id", activity_id)
   end
 
-  @doc """
-  Make unannounce activity data for the given actor and object
-  """
-  def make_unannounce_data(
-        %User{ap_id: ap_id} = user,
-        %Activity{data: %{"context" => context, "object" => object}} = activity,
-        activity_id
-      ) do
-    object = Object.normalize(object)
-
-    %{
-      "type" => "Undo",
-      "actor" => ap_id,
-      "object" => activity.data,
-      "to" => [user.follower_address, object.data["actor"]],
-      "cc" => [Pleroma.Constants.as_public()],
-      "context" => context
-    }
-    |> maybe_put("id", activity_id)
-  end
-
-  def make_unlike_data(
-        %User{ap_id: ap_id} = user,
-        %Activity{data: %{"context" => context, "object" => object}} = activity,
-        activity_id
-      ) do
-    object = Object.normalize(object)
-
-    %{
-      "type" => "Undo",
-      "actor" => ap_id,
-      "object" => activity.data,
-      "to" => [user.follower_address, object.data["actor"]],
-      "cc" => [Pleroma.Constants.as_public()],
-      "context" => context
-    }
-    |> maybe_put("id", activity_id)
-  end
-
   def make_undo_data(
         %User{ap_id: actor, follower_address: follower_address},
         %Activity{
@@ -688,16 +649,6 @@ def make_block_data(blocker, blocked, activity_id) do
     |> maybe_put("id", activity_id)
   end
 
-  def make_unblock_data(blocker, blocked, block_activity, activity_id) do
-    %{
-      "type" => "Undo",
-      "actor" => blocker.ap_id,
-      "to" => [blocked.ap_id],
-      "object" => block_activity.data
-    }
-    |> maybe_put("id", activity_id)
-  end
-
   #### Create-related helpers
 
   def make_create_data(params, additional) do

lib/pleroma/web/admin_api/admin_api_controller.ex

@@ -10,6 +10,7 @@ defmodule Pleroma.Web.AdminAPI.AdminAPIController do
   alias Pleroma.Activity
   alias Pleroma.Config
   alias Pleroma.ConfigDB
+  alias Pleroma.MFA
   alias Pleroma.ModerationLog
   alias Pleroma.Plugs.OAuthScopesPlug
   alias Pleroma.ReportNote
@@ -17,6 +18,8 @@ defmodule Pleroma.Web.AdminAPI.AdminAPIController do
   alias Pleroma.User
   alias Pleroma.UserInviteToken
   alias Pleroma.Web.ActivityPub.ActivityPub
+  alias Pleroma.Web.ActivityPub.Builder
+  alias Pleroma.Web.ActivityPub.Pipeline
   alias Pleroma.Web.ActivityPub.Relay
   alias Pleroma.Web.ActivityPub.Utils
   alias Pleroma.Web.AdminAPI.AccountView
@@ -59,6 +62,7 @@ defmodule Pleroma.Web.AdminAPI.AdminAPIController do
            :right_add,
            :right_add_multiple,
            :right_delete,
+           :disable_mfa,
            :right_delete_multiple,
            :update_user_credentials
          ]
@@ -93,7 +97,7 @@ defmodule Pleroma.Web.AdminAPI.AdminAPIController do
   plug(
     OAuthScopesPlug,
     %{scopes: ["read:statuses"], admin: true}
-    when action in [:list_statuses, :list_user_statuses, :list_instance_statuses]
+    when action in [:list_statuses, :list_user_statuses, :list_instance_statuses, :status_show]
   )
 
   plug(
@@ -133,23 +137,20 @@ defmodule Pleroma.Web.AdminAPI.AdminAPIController do
 
   action_fallback(:errors)
 
-  def user_delete(%{assigns: %{user: admin}} = conn, %{"nickname" => nickname}) do
+  def user_delete(conn, %{"nickname" => nickname}) do
-    user = User.get_cached_by_nickname(nickname)
+    user_delete(conn, %{"nicknames" => [nickname]})
-    User.delete(user)
-
-    ModerationLog.insert_log(%{
-      actor: admin,
-      subject: [user],
-      action: "delete"
-    })
-
-    conn
-    |> json(nickname)
   end
 
   def user_delete(%{assigns: %{user: admin}} = conn, %{"nicknames" => nicknames}) do
-    users = nicknames |> Enum.map(&User.get_cached_by_nickname/1)
+    users =
-    User.delete(users)
+      nicknames
+      |> Enum.map(&User.get_cached_by_nickname/1)
+
+    users
+    |> Enum.each(fn user ->
+      {:ok, delete_data, _} = Builder.delete(admin, user.ap_id)
+      Pipeline.common_pipeline(delete_data, local: true)
+    end)
 
     ModerationLog.insert_log(%{
       actor: admin,
@@ -392,29 +393,12 @@ def list_users(conn, params) do
       email: params["email"]
     }
 
-    with {:ok, users, count} <- Search.user(Map.merge(search_params, filters)),
+    with {:ok, users, count} <- Search.user(Map.merge(search_params, filters)) do
-         {:ok, users, count} <- filter_service_users(users, count),
+      json(
-         do:
+        conn,
-           conn
+        AccountView.render("index.json", users: users, count: count, page_size: page_size)
-           |> json(
+      )
-             AccountView.render("index.json",
+    end
-               users: users,
-               count: count,
-               page_size: page_size
-             )
-           )
-  end
-
-  defp filter_service_users(users, count) do
-    filtered_users = Enum.reject(users, &service_user?/1)
-    count = if Enum.any?(users, &service_user?/1), do: length(filtered_users), else: count
-
-    {:ok, filtered_users, count}
-  end
-
-  defp service_user?(user) do
-    String.match?(user.ap_id, ~r/.*\/relay$/) or
-      String.match?(user.ap_id, ~r/.*\/internal\/fetch$/)
   end
 
   @filters ~w(local external active deactivated is_admin is_moderator)
@@ -692,6 +676,18 @@ def force_password_reset(%{assigns: %{user: admin}} = conn, %{"nicknames" => nic
     json_response(conn, :no_content, "")
   end
 
+  @doc "Disable mfa for user's account."
+  def disable_mfa(conn, %{"nickname" => nickname}) do
+    case User.get_by_nickname(nickname) do
+      %User{} = user ->
+        MFA.disable(user)
+        json(conn, nickname)
+
+      _ ->
+        {:error, :not_found}
+    end
+  end
+
   @doc "Show a given user's credentials"
   def show_user_credentials(%{assigns: %{user: admin}} = conn, %{"nickname" => nickname}) do
     with %User{} = user <- User.get_cached_by_nickname_or_id(nickname) do
@@ -837,6 +833,16 @@ def list_statuses(%{assigns: %{user: _admin}} = conn, params) do
     |> render("index.json", %{activities: activities, as: :activity, skip_relationships: false})
   end
 
+  def status_show(conn, %{"id" => id}) do
+    with %Activity{} = activity <- Activity.get_by_id(id) do
+      conn
+      |> put_view(StatusView)
+      |> render("show.json", %{activity: activity})
+    else
+      _ -> errors(conn, {:error, :not_found})
+    end
+  end
+
   def status_update(%{assigns: %{user: admin}} = conn, %{"id" => id} = params) do
     with {:ok, activity} <- CommonAPI.update_activity_scope(id, params) do
       {:ok, sensitive} = Ecto.Type.cast(:boolean, params["sensitive"])

lib/pleroma/web/admin_api/search.ex

@@ -21,6 +21,7 @@ def user(params \\ %{}) do
     query =
       params
       |> Map.drop([:page, :page_size])
+      |> Map.put(:exclude_service_users, true)
       |> User.Query.build()
       |> order_by([u], u.nickname)
 

lib/pleroma/web/api_spec.ex

@@ -39,7 +39,12 @@ def spec do
               password: %OpenApiSpex.OAuthFlow{
                 authorizationUrl: "/oauth/authorize",
                 tokenUrl: "/oauth/token",
-                scopes: %{"read" => "read", "write" => "write", "follow" => "follow"}
+                scopes: %{
+                  "read" => "read",
+                  "write" => "write",
+                  "follow" => "follow",
+                  "push" => "push"
+                }
               }
             }
           }

lib/pleroma/web/api_spec/cast_and_validate.ex

@@ -0,0 +1,139 @@
+# Pleroma: A lightweight social networking server
+# Copyright © 2019-2020 Moxley Stratton, Mike Buhot <https://github.com/open-api-spex/open_api_spex>, MPL-2.0
+# Copyright © 2020 Pleroma Authors <https://pleroma.social/>
+# SPDX-License-Identifier: AGPL-3.0-only
+
+defmodule Pleroma.Web.ApiSpec.CastAndValidate do
+  @moduledoc """
+  This plug is based on [`OpenApiSpex.Plug.CastAndValidate`]
+  (https://github.com/open-api-spex/open_api_spex/blob/master/lib/open_api_spex/plug/cast_and_validate.ex).
+  The main difference is ignoring unexpected query params instead of throwing
+  an error and a config option (`[Pleroma.Web.ApiSpec.CastAndValidate, :strict]`)
+  to disable this behavior. Also, the default rendering error module
+  is `Pleroma.Web.ApiSpec.RenderError`.
+  """
+
+  @behaviour Plug
+
+  alias Plug.Conn
+
+  @impl Plug
+  def init(opts) do
+    opts
+    |> Map.new()
+    |> Map.put_new(:render_error, Pleroma.Web.ApiSpec.RenderError)
+  end
+
+  @impl Plug
+  def call(%{private: %{open_api_spex: private_data}} = conn, %{
+        operation_id: operation_id,
+        render_error: render_error
+      }) do
+    spec = private_data.spec
+    operation = private_data.operation_lookup[operation_id]
+
+    content_type =
+      case Conn.get_req_header(conn, "content-type") do
+        [header_value | _] ->
+          header_value
+          |> String.split(";")
+          |> List.first()
+
+        _ ->
+          nil
+      end
+
+    private_data = Map.put(private_data, :operation_id, operation_id)
+    conn = Conn.put_private(conn, :open_api_spex, private_data)
+
+    case cast_and_validate(spec, operation, conn, content_type, strict?()) do
+      {:ok, conn} ->
+        conn
+
+      {:error, reason} ->
+        opts = render_error.init(reason)
+
+        conn
+        |> render_error.call(opts)
+        |> Plug.Conn.halt()
+    end
+  end
+
+  def call(
+        %{
+          private: %{
+            phoenix_controller: controller,
+            phoenix_action: action,
+            open_api_spex: private_data
+          }
+        } = conn,
+        opts
+      ) do
+    operation =
+      case private_data.operation_lookup[{controller, action}] do
+        nil ->
+          operation_id = controller.open_api_operation(action).operationId
+          operation = private_data.operation_lookup[operation_id]
+
+          operation_lookup =
+            private_data.operation_lookup
+            |> Map.put({controller, action}, operation)
+
+          OpenApiSpex.Plug.Cache.adapter().put(
+            private_data.spec_module,
+            {private_data.spec, operation_lookup}
+          )
+
+          operation
+
+        operation ->
+          operation
+      end
+
+    if operation.operationId do
+      call(conn, Map.put(opts, :operation_id, operation.operationId))
+    else
+      raise "operationId was not found in action API spec"
+    end
+  end
+
+  def call(conn, opts), do: OpenApiSpex.Plug.CastAndValidate.call(conn, opts)
+
+  defp cast_and_validate(spec, operation, conn, content_type, true = _strict) do
+    OpenApiSpex.cast_and_validate(spec, operation, conn, content_type)
+  end
+
+  defp cast_and_validate(spec, operation, conn, content_type, false = _strict) do
+    case OpenApiSpex.cast_and_validate(spec, operation, conn, content_type) do
+      {:ok, conn} ->
+        {:ok, conn}
+
+      # Remove unexpected query params and cast/validate again
+      {:error, errors} ->
+        query_params =
+          Enum.reduce(errors, conn.query_params, fn
+            %{reason: :unexpected_field, name: name, path: [name]}, params ->
+              Map.delete(params, name)
+
+            %{reason: :invalid_enum, name: nil, path: path, value: value}, params ->
+              path = path |> Enum.reverse() |> tl() |> Enum.reverse() |> list_items_to_string()
+              update_in(params, path, &List.delete(&1, value))
+
+            _, params ->
+              params
+          end)
+
+        conn = %Conn{conn | query_params: query_params}
+        OpenApiSpex.cast_and_validate(spec, operation, conn, content_type)
+    end
+  end
+
+  defp list_items_to_string(list) do
+    Enum.map(list, fn
+      i when is_atom(i) -> to_string(i)
+      i -> i
+    end)
+  end
+
+  defp strict?, do: Pleroma.Config.get([__MODULE__, :strict], false)
+end

lib/pleroma/web/api_spec/helpers.ex

@@ -41,9 +41,17 @@ def pagination_params do
       Operation.parameter(
         :limit,
         :query,
-        %Schema{type: :integer, default: 20, maximum: 40},
+        %Schema{type: :integer, default: 20},
-        "Limit"
+        "Maximum number of items to return. Will be ignored if it's more than 40"
       )
     ]
   end
+
+  def empty_object_response do
+    Operation.response("Empty object", "application/json", %Schema{type: :object, example: %{}})
+  end
+
+  def empty_array_response do
+    Operation.response("Empty array", "application/json", %Schema{type: :array, example: []})
+  end
 end

lib/pleroma/web/api_spec/operations/account_operation.ex

@@ -11,6 +11,7 @@ defmodule Pleroma.Web.ApiSpec.AccountOperation do
   alias Pleroma.Web.ApiSpec.Schemas.ActorType
   alias Pleroma.Web.ApiSpec.Schemas.ApiError
   alias Pleroma.Web.ApiSpec.Schemas.BooleanLike
+  alias Pleroma.Web.ApiSpec.Schemas.List
   alias Pleroma.Web.ApiSpec.Schemas.Status
   alias Pleroma.Web.ApiSpec.Schemas.VisibilityScope
 
@@ -344,7 +345,7 @@ def endorsements_operation do
       description: "Not implemented",
       security: [%{"oAuth" => ["read:accounts"]}],
       responses: %{
-        200 => Operation.response("Empry array", "application/json", %Schema{type: :array})
+        200 => empty_array_response()
       }
     }
   end
@@ -356,7 +357,7 @@ def identity_proofs_operation do
       operationId: "AccountController.identity_proofs",
       description: "Not implemented",
       responses: %{
-        200 => Operation.response("Empry array", "application/json", %Schema{type: :array})
+        200 => empty_array_response()
       }
     }
   end
@@ -555,11 +556,12 @@ defp update_creadentials_request do
     }
   end
 
-  defp array_of_accounts do
+  def array_of_accounts do
     %Schema{
       title: "ArrayOfAccounts",
       type: :array,
-      items: Account
+      items: Account,
+      example: [Account.schema().example]
     }
   end
 
@@ -646,28 +648,12 @@ defp mute_request do
     }
   end
 
-  defp list do
-    %Schema{
-      title: "List",
-      description: "Response schema for a list",
-      type: :object,
-      properties: %{
-        id: %Schema{type: :string},
-        title: %Schema{type: :string}
-      },
-      example: %{
-        "id" => "123",
-        "title" => "my list"
-      }
-    }
-  end
-
   defp array_of_lists do
     %Schema{
       title: "ArrayOfLists",
       description: "Response schema for lists",
       type: :array,
-      items: list(),
+      items: List,
       example: [
         %{"id" => "123", "title" => "my list"},
         %{"id" => "1337", "title" => "anotehr list"}

lib/pleroma/web/api_spec/operations/conversation_operation.ex

@@ -0,0 +1,61 @@
+# Pleroma: A lightweight social networking server
+# Copyright © 2017-2020 Pleroma Authors <https://pleroma.social/>
+# SPDX-License-Identifier: AGPL-3.0-only
+
+defmodule Pleroma.Web.ApiSpec.ConversationOperation do
+  alias OpenApiSpex.Operation
+  alias OpenApiSpex.Schema
+  alias Pleroma.Web.ApiSpec.Schemas.Conversation
+  alias Pleroma.Web.ApiSpec.Schemas.FlakeID
+
+  import Pleroma.Web.ApiSpec.Helpers
+
+  def open_api_operation(action) do
+    operation = String.to_existing_atom("#{action}_operation")
+    apply(__MODULE__, operation, [])
+  end
+
+  def index_operation do
+    %Operation{
+      tags: ["Conversations"],
+      summary: "Show conversation",
+      security: [%{"oAuth" => ["read:statuses"]}],
+      operationId: "ConversationController.index",
+      parameters: [
+        Operation.parameter(
+          :recipients,
+          :query,
+          %Schema{type: :array, items: FlakeID},
+          "Only return conversations with the given recipients (a list of user ids)"
+        )
+        | pagination_params()
+      ],
+      responses: %{
+        200 =>
+          Operation.response("Array of Conversation", "application/json", %Schema{
+            type: :array,
+            items: Conversation,
+            example: [Conversation.schema().example]
+          })
+      }
+    }
+  end
+
+  def mark_as_read_operation do
+    %Operation{
+      tags: ["Conversations"],
+      summary: "Mark as read",
+      operationId: "ConversationController.mark_as_read",
+      parameters: [
+        Operation.parameter(:id, :path, :string, "Conversation ID",
+          example: "123",
+          required: true
+        )
+      ],
+      security: [%{"oAuth" => ["write:conversations"]}],
+      responses: %{
+        200 => Operation.response("Conversation", "application/json", Conversation)
+      }
+    }
+  end
+end

lib/pleroma/web/api_spec/operations/domain_block_operation.ex

@@ -5,7 +5,7 @@
 defmodule Pleroma.Web.ApiSpec.DomainBlockOperation do
   alias OpenApiSpex.Operation
   alias OpenApiSpex.Schema
-  alias Pleroma.Web.ApiSpec.Helpers
+  import Pleroma.Web.ApiSpec.Helpers
 
   def open_api_operation(action) do
     operation = String.to_existing_atom("#{action}_operation")
@@ -46,9 +46,7 @@ def create_operation do
       operationId: "DomainBlockController.create",
       requestBody: domain_block_request(),
       security: [%{"oAuth" => ["follow", "write:blocks"]}],
-      responses: %{
+      responses: %{200 => empty_object_response()}
-        200 => Operation.response("Empty object", "application/json", %Schema{type: :object})
-      }
     }
   end
 
@@ -67,7 +65,7 @@ def delete_operation do
   end
 
   defp domain_block_request do
-    Helpers.request_body(
+    request_body(
       "Parameters",
       %Schema{
         type: :object,

lib/pleroma/web/api_spec/operations/filter_operation.ex

@@ -0,0 +1,227 @@
+# Pleroma: A lightweight social networking server
+# Copyright © 2017-2020 Pleroma Authors <https://pleroma.social/>
+# SPDX-License-Identifier: AGPL-3.0-only
+
+defmodule Pleroma.Web.ApiSpec.FilterOperation do
+  alias OpenApiSpex.Operation
+  alias OpenApiSpex.Schema
+  alias Pleroma.Web.ApiSpec.Helpers
+
+  def open_api_operation(action) do
+    operation = String.to_existing_atom("#{action}_operation")
+    apply(__MODULE__, operation, [])
+  end
+
+  def index_operation do
+    %Operation{
+      tags: ["apps"],
+      summary: "View all filters",
+      operationId: "FilterController.index",
+      security: [%{"oAuth" => ["read:filters"]}],
+      responses: %{
+        200 => Operation.response("Filters", "application/json", array_of_filters())
+      }
+    }
+  end
+
+  def create_operation do
+    %Operation{
+      tags: ["apps"],
+      summary: "Create a filter",
+      operationId: "FilterController.create",
+      requestBody: Helpers.request_body("Parameters", create_request(), required: true),
+      security: [%{"oAuth" => ["write:filters"]}],
+      responses: %{200 => Operation.response("Filter", "application/json", filter())}
+    }
+  end
+
+  def show_operation do
+    %Operation{
+      tags: ["apps"],
+      summary: "View all filters",
+      parameters: [id_param()],
+      operationId: "FilterController.show",
+      security: [%{"oAuth" => ["read:filters"]}],
+      responses: %{
+        200 => Operation.response("Filter", "application/json", filter())
+      }
+    }
+  end
+
+  def update_operation do
+    %Operation{
+      tags: ["apps"],
+      summary: "Update a filter",
+      parameters: [id_param()],
+      operationId: "FilterController.update",
+      requestBody: Helpers.request_body("Parameters", update_request(), required: true),
+      security: [%{"oAuth" => ["write:filters"]}],
+      responses: %{
+        200 => Operation.response("Filter", "application/json", filter())
+      }
+    }
+  end
+
+  def delete_operation do
+    %Operation{
+      tags: ["apps"],
+      summary: "Remove a filter",
+      parameters: [id_param()],
+      operationId: "FilterController.delete",
+      security: [%{"oAuth" => ["write:filters"]}],
+      responses: %{
+        200 =>
+          Operation.response("Filter", "application/json", %Schema{
+            type: :object,
+            description: "Empty object"
+          })
+      }
+    }
+  end
+
+  defp id_param do
+    Operation.parameter(:id, :path, :string, "Filter ID", example: "123", required: true)
+  end
+
+  defp filter do
+    %Schema{
+      title: "Filter",
+      type: :object,
+      properties: %{
+        id: %Schema{type: :string},
+        phrase: %Schema{type: :string, description: "The text to be filtered"},
+        context: %Schema{
+          type: :array,
+          items: %Schema{type: :string, enum: ["home", "notifications", "public", "thread"]},
+          description: "The contexts in which the filter should be applied."
+        },
+        expires_at: %Schema{
+          type: :string,
+          format: :"date-time",
+          description:
+            "When the filter should no longer be applied. String (ISO 8601 Datetime), or null if the filter does not expire.",
+          nullable: true
+        },
+        irreversible: %Schema{
+          type: :boolean,
+          description:
+            "Should matching entities in home and notifications be dropped by the server?"
+        },
+        whole_word: %Schema{
+          type: :boolean,
+          description: "Should the filter consider word boundaries?"
+        }
+      },
+      example: %{
+        "id" => "5580",
+        "phrase" => "@twitter.com",
+        "context" => [
+          "home",
+          "notifications",
+          "public",
+          "thread"
+        ],
+        "whole_word" => false,
+        "expires_at" => nil,
+        "irreversible" => true
+      }
+    }
+  end
+
+  defp array_of_filters do
+    %Schema{
+      title: "ArrayOfFilters",
+      description: "Array of Filters",
+      type: :array,
+      items: filter(),
+      example: [
+        %{
+          "id" => "5580",
+          "phrase" => "@twitter.com",
+          "context" => [
+            "home",
+            "notifications",
+            "public",
+            "thread"
+          ],
+          "whole_word" => false,
+          "expires_at" => nil,
+          "irreversible" => true
+        },
+        %{
+          "id" => "6191",
+          "phrase" => ":eurovision2019:",
+          "context" => [
+            "home"
+          ],
+          "whole_word" => true,
+          "expires_at" => "2019-05-21T13:47:31.333Z",
+          "irreversible" => false
+        }
+      ]
+    }
+  end
+
+  defp create_request do
+    %Schema{
+      title: "FilterCreateRequest",
+      allOf: [
+        update_request(),
+        %Schema{
+          type: :object,
+          properties: %{
+            irreversible: %Schema{
+              type: :bolean,
+              description:
+                "Should the server irreversibly drop matching entities from home and notifications?",
+              default: false
+            }
+          }
+        }
+      ],
+      example: %{
+        "phrase" => "knights",
+        "context" => ["home"]
+      }
+    }
+  end
+
+  defp update_request do
+    %Schema{
+      title: "FilterUpdateRequest",
+      type: :object,
+      properties: %{
+        phrase: %Schema{type: :string, description: "The text to be filtered"},
+        context: %Schema{
+          type: :array,
+          items: %Schema{type: :string, enum: ["home", "notifications", "public", "thread"]},
+          description:
+            "Array of enumerable strings `home`, `notifications`, `public`, `thread`. At least one context must be specified."
+        },
+        irreversible: %Schema{
+          type: :bolean,
+          description:
+            "Should the server irreversibly drop matching entities from home and notifications?"
+        },
+        whole_word: %Schema{
+          type: :bolean,
+          description: "Consider word boundaries?",
+          default: true
+        }
+        # TODO: probably should implement filter expiration
+        # expires_in: %Schema{
+        #   type: :string,
+        #   format: :"date-time",
+        #   description:
+        #     "ISO 8601 Datetime for when the filter expires. Otherwise,
+        #  null for a filter that doesn't expire."
+        # }
+      },
+      required: [:phrase, :context],
+      example: %{
+        "phrase" => "knights",
+        "context" => ["home"]
+      }
+    }
+  end
+end

lib/pleroma/web/api_spec/operations/follow_request_operation.ex

@@ -0,0 +1,65 @@
+# Pleroma: A lightweight social networking server
+# Copyright © 2017-2020 Pleroma Authors <https://pleroma.social/>
+# SPDX-License-Identifier: AGPL-3.0-only
+
+defmodule Pleroma.Web.ApiSpec.FollowRequestOperation do
+  alias OpenApiSpex.Operation
+  alias OpenApiSpex.Schema
+  alias Pleroma.Web.ApiSpec.Schemas.Account
+  alias Pleroma.Web.ApiSpec.Schemas.AccountRelationship
+
+  def open_api_operation(action) do
+    operation = String.to_existing_atom("#{action}_operation")
+    apply(__MODULE__, operation, [])
+  end
+
+  def index_operation do
+    %Operation{
+      tags: ["Follow Requests"],
+      summary: "Pending Follows",
+      security: [%{"oAuth" => ["read:follows", "follow"]}],
+      operationId: "FollowRequestController.index",
+      responses: %{
+        200 =>
+          Operation.response("Array of Account", "application/json", %Schema{
+            type: :array,
+            items: Account,
+            example: [Account.schema().example]
+          })
+      }
+    }
+  end
+
+  def authorize_operation do
+    %Operation{
+      tags: ["Follow Requests"],
+      summary: "Accept Follow",
+      operationId: "FollowRequestController.authorize",
+      parameters: [id_param()],
+      security: [%{"oAuth" => ["follow", "write:follows"]}],
+      responses: %{
+        200 => Operation.response("Relationship", "application/json", AccountRelationship)
+      }
+    }
+  end
+
+  def reject_operation do
+    %Operation{
+      tags: ["Follow Requests"],
+      summary: "Reject Follow",
+      operationId: "FollowRequestController.reject",
+      parameters: [id_param()],
+      security: [%{"oAuth" => ["follow", "write:follows"]}],
+      responses: %{
+        200 => Operation.response("Relationship", "application/json", AccountRelationship)
+      }
+    }
+  end
+
+  defp id_param do
+    Operation.parameter(:id, :path, :string, "Conversation ID",
+      example: "123",
+      required: true
+    )
+  end
+end

lib/pleroma/web/api_spec/operations/instance_operation.ex

@@ -0,0 +1,169 @@
+# Pleroma: A lightweight social networking server
+# Copyright © 2017-2020 Pleroma Authors <https://pleroma.social/>
+# SPDX-License-Identifier: AGPL-3.0-only
+
+defmodule Pleroma.Web.ApiSpec.InstanceOperation do
+  alias OpenApiSpex.Operation
+  alias OpenApiSpex.Schema
+
+  def open_api_operation(action) do
+    operation = String.to_existing_atom("#{action}_operation")
+    apply(__MODULE__, operation, [])
+  end
+
+  def show_operation do
+    %Operation{
+      tags: ["Instance"],
+      summary: "Fetch instance",
+      description: "Information about the server",
+      operationId: "InstanceController.show",
+      responses: %{
+        200 => Operation.response("Instance", "application/json", instance())
+      }
+    }
+  end
+
+  def peers_operation do
+    %Operation{
+      tags: ["Instance"],
+      summary: "List of known hosts",
+      operationId: "InstanceController.peers",
+      responses: %{
+        200 => Operation.response("Array of domains", "application/json", array_of_domains())
+      }
+    }
+  end
+
+  defp instance do
+    %Schema{
+      type: :object,
+      properties: %{
+        uri: %Schema{type: :string, description: "The domain name of the instance"},
+        title: %Schema{type: :string, description: "The title of the website"},
+        description: %Schema{
+          type: :string,
+          description: "Admin-defined description of the Pleroma site"
+        },
+        version: %Schema{
+          type: :string,
+          description: "The version of Pleroma installed on the instance"
+        },
+        email: %Schema{
+          type: :string,
+          description: "An email that may be contacted for any inquiries",
+          format: :email
+        },
+        urls: %Schema{
+          type: :object,
+          description: "URLs of interest for clients apps",
+          properties: %{
+            streaming_api: %Schema{
+              type: :string,
+              description: "Websockets address for push streaming"
+            }
+          }
+        },
+        stats: %Schema{
+          type: :object,
+          description: "Statistics about how much information the instance contains",
+          properties: %{
+            user_count: %Schema{
+              type: :integer,
+              description: "Users registered on this instance"
+            },
+            status_count: %Schema{
+              type: :integer,
+              description: "Statuses authored by users on instance"
+            },
+            domain_count: %Schema{
+              type: :integer,
+              description: "Domains federated with this instance"
+            }
+          }
+        },
+        thumbnail: %Schema{
+          type: :string,
+          description: "Banner image for the website",
+          nullable: true
+        },
+        languages: %Schema{
+          type: :array,
+          items: %Schema{type: :string},
+          description: "Primary langauges of the website and its staff"
+        },
+        registrations: %Schema{type: :boolean, description: "Whether registrations are enabled"},
+        # Extra (not present in Mastodon):
+        max_toot_chars: %Schema{
+          type: :integer,
+          description: ": Posts character limit (CW/Subject included in the counter)"
+        },
+        poll_limits: %Schema{
+          type: :object,
+          description: "A map with poll limits for local polls",
+          properties: %{
+            max_options: %Schema{
+              type: :integer,
+              description: "Maximum number of options."
+            },
+            max_option_chars: %Schema{
+              type: :integer,
+              description: "Maximum number of characters per option."
+            },
+            min_expiration: %Schema{
+              type: :integer,
+              description: "Minimum expiration time (in seconds)."
+            },
+            max_expiration: %Schema{
+              type: :integer,
+              description: "Maximum expiration time (in seconds)."
+            }
+          }
+        },
+        upload_limit: %Schema{
+          type: :integer,
+          description: "File size limit of uploads (except for avatar, background, banner)"
+        },
+        avatar_upload_limit: %Schema{type: :integer, description: "The title of the website"},
+        background_upload_limit: %Schema{type: :integer, description: "The title of the website"},
+        banner_upload_limit: %Schema{type: :integer, description: "The title of the website"}
+      },
+      example: %{
+        "avatar_upload_limit" => 2_000_000,
+        "background_upload_limit" => 4_000_000,
+        "banner_upload_limit" => 4_000_000,
+        "description" => "A Pleroma instance, an alternative fediverse server",
+        "email" => "lain@lain.com",
+        "languages" => ["en"],
+        "max_toot_chars" => 5000,
+        "poll_limits" => %{
+          "max_expiration" => 31_536_000,
+          "max_option_chars" => 200,
+          "max_options" => 20,
+          "min_expiration" => 0
+        },
+        "registrations" => false,
+        "stats" => %{
+          "domain_count" => 2996,
+          "status_count" => 15_802,
+          "user_count" => 5
+        },
+        "thumbnail" => "https://lain.com/instance/thumbnail.jpeg",
+        "title" => "lain.com",
+        "upload_limit" => 16_000_000,
+        "uri" => "https://lain.com",
+        "urls" => %{
+          "streaming_api" => "wss://lain.com"
+        },
+        "version" => "2.7.2 (compatible; Pleroma 2.0.50-536-g25eec6d7-develop)"
+      }
+    }
+  end
+
+  defp array_of_domains do
+    %Schema{
+      type: :array,
+      items: %Schema{type: :string},
+      example: ["pleroma.site", "lain.com", "bikeshed.party"]
+    }
+  end
+end

lib/pleroma/web/api_spec/operations/list_operation.ex

@@ -0,0 +1,188 @@
+# Pleroma: A lightweight social networking server
+# Copyright © 2017-2020 Pleroma Authors <https://pleroma.social/>
+# SPDX-License-Identifier: AGPL-3.0-only
+
+defmodule Pleroma.Web.ApiSpec.ListOperation do
+  alias OpenApiSpex.Operation
+  alias OpenApiSpex.Schema
+  alias Pleroma.Web.ApiSpec.Schemas.Account
+  alias Pleroma.Web.ApiSpec.Schemas.ApiError
+  alias Pleroma.Web.ApiSpec.Schemas.FlakeID
+  alias Pleroma.Web.ApiSpec.Schemas.List
+
+  import Pleroma.Web.ApiSpec.Helpers
+
+  def open_api_operation(action) do
+    operation = String.to_existing_atom("#{action}_operation")
+    apply(__MODULE__, operation, [])
+  end
+
+  def index_operation do
+    %Operation{
+      tags: ["Lists"],
+      summary: "Show user's lists",
+      description: "Fetch all lists that the user owns",
+      security: [%{"oAuth" => ["read:lists"]}],
+      operationId: "ListController.index",
+      responses: %{
+        200 => Operation.response("Array of List", "application/json", array_of_lists())
+      }
+    }
+  end
+
+  def create_operation do
+    %Operation{
+      tags: ["Lists"],
+      summary: "Create  a list",
+      description: "Fetch the list with the given ID. Used for verifying the title of a list.",
+      operationId: "ListController.create",
+      requestBody: create_update_request(),
+      security: [%{"oAuth" => ["write:lists"]}],
+      responses: %{
+        200 => Operation.response("List", "application/json", List),
+        400 => Operation.response("Error", "application/json", ApiError),
+        404 => Operation.response("Error", "application/json", ApiError)
+      }
+    }
+  end
+
+  def show_operation do
+    %Operation{
+      tags: ["Lists"],
+      summary: "Show a single list",
+      description: "Fetch the list with the given ID. Used for verifying the title of a list.",
+      operationId: "ListController.show",
+      parameters: [id_param()],
+      security: [%{"oAuth" => ["read:lists"]}],
+      responses: %{
+        200 => Operation.response("List", "application/json", List),
+        404 => Operation.response("Error", "application/json", ApiError)
+      }
+    }
+  end
+
+  def update_operation do
+    %Operation{
+      tags: ["Lists"],
+      summary: "Update a list",
+      description: "Change the title of a list",
+      operationId: "ListController.update",
+      parameters: [id_param()],
+      requestBody: create_update_request(),
+      security: [%{"oAuth" => ["write:lists"]}],
+      responses: %{
+        200 => Operation.response("List", "application/json", List),
+        422 => Operation.response("Error", "application/json", ApiError)
+      }
+    }
+  end
+
+  def delete_operation do
+    %Operation{
+      tags: ["Lists"],
+      summary: "Delete a list",
+      operationId: "ListController.delete",
+      parameters: [id_param()],
+      security: [%{"oAuth" => ["write:lists"]}],
+      responses: %{
+        200 => Operation.response("Empty object", "application/json", %Schema{type: :object})
+      }
+    }
+  end
+
+  def list_accounts_operation do
+    %Operation{
+      tags: ["Lists"],
+      summary: "View accounts in list",
+      operationId: "ListController.list_accounts",
+      parameters: [id_param()],
+      security: [%{"oAuth" => ["read:lists"]}],
+      responses: %{
+        200 =>
+          Operation.response("Array of Account", "application/json", %Schema{
+            type: :array,
+            items: Account
+          })
+      }
+    }
+  end
+
+  def add_to_list_operation do
+    %Operation{
+      tags: ["Lists"],
+      summary: "Add accounts to list",
+      description: "Add accounts to the given list.",
+      operationId: "ListController.add_to_list",
+      parameters: [id_param()],
+      requestBody: add_remove_accounts_request(),
+      security: [%{"oAuth" => ["write:lists"]}],
+      responses: %{
+        200 => Operation.response("Empty object", "application/json", %Schema{type: :object})
+      }
+    }
+  end
+
+  def remove_from_list_operation do
+    %Operation{
+      tags: ["Lists"],
+      summary: "Remove accounts from list",
+      operationId: "ListController.remove_from_list",
+      parameters: [id_param()],
+      requestBody: add_remove_accounts_request(),
+      security: [%{"oAuth" => ["write:lists"]}],
+      responses: %{
+        200 => Operation.response("Empty object", "application/json", %Schema{type: :object})
+      }
+    }
+  end
+
+  defp array_of_lists do
+    %Schema{
+      title: "ArrayOfLists",
+      description: "Response schema for lists",
+      type: :array,
+      items: List,
+      example: [
+        %{"id" => "123", "title" => "my list"},
+        %{"id" => "1337", "title" => "another list"}
+      ]
+    }
+  end
+
+  defp id_param do
+    Operation.parameter(:id, :path, :string, "List ID",
+      example: "123",
+      required: true
+    )
+  end
+
+  defp create_update_request do
+    request_body(
+      "Parameters",
+      %Schema{
+        description: "POST body for creating or updating a List",
+        type: :object,
+        properties: %{
+          title: %Schema{type: :string, description: "List title"}
+        },
+        required: [:title]
+      },
+      required: true
+    )
+  end
+
+  defp add_remove_accounts_request do
+    request_body(
+      "Parameters",
+      %Schema{
+        description: "POST body for adding/removing accounts to/from a List",
+        type: :object,
+        properties: %{
+          account_ids: %Schema{type: :array, description: "Array of account IDs", items: FlakeID}
+        },
+        required: [:account_ids]
+      },
+      required: true
+    )
+  end
+end

lib/pleroma/web/api_spec/operations/marker_operation.ex

@@ -0,0 +1,140 @@
+# Pleroma: A lightweight social networking server
+# Copyright © 2017-2020 Pleroma Authors <https://pleroma.social/>
+# SPDX-License-Identifier: AGPL-3.0-only
+
+defmodule Pleroma.Web.ApiSpec.MarkerOperation do
+  alias OpenApiSpex.Operation
+  alias OpenApiSpex.Schema
+  alias Pleroma.Web.ApiSpec.Helpers
+
+  def open_api_operation(action) do
+    operation = String.to_existing_atom("#{action}_operation")
+    apply(__MODULE__, operation, [])
+  end
+
+  def index_operation do
+    %Operation{
+      tags: ["Markers"],
+      summary: "Get saved timeline position",
+      security: [%{"oAuth" => ["read:statuses"]}],
+      operationId: "MarkerController.index",
+      parameters: [
+        Operation.parameter(
+          :timeline,
+          :query,
+          %Schema{
+            type: :array,
+            items: %Schema{type: :string, enum: ["home", "notifications"]}
+          },
+          "Array of markers to fetch. If not provided, an empty object will be returned."
+        )
+      ],
+      responses: %{
+        200 => Operation.response("Marker", "application/json", response()),
+        403 => Operation.response("Error", "application/json", api_error())
+      }
+    }
+  end
+
+  def upsert_operation do
+    %Operation{
+      tags: ["Markers"],
+      summary: "Save position in timeline",
+      operationId: "MarkerController.upsert",
+      requestBody: Helpers.request_body("Parameters", upsert_request(), required: true),
+      security: [%{"oAuth" => ["follow", "write:blocks"]}],
+      responses: %{
+        200 => Operation.response("Marker", "application/json", response()),
+        403 => Operation.response("Error", "application/json", api_error())
+      }
+    }
+  end
+
+  defp marker do
+    %Schema{
+      title: "Marker",
+      description: "Schema for a marker",
+      type: :object,
+      properties: %{
+        last_read_id: %Schema{type: :string},
+        version: %Schema{type: :integer},
+        updated_at: %Schema{type: :string},
+        pleroma: %Schema{
+          type: :object,
+          properties: %{
+            unread_count: %Schema{type: :integer}
+          }
+        }
+      },
+      example: %{
+        "last_read_id" => "35098814",
+        "version" => 361,
+        "updated_at" => "2019-11-26T22:37:25.239Z",
+        "pleroma" => %{"unread_count" => 5}
+      }
+    }
+  end
+
+  defp response do
+    %Schema{
+      title: "MarkersResponse",
+      description: "Response schema for markers",
+      type: :object,
+      properties: %{
+        notifications: %Schema{allOf: [marker()], nullable: true},
+        home: %Schema{allOf: [marker()], nullable: true}
+      },
+      items: %Schema{type: :string},
+      example: %{
+        "notifications" => %{
+          "last_read_id" => "35098814",
+          "version" => 361,
+          "updated_at" => "2019-11-26T22:37:25.239Z",
+          "pleroma" => %{"unread_count" => 0}
+        },
+        "home" => %{
+          "last_read_id" => "103206604258487607",
+          "version" => 468,
+          "updated_at" => "2019-11-26T22:37:25.235Z",
+          "pleroma" => %{"unread_count" => 10}
+        }
+      }
+    }
+  end
+
+  defp upsert_request do
+    %Schema{
+      title: "MarkersUpsertRequest",
+      description: "Request schema for marker upsert",
+      type: :object,
+      properties: %{
+        notifications: %Schema{
+          type: :object,
+          properties: %{
+            last_read_id: %Schema{type: :string}
+          }
+        },
+        home: %Schema{
+          type: :object,
+          properties: %{
+            last_read_id: %Schema{type: :string}
+          }
+        }
+      },
+      example: %{
+        "home" => %{
+          "last_read_id" => "103194548672408537",
+          "version" => 462,
+          "updated_at" => "2019-11-24T19:39:39.337Z"
+        }
+      }
+    }
+  end
+
+  defp api_error do
+    %Schema{
+      type: :object,
+      properties: %{error: %Schema{type: :string}}
+    }
+  end
+end

lib/pleroma/web/api_spec/operations/notification_operation.ex

@@ -0,0 +1,211 @@
+# Pleroma: A lightweight social networking server
+# Copyright © 2017-2020 Pleroma Authors <https://pleroma.social/>
+# SPDX-License-Identifier: AGPL-3.0-only
+
+defmodule Pleroma.Web.ApiSpec.NotificationOperation do
+  alias OpenApiSpex.Operation
+  alias OpenApiSpex.Operation
+  alias OpenApiSpex.Schema
+  alias Pleroma.Web.ApiSpec.Schemas.Account
+  alias Pleroma.Web.ApiSpec.Schemas.ApiError
+  alias Pleroma.Web.ApiSpec.Schemas.BooleanLike
+  alias Pleroma.Web.ApiSpec.Schemas.Status
+  alias Pleroma.Web.ApiSpec.Schemas.VisibilityScope
+
+  import Pleroma.Web.ApiSpec.Helpers
+
+  def open_api_operation(action) do
+    operation = String.to_existing_atom("#{action}_operation")
+    apply(__MODULE__, operation, [])
+  end
+
+  def index_operation do
+    %Operation{
+      tags: ["Notifications"],
+      summary: "Get all notifications",
+      description:
+        "Notifications concerning the user. This API returns Link headers containing links to the next/previous page. However, the links can also be constructed dynamically using query params and `id` values.",
+      operationId: "NotificationController.index",
+      security: [%{"oAuth" => ["read:notifications"]}],
+      parameters:
+        [
+          Operation.parameter(
+            :exclude_types,
+            :query,
+            %Schema{type: :array, items: notification_type()},
+            "Array of types to exclude"
+          ),
+          Operation.parameter(
+            :account_id,
+            :query,
+            %Schema{type: :string},
+            "Return only notifications received from this account"
+          ),
+          Operation.parameter(
+            :exclude_visibilities,
+            :query,
+            %Schema{type: :array, items: VisibilityScope},
+            "Exclude the notifications for activities with the given visibilities"
+          ),
+          Operation.parameter(
+            :include_types,
+            :query,
+            %Schema{type: :array, items: notification_type()},
+            "Include the notifications for activities with the given types"
+          ),
+          Operation.parameter(
+            :with_muted,
+            :query,
+            BooleanLike,
+            "Include the notifications from muted users"
+          )
+        ] ++ pagination_params(),
+      responses: %{
+        200 =>
+          Operation.response("Array of notifications", "application/json", %Schema{
+            type: :array,
+            items: notification()
+          }),
+        404 => Operation.response("Error", "application/json", ApiError)
+      }
+    }
+  end
+
+  def show_operation do
+    %Operation{
+      tags: ["Notifications"],
+      summary: "Get a single notification",
+      description: "View information about a notification with a given ID.",
+      operationId: "NotificationController.show",
+      security: [%{"oAuth" => ["read:notifications"]}],
+      parameters: [id_param()],
+      responses: %{
+        200 => Operation.response("Notification", "application/json", notification())
+      }
+    }
+  end
+
+  def clear_operation do
+    %Operation{
+      tags: ["Notifications"],
+      summary: "Dismiss all notifications",
+      description: "Clear all notifications from the server.",
+      operationId: "NotificationController.clear",
+      security: [%{"oAuth" => ["write:notifications"]}],
+      responses: %{200 => empty_object_response()}
+    }
+  end
+
+  def dismiss_operation do
+    %Operation{
+      tags: ["Notifications"],
+      summary: "Dismiss a single notification",
+      description: "Clear a single notification from the server.",
+      operationId: "NotificationController.dismiss",
+      parameters: [id_param()],
+      security: [%{"oAuth" => ["write:notifications"]}],
+      responses: %{200 => empty_object_response()}
+    }
+  end
+
+  def dismiss_via_body_operation do
+    %Operation{
+      tags: ["Notifications"],
+      summary: "Dismiss a single notification",
+      deprecated: true,
+      description: "Clear a single notification from the server.",
+      operationId: "NotificationController.dismiss_via_body",
+      requestBody:
+        request_body(
+          "Parameters",
+          %Schema{type: :object, properties: %{id: %Schema{type: :string}}},
+          required: true
+        ),
+      security: [%{"oAuth" => ["write:notifications"]}],
+      responses: %{200 => empty_object_response()}
+    }
+  end
+
+  def destroy_multiple_operation do
+    %Operation{
+      tags: ["Notifications"],
+      summary: "Dismiss multiple notifications",
+      operationId: "NotificationController.destroy_multiple",
+      security: [%{"oAuth" => ["write:notifications"]}],
+      parameters: [
+        Operation.parameter(
+          :ids,
+          :query,
+          %Schema{type: :array, items: %Schema{type: :string}},
+          "Array of notification IDs to dismiss",
+          required: true
+        )
+      ],
+      responses: %{200 => empty_object_response()}
+    }
+  end
+
+  defp notification do
+    %Schema{
+      title: "Notification",
+      description: "Response schema for a notification",
+      type: :object,
+      properties: %{
+        id: %Schema{type: :string},
+        type: notification_type(),
+        created_at: %Schema{type: :string, format: :"date-time"},
+        account: %Schema{
+          allOf: [Account],
+          description: "The account that performed the action that generated the notification."
+        },
+        status: %Schema{
+          allOf: [Status],
+          description:
+            "Status that was the object of the notification, e.g. in mentions, reblogs, favourites, or polls.",
+          nullable: true
+        }
+      },
+      example: %{
+        "id" => "34975861",
+        "type" => "mention",
+        "created_at" => "2019-11-23T07:49:02.064Z",
+        "account" => Account.schema().example,
+        "status" => Status.schema().example
+      }
+    }
+  end
+
+  defp notification_type do
+    %Schema{
+      type: :string,
+      enum: [
+        "follow",
+        "favourite",
+        "reblog",
+        "mention",
+        "poll",
+        "pleroma:emoji_reaction",
+        "move",
+        "follow_request"
+      ],
+      description: """
+      The type of event that resulted in the notification.
+
+      - `follow` - Someone followed you
+      - `mention` - Someone mentioned you in their status
+      - `reblog` - Someone boosted one of your statuses
+      - `favourite` - Someone favourited one of your statuses
+      - `poll` - A poll you have voted in or created has ended
+      - `move` - Someone moved their account
+      - `pleroma:emoji_reaction` - Someone reacted with emoji to your status
+      """
+    }
+  end
+
+  defp id_param do
+    Operation.parameter(:id, :path, :string, "Notification ID",
+      example: "123",
+      required: true
+    )
+  end
+end

lib/pleroma/web/api_spec/operations/poll_operation.ex

@@ -0,0 +1,76 @@
+# Pleroma: A lightweight social networking server
+# Copyright © 2017-2020 Pleroma Authors <https://pleroma.social/>
+# SPDX-License-Identifier: AGPL-3.0-only
+
+defmodule Pleroma.Web.ApiSpec.PollOperation do
+  alias OpenApiSpex.Operation
+  alias OpenApiSpex.Schema
+  alias Pleroma.Web.ApiSpec.Schemas.ApiError
+  alias Pleroma.Web.ApiSpec.Schemas.FlakeID
+  alias Pleroma.Web.ApiSpec.Schemas.Poll
+
+  import Pleroma.Web.ApiSpec.Helpers
+
+  def open_api_operation(action) do
+    operation = String.to_existing_atom("#{action}_operation")
+    apply(__MODULE__, operation, [])
+  end
+
+  def show_operation do
+    %Operation{
+      tags: ["Polls"],
+      summary: "View a poll",
+      security: [%{"oAuth" => ["read:statuses"]}],
+      parameters: [id_param()],
+      operationId: "PollController.show",
+      responses: %{
+        200 => Operation.response("Poll", "application/json", Poll),
+        404 => Operation.response("Error", "application/json", ApiError)
+      }
+    }
+  end
+
+  def vote_operation do
+    %Operation{
+      tags: ["Polls"],
+      summary: "Vote on a poll",
+      parameters: [id_param()],
+      operationId: "PollController.vote",
+      requestBody: vote_request(),
+      security: [%{"oAuth" => ["write:statuses"]}],
+      responses: %{
+        200 => Operation.response("Poll", "application/json", Poll),
+        422 => Operation.response("Error", "application/json", ApiError),
+        404 => Operation.response("Error", "application/json", ApiError)
+      }
+    }
+  end
+
+  defp id_param do
+    Operation.parameter(:id, :path, FlakeID, "Poll ID",
+      example: "123",
+      required: true
+    )
+  end
+
+  defp vote_request do
+    request_body(
+      "Parameters",
+      %Schema{
+        type: :object,
+        properties: %{
+          choices: %Schema{
+            type: :array,
+            items: %Schema{type: :integer},
+            description: "Array of own votes containing index for each option (starting from 0)"
+          }
+        },
+        required: [:choices]
+      },
+      required: true,
+      example: %{
+        "choices" => [0, 1, 2]
+      }
+    )
+  end
+end

lib/pleroma/web/api_spec/operations/report_operation.ex

@@ -0,0 +1,78 @@
+# Pleroma: A lightweight social networking server
+# Copyright © 2017-2020 Pleroma Authors <https://pleroma.social/>
+# SPDX-License-Identifier: AGPL-3.0-only
+
+defmodule Pleroma.Web.ApiSpec.ReportOperation do
+  alias OpenApiSpex.Operation
+  alias OpenApiSpex.Schema
+  alias Pleroma.Web.ApiSpec.Helpers
+  alias Pleroma.Web.ApiSpec.Schemas.ApiError
+
+  def open_api_operation(action) do
+    operation = String.to_existing_atom("#{action}_operation")
+    apply(__MODULE__, operation, [])
+  end
+
+  def create_operation do
+    %Operation{
+      tags: ["reports"],
+      summary: "File a report",
+      description: "Report problematic users to your moderators",
+      operationId: "ReportController.create",
+      security: [%{"oAuth" => ["follow", "write:reports"]}],
+      requestBody: Helpers.request_body("Parameters", create_request(), required: true),
+      responses: %{
+        200 => Operation.response("Report", "application/json", create_response()),
+        400 => Operation.response("Report", "application/json", ApiError)
+      }
+    }
+  end
+
+  defp create_request do
+    %Schema{
+      title: "ReportCreateRequest",
+      description: "POST body for creating a report",
+      type: :object,
+      properties: %{
+        account_id: %Schema{type: :string, description: "ID of the account to report"},
+        status_ids: %Schema{
+          type: :array,
+          items: %Schema{type: :string},
+          description: "Array of Statuses to attach to the report, for context"
+        },
+        comment: %Schema{
+          type: :string,
+          description: "Reason for the report"
+        },
+        forward: %Schema{
+          type: :boolean,
+          default: false,
+          description:
+            "If the account is remote, should the report be forwarded to the remote admin?"
+        }
+      },
+      required: [:account_id],
+      example: %{
+        "account_id" => "123",
+        "status_ids" => ["1337"],
+        "comment" => "bad status!",
+        "forward" => "false"
+      }
+    }
+  end
+
+  defp create_response do
+    %Schema{
+      title: "ReportResponse",
+      type: :object,
+      properties: %{
+        id: %Schema{type: :string, description: "Report ID"},
+        action_taken: %Schema{type: :boolean, description: "Is action taken?"}
+      },
+      example: %{
+        "id" => "123",
+        "action_taken" => false
+      }
+    }
+  end
+end

lib/pleroma/web/api_spec/operations/scheduled_activity_operation.ex

@@ -0,0 +1,96 @@
+# Pleroma: A lightweight social networking server
+# Copyright © 2017-2020 Pleroma Authors <https://pleroma.social/>
+# SPDX-License-Identifier: AGPL-3.0-only
+
+defmodule Pleroma.Web.ApiSpec.ScheduledActivityOperation do
+  alias OpenApiSpex.Operation
+  alias OpenApiSpex.Schema
+  alias Pleroma.Web.ApiSpec.Schemas.ApiError
+  alias Pleroma.Web.ApiSpec.Schemas.FlakeID
+  alias Pleroma.Web.ApiSpec.Schemas.ScheduledStatus
+
+  import Pleroma.Web.ApiSpec.Helpers
+
+  def open_api_operation(action) do
+    operation = String.to_existing_atom("#{action}_operation")
+    apply(__MODULE__, operation, [])
+  end
+
+  def index_operation do
+    %Operation{
+      tags: ["Scheduled Statuses"],
+      summary: "View scheduled statuses",
+      security: [%{"oAuth" => ["read:statuses"]}],
+      parameters: pagination_params(),
+      operationId: "ScheduledActivity.index",
+      responses: %{
+        200 =>
+          Operation.response("Array of ScheduledStatus", "application/json", %Schema{
+            type: :array,
+            items: ScheduledStatus
+          })
+      }
+    }
+  end
+
+  def show_operation do
+    %Operation{
+      tags: ["Scheduled Statuses"],
+      summary: "View a single scheduled status",
+      security: [%{"oAuth" => ["read:statuses"]}],
+      parameters: [id_param()],
+      operationId: "ScheduledActivity.show",
+      responses: %{
+        200 => Operation.response("Scheduled Status", "application/json", ScheduledStatus),
+        404 => Operation.response("Error", "application/json", ApiError)
+      }
+    }
+  end
+
+  def update_operation do
+    %Operation{
+      tags: ["Scheduled Statuses"],
+      summary: "Schedule a status",
+      operationId: "ScheduledActivity.update",
+      security: [%{"oAuth" => ["write:statuses"]}],
+      parameters: [id_param()],
+      requestBody:
+        request_body("Parameters", %Schema{
+          type: :object,
+          properties: %{
+            scheduled_at: %Schema{
+              type: :string,
+              format: :"date-time",
+              description:
+                "ISO 8601 Datetime at which the status will be published. Must be at least 5 minutes into the future."
+            }
+          }
+        }),
+      responses: %{
+        200 => Operation.response("Scheduled Status", "application/json", ScheduledStatus),
+        404 => Operation.response("Error", "application/json", ApiError)
+      }
+    }
+  end
+
+  def delete_operation do
+    %Operation{
+      tags: ["Scheduled Statuses"],
+      summary: "Cancel a scheduled status",
+      security: [%{"oAuth" => ["write:statuses"]}],
+      parameters: [id_param()],
+      operationId: "ScheduledActivity.delete",
+      responses: %{
+        200 => Operation.response("Empty object", "application/json", %Schema{type: :object}),
+        404 => Operation.response("Error", "application/json", ApiError)
+      }
+    }
+  end
+
+  defp id_param do
+    Operation.parameter(:id, :path, FlakeID, "Poll ID",
+      example: "123",
+      required: true
+    )
+  end
+end

lib/pleroma/web/api_spec/operations/search_operation.ex

@@ -0,0 +1,207 @@
+# Pleroma: A lightweight social networking server
+# Copyright © 2017-2020 Pleroma Authors <https://pleroma.social/>
+# SPDX-License-Identifier: AGPL-3.0-only
+
+defmodule Pleroma.Web.ApiSpec.SearchOperation do
+  alias OpenApiSpex.Operation
+  alias OpenApiSpex.Schema
+  alias Pleroma.Web.ApiSpec.AccountOperation
+  alias Pleroma.Web.ApiSpec.Schemas.Account
+  alias Pleroma.Web.ApiSpec.Schemas.BooleanLike
+  alias Pleroma.Web.ApiSpec.Schemas.FlakeID
+  alias Pleroma.Web.ApiSpec.Schemas.Status
+  alias Pleroma.Web.ApiSpec.Schemas.Tag
+
+  import Pleroma.Web.ApiSpec.Helpers
+
+  def open_api_operation(action) do
+    operation = String.to_existing_atom("#{action}_operation")
+    apply(__MODULE__, operation, [])
+  end
+
+  def account_search_operation do
+    %Operation{
+      tags: ["Search"],
+      summary: "Search for matching accounts by username or display name",
+      operationId: "SearchController.account_search",
+      parameters: [
+        Operation.parameter(:q, :query, %Schema{type: :string}, "What to search for",
+          required: true
+        ),
+        Operation.parameter(
+          :limit,
+          :query,
+          %Schema{type: :integer, default: 40},
+          "Maximum number of results"
+        ),
+        Operation.parameter(
+          :resolve,
+          :query,
+          %Schema{allOf: [BooleanLike], default: false},
+          "Attempt WebFinger lookup. Use this when `q` is an exact address."
+        ),
+        Operation.parameter(
+          :following,
+          :query,
+          %Schema{allOf: [BooleanLike], default: false},
+          "Only include accounts that the user is following"
+        )
+      ],
+      responses: %{
+        200 =>
+          Operation.response(
+            "Array of Account",
+            "application/json",
+            AccountOperation.array_of_accounts()
+          )
+      }
+    }
+  end
+
+  def search_operation do
+    %Operation{
+      tags: ["Search"],
+      summary: "Search results",
+      security: [%{"oAuth" => ["read:search"]}],
+      operationId: "SearchController.search",
+      deprecated: true,
+      parameters: [
+        Operation.parameter(
+          :account_id,
+          :query,
+          FlakeID,
+          "If provided, statuses returned will be authored only by this account"
+        ),
+        Operation.parameter(
+          :type,
+          :query,
+          %Schema{type: :string, enum: ["accounts", "hashtags", "statuses"]},
+          "Search type"
+        ),
+        Operation.parameter(:q, :query, %Schema{type: :string}, "The search query", required: true),
+        Operation.parameter(
+          :resolve,
+          :query,
+          %Schema{allOf: [BooleanLike], default: false},
+          "Attempt WebFinger lookup"
+        ),
+        Operation.parameter(
+          :following,
+          :query,
+          %Schema{allOf: [BooleanLike], default: false},
+          "Only include accounts that the user is following"
+        ),
+        Operation.parameter(
+          :offset,
+          :query,
+          %Schema{type: :integer},
+          "Offset"
+        )
+        | pagination_params()
+      ],
+      responses: %{
+        200 => Operation.response("Results", "application/json", results())
+      }
+    }
+  end
+
+  def search2_operation do
+    %Operation{
+      tags: ["Search"],
+      summary: "Search results",
+      security: [%{"oAuth" => ["read:search"]}],
+      operationId: "SearchController.search2",
+      parameters: [
+        Operation.parameter(
+          :account_id,
+          :query,
+          FlakeID,
+          "If provided, statuses returned will be authored only by this account"
+        ),
+        Operation.parameter(
+          :type,
+          :query,
+          %Schema{type: :string, enum: ["accounts", "hashtags", "statuses"]},
+          "Search type"
+        ),
+        Operation.parameter(:q, :query, %Schema{type: :string}, "What to search for",
+          required: true
+        ),
+        Operation.parameter(
+          :resolve,
+          :query,
+          %Schema{allOf: [BooleanLike], default: false},
+          "Attempt WebFinger lookup"
+        ),
+        Operation.parameter(
+          :following,
+          :query,
+          %Schema{allOf: [BooleanLike], default: false},
+          "Only include accounts that the user is following"
+        )
+        | pagination_params()
+      ],
+      responses: %{
+        200 => Operation.response("Results", "application/json", results2())
+      }
+    }
+  end
+
+  defp results2 do
+    %Schema{
+      title: "SearchResults",
+      type: :object,
+      properties: %{
+        accounts: %Schema{
+          type: :array,
+          items: Account,
+          description: "Accounts which match the given query"
+        },
+        statuses: %Schema{
+          type: :array,
+          items: Status,
+          description: "Statuses which match the given query"
+        },
+        hashtags: %Schema{
+          type: :array,
+          items: Tag,
+          description: "Hashtags which match the given query"
+        }
+      },
+      example: %{
+        "accounts" => [Account.schema().example],
+        "statuses" => [Status.schema().example],
+        "hashtags" => [Tag.schema().example]
+      }
+    }
+  end
+
+  defp results do
+    %Schema{
+      title: "SearchResults",
+      type: :object,
+      properties: %{
+        accounts: %Schema{
+          type: :array,
+          items: Account,
+          description: "Accounts which match the given query"
+        },
+        statuses: %Schema{
+          type: :array,
+          items: Status,
+          description: "Statuses which match the given query"
+        },
+        hashtags: %Schema{
+          type: :array,
+          items: %Schema{type: :string},
+          description: "Hashtags which match the given query"
+        }
+      },
+      example: %{
+        "accounts" => [Account.schema().example],
+        "statuses" => [Status.schema().example],
+        "hashtags" => ["cofe"]
+      }
+    }
+  end
+end

lib/pleroma/web/api_spec/operations/subscription_operation.ex

@@ -0,0 +1,188 @@
+# Pleroma: A lightweight social networking server
+# Copyright © 2017-2020 Pleroma Authors <https://pleroma.social/>
+# SPDX-License-Identifier: AGPL-3.0-only
+
+defmodule Pleroma.Web.ApiSpec.SubscriptionOperation do
+  alias OpenApiSpex.Operation
+  alias OpenApiSpex.Schema
+  alias Pleroma.Web.ApiSpec.Helpers
+  alias Pleroma.Web.ApiSpec.Schemas.ApiError
+  alias Pleroma.Web.ApiSpec.Schemas.PushSubscription
+
+  def open_api_operation(action) do
+    operation = String.to_existing_atom("#{action}_operation")
+    apply(__MODULE__, operation, [])
+  end
+
+  def create_operation do
+    %Operation{
+      tags: ["Push Subscriptions"],
+      summary: "Subscribe to push notifications",
+      description:
+        "Add a Web Push API subscription to receive notifications. Each access token can have one push subscription. If you create a new subscription, the old subscription is deleted.",
+      operationId: "SubscriptionController.create",
+      security: [%{"oAuth" => ["push"]}],
+      requestBody: Helpers.request_body("Parameters", create_request(), required: true),
+      responses: %{
+        200 => Operation.response("Push Subscription", "application/json", PushSubscription),
+        400 => Operation.response("Error", "application/json", ApiError),
+        403 => Operation.response("Error", "application/json", ApiError)
+      }
+    }
+  end
+
+  def show_operation do
+    %Operation{
+      tags: ["Push Subscriptions"],
+      summary: "Get current subscription",
+      description: "View the PushSubscription currently associated with this access token.",
+      operationId: "SubscriptionController.show",
+      security: [%{"oAuth" => ["push"]}],
+      responses: %{
+        200 => Operation.response("Push Subscription", "application/json", PushSubscription),
+        403 => Operation.response("Error", "application/json", ApiError),
+        404 => Operation.response("Error", "application/json", ApiError)
+      }
+    }
+  end
+
+  def update_operation do
+    %Operation{
+      tags: ["Push Subscriptions"],
+      summary: "Change types of notifications",
+      description:
+        "Updates the current push subscription. Only the data part can be updated. To change fundamentals, a new subscription must be created instead.",
+      operationId: "SubscriptionController.update",
+      security: [%{"oAuth" => ["push"]}],
+      requestBody: Helpers.request_body("Parameters", update_request(), required: true),
+      responses: %{
+        200 => Operation.response("Push Subscription", "application/json", PushSubscription),
+        403 => Operation.response("Error", "application/json", ApiError)
+      }
+    }
+  end
+
+  def delete_operation do
+    %Operation{
+      tags: ["Push Subscriptions"],
+      summary: "Remove current subscription",
+      description: "Removes the current Web Push API subscription.",
+      operationId: "SubscriptionController.delete",
+      security: [%{"oAuth" => ["push"]}],
+      responses: %{
+        200 => Operation.response("Empty object", "application/json", %Schema{type: :object}),
+        403 => Operation.response("Error", "application/json", ApiError),
+        404 => Operation.response("Error", "application/json", ApiError)
+      }
+    }
+  end
+
+  defp create_request do
+    %Schema{
+      title: "SubscriptionCreateRequest",
+      description: "POST body for creating a push subscription",
+      type: :object,
+      properties: %{
+        subscription: %Schema{
+          type: :object,
+          properties: %{
+            endpoint: %Schema{
+              type: :string,
+              description: "Endpoint URL that is called when a notification event occurs."
+            },
+            keys: %Schema{
+              type: :object,
+              properties: %{
+                p256dh: %Schema{
+                  type: :string,
+                  description:
+                    "User agent public key. Base64 encoded string of public key of ECDH key using `prime256v1` curve."
+                },
+                auth: %Schema{
+                  type: :string,
+                  description: "Auth secret. Base64 encoded string of 16 bytes of random data."
+                }
+              },
+              required: [:p256dh, :auth]
+            }
+          },
+          required: [:endpoint, :keys]
+        },
+        data: %Schema{
+          type: :object,
+          properties: %{
+            alerts: %Schema{
+              type: :object,
+              properties: %{
+                follow: %Schema{type: :boolean, description: "Receive follow notifications?"},
+                favourite: %Schema{
+                  type: :boolean,
+                  description: "Receive favourite notifications?"
+                },
+                reblog: %Schema{type: :boolean, description: "Receive reblog notifications?"},
+                mention: %Schema{type: :boolean, description: "Receive mention notifications?"},
+                poll: %Schema{type: :boolean, description: "Receive poll notifications?"}
+              }
+            }
+          }
+        }
+      },
+      required: [:subscription],
+      example: %{
+        "subscription" => %{
+          "endpoint" => "https://example.com/example/1234",
+          "keys" => %{
+            "auth" => "8eDyX_uCN0XRhSbY5hs7Hg==",
+            "p256dh" =>
+              "BCIWgsnyXDv1VkhqL2P7YRBvdeuDnlwAPT2guNhdIoW3IP7GmHh1SMKPLxRf7x8vJy6ZFK3ol2ohgn_-0yP7QQA="
+          }
+        },
+        "data" => %{
+          "alerts" => %{
+            "follow" => true,
+            "mention" => true,
+            "poll" => false
+          }
+        }
+      }
+    }
+  end
+
+  defp update_request do
+    %Schema{
+      title: "SubscriptionUpdateRequest",
+      type: :object,
+      properties: %{
+        data: %Schema{
+          type: :object,
+          properties: %{
+            alerts: %Schema{
+              type: :object,
+              properties: %{
+                follow: %Schema{type: :boolean, description: "Receive follow notifications?"},
+                favourite: %Schema{
+                  type: :boolean,
+                  description: "Receive favourite notifications?"
+                },
+                reblog: %Schema{type: :boolean, description: "Receive reblog notifications?"},
+                mention: %Schema{type: :boolean, description: "Receive mention notifications?"},
+                poll: %Schema{type: :boolean, description: "Receive poll notifications?"}
+              }
+            }
+          }
+        }
+      },
+      example: %{
+        "data" => %{
+          "alerts" => %{
+            "follow" => true,
+            "favourite" => true,
+            "reblog" => true,
+            "mention" => true,
+            "poll" => true
+          }
+        }
+      }
+    }
+  end
+end

lib/pleroma/web/api_spec/render_error.ex

@@ -17,6 +17,9 @@ def init(opts), do: opts
   def call(conn, errors) do
     errors =
       Enum.map(errors, fn
+        %{name: nil, reason: :invalid_enum} = err ->
+          %OpenApiSpex.Cast.Error{err | name: err.value}
+
         %{name: nil} = err ->
           %OpenApiSpex.Cast.Error{err | name: List.last(err.path)}
 

lib/pleroma/web/api_spec/schemas/attachment.ex

@@ -0,0 +1,68 @@
+# Pleroma: A lightweight social networking server
+# Copyright © 2017-2020 Pleroma Authors <https://pleroma.social/>
+# SPDX-License-Identifier: AGPL-3.0-only
+
+defmodule Pleroma.Web.ApiSpec.Schemas.Attachment do
+  alias OpenApiSpex.Schema
+
+  require OpenApiSpex
+
+  OpenApiSpex.schema(%{
+    title: "Attachment",
+    description: "Represents a file or media attachment that can be added to a status.",
+    type: :object,
+    requried: [:id, :url, :preview_url],
+    properties: %{
+      id: %Schema{type: :string},
+      url: %Schema{
+        type: :string,
+        format: :uri,
+        description: "The location of the original full-size attachment"
+      },
+      remote_url: %Schema{
+        type: :string,
+        format: :uri,
+        description:
+          "The location of the full-size original attachment on the remote website. String (URL), or null if the attachment is local",
+        nullable: true
+      },
+      preview_url: %Schema{
+        type: :string,
+        format: :uri,
+        description: "The location of a scaled-down preview of the attachment"
+      },
+      text_url: %Schema{
+        type: :string,
+        format: :uri,
+        description: "A shorter URL for the attachment"
+      },
+      description: %Schema{
+        type: :string,
+        nullable: true,
+        description:
+          "Alternate text that describes what is in the media attachment, to be used for the visually impaired or when media attachments do not load"
+      },
+      type: %Schema{
+        type: :string,
+        enum: ["image", "video", "audio", "unknown"],
+        description: "The type of the attachment"
+      },
+      pleroma: %Schema{
+        type: :object,
+        properties: %{
+          mime_type: %Schema{type: :string, description: "mime type of the attachment"}
+        }
+      }
+    },
+    example: %{
+      id: "1638338801",
+      type: "image",
+      url: "someurl",
+      remote_url: "someurl",
+      preview_url: "someurl",
+      text_url: "someurl",
+      description: nil,
+      pleroma: %{mime_type: "image/png"}
+    }
+  })
+end

lib/pleroma/web/api_spec/schemas/conversation.ex

@@ -0,0 +1,41 @@
+# Pleroma: A lightweight social networking server
+# Copyright © 2017-2020 Pleroma Authors <https://pleroma.social/>
+# SPDX-License-Identifier: AGPL-3.0-only
+
+defmodule Pleroma.Web.ApiSpec.Schemas.Conversation do
+  alias OpenApiSpex.Schema
+  alias Pleroma.Web.ApiSpec.Schemas.Account
+  alias Pleroma.Web.ApiSpec.Schemas.Status
+
+  require OpenApiSpex
+
+  OpenApiSpex.schema(%{
+    title: "Conversation",
+    description: "Represents a conversation with \"direct message\" visibility.",
+    type: :object,
+    required: [:id, :accounts, :unread],
+    properties: %{
+      id: %Schema{type: :string},
+      accounts: %Schema{
+        type: :array,
+        items: Account,
+        description: "Participants in the conversation"
+      },
+      unread: %Schema{
+        type: :boolean,
+        description: "Is the conversation currently marked as unread?"
+      },
+      # last_status: Status
+      last_status: %Schema{
+        allOf: [Status],
+        description: "The last status in the conversation, to be used for optional display"
+      }
+    },
+    example: %{
+      "id" => "418450",
+      "unread" => true,
+      "accounts" => [Account.schema().example],
+      "last_status" => Status.schema().example
+    }
+  })
+end

lib/pleroma/web/api_spec/schemas/list.ex

@@ -0,0 +1,23 @@
+# Pleroma: A lightweight social networking server
+# Copyright © 2017-2020 Pleroma Authors <https://pleroma.social/>
+# SPDX-License-Identifier: AGPL-3.0-only
+
+defmodule Pleroma.Web.ApiSpec.Schemas.List do
+  alias OpenApiSpex.Schema
+
+  require OpenApiSpex
+
+  OpenApiSpex.schema(%{
+    title: "List",
+    description: "Represents a list of users",
+    type: :object,
+    properties: %{
+      id: %Schema{type: :string, description: "The internal database ID of the list"},
+      title: %Schema{type: :string, description: "The user-defined title of the list"}
+    },
+    example: %{
+      "id" => "12249",
+      "title" => "Friends"
+    }
+  })
+end

lib/pleroma/web/api_spec/schemas/poll.ex

@@ -11,26 +11,72 @@ defmodule Pleroma.Web.ApiSpec.Schemas.Poll do
 
   OpenApiSpex.schema(%{
     title: "Poll",
-    description: "Response schema for account custom fields",
+    description: "Represents a poll attached to a status",
     type: :object,
     properties: %{
       id: FlakeID,
-      expires_at: %Schema{type: :string, format: "date-time"},
+      expires_at: %Schema{
-      expired: %Schema{type: :boolean},
+        type: :string,
-      multiple: %Schema{type: :boolean},
+        format: :"date-time",
-      votes_count: %Schema{type: :integer},
+        nullable: true,
-      voted: %Schema{type: :boolean},
+        description: "When the poll ends"
-      emojis: %Schema{type: :array, items: Emoji},
+      },
+      expired: %Schema{type: :boolean, description: "Is the poll currently expired?"},
+      multiple: %Schema{
+        type: :boolean,
+        description: "Does the poll allow multiple-choice answers?"
+      },
+      votes_count: %Schema{
+        type: :integer,
+        nullable: true,
+        description: "How many votes have been received. Number, or null if `multiple` is false."
+      },
+      voted: %Schema{
+        type: :boolean,
+        nullable: true,
+        description:
+          "When called with a user token, has the authorized user voted? Boolean, or null if no current user."
+      },
+      emojis: %Schema{
+        type: :array,
+        items: Emoji,
+        description: "Custom emoji to be used for rendering poll options."
+      },
       options: %Schema{
         type: :array,
         items: %Schema{
+          title: "PollOption",
           type: :object,
           properties: %{
             title: %Schema{type: :string},
             votes_count: %Schema{type: :integer}
           }
-        }
+        },
+        description: "Possible answers for the poll."
       }
+    },
+    example: %{
+      id: "34830",
+      expires_at: "2019-12-05T04:05:08.302Z",
+      expired: true,
+      multiple: false,
+      votes_count: 10,
+      voters_count: nil,
+      voted: true,
+      own_votes: [
+        1
+      ],
+      options: [
+        %{
+          title: "accept",
+          votes_count: 6
+        },
+        %{
+          title: "deny",
+          votes_count: 4
+        }
+      ],
+      emojis: []
     }
   })
 end

lib/pleroma/web/api_spec/schemas/push_subscription.ex

@@ -0,0 +1,66 @@
+# Pleroma: A lightweight social networking server
+# Copyright © 2017-2020 Pleroma Authors <https://pleroma.social/>
+# SPDX-License-Identifier: AGPL-3.0-only
+
+defmodule Pleroma.Web.ApiSpec.Schemas.PushSubscription do
+  alias OpenApiSpex.Schema
+
+  require OpenApiSpex
+
+  OpenApiSpex.schema(%{
+    title: "PushSubscription",
+    description: "Response schema for a push subscription",
+    type: :object,
+    properties: %{
+      id: %Schema{
+        anyOf: [%Schema{type: :string}, %Schema{type: :integer}],
+        description: "The id of the push subscription in the database."
+      },
+      endpoint: %Schema{type: :string, description: "Where push alerts will be sent to."},
+      server_key: %Schema{type: :string, description: "The streaming server's VAPID key."},
+      alerts: %Schema{
+        type: :object,
+        description: "Which alerts should be delivered to the endpoint.",
+        properties: %{
+          follow: %Schema{
+            type: :boolean,
+            description: "Receive a push notification when someone has followed you?"
+          },
+          favourite: %Schema{
+            type: :boolean,
+            description:
+              "Receive a push notification when a status you created has been favourited by someone else?"
+          },
+          reblog: %Schema{
+            type: :boolean,
+            description:
+              "Receive a push notification when a status you created has been boosted by someone else?"
+          },
+          mention: %Schema{
+            type: :boolean,
+            description:
+              "Receive a push notification when someone else has mentioned you in a status?"
+          },
+          poll: %Schema{
+            type: :boolean,
+            description:
+              "Receive a push notification when a poll you voted in or created has ended? "
+          }
+        }
+      }
+    },
+    example: %{
+      "id" => "328_183",
+      "endpoint" => "https://yourdomain.example/listener",
+      "alerts" => %{
+        "follow" => true,
+        "favourite" => true,
+        "reblog" => true,
+        "mention" => true,
+        "poll" => true
+      },
+      "server_key" =>
+        "BCk-QqERU0q-CfYZjcuB6lnyyOYfJ2AifKqfeGIm7Z-HiTU5T9eTG5GxVA0_OH5mMlI4UkkDTpaZwozy0TzdZ2M="
+    }
+  })
+end

lib/pleroma/web/api_spec/schemas/scheduled_status.ex

@@ -0,0 +1,54 @@
+# Pleroma: A lightweight social networking server
+# Copyright © 2017-2020 Pleroma Authors <https://pleroma.social/>
+# SPDX-License-Identifier: AGPL-3.0-only
+
+defmodule Pleroma.Web.ApiSpec.Schemas.ScheduledStatus do
+  alias OpenApiSpex.Schema
+  alias Pleroma.Web.ApiSpec.Schemas.Attachment
+  alias Pleroma.Web.ApiSpec.Schemas.Poll
+  alias Pleroma.Web.ApiSpec.Schemas.VisibilityScope
+
+  require OpenApiSpex
+
+  OpenApiSpex.schema(%{
+    title: "ScheduledStatus",
+    description: "Represents a status that will be published at a future scheduled date.",
+    type: :object,
+    required: [:id, :scheduled_at, :params],
+    properties: %{
+      id: %Schema{type: :string},
+      scheduled_at: %Schema{type: :string, format: :"date-time"},
+      media_attachments: %Schema{type: :array, items: Attachment},
+      params: %Schema{
+        type: :object,
+        required: [:text, :visibility],
+        properties: %{
+          text: %Schema{type: :string, nullable: true},
+          media_ids: %Schema{type: :array, nullable: true, items: %Schema{type: :string}},
+          sensitive: %Schema{type: :boolean, nullable: true},
+          spoiler_text: %Schema{type: :string, nullable: true},
+          visibility: %Schema{type: VisibilityScope, nullable: true},
+          scheduled_at: %Schema{type: :string, format: :"date-time", nullable: true},
+          poll: %Schema{type: Poll, nullable: true},
+          in_reply_to_id: %Schema{type: :string, nullable: true}
+        }
+      }
+    },
+    example: %{
+      id: "3221",
+      scheduled_at: "2019-12-05T12:33:01.000Z",
+      params: %{
+        text: "test content",
+        media_ids: nil,
+        sensitive: nil,
+        spoiler_text: nil,
+        visibility: nil,
+        scheduled_at: nil,
+        poll: nil,
+        idempotency: nil,
+        in_reply_to_id: nil
+      },
+      media_attachments: [Attachment.schema().example]
+    }
+  })
+end

lib/pleroma/web/api_spec/schemas/status.ex

@@ -5,9 +5,11 @@
 defmodule Pleroma.Web.ApiSpec.Schemas.Status do
   alias OpenApiSpex.Schema
   alias Pleroma.Web.ApiSpec.Schemas.Account
+  alias Pleroma.Web.ApiSpec.Schemas.Attachment
   alias Pleroma.Web.ApiSpec.Schemas.Emoji
   alias Pleroma.Web.ApiSpec.Schemas.FlakeID
   alias Pleroma.Web.ApiSpec.Schemas.Poll
+  alias Pleroma.Web.ApiSpec.Schemas.Tag
   alias Pleroma.Web.ApiSpec.Schemas.VisibilityScope
 
   require OpenApiSpex
@@ -50,22 +52,7 @@ defmodule Pleroma.Web.ApiSpec.Schemas.Status do
       language: %Schema{type: :string, nullable: true},
       media_attachments: %Schema{
         type: :array,
-        items: %Schema{
+        items: Attachment
-          type: :object,
-          properties: %{
-            id: %Schema{type: :string},
-            url: %Schema{type: :string, format: :uri},
-            remote_url: %Schema{type: :string, format: :uri},
-            preview_url: %Schema{type: :string, format: :uri},
-            text_url: %Schema{type: :string, format: :uri},
-            description: %Schema{type: :string},
-            type: %Schema{type: :string, enum: ["image", "video", "audio", "unknown"]},
-            pleroma: %Schema{
-              type: :object,
-              properties: %{mime_type: %Schema{type: :string}}
-            }
-          }
-        }
       },
       mentions: %Schema{
         type: :array,
@@ -86,7 +73,12 @@ defmodule Pleroma.Web.ApiSpec.Schemas.Status do
         properties: %{
           content: %Schema{type: :object, additionalProperties: %Schema{type: :string}},
           conversation_id: %Schema{type: :integer},
-          direct_conversation_id: %Schema{type: :string, nullable: true},
+          direct_conversation_id: %Schema{
+            type: :integer,
+            nullable: true,
+            description:
+              "The ID of the Mastodon direct message conversation the status is associated with (if any)"
+          },
           emoji_reactions: %Schema{
             type: :array,
             items: %Schema{
@@ -115,16 +107,7 @@ defmodule Pleroma.Web.ApiSpec.Schemas.Status do
       replies_count: %Schema{type: :integer},
       sensitive: %Schema{type: :boolean},
       spoiler_text: %Schema{type: :string},
-      tags: %Schema{
+      tags: %Schema{type: :array, items: Tag},
-        type: :array,
-        items: %Schema{
-          type: :object,
-          properties: %{
-            name: %Schema{type: :string},
-            url: %Schema{type: :string, format: :uri}
-          }
-        }
-      },
       uri: %Schema{type: :string, format: :uri},
       url: %Schema{type: :string, nullable: true, format: :uri},
       visibility: VisibilityScope

lib/pleroma/web/api_spec/schemas/tag.ex

@@ -0,0 +1,27 @@
+# Pleroma: A lightweight social networking server
+# Copyright © 2017-2020 Pleroma Authors <https://pleroma.social/>
+# SPDX-License-Identifier: AGPL-3.0-only
+
+defmodule Pleroma.Web.ApiSpec.Schemas.Tag do
+  alias OpenApiSpex.Schema
+
+  require OpenApiSpex
+
+  OpenApiSpex.schema(%{
+    title: "Tag",
+    description: "Represents a hashtag used within the content of a status",
+    type: :object,
+    properties: %{
+      name: %Schema{type: :string, description: "The value of the hashtag after the # sign"},
+      url: %Schema{
+        type: :string,
+        format: :uri,
+        description: "A link to the hashtag on the instance"
+      }
+    },
+    example: %{
+      name: "cofe",
+      url: "https://lain.com/tag/cofe"
+    }
+  })
+end

lib/pleroma/web/auth/pleroma_authenticator.ex

@@ -19,8 +19,8 @@ def get_user(%Plug.Conn{} = conn) do
          {_, true} <- {:checkpw, AuthenticationPlug.checkpw(password, user.password_hash)} do
       {:ok, user}
     else
-      error ->
+      {:error, _reason} = error -> error
-        {:error, error}
+      error -> {:error, error}
     end
   end
 

lib/pleroma/web/auth/totp_authenticator.ex

@@ -0,0 +1,45 @@
+# Pleroma: A lightweight social networking server
+# Copyright © 2017-2019 Pleroma Authors <https://pleroma.social/>
+# SPDX-License-Identifier: AGPL-3.0-only
+
+defmodule Pleroma.Web.Auth.TOTPAuthenticator do
+  alias Comeonin.Pbkdf2
+  alias Pleroma.MFA
+  alias Pleroma.MFA.TOTP
+  alias Pleroma.User
+
+  @doc "Verify code or check backup code."
+  @spec verify(String.t(), User.t()) ::
+          {:ok, :pass} | {:error, :invalid_token | :invalid_secret_and_token}
+  def verify(
+        token,
+        %User{
+          multi_factor_authentication_settings:
+            %{enabled: true, totp: %{secret: secret, confirmed: true}} = _
+        } = _user
+      )
+      when is_binary(token) and byte_size(token) > 0 do
+    TOTP.validate_token(secret, token)
+  end
+
+  def verify(_, _), do: {:error, :invalid_token}
+
+  @spec verify_recovery_code(User.t(), String.t()) ::
+          {:ok, :pass} | {:error, :invalid_token}
+  def verify_recovery_code(
+        %User{multi_factor_authentication_settings: %{enabled: true, backup_codes: codes}} = user,
+        code
+      )
+      when is_list(codes) and is_binary(code) do
+    hash_code = Enum.find(codes, fn hash -> Pbkdf2.checkpw(code, hash) end)
+
+    if hash_code do
+      MFA.invalidate_backup_code(user, hash_code)
+      {:ok, :pass}
+    else
+      {:error, :invalid_token}
+    end
+  end
+
+  def verify_recovery_code(_, _), do: {:error, :invalid_token}
+end

lib/pleroma/web/common_api/common_api.ex

@@ -7,6 +7,7 @@ defmodule Pleroma.Web.CommonAPI do
   alias Pleroma.ActivityExpiration
   alias Pleroma.Conversation.Participation
   alias Pleroma.FollowingRelationship
+  alias Pleroma.Notification
   alias Pleroma.Object
   alias Pleroma.ThreadMute
   alias Pleroma.User
@@ -23,6 +24,14 @@ defmodule Pleroma.Web.CommonAPI do
   require Pleroma.Constants
   require Logger
 
+  def unblock(blocker, blocked) do
+    with %Activity{} = block <- Utils.fetch_latest_block(blocker, blocked),
+         {:ok, unblock_data, _} <- Builder.undo(blocker, block),
+         {:ok, unblock, _} <- Pipeline.common_pipeline(unblock_data, local: true) do
+      {:ok, unblock}
+    end
+  end
+
   def follow(follower, followed) do
     timeout = Pleroma.Config.get([:activitypub, :follow_handshake_timeout])
 
@@ -42,8 +51,8 @@ def unfollow(follower, unfollowed) do
   end
 
   def accept_follow_request(follower, followed) do
-    with {:ok, follower} <- User.follow(follower, followed),
+    with %Activity{} = follow_activity <- Utils.fetch_latest_follow(follower, followed),
-         %Activity{} = follow_activity <- Utils.fetch_latest_follow(follower, followed),
+         {:ok, follower} <- User.follow(follower, followed),
          {:ok, follow_activity} <- Utils.update_follow_state_for_all(follow_activity, "accept"),
          {:ok, _relationship} <- FollowingRelationship.update(follower, followed, :follow_accept),
          {:ok, _activity} <-
@@ -61,6 +70,7 @@ def reject_follow_request(follower, followed) do
     with %Activity{} = follow_activity <- Utils.fetch_latest_follow(follower, followed),
          {:ok, follow_activity} <- Utils.update_follow_state_for_all(follow_activity, "reject"),
          {:ok, _relationship} <- FollowingRelationship.update(follower, followed, :follow_reject),
+         {:ok, _notifications} <- Notification.dismiss(follow_activity),
          {:ok, _activity} <-
            ActivityPub.reject(%{
              to: [follower.ap_id],
@@ -77,8 +87,8 @@ def delete(activity_id, user) do
            {:find_activity, Activity.get_by_id_with_object(activity_id)},
          %Object{} = object <- Object.normalize(activity),
          true <- User.superuser?(user) || user.ap_id == object.data["actor"],
-         {:ok, _} <- unpin(activity_id, user),
+         {:ok, delete_data, _} <- Builder.delete(user, object.data["id"]),
-         {:ok, delete} <- ActivityPub.delete(object) do
+         {:ok, delete, _} <- Pipeline.common_pipeline(delete_data, local: true) do
       {:ok, delete}
     else
       {:find_activity, _} -> {:error, :not_found}
@@ -105,9 +115,12 @@ def repeat(id, user, params \\ %{}) do
 
   def unrepeat(id, user) do
     with {_, %Activity{data: %{"type" => "Create"}} = activity} <-
-           {:find_activity, Activity.get_by_id(id)} do
+           {:find_activity, Activity.get_by_id(id)},
-      object = Object.normalize(activity)
+         %Object{} = note <- Object.normalize(activity, false),
-      ActivityPub.unannounce(user, object)
+         %Activity{} = announce <- Utils.get_existing_announce(user.ap_id, note),
+         {:ok, undo, _} <- Builder.undo(user, announce),
+         {:ok, activity, _} <- Pipeline.common_pipeline(undo, local: true) do
+      {:ok, activity}
     else
       {:find_activity, _} -> {:error, :not_found}
       _ -> {:error, dgettext("errors", "Could not unrepeat")}
@@ -164,9 +177,12 @@ def favorite_helper(user, id) do
 
   def unfavorite(id, user) do
     with {_, %Activity{data: %{"type" => "Create"}} = activity} <-
-           {:find_activity, Activity.get_by_id(id)} do
+           {:find_activity, Activity.get_by_id(id)},
-      object = Object.normalize(activity)
+         %Object{} = note <- Object.normalize(activity, false),
-      ActivityPub.unlike(user, object)
+         %Activity{} = like <- Utils.get_existing_like(user.ap_id, note),
+         {:ok, undo, _} <- Builder.undo(user, like),
+         {:ok, activity, _} <- Pipeline.common_pipeline(undo, local: true) do
+      {:ok, activity}
     else
       {:find_activity, _} -> {:error, :not_found}
       _ -> {:error, dgettext("errors", "Could not unfavorite")}
@@ -175,8 +191,10 @@ def unfavorite(id, user) do
 
   def react_with_emoji(id, user, emoji) do
     with %Activity{} = activity <- Activity.get_by_id(id),
-         object <- Object.normalize(activity) do
+         object <- Object.normalize(activity),
-      ActivityPub.react_with_emoji(user, object, emoji)
+         {:ok, emoji_react, _} <- Builder.emoji_react(user, object, emoji),
+         {:ok, activity, _} <- Pipeline.common_pipeline(emoji_react, local: true) do
+      {:ok, activity}
     else
       _ ->
         {:error, dgettext("errors", "Could not add reaction emoji")}
@@ -184,8 +202,10 @@ def react_with_emoji(id, user, emoji) do
   end
 
   def unreact_with_emoji(id, user, emoji) do
-    with %Activity{} = reaction_activity <- Utils.get_latest_reaction(id, user, emoji) do
+    with %Activity{} = reaction_activity <- Utils.get_latest_reaction(id, user, emoji),
-      ActivityPub.unreact_with_emoji(user, reaction_activity.data["id"])
+         {:ok, undo, _} <- Builder.undo(user, reaction_activity),
+         {:ok, activity, _} <- Pipeline.common_pipeline(undo, local: true) do
+      {:ok, activity}
     else
       _ ->
         {:error, dgettext("errors", "Could not remove reaction emoji")}
@@ -380,9 +400,9 @@ def thread_muted?(user, activity) do
     ThreadMute.exists?(user.id, activity.data["context"])
   end
 
-  def report(user, %{"account_id" => account_id} = data) do
+  def report(user, data) do
-    with {:ok, account} <- get_reported_account(account_id),
+    with {:ok, account} <- get_reported_account(data.account_id),
-         {:ok, {content_html, _, _}} <- make_report_content_html(data["comment"]),
+         {:ok, {content_html, _, _}} <- make_report_content_html(data[:comment]),
          {:ok, statuses} <- get_report_statuses(account, data) do
       ActivityPub.flag(%{
         context: Utils.generate_context_id(),
@@ -390,13 +410,11 @@ def report(user, %{"account_id" => account_id} = data) do
         account: account,
         statuses: statuses,
         content: content_html,
-        forward: data["forward"] || false
+        forward: Map.get(data, :forward, false)
       })
     end
   end
 
-  def report(_user, _params), do: {:error, dgettext("errors", "Valid `account_id` required")}
-
   defp get_reported_account(account_id) do
     case User.get_cached_by_id(account_id) do
       %User{} = account -> {:ok, account}

lib/pleroma/web/common_api/utils.ex

@@ -402,6 +402,7 @@ defp shortname(name) do
     end
   end
 
+  @spec confirm_current_password(User.t(), String.t()) :: {:ok, User.t()} | {:error, String.t()}
   def confirm_current_password(user, password) do
     with %User{local: true} = db_user <- User.get_cached_by_id(user.id),
          true <- AuthenticationPlug.checkpw(password, db_user.password_hash) do
@@ -504,7 +505,8 @@ def make_report_content_html(comment) do
     end
   end
 
-  def get_report_statuses(%User{ap_id: actor}, %{"status_ids" => status_ids}) do
+  def get_report_statuses(%User{ap_id: actor}, %{status_ids: status_ids})
+      when is_list(status_ids) do
     {:ok, Activity.all_by_actor_and_id(actor, status_ids)}
   end
 

lib/pleroma/web/endpoint.ex

@@ -5,6 +5,8 @@
 defmodule Pleroma.Web.Endpoint do
   use Phoenix.Endpoint, otp_app: :pleroma
 
+  require Pleroma.Constants
+
   socket("/socket", Pleroma.Web.UserSocket)
 
   plug(Pleroma.Plugs.SetLocalePlug)
@@ -34,8 +36,7 @@ defmodule Pleroma.Web.Endpoint do
     Plug.Static,
     at: "/",
     from: :pleroma,
-    only:
+    only: Pleroma.Constants.static_only_files(),
-      ~w(index.html robots.txt static finmoji emoji packs sounds images instance sw.js sw-pleroma.js favicon.png schemas doc),
     # credo:disable-for-previous-line Credo.Check.Readability.MaxLineLength
     gzip: true,
     cache_control_for_etags: @static_cache_control,

lib/pleroma/web/feed/user_controller.ex

@@ -27,7 +27,7 @@ def feed_redirect(%{assigns: %{format: format}} = conn, _params)
       when format in ["json", "activity+json"] do
     with %{halted: false} = conn <-
            Pleroma.Plugs.EnsureAuthenticatedPlug.call(conn,
-             unless_func: &Pleroma.Web.FederatingPlug.federating?/0
+             unless_func: &Pleroma.Web.FederatingPlug.federating?/1
            ) do
       ActivityPubController.call(conn, :user)
     end

lib/pleroma/web/mastodon_api/controllers/account_controller.ex

@@ -27,7 +27,7 @@ defmodule Pleroma.Web.MastodonAPI.AccountController do
   alias Pleroma.Web.OAuth.Token
   alias Pleroma.Web.TwitterAPI.TwitterAPI
 
-  plug(OpenApiSpex.Plug.CastAndValidate, render_error: Pleroma.Web.ApiSpec.RenderError)
+  plug(Pleroma.Web.ApiSpec.CastAndValidate)
 
   plug(:skip_plug, [OAuthScopesPlug, EnsurePublicOrAuthenticatedPlug] when action == :create)
 
@@ -94,24 +94,8 @@ defmodule Pleroma.Web.MastodonAPI.AccountController do
 
   @doc "POST /api/v1/accounts"
   def create(%{assigns: %{app: app}, body_params: params} = conn, _params) do
-    params =
-      params
-      |> Map.take([
-        :email,
-        :bio,
-        :captcha_solution,
-        :captcha_token,
-        :captcha_answer_data,
-        :token,
-        :password,
-        :fullname
-      ])
-      |> Map.put(:nickname, params.username)
-      |> Map.put(:fullname, Map.get(params, :fullname, params.username))
-      |> Map.put(:confirm, params.password)
-      |> Map.put(:trusted_app, app.trusted)
-
     with :ok <- validate_email_param(params),
+         :ok <- TwitterAPI.validate_captcha(app, params),
          {:ok, user} <- TwitterAPI.register_user(params, need_confirmation: true),
          {:ok, token} <- Token.create_token(app, user, %{scopes: app.scopes}) do
       json(conn, %{
@@ -121,7 +105,7 @@ def create(%{assigns: %{app: app}, body_params: params} = conn, _params) do
         created_at: Token.Utils.format_created_at(token)
       })
     else
-      {:error, errors} -> json_response(conn, :bad_request, errors)
+      {:error, error} -> json_response(conn, :bad_request, %{error: error})
     end
   end
 
@@ -133,11 +117,11 @@ def create(conn, _) do
     render_error(conn, :forbidden, "Invalid credentials")
   end
 
-  defp validate_email_param(%{:email => email}) when not is_nil(email), do: :ok
+  defp validate_email_param(%{email: email}) when not is_nil(email), do: :ok
 
   defp validate_email_param(_) do
     case Pleroma.Config.get([:instance, :account_activation_required]) do
-      true -> {:error, %{"error" => "Missing parameters"}}
+      true -> {:error, dgettext("errors", "Missing parameter: %{name}", name: "email")}
       _ -> :ok
     end
   end
@@ -372,8 +356,7 @@ def block(%{assigns: %{user: blocker, account: blocked}} = conn, _params) do
 
   @doc "POST /api/v1/accounts/:id/unblock"
   def unblock(%{assigns: %{user: blocker, account: blocked}} = conn, _params) do
-    with {:ok, _user_block} <- User.unblock(blocker, blocked),
+    with {:ok, _activity} <- CommonAPI.unblock(blocker, blocked) do
-         {:ok, _activity} <- ActivityPub.unblock(blocker, blocked) do
       render(conn, "relationship.json", user: blocker, target: blocked)
     else
       {:error, message} -> json_response(conn, :forbidden, %{error: message})

lib/pleroma/web/mastodon_api/controllers/app_controller.ex

@@ -22,7 +22,7 @@ defmodule Pleroma.Web.MastodonAPI.AppController do
 
   plug(OAuthScopesPlug, %{scopes: ["read"]} when action == :verify_credentials)
 
-  plug(OpenApiSpex.Plug.CastAndValidate)
+  plug(Pleroma.Web.ApiSpec.CastAndValidate)
 
   @local_mastodon_name "Mastodon-Local"
 

lib/pleroma/web/mastodon_api/controllers/conversation_controller.ex

@@ -13,9 +13,12 @@ defmodule Pleroma.Web.MastodonAPI.ConversationController do
 
   action_fallback(Pleroma.Web.MastodonAPI.FallbackController)
 
+  plug(Pleroma.Web.ApiSpec.CastAndValidate)
   plug(OAuthScopesPlug, %{scopes: ["read:statuses"]} when action == :index)
   plug(OAuthScopesPlug, %{scopes: ["write:conversations"]} when action != :index)
 
+  defdelegate open_api_operation(action), to: Pleroma.Web.ApiSpec.ConversationOperation
+
   @doc "GET /api/v1/conversations"
   def index(%{assigns: %{user: user}} = conn, params) do
     participations = Participation.for_user_with_last_activity_id(user, params)
@@ -26,7 +29,7 @@ def index(%{assigns: %{user: user}} = conn, params) do
   end
 
   @doc "POST /api/v1/conversations/:id/read"
-  def mark_as_read(%{assigns: %{user: user}} = conn, %{"id" => participation_id}) do
+  def mark_as_read(%{assigns: %{user: user}} = conn, %{id: participation_id}) do
     with %Participation{} = participation <-
            Repo.get_by(Participation, id: participation_id, user_id: user.id),
          {:ok, participation} <- Participation.mark_as_read(participation) do

lib/pleroma/web/mastodon_api/controllers/custom_emoji_controller.ex

@@ -5,7 +5,7 @@
 defmodule Pleroma.Web.MastodonAPI.CustomEmojiController do
   use Pleroma.Web, :controller
 
-  plug(OpenApiSpex.Plug.CastAndValidate)
+  plug(Pleroma.Web.ApiSpec.CastAndValidate)
 
   plug(
     :skip_plug,

lib/pleroma/web/mastodon_api/controllers/domain_block_controller.ex

@@ -8,7 +8,7 @@ defmodule Pleroma.Web.MastodonAPI.DomainBlockController do
   alias Pleroma.Plugs.OAuthScopesPlug
   alias Pleroma.User
 
-  plug(OpenApiSpex.Plug.CastAndValidate)
+  plug(Pleroma.Web.ApiSpec.CastAndValidate)
   defdelegate open_api_operation(action), to: Pleroma.Web.ApiSpec.DomainBlockOperation
 
   plug(

lib/pleroma/web/mastodon_api/controllers/filter_controller.ex

@@ -10,6 +10,7 @@ defmodule Pleroma.Web.MastodonAPI.FilterController do
 
   @oauth_read_actions [:show, :index]
 
+  plug(Pleroma.Web.ApiSpec.CastAndValidate)
   plug(OAuthScopesPlug, %{scopes: ["read:filters"]} when action in @oauth_read_actions)
 
   plug(
@@ -17,60 +18,60 @@ defmodule Pleroma.Web.MastodonAPI.FilterController do
     %{scopes: ["write:filters"]} when action not in @oauth_read_actions
   )
 
+  defdelegate open_api_operation(action), to: Pleroma.Web.ApiSpec.FilterOperation
+
   @doc "GET /api/v1/filters"
   def index(%{assigns: %{user: user}} = conn, _) do
     filters = Filter.get_filters(user)
 
-    render(conn, "filters.json", filters: filters)
+    render(conn, "index.json", filters: filters)
   end
 
   @doc "POST /api/v1/filters"
-  def create(
+  def create(%{assigns: %{user: user}, body_params: params} = conn, _) do
-        %{assigns: %{user: user}} = conn,
-        %{"phrase" => phrase, "context" => context} = params
-      ) do
     query = %Filter{
       user_id: user.id,
-      phrase: phrase,
+      phrase: params.phrase,
-      context: context,
+      context: params.context,
-      hide: Map.get(params, "irreversible", false),
+      hide: params.irreversible,
-      whole_word: Map.get(params, "boolean", true)
+      whole_word: params.whole_word
-      # expires_at
+      # TODO: support `expires_in` parameter (as in Mastodon API)
     }
 
     {:ok, response} = Filter.create(query)
 
-    render(conn, "filter.json", filter: response)
+    render(conn, "show.json", filter: response)
   end
 
   @doc "GET /api/v1/filters/:id"
-  def show(%{assigns: %{user: user}} = conn, %{"id" => filter_id}) do
+  def show(%{assigns: %{user: user}} = conn, %{id: filter_id}) do
     filter = Filter.get(filter_id, user)
 
-    render(conn, "filter.json", filter: filter)
+    render(conn, "show.json", filter: filter)
   end
 
   @doc "PUT /api/v1/filters/:id"
   def update(
-        %{assigns: %{user: user}} = conn,
+        %{assigns: %{user: user}, body_params: params} = conn,
-        %{"phrase" => phrase, "context" => context, "id" => filter_id} = params
+        %{id: filter_id}
       ) do
-    query = %Filter{
+    params =
-      user_id: user.id,
+      params
-      filter_id: filter_id,
+      |> Map.delete(:irreversible)
-      phrase: phrase,
+      |> Map.put(:hide, params[:irreversible])
-      context: context,
+      |> Enum.reject(fn {_key, value} -> is_nil(value) end)
-      hide: Map.get(params, "irreversible", nil),
+      |> Map.new()
-      whole_word: Map.get(params, "boolean", true)
-      # expires_at
-    }
 
-    {:ok, response} = Filter.update(query)
+    # TODO: support `expires_in` parameter (as in Mastodon API)
-    render(conn, "filter.json", filter: response)
+
+    with %Filter{} = filter <- Filter.get(filter_id, user),
+         {:ok, %Filter{} = filter} <- Filter.update(filter, params) do
+      render(conn, "show.json", filter: filter)
+    end
   end
 
   @doc "DELETE /api/v1/filters/:id"
-  def delete(%{assigns: %{user: user}} = conn, %{"id" => filter_id}) do
+  def delete(%{assigns: %{user: user}} = conn, %{id: filter_id}) do
     query = %Filter{
       user_id: user.id,
       filter_id: filter_id

lib/pleroma/web/mastodon_api/controllers/follow_request_controller.ex

@@ -10,6 +10,7 @@ defmodule Pleroma.Web.MastodonAPI.FollowRequestController do
   alias Pleroma.Web.CommonAPI
 
   plug(:put_view, Pleroma.Web.MastodonAPI.AccountView)
+  plug(Pleroma.Web.ApiSpec.CastAndValidate)
   plug(:assign_follower when action != :index)
 
   action_fallback(:errors)
@@ -21,6 +22,8 @@ defmodule Pleroma.Web.MastodonAPI.FollowRequestController do
     %{scopes: ["follow", "write:follows"]} when action != :index
   )
 
+  defdelegate open_api_operation(action), to: Pleroma.Web.ApiSpec.FollowRequestOperation
+
   @doc "GET /api/v1/follow_requests"
   def index(%{assigns: %{user: followed}} = conn, _params) do
     follow_requests = User.get_follow_requests(followed)
@@ -42,7 +45,7 @@ def reject(%{assigns: %{user: followed, follower: follower}} = conn, _params) do
     end
   end
 
-  defp assign_follower(%{params: %{"id" => id}} = conn, _) do
+  defp assign_follower(%{params: %{id: id}} = conn, _) do
     case User.get_cached_by_id(id) do
       %User{} = follower -> assign(conn, :follower, follower)
       nil -> Pleroma.Web.MastodonAPI.FallbackController.call(conn, {:error, :not_found}) |> halt()

lib/pleroma/web/mastodon_api/controllers/instance_controller.ex

@@ -5,12 +5,16 @@
 defmodule Pleroma.Web.MastodonAPI.InstanceController do
   use Pleroma.Web, :controller
 
+  plug(OpenApiSpex.Plug.CastAndValidate)
+
   plug(
     :skip_plug,
     [Pleroma.Plugs.OAuthScopesPlug, Pleroma.Plugs.EnsurePublicOrAuthenticatedPlug]
     when action in [:show, :peers]
   )
 
+  defdelegate open_api_operation(action), to: Pleroma.Web.ApiSpec.InstanceOperation
+
   @doc "GET /api/v1/instance"
   def show(conn, _params) do
     render(conn, "show.json")

lib/pleroma/web/mastodon_api/controllers/list_controller.ex

@@ -9,20 +9,17 @@ defmodule Pleroma.Web.MastodonAPI.ListController do
   alias Pleroma.User
   alias Pleroma.Web.MastodonAPI.AccountView
 
-  plug(:list_by_id_and_user when action not in [:index, :create])
-
   @oauth_read_actions [:index, :show, :list_accounts]
 
+  plug(Pleroma.Web.ApiSpec.CastAndValidate)
+  plug(:list_by_id_and_user when action not in [:index, :create])
   plug(OAuthScopesPlug, %{scopes: ["read:lists"]} when action in @oauth_read_actions)
-
+  plug(OAuthScopesPlug, %{scopes: ["write:lists"]} when action not in @oauth_read_actions)
-  plug(
-    OAuthScopesPlug,
-    %{scopes: ["write:lists"]}
-    when action not in @oauth_read_actions
-  )
 
   action_fallback(Pleroma.Web.MastodonAPI.FallbackController)
 
+  defdelegate open_api_operation(action), to: Pleroma.Web.ApiSpec.ListOperation
+
   # GET /api/v1/lists
   def index(%{assigns: %{user: user}} = conn, opts) do
     lists = Pleroma.List.for_user(user, opts)
@@ -30,7 +27,7 @@ def index(%{assigns: %{user: user}} = conn, opts) do
   end
 
   # POST /api/v1/lists
-  def create(%{assigns: %{user: user}} = conn, %{"title" => title}) do
+  def create(%{assigns: %{user: user}, body_params: %{title: title}} = conn, _) do
     with {:ok, %Pleroma.List{} = list} <- Pleroma.List.create(title, user) do
       render(conn, "show.json", list: list)
     end
@@ -42,7 +39,7 @@ def show(%{assigns: %{list: list}} = conn, _) do
   end
 
   # PUT /api/v1/lists/:id
-  def update(%{assigns: %{list: list}} = conn, %{"title" => title}) do
+  def update(%{assigns: %{list: list}, body_params: %{title: title}} = conn, _) do
     with {:ok, list} <- Pleroma.List.rename(list, title) do
       render(conn, "show.json", list: list)
     end
@@ -65,7 +62,7 @@ def list_accounts(%{assigns: %{user: user, list: list}} = conn, _) do
   end
 
   # POST /api/v1/lists/:id/accounts
-  def add_to_list(%{assigns: %{list: list}} = conn, %{"account_ids" => account_ids}) do
+  def add_to_list(%{assigns: %{list: list}, body_params: %{account_ids: account_ids}} = conn, _) do
     Enum.each(account_ids, fn account_id ->
       with %User{} = followed <- User.get_cached_by_id(account_id) do
         Pleroma.List.follow(list, followed)
@@ -76,7 +73,10 @@ def add_to_list(%{assigns: %{list: list}} = conn, %{"account_ids" => account_ids
   end
 
   # DELETE /api/v1/lists/:id/accounts
-  def remove_from_list(%{assigns: %{list: list}} = conn, %{"account_ids" => account_ids}) do
+  def remove_from_list(
+        %{assigns: %{list: list}, body_params: %{account_ids: account_ids}} = conn,
+        _
+      ) do
     Enum.each(account_ids, fn account_id ->
       with %User{} = followed <- User.get_cached_by_id(account_id) do
         Pleroma.List.unfollow(list, followed)
@@ -86,7 +86,7 @@ def remove_from_list(%{assigns: %{list: list}} = conn, %{"account_ids" => accoun
     json(conn, %{})
   end
 
-  defp list_by_id_and_user(%{assigns: %{user: user}, params: %{"id" => id}} = conn, _) do
+  defp list_by_id_and_user(%{assigns: %{user: user}, params: %{id: id}} = conn, _) do
     case Pleroma.List.get(id, user) do
       %Pleroma.List{} = list -> assign(conn, :list, list)
       nil -> conn |> render_error(:not_found, "List not found") |> halt()

lib/pleroma/web/mastodon_api/controllers/marker_controller.ex

@@ -6,6 +6,8 @@ defmodule Pleroma.Web.MastodonAPI.MarkerController do
   use Pleroma.Web, :controller
   alias Pleroma.Plugs.OAuthScopesPlug
 
+  plug(Pleroma.Web.ApiSpec.CastAndValidate)
+
   plug(
     OAuthScopesPlug,
     %{scopes: ["read:statuses"]}
@@ -16,14 +18,18 @@ defmodule Pleroma.Web.MastodonAPI.MarkerController do
 
   action_fallback(Pleroma.Web.MastodonAPI.FallbackController)
 
+  defdelegate open_api_operation(action), to: Pleroma.Web.ApiSpec.MarkerOperation
+
   # GET /api/v1/markers
   def index(%{assigns: %{user: user}} = conn, params) do
-    markers = Pleroma.Marker.get_markers(user, params["timeline"])
+    markers = Pleroma.Marker.get_markers(user, params[:timeline])
     render(conn, "markers.json", %{markers: markers})
   end
 
   # POST /api/v1/markers
-  def upsert(%{assigns: %{user: user}} = conn, params) do
+  def upsert(%{assigns: %{user: user}, body_params: params} = conn, _) do
+    params = Map.new(params, fn {key, value} -> {to_string(key), value} end)
+
     with {:ok, result} <- Pleroma.Marker.upsert(user, params),
          markers <- Map.values(result) do
       render(conn, "markers.json", %{markers: markers})

lib/pleroma/web/mastodon_api/controllers/notification_controller.ex

@@ -13,6 +13,8 @@ defmodule Pleroma.Web.MastodonAPI.NotificationController do
 
   @oauth_read_actions [:show, :index]
 
+  plug(Pleroma.Web.ApiSpec.CastAndValidate)
+
   plug(
     OAuthScopesPlug,
     %{scopes: ["read:notifications"]} when action in @oauth_read_actions
@@ -20,14 +22,16 @@ defmodule Pleroma.Web.MastodonAPI.NotificationController do
 
   plug(OAuthScopesPlug, %{scopes: ["write:notifications"]} when action not in @oauth_read_actions)
 
+  defdelegate open_api_operation(action), to: Pleroma.Web.ApiSpec.NotificationOperation
+
   # GET /api/v1/notifications
-  def index(conn, %{"account_id" => account_id} = params) do
+  def index(conn, %{account_id: account_id} = params) do
     case Pleroma.User.get_cached_by_id(account_id) do
       %{ap_id: account_ap_id} ->
         params =
           params
-          |> Map.delete("account_id")
+          |> Map.delete(:account_id)
-          |> Map.put("account_ap_id", account_ap_id)
+          |> Map.put(:account_ap_id, account_ap_id)
 
         index(conn, params)
 
@@ -39,6 +43,7 @@ def index(conn, %{"account_id" => account_id} = params) do
   end
 
   def index(%{assigns: %{user: user}} = conn, params) do
+    params = Map.new(params, fn {k, v} -> {to_string(k), v} end)
     notifications = MastodonAPI.get_notifications(user, params)
 
     conn
@@ -51,7 +56,7 @@ def index(%{assigns: %{user: user}} = conn, params) do
   end
 
   # GET /api/v1/notifications/:id
-  def show(%{assigns: %{user: user}} = conn, %{"id" => id}) do
+  def show(%{assigns: %{user: user}} = conn, %{id: id}) do
     with {:ok, notification} <- Notification.get(user, id) do
       render(conn, "show.json", notification: notification, for: user)
     else
@@ -69,8 +74,8 @@ def clear(%{assigns: %{user: user}} = conn, _params) do
   end
 
   # POST /api/v1/notifications/:id/dismiss
-  # POST /api/v1/notifications/dismiss (deprecated)
+
-  def dismiss(%{assigns: %{user: user}} = conn, %{"id" => id} = _params) do
+  def dismiss(%{assigns: %{user: user}} = conn, %{id: id} = _params) do
     with {:ok, _notif} <- Notification.dismiss(user, id) do
       json(conn, %{})
     else
@@ -81,8 +86,13 @@ def dismiss(%{assigns: %{user: user}} = conn, %{"id" => id} = _params) do
     end
   end
 
+  # POST /api/v1/notifications/dismiss (deprecated)
+  def dismiss_via_body(%{body_params: params} = conn, _) do
+    dismiss(conn, params)
+  end
+
   # DELETE /api/v1/notifications/destroy_multiple
-  def destroy_multiple(%{assigns: %{user: user}} = conn, %{"ids" => ids} = _params) do
+  def destroy_multiple(%{assigns: %{user: user}} = conn, %{ids: ids} = _params) do
     Notification.destroy_multiple(user, ids)
     json(conn, %{})
   end

lib/pleroma/web/mastodon_api/controllers/poll_controller.ex

@@ -15,6 +15,8 @@ defmodule Pleroma.Web.MastodonAPI.PollController do
 
   action_fallback(Pleroma.Web.MastodonAPI.FallbackController)
 
+  plug(Pleroma.Web.ApiSpec.CastAndValidate)
+
   plug(
     OAuthScopesPlug,
     %{scopes: ["read:statuses"], fallback: :proceed_unauthenticated} when action == :show
@@ -22,8 +24,10 @@ defmodule Pleroma.Web.MastodonAPI.PollController do
 
   plug(OAuthScopesPlug, %{scopes: ["write:statuses"]} when action == :vote)
 
+  defdelegate open_api_operation(action), to: Pleroma.Web.ApiSpec.PollOperation
+
   @doc "GET /api/v1/polls/:id"
-  def show(%{assigns: %{user: user}} = conn, %{"id" => id}) do
+  def show(%{assigns: %{user: user}} = conn, %{id: id}) do
     with %Object{} = object <- Object.get_by_id_and_maybe_refetch(id, interval: 60),
          %Activity{} = activity <- Activity.get_create_by_object_ap_id(object.data["id"]),
          true <- Visibility.visible_for_user?(activity, user) do
@@ -35,7 +39,7 @@ def show(%{assigns: %{user: user}} = conn, %{"id" => id}) do
   end
 
   @doc "POST /api/v1/polls/:id/votes"
-  def vote(%{assigns: %{user: user}} = conn, %{"id" => id, "choices" => choices}) do
+  def vote(%{assigns: %{user: user}, body_params: %{choices: choices}} = conn, %{id: id}) do
     with %Object{data: %{"type" => "Question"}} = object <- Object.get_by_id(id),
          %Activity{} = activity <- Activity.get_create_by_object_ap_id(object.data["id"]),
          true <- Visibility.visible_for_user?(activity, user),

lib/pleroma/web/mastodon_api/controllers/report_controller.ex

@@ -9,10 +9,13 @@ defmodule Pleroma.Web.MastodonAPI.ReportController do
 
   action_fallback(Pleroma.Web.MastodonAPI.FallbackController)
 
+  plug(Pleroma.Web.ApiSpec.CastAndValidate)
   plug(OAuthScopesPlug, %{scopes: ["write:reports"]} when action == :create)
 
+  defdelegate open_api_operation(action), to: Pleroma.Web.ApiSpec.ReportOperation
+
   @doc "POST /api/v1/reports"
-  def create(%{assigns: %{user: user}} = conn, params) do
+  def create(%{assigns: %{user: user}, body_params: params} = conn, _) do
     with {:ok, activity} <- Pleroma.Web.CommonAPI.report(user, params) do
       render(conn, "show.json", activity: activity)
     end

lib/pleroma/web/mastodon_api/controllers/scheduled_activity_controller.ex

@@ -11,17 +11,21 @@ defmodule Pleroma.Web.MastodonAPI.ScheduledActivityController do
   alias Pleroma.ScheduledActivity
   alias Pleroma.Web.MastodonAPI.MastodonAPI
 
-  plug(:assign_scheduled_activity when action != :index)
-
   @oauth_read_actions [:show, :index]
 
+  plug(Pleroma.Web.ApiSpec.CastAndValidate)
   plug(OAuthScopesPlug, %{scopes: ["read:statuses"]} when action in @oauth_read_actions)
   plug(OAuthScopesPlug, %{scopes: ["write:statuses"]} when action not in @oauth_read_actions)
+  plug(:assign_scheduled_activity when action != :index)
 
   action_fallback(Pleroma.Web.MastodonAPI.FallbackController)
 
+  defdelegate open_api_operation(action), to: Pleroma.Web.ApiSpec.ScheduledActivityOperation
+
   @doc "GET /api/v1/scheduled_statuses"
   def index(%{assigns: %{user: user}} = conn, params) do
+    params = Map.new(params, fn {key, value} -> {to_string(key), value} end)
+
     with scheduled_activities <- MastodonAPI.get_scheduled_activities(user, params) do
       conn
       |> add_link_headers(scheduled_activities)
@@ -35,7 +39,7 @@ def show(%{assigns: %{scheduled_activity: scheduled_activity}} = conn, _params)
   end
 
   @doc "PUT /api/v1/scheduled_statuses/:id"
-  def update(%{assigns: %{scheduled_activity: scheduled_activity}} = conn, params) do
+  def update(%{assigns: %{scheduled_activity: scheduled_activity}, body_params: params} = conn, _) do
     with {:ok, scheduled_activity} <- ScheduledActivity.update(scheduled_activity, params) do
       render(conn, "show.json", scheduled_activity: scheduled_activity)
     end
@@ -48,7 +52,7 @@ def delete(%{assigns: %{scheduled_activity: scheduled_activity}} = conn, _params
     end
   end
 
-  defp assign_scheduled_activity(%{assigns: %{user: user}, params: %{"id" => id}} = conn, _) do
+  defp assign_scheduled_activity(%{assigns: %{user: user}, params: %{id: id}} = conn, _) do
     case ScheduledActivity.get(user, id) do
       %ScheduledActivity{} = activity -> assign(conn, :scheduled_activity, activity)
       nil -> Pleroma.Web.MastodonAPI.FallbackController.call(conn, {:error, :not_found}) |> halt()