Merge branch 'develop' of https://git.pleroma.social/pleroma/pleroma into develop

CHANGELOG.md

@@ -37,16 +37,21 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/).
 - Rich Media: The crawled URL is now spliced into the rich media data.
 - ActivityPub S2S: sharedInbox usage has been mostly aligned with the rules in the AP specification.
 - ActivityPub S2S: remote user deletions now work the same as local user deletions.
+- ActivityPub S2S: POST requests are now signed with `(request-target)` pseudo-header.
 - Not being able to access the Mastodon FE login page on private instances
 - Invalid SemVer version generation, when the current branch does not have commits ahead of tag/checked out on a tag
 - Pleroma.Upload base_url was not automatically whitelisted by MediaProxy. Now your custom CDN or file hosting will be accessed directly as expected.
 - Report email not being sent to admins when the reporter is a remote user
 - MRF: ensure that subdomain_match calls are case-insensitive
+- MRF: fix use of unserializable keyword lists in describe() implementations
 
 ### Added
+- **Breaking:** MRF describe API, which adds support for exposing configuration information about MRF policies to NodeInfo.
+  Custom modules will need to be updated by adding, at the very least, `def describe, do: {:ok, %{}}` to the MRF policy modules.
 - MRF: Support for priming the mediaproxy cache (`Pleroma.Web.ActivityPub.MRF.MediaProxyWarmingPolicy`)
 - MRF: Support for excluding specific domains from Transparency.
 - MRF: Support for filtering posts based on who they mention (`Pleroma.Web.ActivityPub.MRF.MentionPolicy`)
+- MRF: Support for filtering posts based on ActivityStreams vocabulary (`Pleroma.Web.ActivityPub.MRF.VocabularyPolicy`)
 - MRF (Simple Policy): Support for wildcard domains.
 - Support for wildcard domains in user domain blocks setting.
 - Configuration: `quarantined_instances` support wildcard domains.
@@ -69,6 +74,7 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/).
 - Added synchronization of following/followers counters for external users
 - Configuration: `enabled` option for `Pleroma.Emails.Mailer`, defaulting to `false`.
 - Configuration: Pleroma.Plugs.RateLimiter `bucket_name`, `params` options.
+- Configuration: `user_bio_length` and `user_name_length` options.
 - Addressable lists
 - Twitter API: added rate limit for `/api/account/password_reset` endpoint.
 - ActivityPub: Add an internal service actor for fetching ActivityPub objects.

config/config.exs

@@ -253,6 +253,8 @@
   skip_thread_containment: true,
   limit_to_local_content: :unauthenticated,
   dynamic_configuration: false,
+  user_bio_length: 5000,
+  user_name_length: 100,
   external_user_synchronization: true
 
 config :pleroma, :markup,
@@ -336,6 +338,10 @@
 
 config :pleroma, :mrf_subchain, match_actor: %{}
 
+config :pleroma, :mrf_vocabulary,
+  accept: [],
+  reject: []
+
 config :pleroma, :rich_media,
   enabled: true,
   ignore_hosts: [],

docs/config.md

@@ -103,6 +103,7 @@ config :pleroma, Pleroma.Emails.Mailer,
   * `Pleroma.Web.ActivityPub.MRF.AntiLinkSpamPolicy`: Rejects posts from likely spambots by rejecting posts from new users that contain links.
   * `Pleroma.Web.ActivityPub.MRF.MediaProxyWarmingPolicy`: Crawls attachments using their MediaProxy URLs so that the MediaProxy cache is primed.
   * `Pleroma.Web.ActivityPub.MRF.MentionPolicy`: Drops posts mentioning configurable users. (see `:mrf_mention` section)
+  * `Pleroma.Web.ActivityPub.MRF.VocabularyPolicy`: Restricts activities to a configured set of vocabulary. (see `:mrf_vocabulary` section)
 * `public`: Makes the client API in authentificated mode-only except for user-profiles. Useful for disabling the Local Timeline and The Whole Known Network.
 * `quarantined_instances`: List of ActivityPub instances where private(DMs, followers-only) activities will not be send.
 * `managed_config`: Whenether the config for pleroma-fe is configured in this config or in ``static/config.json``
@@ -126,6 +127,8 @@ config :pleroma, Pleroma.Emails.Mailer,
 * `safe_dm_mentions`: If set to true, only mentions at the beginning of a post will be used to address people in direct messages. This is to prevent accidental mentioning of people when talking about them (e.g. "@friend hey i really don't like @enemy"). Default: `false`.
 * `healthcheck`: If set to true, system data will be shown on ``/api/pleroma/healthcheck``.
 * `remote_post_retention_days`: The default amount of days to retain remote posts when pruning the database.
+* `user_bio_length`: A user bio maximum length (default: `5000`)
+* `user_name_length`: A user name maximum length (default: `100`)
 * `skip_thread_containment`: Skip filter out broken threads. The default is `false`.
 * `limit_to_local_content`: Limit unauthenticated users to search for local statutes and users only. Possible values: `:unauthenticated`, `:all` and `false`. The default is `:unauthenticated`.
 * `dynamic_configuration`: Allow transferring configuration to DB with the subsequent customization from Admin api.
@@ -276,6 +279,10 @@ config :pleroma, :mrf_subchain,
 ## :mrf_mention
 * `actors`: A list of actors, for which to drop any posts mentioning.
 
+## :mrf_vocabulary
+* `accept`: A list of ActivityStreams terms to accept.  If empty, all supported messages are accepted.
+* `reject`: A list of ActivityStreams terms to reject.  If empty, no messages are rejected.
+
 ## :media_proxy
 * `enabled`: Enables proxying of remote media to the instance’s proxy
 * `base_url`: The base URL to access a user-uploaded file. Useful when you want to proxy the media files via another host/CDN fronts.

lib/mix/tasks/pleroma/benchmark.ex

@@ -26,4 +26,48 @@ def run(["tag"]) do
       end
     })
   end
+
+  def run(["render_timeline", nickname]) do
+    start_pleroma()
+    user = Pleroma.User.get_by_nickname(nickname)
+
+    activities =
+      %{}
+      |> Map.put("type", ["Create", "Announce"])
+      |> Map.put("blocking_user", user)
+      |> Map.put("muting_user", user)
+      |> Map.put("user", user)
+      |> Map.put("limit", 80)
+      |> Pleroma.Web.ActivityPub.ActivityPub.fetch_public_activities()
+      |> Enum.reverse()
+
+    inputs = %{
+      "One activity" => Enum.take_random(activities, 1),
+      "Ten activities" => Enum.take_random(activities, 10),
+      "Twenty activities" => Enum.take_random(activities, 20),
+      "Forty activities" => Enum.take_random(activities, 40),
+      "Eighty activities" => Enum.take_random(activities, 80)
+    }
+
+    Benchee.run(
+      %{
+        "Parallel rendering" => fn activities ->
+          Pleroma.Web.MastodonAPI.StatusView.render("index.json", %{
+            activities: activities,
+            for: user,
+            as: :activity
+          })
+        end,
+        "Standart rendering" => fn activities ->
+          Pleroma.Web.MastodonAPI.StatusView.render("index.json", %{
+            activities: activities,
+            for: user,
+            as: :activity,
+            parallel: false
+          })
+        end
+      },
+      inputs: inputs
+    )
+  end
 end

lib/pleroma/activity.ex

@@ -224,6 +224,29 @@ def get_create_by_object_ap_id(ap_id) when is_binary(ap_id) do
 
   def get_create_by_object_ap_id(_), do: nil
 
+  def create_by_object_ap_id_with_object(ap_ids) when is_list(ap_ids) do
+    from(
+      activity in Activity,
+      where:
+        fragment(
+          "coalesce((?)->'object'->>'id', (?)->>'object') = ANY(?)",
+          activity.data,
+          activity.data,
+          ^ap_ids
+        ),
+      where: fragment("(?)->>'type' = 'Create'", activity.data),
+      inner_join: o in Object,
+      on:
+        fragment(
+          "(?->>'id') = COALESCE(?->'object'->>'id', ?->>'object')",
+          o.data,
+          activity.data,
+          activity.data
+        ),
+      preload: [object: o]
+    )
+  end
+
   def create_by_object_ap_id_with_object(ap_id) when is_binary(ap_id) do
     from(
       activity in Activity,
@@ -263,8 +286,8 @@ defp get_in_reply_to_activity_from_object(%Object{data: %{"inReplyTo" => ap_id}}
 
   defp get_in_reply_to_activity_from_object(_), do: nil
 
-  def get_in_reply_to_activity(%Activity{data: %{"object" => object}}) do
+  def get_in_reply_to_activity(%Activity{} = activity) do
-    get_in_reply_to_activity_from_object(Object.normalize(object))
+    get_in_reply_to_activity_from_object(Object.normalize(activity))
   end
 
   def normalize(obj) when is_map(obj), do: get_by_ap_id_with_object(obj["id"])

lib/pleroma/application.ex

@@ -3,11 +3,14 @@
 # SPDX-License-Identifier: AGPL-3.0-only
 
 defmodule Pleroma.Application do
+  import Cachex.Spec
   use Application
 
   @name Mix.Project.config()[:name]
   @version Mix.Project.config()[:version]
   @repository Mix.Project.config()[:source_url]
+  @env Mix.env()
+
   def name, do: @name
   def version, do: @version
   def named_version, do: @name <> " " <> @version
@@ -21,116 +24,24 @@ def user_agent do
   # See http://elixir-lang.org/docs/stable/elixir/Application.html
   # for more information on OTP Applications
   def start(_type, _args) do
-    import Cachex.Spec
-
     Pleroma.Config.DeprecationWarnings.warn()
     setup_instrumenters()
 
     # Define workers and child supervisors to be supervised
     children =
       [
-        # Start the Ecto repository
+        Pleroma.Repo,
-        %{id: Pleroma.Repo, start: {Pleroma.Repo, :start_link, []}, type: :supervisor},
+        Pleroma.Config.TransferTask,
-        %{id: Pleroma.Config.TransferTask, start: {Pleroma.Config.TransferTask, :start_link, []}},
+        Pleroma.Emoji,
-        %{id: Pleroma.Emoji, start: {Pleroma.Emoji, :start_link, []}},
+        Pleroma.Captcha,
-        %{id: Pleroma.Captcha, start: {Pleroma.Captcha, :start_link, []}},
+        Pleroma.FlakeId,
-        %{
+        Pleroma.ScheduledActivityWorker
-          id: :cachex_used_captcha_cache,
-          start:
-            {Cachex, :start_link,
-             [
-               :used_captcha_cache,
-               [
-                 ttl_interval:
-                   :timer.seconds(Pleroma.Config.get!([Pleroma.Captcha, :seconds_valid]))
-               ]
-             ]}
-        },
-        %{
-          id: :cachex_user,
-          start:
-            {Cachex, :start_link,
-             [
-               :user_cache,
-               [
-                 default_ttl: 25_000,
-                 ttl_interval: 1000,
-                 limit: 2500
-               ]
-             ]}
-        },
-        %{
-          id: :cachex_object,
-          start:
-            {Cachex, :start_link,
-             [
-               :object_cache,
-               [
-                 default_ttl: 25_000,
-                 ttl_interval: 1000,
-                 limit: 2500
-               ]
-             ]}
-        },
-        %{
-          id: :cachex_rich_media,
-          start:
-            {Cachex, :start_link,
-             [
-               :rich_media_cache,
-               [
-                 default_ttl: :timer.minutes(120),
-                 limit: 5000
-               ]
-             ]}
-        },
-        %{
-          id: :cachex_scrubber,
-          start:
-            {Cachex, :start_link,
-             [
-               :scrubber_cache,
-               [
-                 limit: 2500
-               ]
-             ]}
-        },
-        %{
-          id: :cachex_idem,
-          start:
-            {Cachex, :start_link,
-             [
-               :idempotency_cache,
-               [
-                 expiration:
-                   expiration(
-                     default: :timer.seconds(6 * 60 * 60),
-                     interval: :timer.seconds(60)
-                   ),
-                 limit: 2500
-               ]
-             ]}
-        },
-        %{id: Pleroma.FlakeId, start: {Pleroma.FlakeId, :start_link, []}},
-        %{
-          id: Pleroma.ScheduledActivityWorker,
-          start: {Pleroma.ScheduledActivityWorker, :start_link, []}
-        }
       ] ++
+        cachex_children() ++
         hackney_pool_children() ++
         [
-          %{
+          Pleroma.Web.Federator.RetryQueue,
-            id: Pleroma.Web.Federator.RetryQueue,
+          Pleroma.Stats,
-            start: {Pleroma.Web.Federator.RetryQueue, :start_link, []}
-          },
-          %{
-            id: Pleroma.Web.OAuth.Token.CleanWorker,
-            start: {Pleroma.Web.OAuth.Token.CleanWorker, :start_link, []}
-          },
-          %{
-            id: Pleroma.Stats,
-            start: {Pleroma.Stats, :start_link, []}
-          },
           %{
             id: :web_push_init,
             start: {Task, :start_link, [&Pleroma.Web.Push.init/0]},
@@ -147,16 +58,12 @@ def start(_type, _args) do
             restart: :temporary
           }
         ] ++
-        streamer_child() ++
+        oauth_cleanup_child(oauth_cleanup_enabled?()) ++
-        chat_child() ++
+        streamer_child(@env) ++
+        chat_child(@env, chat_enabled?()) ++
         [
-          # Start the endpoint when the application starts
+          Pleroma.Web.Endpoint,
-          %{
+          Pleroma.Gopher.Server
-            id: Pleroma.Web.Endpoint,
-            start: {Pleroma.Web.Endpoint, :start_link, []},
-            type: :supervisor
-          },
-          %{id: Pleroma.Gopher.Server, start: {Pleroma.Gopher.Server, :start_link, []}}
         ]
 
     # See http://elixir-lang.org/docs/stable/elixir/Supervisor.html
@@ -201,28 +108,54 @@ def enabled_hackney_pools do
       end
   end
 
-  if Pleroma.Config.get(:env) == :test do
+  defp cachex_children do
-    defp streamer_child, do: []
+    [
-    defp chat_child, do: []
+      build_cachex("used_captcha", ttl_interval: seconds_valid_interval()),
-  else
+      build_cachex("user", default_ttl: 25_000, ttl_interval: 1000, limit: 2500),
-    defp streamer_child do
+      build_cachex("object", default_ttl: 25_000, ttl_interval: 1000, limit: 2500),
-      [%{id: Pleroma.Web.Streamer, start: {Pleroma.Web.Streamer, :start_link, []}}]
+      build_cachex("rich_media", default_ttl: :timer.minutes(120), limit: 5000),
-    end
+      build_cachex("scrubber", limit: 2500),
-
+      build_cachex("idempotency", expiration: idempotency_expiration(), limit: 2500)
-    defp chat_child do
+    ]
-      if Pleroma.Config.get([:chat, :enabled]) do
-        [
-          %{
-            id: Pleroma.Web.ChatChannel.ChatChannelState,
-            start: {Pleroma.Web.ChatChannel.ChatChannelState, :start_link, []}
-          }
-        ]
-      else
-        []
-      end
-    end
   end
 
+  defp idempotency_expiration,
+    do: expiration(default: :timer.seconds(6 * 60 * 60), interval: :timer.seconds(60))
+
+  defp seconds_valid_interval,
+    do: :timer.seconds(Pleroma.Config.get!([Pleroma.Captcha, :seconds_valid]))
+
+  defp build_cachex(type, opts),
+    do: %{
+      id: String.to_atom("cachex_" <> type),
+      start: {Cachex, :start_link, [String.to_atom(type <> "_cache"), opts]},
+      type: :worker
+    }
+
+  defp chat_enabled?, do: Pleroma.Config.get([:chat, :enabled])
+
+  defp oauth_cleanup_enabled?,
+    do: Pleroma.Config.get([:oauth2, :clean_expired_tokens], false)
+
+  defp streamer_child(:test), do: []
+
+  defp streamer_child(_) do
+    [Pleroma.Web.Streamer]
+  end
+
+  defp oauth_cleanup_child(true),
+    do: [Pleroma.Web.OAuth.Token.CleanWorker]
+
+  defp oauth_cleanup_child(_), do: []
+
+  defp chat_child(:test, _), do: []
+
+  defp chat_child(_env, true) do
+    [Pleroma.Web.ChatChannel.ChatChannelState]
+  end
+
+  defp chat_child(_, _), do: []
+
   defp hackney_pool_children do
     for pool <- enabled_hackney_pools() do
       options = Pleroma.Config.get([:hackney_pools, pool])

lib/pleroma/captcha/captcha.ex

@@ -12,7 +12,7 @@ defmodule Pleroma.Captcha do
   use GenServer
 
   @doc false
-  def start_link do
+  def start_link(_) do
     GenServer.start_link(__MODULE__, [], name: __MODULE__)
   end
 

lib/pleroma/config/transfer_task.ex

@@ -6,7 +6,7 @@ defmodule Pleroma.Config.TransferTask do
   use Task
   alias Pleroma.Web.AdminAPI.Config
 
-  def start_link do
+  def start_link(_) do
     load_and_update_env()
     if Pleroma.Config.get(:env) == :test, do: Ecto.Adapters.SQL.Sandbox.checkin(Pleroma.Repo)
     :ignore

lib/pleroma/emoji.ex

@@ -24,7 +24,7 @@ defmodule Pleroma.Emoji do
   @ets_options [:ordered_set, :protected, :named_table, {:read_concurrency, true}]
 
   @doc false
-  def start_link do
+  def start_link(_) do
     GenServer.start_link(__MODULE__, [], name: __MODULE__)
   end
 

lib/pleroma/flake_id.ex

@@ -98,7 +98,7 @@ def dump(value) do
   def autogenerate, do: get()
 
   # -- GenServer API
-  def start_link do
+  def start_link(_) do
     :gen_server.start_link({:local, :flake}, __MODULE__, [], [])
   end
 

lib/pleroma/gopher/server.ex

@@ -6,7 +6,7 @@ defmodule Pleroma.Gopher.Server do
   use GenServer
   require Logger
 
-  def start_link do
+  def start_link(_) do
     config = Pleroma.Config.get(:gopher, [])
     ip = Keyword.get(config, :ip, {0, 0, 0, 0})
     port = Keyword.get(config, :port, 1234)

lib/pleroma/html.ex

@@ -203,6 +203,8 @@ defmodule Pleroma.HTML.Scrubber.Default do
   Meta.allow_tag_with_these_attributes("p", [])
   Meta.allow_tag_with_these_attributes("pre", [])
   Meta.allow_tag_with_these_attributes("strong", [])
+  Meta.allow_tag_with_these_attributes("sub", [])
+  Meta.allow_tag_with_these_attributes("sup", [])
   Meta.allow_tag_with_these_attributes("u", [])
   Meta.allow_tag_with_these_attributes("ul", [])
 

lib/pleroma/scheduled_activity_worker.ex

@@ -16,7 +16,7 @@ defmodule Pleroma.ScheduledActivityWorker do
 
   @schedule_interval :timer.minutes(1)
 
-  def start_link do
+  def start_link(_) do
     GenServer.start_link(__MODULE__, nil)
   end
 

lib/pleroma/stats.ex

@@ -7,31 +7,56 @@ defmodule Pleroma.Stats do
   alias Pleroma.Repo
   alias Pleroma.User
 
-  def start_link do
+  use GenServer
-    agent = Agent.start_link(fn -> {[], %{}} end, name: __MODULE__)
+
-    spawn(fn -> schedule_update() end)
+  @interval 1000 * 60 * 60
-    agent
+
+  def start_link(_) do
+    GenServer.start_link(__MODULE__, initial_data(), name: __MODULE__)
+  end
+
+  def force_update do
+    GenServer.call(__MODULE__, :force_update)
   end
 
   def get_stats do
-    Agent.get(__MODULE__, fn {_, stats} -> stats end)
+    %{stats: stats} = GenServer.call(__MODULE__, :get_state)
+
+    stats
   end
 
   def get_peers do
-    Agent.get(__MODULE__, fn {peers, _} -> peers end)
+    %{peers: peers} = GenServer.call(__MODULE__, :get_state)
+
+    peers
   end
 
-  def schedule_update do
+  def init(args) do
-    spawn(fn ->
+    Process.send(self(), :run_update, [])
-      # 1 hour
+    {:ok, args}
-      Process.sleep(1000 * 60 * 60)
-      schedule_update()
-    end)
-
-    update_stats()
   end
 
-  def update_stats do
+  def handle_call(:force_update, _from, _state) do
+    new_stats = get_stat_data()
+    {:reply, new_stats, new_stats}
+  end
+
+  def handle_call(:get_state, _from, state) do
+    {:reply, state, state}
+  end
+
+  def handle_info(:run_update, _state) do
+    new_stats = get_stat_data()
+
+    Process.send_after(self(), :run_update, @interval)
+    {:noreply, new_stats}
+  end
+
+  defp initial_data do
+    %{peers: [], stats: %{}}
+  end
+
+  defp get_stat_data do
     peers =
       from(
         u in User,
@@ -52,8 +77,9 @@ def update_stats do
 
     user_count = Repo.aggregate(User.Query.build(%{local: true, active: true}), :count, :id)
 
-    Agent.update(__MODULE__, fn _ ->
+    %{
-      {peers, %{domain_count: domain_count, status_count: status_count, user_count: user_count}}
+      peers: peers,
-    end)
+      stats: %{domain_count: domain_count, status_count: status_count, user_count: user_count}
+    }
   end
 end

lib/pleroma/user.ex

@@ -132,6 +132,28 @@ def user_info(%User{} = user, args \\ %{}) do
     |> Map.put(:follower_count, follower_count)
   end
 
+  def follow_state(%User{} = user, %User{} = target) do
+    follow_activity = Utils.fetch_latest_follow(user, target)
+
+    if follow_activity,
+      do: follow_activity.data["state"],
+      # Ideally this would be nil, but then Cachex does not commit the value
+      else: false
+  end
+
+  def get_cached_follow_state(user, target) do
+    key = "follow_state:#{user.ap_id}|#{target.ap_id}"
+    Cachex.fetch!(:user_cache, key, fn _ -> {:commit, follow_state(user, target)} end)
+  end
+
+  def set_follow_state_cache(user_ap_id, target_ap_id, state) do
+    Cachex.put(
+      :user_cache,
+      "follow_state:#{user_ap_id}|#{target_ap_id}",
+      state
+    )
+  end
+
   def set_info_cache(user, args) do
     Cachex.put(:user_cache, "user_info:#{user.id}", user_info(user, args))
   end
@@ -152,10 +174,10 @@ def following_count(%User{} = user) do
   end
 
   def remote_user_creation(params) do
-    params =
+    bio_limit = Pleroma.Config.get([:instance, :user_bio_length], 5000)
-      params
+    name_limit = Pleroma.Config.get([:instance, :user_name_length], 100)
-      |> Map.put(:info, params[:info] || %{})
 
+    params = Map.put(params, :info, params[:info] || %{})
     info_cng = User.Info.remote_user_creation(%User.Info{}, params[:info])
 
     changes =
@@ -164,8 +186,8 @@ def remote_user_creation(params) do
       |> validate_required([:name, :ap_id])
       |> unique_constraint(:nickname)
       |> validate_format(:nickname, @email_regex)
-      |> validate_length(:bio, max: 5000)
+      |> validate_length(:bio, max: bio_limit)
-      |> validate_length(:name, max: 100)
+      |> validate_length(:name, max: name_limit)
       |> put_change(:local, false)
       |> put_embed(:info, info_cng)
 
@@ -188,22 +210,23 @@ def remote_user_creation(params) do
   end
 
   def update_changeset(struct, params \\ %{}) do
+    bio_limit = Pleroma.Config.get([:instance, :user_bio_length], 5000)
+    name_limit = Pleroma.Config.get([:instance, :user_name_length], 100)
+
     struct
     |> cast(params, [:bio, :name, :avatar, :following])
     |> unique_constraint(:nickname)
     |> validate_format(:nickname, local_nickname_regex())
-    |> validate_length(:bio, max: 5000)
+    |> validate_length(:bio, max: bio_limit)
-    |> validate_length(:name, min: 1, max: 100)
+    |> validate_length(:name, min: 1, max: name_limit)
   end
 
   def upgrade_changeset(struct, params \\ %{}) do
-    params =
+    bio_limit = Pleroma.Config.get([:instance, :user_bio_length], 5000)
-      params
+    name_limit = Pleroma.Config.get([:instance, :user_name_length], 100)
-      |> Map.put(:last_refreshed_at, NaiveDateTime.utc_now())
 
-    info_cng =
+    params = Map.put(params, :last_refreshed_at, NaiveDateTime.utc_now())
-      struct.info
+    info_cng = User.Info.user_upgrade(struct.info, params[:info])
-      |> User.Info.user_upgrade(params[:info])
 
     struct
     |> cast(params, [
@@ -216,8 +239,8 @@ def upgrade_changeset(struct, params \\ %{}) do
     ])
     |> unique_constraint(:nickname)
     |> validate_format(:nickname, local_nickname_regex())
-    |> validate_length(:bio, max: 5000)
+    |> validate_length(:bio, max: bio_limit)
-    |> validate_length(:name, max: 100)
+    |> validate_length(:name, max: name_limit)
     |> put_embed(:info, info_cng)
   end
 
@@ -244,6 +267,9 @@ def reset_password(%User{id: user_id} = user, data) do
   end
 
   def register_changeset(struct, params \\ %{}, opts \\ []) do
+    bio_limit = Pleroma.Config.get([:instance, :user_bio_length], 5000)
+    name_limit = Pleroma.Config.get([:instance, :user_name_length], 100)
+
     need_confirmation? =
       if is_nil(opts[:need_confirmation]) do
         Pleroma.Config.get([:instance, :account_activation_required])
@@ -264,8 +290,8 @@ def register_changeset(struct, params \\ %{}, opts \\ []) do
       |> validate_exclusion(:nickname, Pleroma.Config.get([User, :restricted_nicknames]))
       |> validate_format(:nickname, local_nickname_regex())
       |> validate_format(:email, @email_regex)
-      |> validate_length(:bio, max: 1000)
+      |> validate_length(:bio, max: bio_limit)
-      |> validate_length(:name, min: 1, max: 100)
+      |> validate_length(:name, min: 1, max: name_limit)
       |> put_change(:info, info_change)
 
     changeset =

lib/pleroma/web/activity_pub/activity_pub.ex

@@ -388,7 +388,8 @@ def unannounce(
   def follow(follower, followed, activity_id \\ nil, local \\ true) do
     with data <- make_follow_data(follower, followed, activity_id),
          {:ok, activity} <- insert(data, local),
-         :ok <- maybe_federate(activity) do
+         :ok <- maybe_federate(activity),
+         _ <- User.set_follow_state_cache(follower.ap_id, followed.ap_id, activity.data["state"]) do
       {:ok, activity}
     end
   end
@@ -518,6 +519,8 @@ defp fetch_activities_for_context_query(context, opts) do
 
     from(activity in Activity)
     |> maybe_preload_objects(opts)
+    |> maybe_preload_bookmarks(opts)
+    |> maybe_set_thread_muted_field(opts)
     |> restrict_blocked(opts)
     |> restrict_recipients(recipients, opts["user"])
     |> where(
@@ -531,6 +534,7 @@ defp fetch_activities_for_context_query(context, opts) do
       )
     )
     |> exclude_poll_votes(opts)
+    |> exclude_id(opts)
     |> order_by([activity], desc: activity.id)
   end
 
@@ -623,6 +627,7 @@ def fetch_user_activities(user, reading_user, params \\ %{}) do
     params =
       params
       |> Map.put("type", ["Create", "Announce"])
+      |> Map.put("user", reading_user)
       |> Map.put("actor_id", user.ap_id)
       |> Map.put("whole_db", true)
       |> Map.put("pinned_activity_ids", user.info.pinned_activities)
@@ -870,6 +875,12 @@ defp exclude_poll_votes(query, _) do
     end
   end
 
+  defp exclude_id(query, %{"exclude_id" => id}) when is_binary(id) do
+    from(activity in query, where: activity.id != ^id)
+  end
+
+  defp exclude_id(query, _), do: query
+
   defp maybe_preload_objects(query, %{"skip_preload" => true}), do: query
 
   defp maybe_preload_objects(query, _) do

lib/pleroma/web/activity_pub/mrf.ex

@@ -35,4 +35,36 @@ def subdomains_regex(domains) when is_list(domains) do
   def subdomain_match?(domains, host) do
     Enum.any?(domains, fn domain -> Regex.match?(domain, host) end)
   end
+
+  @callback describe() :: {:ok | :error, Map.t()}
+
+  def describe(policies) do
+    {:ok, policy_configs} =
+      policies
+      |> Enum.reduce({:ok, %{}}, fn
+        policy, {:ok, data} ->
+          {:ok, policy_data} = policy.describe()
+          {:ok, Map.merge(data, policy_data)}
+
+        _, error ->
+          error
+      end)
+
+    mrf_policies =
+      get_policies()
+      |> Enum.map(fn policy -> to_string(policy) |> String.split(".") |> List.last() end)
+
+    exclusions = Pleroma.Config.get([:instance, :mrf_transparency_exclusions])
+
+    base =
+      %{
+        mrf_policies: mrf_policies,
+        exclusions: length(exclusions) > 0
+      }
+      |> Map.merge(policy_configs)
+
+    {:ok, base}
+  end
+
+  def describe, do: get_policies() |> describe()
 end

lib/pleroma/web/activity_pub/mrf/anti_followbot_policy.ex

@@ -62,4 +62,7 @@ def filter(%{"type" => "Follow", "actor" => actor_id} = message) do
 
   @impl true
   def filter(message), do: {:ok, message}
+
+  @impl true
+  def describe, do: {:ok, %{}}
 end

lib/pleroma/web/activity_pub/mrf/anti_link_spam_policy.ex

@@ -5,6 +5,8 @@
 defmodule Pleroma.Web.ActivityPub.MRF.AntiLinkSpamPolicy do
   alias Pleroma.User
 
+  @behaviour Pleroma.Web.ActivityPub.MRF
+
   require Logger
 
   # has the user successfully posted before?
@@ -22,6 +24,7 @@ defp contains_links?(%{"content" => content} = _object) do
 
   defp contains_links?(_), do: false
 
+  @impl true
   def filter(%{"type" => "Create", "actor" => actor, "object" => object} = message) do
     with {:ok, %User{} = u} <- User.get_or_fetch_by_ap_id(actor),
          {:contains_links, true} <- {:contains_links, contains_links?(object)},
@@ -45,4 +48,7 @@ def filter(%{"type" => "Create", "actor" => actor, "object" => object} = message
 
   # in all other cases, pass through
   def filter(message), do: {:ok, message}
+
+  @impl true
+  def describe, do: {:ok, %{}}
 end

lib/pleroma/web/activity_pub/mrf/drop_policy.ex

@@ -12,4 +12,7 @@ def filter(object) do
     Logger.info("REJECTING #{inspect(object)}")
     {:reject, object}
   end
+
+  @impl true
+  def describe, do: {:ok, %{}}
 end

lib/pleroma/web/activity_pub/mrf/ensure_re_prepended.ex

@@ -39,4 +39,6 @@ def filter(%{"type" => "Create", "object" => child_object} = object) do
   end
 
   def filter(object), do: {:ok, object}
+
+  def describe, do: {:ok, %{}}
 end

lib/pleroma/web/activity_pub/mrf/hellthread_policy.ex

@@ -90,4 +90,8 @@ def filter(%{"type" => "Create"} = message) do
 
   @impl true
   def filter(message), do: {:ok, message}
+
+  @impl true
+  def describe,
+    do: {:ok, %{mrf_hellthread: Pleroma.Config.get(:mrf_hellthread) |> Enum.into(%{})}}
 end

lib/pleroma/web/activity_pub/mrf/keyword_policy.ex

@@ -96,4 +96,36 @@ def filter(%{"type" => "Create", "object" => %{"content" => _content}} = message
 
   @impl true
   def filter(message), do: {:ok, message}
+
+  @impl true
+  def describe do
+    # This horror is needed to convert regex sigils to strings
+    mrf_keyword =
+      Pleroma.Config.get(:mrf_keyword, [])
+      |> Enum.map(fn {key, value} ->
+        {key,
+         Enum.map(value, fn
+           {pattern, replacement} ->
+             %{
+               "pattern" =>
+                 if not is_binary(pattern) do
+                   inspect(pattern)
+                 else
+                   pattern
+                 end,
+               "replacement" => replacement
+             }
+
+           pattern ->
+             if not is_binary(pattern) do
+               inspect(pattern)
+             else
+               pattern
+             end
+         end)}
+      end)
+      |> Enum.into(%{})
+
+    {:ok, %{mrf_keyword: mrf_keyword}}
+  end
 end

lib/pleroma/web/activity_pub/mrf/mediaproxy_warming_policy.ex

@@ -53,4 +53,7 @@ def filter(
 
   @impl true
   def filter(message), do: {:ok, message}
+
+  @impl true
+  def describe, do: {:ok, %{}}
 end

lib/pleroma/web/activity_pub/mrf/mention_policy.ex

@@ -21,4 +21,7 @@ def filter(%{"type" => "Create"} = message) do
 
   @impl true
   def filter(message), do: {:ok, message}
+
+  @impl true
+  def describe, do: {:ok, %{}}
 end

lib/pleroma/web/activity_pub/mrf/no_placeholder_text_policy.ex

@@ -19,4 +19,7 @@ def filter(
 
   @impl true
   def filter(object), do: {:ok, object}
+
+  @impl true
+  def describe, do: {:ok, %{}}
 end

lib/pleroma/web/activity_pub/mrf/noop_policy.ex

@@ -10,4 +10,7 @@ defmodule Pleroma.Web.ActivityPub.MRF.NoOpPolicy do
   def filter(object) do
     {:ok, object}
   end
+
+  @impl true
+  def describe, do: {:ok, %{}}
 end

lib/pleroma/web/activity_pub/mrf/normalize_markup.ex

@@ -21,4 +21,6 @@ def filter(%{"type" => "Create", "object" => child_object} = object) do
   end
 
   def filter(object), do: {:ok, object}
+
+  def describe, do: {:ok, %{}}
 end

lib/pleroma/web/activity_pub/mrf/reject_non_public.ex

@@ -44,4 +44,8 @@ def filter(%{"type" => "Create"} = object) do
 
   @impl true
   def filter(object), do: {:ok, object}
+
+  @impl true
+  def describe,
+    do: {:ok, %{mrf_rejectnonpublic: Pleroma.Config.get(:mrf_rejectnonpublic) |> Enum.into(%{})}}
 end

lib/pleroma/web/activity_pub/mrf/simple_policy.ex

@@ -177,4 +177,16 @@ def filter(%{"id" => actor, "type" => obj_type} = object)
   end
 
   def filter(object), do: {:ok, object}
+
+  @impl true
+  def describe do
+    exclusions = Pleroma.Config.get([:instance, :mrf_transparency_exclusions])
+
+    mrf_simple =
+      Pleroma.Config.get(:mrf_simple)
+      |> Enum.map(fn {k, v} -> {k, Enum.reject(v, fn v -> v in exclusions end)} end)
+      |> Enum.into(%{})
+
+    {:ok, %{mrf_simple: mrf_simple}}
+  end
 end

lib/pleroma/web/activity_pub/mrf/subchain_policy.ex

@@ -37,4 +37,7 @@ def filter(%{"actor" => actor} = message) do
 
   @impl true
   def filter(message), do: {:ok, message}
+
+  @impl true
+  def describe, do: {:ok, %{}}
 end

lib/pleroma/web/activity_pub/mrf/tag_policy.ex

@@ -165,4 +165,7 @@ def filter(%{"actor" => actor, "type" => "Create"} = message),
 
   @impl true
   def filter(message), do: {:ok, message}
+
+  @impl true
+  def describe, do: {:ok, %{}}
 end

lib/pleroma/web/activity_pub/mrf/user_allowlist_policy.ex

@@ -32,4 +32,13 @@ def filter(%{"actor" => actor} = object) do
   end
 
   def filter(object), do: {:ok, object}
+
+  @impl true
+  def describe do
+    mrf_user_allowlist =
+      Config.get([:mrf_user_allowlist], [])
+      |> Enum.into(%{}, fn {k, v} -> {k, length(v)} end)
+
+    {:ok, %{mrf_user_allowlist: mrf_user_allowlist}}
+  end
 end

lib/pleroma/web/activity_pub/mrf/vocabulary_policy.ex

@@ -0,0 +1,37 @@
+# Pleroma: A lightweight social networking server
+# Copyright © 2017-2019 Pleroma Authors <https://pleroma.social/>
+# SPDX-License-Identifier: AGPL-3.0-only
+
+defmodule Pleroma.Web.ActivityPub.MRF.VocabularyPolicy do
+  @moduledoc "Filter messages which belong to certain activity vocabularies"
+
+  @behaviour Pleroma.Web.ActivityPub.MRF
+
+  def filter(%{"type" => "Undo", "object" => child_message} = message) do
+    with {:ok, _} <- filter(child_message) do
+      {:ok, message}
+    else
+      {:reject, nil} ->
+        {:reject, nil}
+    end
+  end
+
+  def filter(%{"type" => message_type} = message) do
+    with accepted_vocabulary <- Pleroma.Config.get([:mrf_vocabulary, :accept]),
+         rejected_vocabulary <- Pleroma.Config.get([:mrf_vocabulary, :reject]),
+         true <-
+           length(accepted_vocabulary) == 0 || Enum.member?(accepted_vocabulary, message_type),
+         false <-
+           length(rejected_vocabulary) > 0 && Enum.member?(rejected_vocabulary, message_type),
+         {:ok, _} <- filter(message["object"]) do
+      {:ok, message}
+    else
+      _ -> {:reject, nil}
+    end
+  end
+
+  def filter(message), do: {:ok, message}
+
+  def describe,
+    do: {:ok, %{mrf_vocabulary: Pleroma.Config.get(:mrf_vocabulary) |> Enum.into(%{})}}
+end

lib/pleroma/web/activity_pub/publisher.ex

@@ -46,7 +46,7 @@ def is_representable?(%Activity{} = activity) do
   """
   def publish_one(%{inbox: inbox, json: json, actor: %User{} = actor, id: id} = params) do
     Logger.info("Federating #{id} to #{inbox}")
-    host = URI.parse(inbox).host
+    %{host: host, path: path} = URI.parse(inbox)
 
     digest = "SHA-256=" <> (:crypto.hash(:sha256, json) |> Base.encode64())
 
@@ -56,6 +56,7 @@ def publish_one(%{inbox: inbox, json: json, actor: %User{} = actor, id: id} = pa
 
     signature =
       Pleroma.Signature.sign(actor, %{
+        "(request-target)": "post #{path}",
         host: host,
         "content-length": byte_size(json),
         digest: digest,

lib/pleroma/web/activity_pub/relay.ex

@@ -14,6 +14,7 @@ def get_actor do
     |> User.get_or_create_service_actor_by_ap_id()
   end
 
+  @spec follow(String.t()) :: {:ok, Activity.t()} | {:error, any()}
   def follow(target_instance) do
     with %User{} = local_user <- get_actor(),
          {:ok, %User{} = target_user} <- User.get_or_fetch_by_ap_id(target_instance),
@@ -21,12 +22,17 @@ def follow(target_instance) do
       Logger.info("relay: followed instance: #{target_instance}; id=#{activity.data["id"]}")
       {:ok, activity}
     else
+      {:error, _} = error ->
+        Logger.error("error: #{inspect(error)}")
+        error
+
       e ->
         Logger.error("error: #{inspect(e)}")
         {:error, e}
     end
   end
 
+  @spec unfollow(String.t()) :: {:ok, Activity.t()} | {:error, any()}
   def unfollow(target_instance) do
     with %User{} = local_user <- get_actor(),
          {:ok, %User{} = target_user} <- User.get_or_fetch_by_ap_id(target_instance),
@@ -34,20 +40,27 @@ def unfollow(target_instance) do
       Logger.info("relay: unfollowed instance: #{target_instance}: id=#{activity.data["id"]}")
       {:ok, activity}
     else
+      {:error, _} = error ->
+        Logger.error("error: #{inspect(error)}")
+        error
+
       e ->
         Logger.error("error: #{inspect(e)}")
         {:error, e}
     end
   end
 
+  @spec publish(any()) :: {:ok, Activity.t(), Object.t()} | {:error, any()}
   def publish(%Activity{data: %{"type" => "Create"}} = activity) do
     with %User{} = user <- get_actor(),
          %Object{} = object <- Object.normalize(activity) do
       ActivityPub.announce(user, object, nil, true, false)
     else
-      e -> Logger.error("error: #{inspect(e)}")
+      e ->
+        Logger.error("error: #{inspect(e)}")
+        {:error, inspect(e)}
     end
   end
 
-  def publish(_), do: nil
+  def publish(_), do: {:error, "Not implemented"}
 end

lib/pleroma/web/activity_pub/utils.ex

@@ -374,6 +374,7 @@ def update_follow_state_for_all(
         [state, actor, object]
       )
 
+      User.set_follow_state_cache(actor, object, state)
       activity = Activity.get_by_id(activity.id)
       {:ok, activity}
     rescue
@@ -382,12 +383,16 @@ def update_follow_state_for_all(
     end
   end
 
-  def update_follow_state(%Activity{} = activity, state) do
+  def update_follow_state(
+        %Activity{data: %{"actor" => actor, "object" => object}} = activity,
+        state
+      ) do
     with new_data <-
            activity.data
            |> Map.put("state", state),
          changeset <- Changeset.change(activity, data: new_data),
-         {:ok, activity} <- Repo.update(changeset) do
+         {:ok, activity} <- Repo.update(changeset),
+         _ <- User.set_follow_state_cache(actor, object, state) do
       {:ok, activity}
     end
   end

lib/pleroma/web/chat_channel.ex

@@ -33,9 +33,11 @@ def handle_in("new_msg", %{"text" => text}, %{assigns: %{user_name: user_name}}
 end
 
 defmodule Pleroma.Web.ChatChannel.ChatChannelState do
+  use Agent
+
   @max_messages 20
 
-  def start_link do
+  def start_link(_) do
     Agent.start_link(fn -> %{max_id: 1, messages: []} end, name: __MODULE__)
   end
 

lib/pleroma/web/federator/retry_queue.ex

@@ -13,7 +13,7 @@ def init(args) do
     {:ok, %{args | queue_table: queue_table, running_jobs: :sets.new()}}
   end
 
-  def start_link do
+  def start_link(_) do
     enabled =
       if Pleroma.Config.get(:env) == :test,
         do: true,

lib/pleroma/web/mastodon_api/mastodon_api.ex

@@ -13,10 +13,8 @@ defmodule Pleroma.Web.MastodonAPI.MastodonAPI do
   alias Pleroma.User
   alias Pleroma.Web.CommonAPI
 
+  @spec follow(User.t(), User.t(), map) :: {:ok, User.t()} | {:error, String.t()}
   def follow(follower, followed, params \\ %{}) do
-    options = cast_params(params)
-    reblogs = options[:reblogs]
-
     result =
       if not User.following?(follower, followed) do
         CommonAPI.follow(follower, followed)
@@ -24,19 +22,25 @@ def follow(follower, followed, params \\ %{}) do
         {:ok, follower, followed, nil}
       end
 
-    with {:ok, follower, followed, _} <- result do
+    with {:ok, follower, _followed, _} <- result do
-      reblogs
+      options = cast_params(params)
-      |> case do
+
-        false -> CommonAPI.hide_reblogs(follower, followed)
+      case reblogs_visibility(options[:reblogs], result) do
-        _ -> CommonAPI.show_reblogs(follower, followed)
-      end
-      |> case do
         {:ok, follower} -> {:ok, follower}
         _ -> {:ok, follower}
       end
     end
   end
 
+  defp reblogs_visibility(false, {:ok, follower, followed, _}) do
+    CommonAPI.hide_reblogs(follower, followed)
+  end
+
+  defp reblogs_visibility(_, {:ok, follower, followed, _}) do
+    CommonAPI.show_reblogs(follower, followed)
+  end
+
+  @spec get_followers(User.t(), map()) :: list(User.t())
   def get_followers(user, params \\ %{}) do
     user
     |> User.get_followers_query()

lib/pleroma/web/mastodon_api/mastodon_api_controller.ex

@@ -435,6 +435,7 @@ def public_timeline(%{assigns: %{user: user}} = conn, params) do
       |> Map.put("local_only", local_only)
       |> Map.put("blocking_user", user)
       |> Map.put("muting_user", user)
+      |> Map.put("user", user)
       |> ActivityPub.fetch_public_activities()
       |> Enum.reverse()
 
@@ -496,12 +497,9 @@ def get_context(%{assigns: %{user: user}} = conn, %{"id" => id}) do
          activities <-
            ActivityPub.fetch_activities_for_context(activity.data["context"], %{
              "blocking_user" => user,
-             "user" => user
+             "user" => user,
+             "exclude_id" => activity.id
            }),
-         activities <-
-           activities |> Enum.filter(fn %{id: aid} -> to_string(aid) != to_string(id) end),
-         activities <-
-           activities |> Enum.filter(fn %{data: %{"type" => type}} -> type == "Create" end),
          grouped_activities <- Enum.group_by(activities, fn %{id: id} -> id < activity.id end) do
       result = %{
         ancestors:
@@ -536,8 +534,8 @@ def get_poll(%{assigns: %{user: user}} = conn, %{"id" => id}) do
       |> put_view(StatusView)
       |> try_render("poll.json", %{object: object, for: user})
     else
-      nil -> render_error(conn, :not_found, "Record not found")
+      error when is_nil(error) or error == false ->
-      false -> render_error(conn, :not_found, "Record not found")
+        render_error(conn, :not_found, "Record not found")
     end
   end
 
@@ -885,8 +883,8 @@ def get_mascot(%{assigns: %{user: user}} = conn, _params) do
   end
 
   def favourited_by(%{assigns: %{user: user}} = conn, %{"id" => id}) do
-    with %Activity{data: %{"object" => object}} <- Activity.get_by_id(id),
+    with %Activity{} = activity <- Activity.get_by_id_with_object(id),
-         %Object{data: %{"likes" => likes}} <- Object.normalize(object) do
+         %Object{data: %{"likes" => likes}} <- Object.normalize(activity) do
       q = from(u in User, where: u.ap_id in ^likes)
 
       users =
@@ -902,8 +900,8 @@ def favourited_by(%{assigns: %{user: user}} = conn, %{"id" => id}) do
   end
 
   def reblogged_by(%{assigns: %{user: user}} = conn, %{"id" => id}) do
-    with %Activity{data: %{"object" => object}} <- Activity.get_by_id(id),
+    with %Activity{} = activity <- Activity.get_by_id_with_object(id),
-         %Object{data: %{"announcements" => announces}} <- Object.normalize(object) do
+         %Object{data: %{"announcements" => announces}} <- Object.normalize(activity) do
       q = from(u in User, where: u.ap_id in ^announces)
 
       users =
@@ -944,6 +942,7 @@ def hashtag_timeline(%{assigns: %{user: user}} = conn, params) do
       |> Map.put("local_only", local_only)
       |> Map.put("blocking_user", user)
       |> Map.put("muting_user", user)
+      |> Map.put("user", user)
       |> Map.put("tag", tags)
       |> Map.put("tag_all", tag_all)
       |> Map.put("tag_reject", tag_reject)
@@ -1350,6 +1349,7 @@ def list_timeline(%{assigns: %{user: user}} = conn, %{"list_id" => id} = params)
         params
         |> Map.put("type", "Create")
         |> Map.put("blocking_user", user)
+        |> Map.put("user", user)
         |> Map.put("muting_user", user)
 
       # we must filter the following list for the user to avoid leaking statuses the user
@@ -1690,45 +1690,35 @@ def suggestions(%{assigns: %{user: user}} = conn, _) do
         |> String.replace("{{user}}", user)
 
       with {:ok, %{status: 200, body: body}} <-
-             HTTP.get(
+             HTTP.get(url, [], adapter: [recv_timeout: timeout, pool: :default]),
-               url,
-               [],
-               adapter: [
-                 recv_timeout: timeout,
-                 pool: :default
-               ]
-             ),
            {:ok, data} <- Jason.decode(body) do
         data =
           data
           |> Enum.slice(0, limit)
           |> Enum.map(fn x ->
-            Map.put(
+            x
-              x,
+            |> Map.put("id", fetch_suggestion_id(x))
-              "id",
+            |> Map.put("avatar", MediaProxy.url(x["avatar"]))
-              case User.get_or_fetch(x["acct"]) do
+            |> Map.put("avatar_static", MediaProxy.url(x["avatar_static"]))
-                {:ok, %User{id: id}} -> id
-                _ -> 0
-              end
-            )
-          end)
-          |> Enum.map(fn x ->
-            Map.put(x, "avatar", MediaProxy.url(x["avatar"]))
-          end)
-          |> Enum.map(fn x ->
-            Map.put(x, "avatar_static", MediaProxy.url(x["avatar_static"]))
           end)
 
-        conn
+        json(conn, data)
-        |> json(data)
       else
-        e -> Logger.error("Could not retrieve suggestions at fetch #{url}, #{inspect(e)}")
+        e ->
+          Logger.error("Could not retrieve suggestions at fetch #{url}, #{inspect(e)}")
       end
     else
       json(conn, [])
     end
   end
 
+  defp fetch_suggestion_id(attrs) do
+    case User.get_or_fetch(attrs["acct"]) do
+      {:ok, %User{id: id}} -> id
+      _ -> 0
+    end
+  end
+
   def status_card(%{assigns: %{user: user}} = conn, %{"id" => status_id}) do
     with %Activity{} = activity <- Activity.get_by_id(status_id),
          true <- Visibility.visible_for_user?(activity, user) do

lib/pleroma/web/mastodon_api/views/account_view.ex

@@ -37,11 +37,11 @@ def render("relationship.json", %{user: nil, target: _target}) do
   end
 
   def render("relationship.json", %{user: %User{} = user, target: %User{} = target}) do
-    follow_activity = Pleroma.Web.ActivityPub.Utils.fetch_latest_follow(user, target)
+    follow_state = User.get_cached_follow_state(user, target)
 
     requested =
-      if follow_activity && !User.following?(target, user) do
+      if follow_state && !User.following?(user, target) do
-        follow_activity.data["state"] == "pending"
+        follow_state == "pending"
       else
         false
       end

lib/pleroma/web/mastodon_api/views/status_view.ex

@@ -5,6 +5,8 @@
 defmodule Pleroma.Web.MastodonAPI.StatusView do
   use Pleroma.Web, :view
 
+  require Pleroma.Constants
+
   alias Pleroma.Activity
   alias Pleroma.HTML
   alias Pleroma.Object
@@ -24,19 +26,19 @@ defp get_replied_to_activities([]), do: %{}
   defp get_replied_to_activities(activities) do
     activities
     |> Enum.map(fn
-      %{data: %{"type" => "Create", "object" => object}} ->
+      %{data: %{"type" => "Create"}} = activity ->
-        object = Object.normalize(object)
+        object = Object.normalize(activity)
-        object.data["inReplyTo"] != "" && object.data["inReplyTo"]
+        object && object.data["inReplyTo"] != "" && object.data["inReplyTo"]
 
       _ ->
         nil
     end)
     |> Enum.filter(& &1)
-    |> Activity.create_by_object_ap_id()
+    |> Activity.create_by_object_ap_id_with_object()
     |> Repo.all()
     |> Enum.reduce(%{}, fn activity, acc ->
       object = Object.normalize(activity)
-      Map.put(acc, object.data["id"], activity)
+      if object, do: Map.put(acc, object.data["id"], activity), else: acc
     end)
   end
 
@@ -68,12 +70,14 @@ defp reblogged?(activity, user) do
 
   def render("index.json", opts) do
     replied_to_activities = get_replied_to_activities(opts.activities)
+    parallel = unless is_nil(opts[:parallel]), do: opts[:parallel], else: true
 
     opts.activities
     |> safe_render_many(
       StatusView,
       "status.json",
-      Map.put(opts, :replied_to_activities, replied_to_activities)
+      Map.put(opts, :replied_to_activities, replied_to_activities),
+      parallel
     )
   end
 
@@ -88,6 +92,7 @@ def render(
     reblogged_activity =
       Activity.create_by_object_ap_id(activity_object.data["id"])
       |> Activity.with_preloaded_bookmark(opts[:for])
+      |> Activity.with_set_thread_muted_field(opts[:for])
       |> Repo.one()
 
     reblogged = render("status.json", Map.put(opts, :activity, reblogged_activity))
@@ -142,6 +147,7 @@ def render("status.json", %{activity: %{data: %{"object" => _object}} = activity
     object = Object.normalize(activity)
 
     user = get_user(activity.data["actor"])
+    user_follower_address = user.follower_address
 
     like_count = object.data["like_count"] || 0
     announcement_count = object.data["announcement_count"] || 0
@@ -157,7 +163,11 @@ def render("status.json", %{activity: %{data: %{"object" => _object}} = activity
     mentions =
       (object.data["to"] ++ tag_mentions)
       |> Enum.uniq()
-      |> Enum.map(fn ap_id -> User.get_cached_by_ap_id(ap_id) end)
+      |> Enum.map(fn
+        Pleroma.Constants.as_public() -> nil
+        ^user_follower_address -> nil
+        ap_id -> User.get_cached_by_ap_id(ap_id)
+      end)
       |> Enum.filter(& &1)
       |> Enum.map(fn user -> AccountView.render("mention.json", %{user: user}) end)
 

lib/pleroma/web/nodeinfo/nodeinfo_controller.ex

@@ -34,64 +34,18 @@ def schemas(conn, _params) do
   def raw_nodeinfo do
     stats = Stats.get_stats()
 
-    exclusions = Config.get([:instance, :mrf_transparency_exclusions])
-
-    mrf_simple =
-      Config.get(:mrf_simple)
-      |> Enum.map(fn {k, v} -> {k, Enum.reject(v, fn v -> v in exclusions end)} end)
-      |> Enum.into(%{})
-
-    # This horror is needed to convert regex sigils to strings
-    mrf_keyword =
-      Config.get(:mrf_keyword, [])
-      |> Enum.map(fn {key, value} ->
-        {key,
-         Enum.map(value, fn
-           {pattern, replacement} ->
-             %{
-               "pattern" =>
-                 if not is_binary(pattern) do
-                   inspect(pattern)
-                 else
-                   pattern
-                 end,
-               "replacement" => replacement
-             }
-
-           pattern ->
-             if not is_binary(pattern) do
-               inspect(pattern)
-             else
-               pattern
-             end
-         end)}
-      end)
-      |> Enum.into(%{})
-
-    mrf_policies =
-      MRF.get_policies()
-      |> Enum.map(fn policy -> to_string(policy) |> String.split(".") |> List.last() end)
-
     quarantined = Config.get([:instance, :quarantined_instances], [])
 
     staff_accounts =
       User.all_superusers()
       |> Enum.map(fn u -> u.ap_id end)
 
-    mrf_user_allowlist =
-      Config.get([:mrf_user_allowlist], [])
-      |> Enum.into(%{}, fn {k, v} -> {k, length(v)} end)
-
     federation_response =
       if Config.get([:instance, :mrf_transparency]) do
-        %{
+        {:ok, data} = MRF.describe()
-          mrf_policies: mrf_policies,
+
-          mrf_simple: mrf_simple,
+        data
-          mrf_keyword: mrf_keyword,
+        |> Map.merge(%{quarantined_instances: quarantined})
-          mrf_user_allowlist: mrf_user_allowlist,
-          quarantined_instances: quarantined,
-          exclusions: length(exclusions) > 0
-        }
       else
         %{}
       end

lib/pleroma/web/oauth/token/clean_worker.ex

@@ -6,36 +6,30 @@ defmodule Pleroma.Web.OAuth.Token.CleanWorker do
   @moduledoc """
   The module represents functions to clean an expired oauth tokens.
   """
+  use GenServer
+
+  @ten_seconds 10_000
+  @one_day 86_400_000
 
-  # 10 seconds
-  @start_interval 10_000
   @interval Pleroma.Config.get(
-              # 24 hours
               [:oauth2, :clean_expired_tokens_interval],
-              86_400_000
+              @one_day
             )
-  @queue :background
 
   alias Pleroma.Web.OAuth.Token
 
-  def start_link, do: GenServer.start_link(__MODULE__, nil)
+  def start_link(_), do: GenServer.start_link(__MODULE__, %{})
 
   def init(_) do
-    if Pleroma.Config.get([:oauth2, :clean_expired_tokens], false) do
+    Process.send_after(self(), :perform, @ten_seconds)
-      Process.send_after(self(), :perform, @start_interval)
+    {:ok, nil}
-      {:ok, nil}
-    else
-      :ignore
-    end
   end
 
   @doc false
   def handle_info(:perform, state) do
+    Token.delete_expired_tokens()
+
     Process.send_after(self(), :perform, @interval)
-    PleromaJobQueue.enqueue(@queue, __MODULE__, [:clean])
     {:noreply, state}
   end
-
-  # Job Worker Callbacks
-  def perform(:clean), do: Token.delete_expired_tokens()
 end

lib/pleroma/web/streamer.ex

@@ -18,7 +18,7 @@ defmodule Pleroma.Web.Streamer do
 
   @keepalive_interval :timer.seconds(30)
 
-  def start_link do
+  def start_link(_) do
     GenServer.start_link(__MODULE__, %{}, name: __MODULE__)
   end
 
@@ -35,28 +35,21 @@ def stream(topic, item) do
   end
 
   def init(args) do
-    spawn(fn ->
+    Process.send_after(self(), %{action: :ping}, @keepalive_interval)
-      # 30 seconds
-      Process.sleep(@keepalive_interval)
-      GenServer.cast(__MODULE__, %{action: :ping})
-    end)
 
     {:ok, args}
   end
 
-  def handle_cast(%{action: :ping}, topics) do
+  def handle_info(%{action: :ping}, topics) do
-    Map.values(topics)
+    topics
+    |> Map.values()
     |> List.flatten()
     |> Enum.each(fn socket ->
       Logger.debug("Sending keepalive ping")
       send(socket.transport_pid, {:text, ""})
     end)
 
-    spawn(fn ->
+    Process.send_after(self(), %{action: :ping}, @keepalive_interval)
-      # 30 seconds
-      Process.sleep(@keepalive_interval)
-      GenServer.cast(__MODULE__, %{action: :ping})
-    end)
 
     {:noreply, topics}
   end

lib/pleroma/web/web.ex

@@ -58,17 +58,31 @@ def safe_render(view, template, assigns \\ %{}) do
       rescue
         error ->
           Logger.error(
-            "#{__MODULE__} failed to render #{inspect({view, template})}: #{inspect(error)}"
+            "#{__MODULE__} failed to render #{inspect({view, template})}\n" <>
+              Exception.format(:error, error, __STACKTRACE__)
           )
 
-          Logger.error(inspect(__STACKTRACE__))
           nil
       end
 
       @doc """
-      Same as `render_many/4` but wrapped in rescue block.
+      Same as `render_many/4` but wrapped in rescue block and parallelized (unless disabled by passing false as a fifth argument).
       """
-      def safe_render_many(collection, view, template, assigns \\ %{}) do
+      def safe_render_many(collection, view, template, assigns \\ %{}, parallel \\ true)
+
+      def safe_render_many(collection, view, template, assigns, true) do
+        Enum.map(collection, fn resource ->
+          Task.async(fn ->
+            as = Map.get(assigns, :as) || view.__resource__
+            assigns = Map.put(assigns, as, resource)
+            safe_render(view, template, assigns)
+          end)
+        end)
+        |> Enum.map(&Task.await(&1, :infinity))
+        |> Enum.filter(& &1)
+      end
+
+      def safe_render_many(collection, view, template, assigns, false) do
         Enum.map(collection, fn resource ->
           as = Map.get(assigns, :as) || view.__resource__
           assigns = Map.put(assigns, as, resource)

mix.exs

@@ -95,7 +95,7 @@ defp oauth_deps do
   defp deps do
     [
       {:phoenix, "~> 1.4.8"},
-      {:tzdata, "~> 1.0"},
+      {:tzdata, "~> 0.5.21"},
       {:plug_cowboy, "~> 2.0"},
       {:phoenix_pubsub, "~> 1.1"},
       {:phoenix_ecto, "~> 4.0"},

mix.lock

@@ -87,7 +87,7 @@
   "tesla": {:hex, :tesla, "1.2.1", "864783cc27f71dd8c8969163704752476cec0f3a51eb3b06393b3971dc9733ff", [:mix], [{:exjsx, ">= 3.0.0", [hex: :exjsx, repo: "hexpm", optional: true]}, {:fuse, "~> 2.4", [hex: :fuse, repo: "hexpm", optional: true]}, {:hackney, "~> 1.6", [hex: :hackney, repo: "hexpm", optional: true]}, {:ibrowse, "~> 4.4.0", [hex: :ibrowse, repo: "hexpm", optional: true]}, {:jason, ">= 1.0.0", [hex: :jason, repo: "hexpm", optional: true]}, {:mime, "~> 1.0", [hex: :mime, repo: "hexpm", optional: false]}, {:poison, ">= 1.0.0", [hex: :poison, repo: "hexpm", optional: true]}], "hexpm"},
   "timex": {:hex, :timex, "3.6.1", "efdf56d0e67a6b956cc57774353b0329c8ab7726766a11547e529357ffdc1d56", [:mix], [{:combine, "~> 0.10", [hex: :combine, repo: "hexpm", optional: false]}, {:gettext, "~> 0.10", [hex: :gettext, repo: "hexpm", optional: false]}, {:tzdata, "~> 0.1.8 or ~> 0.5 or ~> 1.0.0", [hex: :tzdata, repo: "hexpm", optional: false]}], "hexpm"},
   "trailing_format_plug": {:hex, :trailing_format_plug, "0.0.7", "64b877f912cf7273bed03379936df39894149e35137ac9509117e59866e10e45", [:mix], [{:plug, "> 0.12.0", [hex: :plug, repo: "hexpm", optional: false]}], "hexpm"},
-  "tzdata": {:hex, :tzdata, "1.0.1", "f6027a331af7d837471248e62733c6ebee86a72e57c613aa071ebb1f750fc71a", [:mix], [{:hackney, "~> 1.0", [hex: :hackney, repo: "hexpm", optional: false]}], "hexpm"},
+  "tzdata": {:hex, :tzdata, "0.5.21", "8cbf3607fcce69636c672d5be2bbb08687fe26639a62bdcc283d267277db7cf0", [:mix], [{:hackney, "~> 1.0", [hex: :hackney, repo: "hexpm", optional: false]}], "hexpm"},
   "ueberauth": {:hex, :ueberauth, "0.6.1", "9e90d3337dddf38b1ca2753aca9b1e53d8a52b890191cdc55240247c89230412", [:mix], [{:plug, "~> 1.5", [hex: :plug, repo: "hexpm", optional: false]}], "hexpm"},
   "unicode_util_compat": {:hex, :unicode_util_compat, "0.4.1", "d869e4c68901dd9531385bb0c8c40444ebf624e60b6962d95952775cac5e90cd", [:rebar3], [], "hexpm"},
   "unsafe": {:hex, :unsafe, "1.0.1", "a27e1874f72ee49312e0a9ec2e0b27924214a05e3ddac90e91727bc76f8613d8", [:mix], [], "hexpm"},

test/config/transfer_task_test.exs

@@ -31,7 +31,7 @@ test "transfer config values from db to env" do
       value: [live: 15, com: 35]
     })
 
-    Pleroma.Config.TransferTask.start_link()
+    Pleroma.Config.TransferTask.start_link([])
 
     assert Application.get_env(:pleroma, :test_key) == [live: 2, com: 3]
     assert Application.get_env(:idna, :test_key) == [live: 15, com: 35]
@@ -50,7 +50,7 @@ test "non existing atom" do
     })
 
     assert ExUnit.CaptureLog.capture_log(fn ->
-             Pleroma.Config.TransferTask.start_link()
+             Pleroma.Config.TransferTask.start_link([])
            end) =~
              "updating env causes error, key: \"undefined_atom_key\", error: %ArgumentError{message: \"argument error\"}"
   end

test/support/mrf_module_mock.ex

@@ -0,0 +1,13 @@
+# Pleroma: A lightweight social networking server
+# Copyright © 2017-2018 Pleroma Authors <https://pleroma.social/>
+# SPDX-License-Identifier: AGPL-3.0-only
+
+defmodule MRFModuleMock do
+  @behaviour Pleroma.Web.ActivityPub.MRF
+
+  @impl true
+  def filter(message), do: {:ok, message}
+
+  @impl true
+  def describe, do: {:ok, %{mrf_module_mock: "some config data"}}
+end

test/user_test.exs

@@ -525,7 +525,10 @@ test "it has required fields" do
     end
 
     test "it restricts some sizes" do
-      [bio: 5000, name: 100]
+      bio_limit = Pleroma.Config.get([:instance, :user_bio_length], 5000)
+      name_limit = Pleroma.Config.get([:instance, :user_name_length], 100)
+
+      [bio: bio_limit, name: name_limit]
       |> Enum.each(fn {field, size} ->
         string = String.pad_leading(".", size)
         cs = User.remote_user_creation(Map.put(@valid_remote, field, string))

test/web/activity_pub/mrf/mrf_test.exs

@@ -57,4 +57,30 @@ test "matches are case-insensitive" do
       refute MRF.subdomain_match?(regexes, "example.com")
     end
   end
+
+  describe "describe/0" do
+    test "it works as expected with noop policy" do
+      expected = %{
+        mrf_policies: ["NoOpPolicy"],
+        exclusions: false
+      }
+
+      {:ok, ^expected} = MRF.describe()
+    end
+
+    test "it works as expected with mock policy" do
+      config = Pleroma.Config.get([:instance, :rewrite_policy])
+      Pleroma.Config.put([:instance, :rewrite_policy], [MRFModuleMock])
+
+      expected = %{
+        mrf_policies: ["MRFModuleMock"],
+        mrf_module_mock: "some config data",
+        exclusions: false
+      }
+
+      {:ok, ^expected} = MRF.describe()
+
+      Pleroma.Config.put([:instance, :rewrite_policy], config)
+    end
+  end
 end

test/web/activity_pub/mrf/vocabulary_policy_test.exs

@@ -0,0 +1,123 @@
+# Pleroma: A lightweight social networking server
+# Copyright © 2017-2019 Pleroma Authors <https://pleroma.social/>
+# SPDX-License-Identifier: AGPL-3.0-only
+
+defmodule Pleroma.Web.ActivityPub.MRF.VocabularyPolicyTest do
+  use Pleroma.DataCase
+
+  alias Pleroma.Web.ActivityPub.MRF.VocabularyPolicy
+
+  describe "accept" do
+    test "it accepts based on parent activity type" do
+      config = Pleroma.Config.get([:mrf_vocabulary, :accept])
+      Pleroma.Config.put([:mrf_vocabulary, :accept], ["Like"])
+
+      message = %{
+        "type" => "Like",
+        "object" => "whatever"
+      }
+
+      {:ok, ^message} = VocabularyPolicy.filter(message)
+
+      Pleroma.Config.put([:mrf_vocabulary, :accept], config)
+    end
+
+    test "it accepts based on child object type" do
+      config = Pleroma.Config.get([:mrf_vocabulary, :accept])
+      Pleroma.Config.put([:mrf_vocabulary, :accept], ["Create", "Note"])
+
+      message = %{
+        "type" => "Create",
+        "object" => %{
+          "type" => "Note",
+          "content" => "whatever"
+        }
+      }
+
+      {:ok, ^message} = VocabularyPolicy.filter(message)
+
+      Pleroma.Config.put([:mrf_vocabulary, :accept], config)
+    end
+
+    test "it does not accept disallowed child objects" do
+      config = Pleroma.Config.get([:mrf_vocabulary, :accept])
+      Pleroma.Config.put([:mrf_vocabulary, :accept], ["Create", "Note"])
+
+      message = %{
+        "type" => "Create",
+        "object" => %{
+          "type" => "Article",
+          "content" => "whatever"
+        }
+      }
+
+      {:reject, nil} = VocabularyPolicy.filter(message)
+
+      Pleroma.Config.put([:mrf_vocabulary, :accept], config)
+    end
+
+    test "it does not accept disallowed parent types" do
+      config = Pleroma.Config.get([:mrf_vocabulary, :accept])
+      Pleroma.Config.put([:mrf_vocabulary, :accept], ["Announce", "Note"])
+
+      message = %{
+        "type" => "Create",
+        "object" => %{
+          "type" => "Note",
+          "content" => "whatever"
+        }
+      }
+
+      {:reject, nil} = VocabularyPolicy.filter(message)
+
+      Pleroma.Config.put([:mrf_vocabulary, :accept], config)
+    end
+  end
+
+  describe "reject" do
+    test "it rejects based on parent activity type" do
+      config = Pleroma.Config.get([:mrf_vocabulary, :reject])
+      Pleroma.Config.put([:mrf_vocabulary, :reject], ["Like"])
+
+      message = %{
+        "type" => "Like",
+        "object" => "whatever"
+      }
+
+      {:reject, nil} = VocabularyPolicy.filter(message)
+
+      Pleroma.Config.put([:mrf_vocabulary, :reject], config)
+    end
+
+    test "it rejects based on child object type" do
+      config = Pleroma.Config.get([:mrf_vocabulary, :reject])
+      Pleroma.Config.put([:mrf_vocabulary, :reject], ["Note"])
+
+      message = %{
+        "type" => "Create",
+        "object" => %{
+          "type" => "Note",
+          "content" => "whatever"
+        }
+      }
+
+      {:reject, nil} = VocabularyPolicy.filter(message)
+
+      Pleroma.Config.put([:mrf_vocabulary, :reject], config)
+    end
+
+    test "it passes through objects that aren't disallowed" do
+      config = Pleroma.Config.get([:mrf_vocabulary, :reject])
+      Pleroma.Config.put([:mrf_vocabulary, :reject], ["Like"])
+
+      message = %{
+        "type" => "Announce",
+        "object" => "whatever"
+      }
+
+      {:ok, ^message} = VocabularyPolicy.filter(message)
+
+      Pleroma.Config.put([:mrf_vocabulary, :reject], config)
+    end
+  end
+end

test/web/activity_pub/relay_test.exs

@@ -5,11 +5,71 @@
 defmodule Pleroma.Web.ActivityPub.RelayTest do
   use Pleroma.DataCase
 
+  alias Pleroma.Activity
+  alias Pleroma.Object
+  alias Pleroma.Web.ActivityPub.ActivityPub
   alias Pleroma.Web.ActivityPub.Relay
 
+  import Pleroma.Factory
+
   test "gets an actor for the relay" do
     user = Relay.get_actor()
+    assert user.ap_id == "#{Pleroma.Web.Endpoint.url()}/relay"
+  end
 
-    assert user.ap_id =~ "/relay"
+  describe "follow/1" do
+    test "returns errors when user not found" do
+      assert Relay.follow("test-ap-id") == {:error, "Could not fetch by AP id"}
+    end
+
+    test "returns activity" do
+      user = insert(:user)
+      service_actor = Relay.get_actor()
+      assert {:ok, %Activity{} = activity} = Relay.follow(user.ap_id)
+      assert activity.actor == "#{Pleroma.Web.Endpoint.url()}/relay"
+      assert user.ap_id in activity.recipients
+      assert activity.data["type"] == "Follow"
+      assert activity.data["actor"] == service_actor.ap_id
+      assert activity.data["object"] == user.ap_id
+    end
+  end
+
+  describe "unfollow/1" do
+    test "returns errors when user not found" do
+      assert Relay.unfollow("test-ap-id") == {:error, "Could not fetch by AP id"}
+    end
+
+    test "returns activity" do
+      user = insert(:user)
+      service_actor = Relay.get_actor()
+      ActivityPub.follow(service_actor, user)
+      assert {:ok, %Activity{} = activity} = Relay.unfollow(user.ap_id)
+      assert activity.actor == "#{Pleroma.Web.Endpoint.url()}/relay"
+      assert user.ap_id in activity.recipients
+      assert activity.data["type"] == "Undo"
+      assert activity.data["actor"] == service_actor.ap_id
+      assert activity.data["to"] == [user.ap_id]
+    end
+  end
+
+  describe "publish/1" do
+    test "returns error when activity not `Create` type" do
+      activity = insert(:like_activity)
+      assert Relay.publish(activity) == {:error, "Not implemented"}
+    end
+
+    test "returns error when activity not public" do
+      activity = insert(:direct_note_activity)
+      assert Relay.publish(activity) == {:error, false}
+    end
+
+    test "returns announce activity" do
+      service_actor = Relay.get_actor()
+      note = insert(:note_activity)
+      assert {:ok, %Activity{} = activity, %Object{} = obj} = Relay.publish(note)
+      assert activity.data["type"] == "Announce"
+      assert activity.data["actor"] == service_actor.ap_id
+      assert activity.data["object"] == obj.data["id"]
+    end
   end
 end

test/web/mastodon_api/mastodon_api_controller_test.exs

@@ -7,6 +7,7 @@ defmodule Pleroma.Web.MastodonAPI.MastodonAPIControllerTest do
 
   alias Ecto.Changeset
   alias Pleroma.Activity
+  alias Pleroma.Config
   alias Pleroma.Notification
   alias Pleroma.Object
   alias Pleroma.Repo
@@ -85,11 +86,11 @@ test "the public timeline", %{conn: conn} do
   end
 
   test "the public timeline when public is set to false", %{conn: conn} do
-    public = Pleroma.Config.get([:instance, :public])
+    public = Config.get([:instance, :public])
-    Pleroma.Config.put([:instance, :public], false)
+    Config.put([:instance, :public], false)
 
     on_exit(fn ->
-      Pleroma.Config.put([:instance, :public], public)
+      Config.put([:instance, :public], public)
     end)
 
     assert conn
@@ -250,7 +251,7 @@ test "posting a fake status", %{conn: conn} do
     end
 
     test "posting a status with OGP link preview", %{conn: conn} do
-      Pleroma.Config.put([:rich_media, :enabled], true)
+      Config.put([:rich_media, :enabled], true)
 
       conn =
         conn
@@ -260,7 +261,7 @@ test "posting a status with OGP link preview", %{conn: conn} do
 
       assert %{"id" => id, "card" => %{"title" => "The Rock"}} = json_response(conn, 200)
       assert Activity.get_by_id(id)
-      Pleroma.Config.put([:rich_media, :enabled], false)
+      Config.put([:rich_media, :enabled], false)
     end
 
     test "posting a direct status", %{conn: conn} do
@@ -304,7 +305,7 @@ test "posting a poll", %{conn: conn} do
 
     test "option limit is enforced", %{conn: conn} do
       user = insert(:user)
-      limit = Pleroma.Config.get([:instance, :poll_limits, :max_options])
+      limit = Config.get([:instance, :poll_limits, :max_options])
 
       conn =
         conn
@@ -320,7 +321,7 @@ test "option limit is enforced", %{conn: conn} do
 
     test "option character limit is enforced", %{conn: conn} do
       user = insert(:user)
-      limit = Pleroma.Config.get([:instance, :poll_limits, :max_option_chars])
+      limit = Config.get([:instance, :poll_limits, :max_option_chars])
 
       conn =
         conn
@@ -339,7 +340,7 @@ test "option character limit is enforced", %{conn: conn} do
 
     test "minimal date limit is enforced", %{conn: conn} do
       user = insert(:user)
-      limit = Pleroma.Config.get([:instance, :poll_limits, :min_expiration])
+      limit = Config.get([:instance, :poll_limits, :min_expiration])
 
       conn =
         conn
@@ -358,7 +359,7 @@ test "minimal date limit is enforced", %{conn: conn} do
 
     test "maximum date limit is enforced", %{conn: conn} do
       user = insert(:user)
-      limit = Pleroma.Config.get([:instance, :poll_limits, :max_expiration])
+      limit = Config.get([:instance, :poll_limits, :max_expiration])
 
       conn =
         conn
@@ -1633,12 +1634,12 @@ test "returns the relationships for the current user", %{conn: conn} do
 
   describe "media upload" do
     setup do
-      upload_config = Pleroma.Config.get([Pleroma.Upload])
+      upload_config = Config.get([Pleroma.Upload])
-      proxy_config = Pleroma.Config.get([:media_proxy])
+      proxy_config = Config.get([:media_proxy])
 
       on_exit(fn ->
-        Pleroma.Config.put([Pleroma.Upload], upload_config)
+        Config.put([Pleroma.Upload], upload_config)
-        Pleroma.Config.put([:media_proxy], proxy_config)
+        Config.put([:media_proxy], proxy_config)
       end)
 
       user = insert(:user)
@@ -2581,7 +2582,7 @@ test "get instance information", %{conn: conn} do
     conn = get(conn, "/api/v1/instance")
     assert result = json_response(conn, 200)
 
-    email = Pleroma.Config.get([:instance, :email])
+    email = Config.get([:instance, :email])
     # Note: not checking for "max_toot_chars" since it's optional
     assert %{
              "uri" => _,
@@ -2623,7 +2624,7 @@ test "get instance stats", %{conn: conn} do
       |> Changeset.put_embed(:info, info_change)
       |> User.update_and_set_cache()
 
-    Pleroma.Stats.update_stats()
+    Pleroma.Stats.force_update()
 
     conn = get(conn, "/api/v1/instance")
 
@@ -2641,7 +2642,7 @@ test "get peers", %{conn: conn} do
     insert(:user, %{local: false, nickname: "u@peer1.com"})
     insert(:user, %{local: false, nickname: "u@peer2.com"})
 
-    Pleroma.Stats.update_stats()
+    Pleroma.Stats.force_update()
 
     conn = get(conn, "/api/v1/instance/peers")
 
@@ -2666,7 +2667,7 @@ test "put settings", %{conn: conn} do
 
   describe "pinned statuses" do
     setup do
-      Pleroma.Config.put([:instance, :max_pinned_statuses], 1)
+      Config.put([:instance, :max_pinned_statuses], 1)
 
       user = insert(:user)
       {:ok, activity} = CommonAPI.post(user, %{"status" => "HI!!!"})
@@ -2766,10 +2767,10 @@ test "max pinned statuses", %{conn: conn, user: user, activity: activity_one} do
 
   describe "cards" do
     setup do
-      Pleroma.Config.put([:rich_media, :enabled], true)
+      Config.put([:rich_media, :enabled], true)
 
       on_exit(fn ->
-        Pleroma.Config.put([:rich_media, :enabled], false)
+        Config.put([:rich_media, :enabled], false)
       end)
 
       user = insert(:user)
@@ -2997,7 +2998,7 @@ test "comment must be up to the size specified in the config", %{
       reporter: reporter,
       target_user: target_user
     } do
-      max_size = Pleroma.Config.get([:instance, :max_report_comment_size], 1000)
+      max_size = Config.get([:instance, :max_report_comment_size], 1000)
       comment = String.pad_trailing("a", max_size + 1, "a")
 
       error = %{"error" => "Comment must be up to #{max_size} characters"}
@@ -3126,15 +3127,15 @@ test "redirects not logged-in users to the login page on private instances", %{
       conn: conn,
       path: path
     } do
-      is_public = Pleroma.Config.get([:instance, :public])
+      is_public = Config.get([:instance, :public])
-      Pleroma.Config.put([:instance, :public], false)
+      Config.put([:instance, :public], false)
 
       conn = get(conn, path)
 
       assert conn.status == 302
       assert redirected_to(conn) == "/web/login"
 
-      Pleroma.Config.put([:instance, :public], is_public)
+      Config.put([:instance, :public], is_public)
     end
 
     test "does not redirect logged in users to the login page", %{conn: conn, path: path} do
@@ -3876,8 +3877,8 @@ test "it sends an email to user", %{user: user} do
       token_record = Repo.get_by(Pleroma.PasswordResetToken, user_id: user.id)
 
       email = Pleroma.Emails.UserEmail.password_reset_email(user, token_record.token)
-      notify_email = Pleroma.Config.get([:instance, :notify_email])
+      notify_email = Config.get([:instance, :notify_email])
-      instance_name = Pleroma.Config.get([:instance, :name])
+      instance_name = Config.get([:instance, :name])
 
       assert_email_sent(
         from: {instance_name, notify_email},
@@ -3909,11 +3910,11 @@ test "it returns 400 when user is not local", %{conn: conn, user: user} do
 
   describe "POST /api/v1/pleroma/accounts/confirmation_resend" do
     setup do
-      setting = Pleroma.Config.get([:instance, :account_activation_required])
+      setting = Config.get([:instance, :account_activation_required])
 
       unless setting do
-        Pleroma.Config.put([:instance, :account_activation_required], true)
+        Config.put([:instance, :account_activation_required], true)
-        on_exit(fn -> Pleroma.Config.put([:instance, :account_activation_required], setting) end)
+        on_exit(fn -> Config.put([:instance, :account_activation_required], setting) end)
       end
 
       user = insert(:user)
@@ -3937,8 +3938,8 @@ test "resend account confirmation email", %{conn: conn, user: user} do
       |> json_response(:no_content)
 
       email = Pleroma.Emails.UserEmail.account_confirmation_email(user)
-      notify_email = Pleroma.Config.get([:instance, :notify_email])
+      notify_email = Config.get([:instance, :notify_email])
-      instance_name = Pleroma.Config.get([:instance, :name])
+      instance_name = Config.get([:instance, :name])
 
       assert_email_sent(
         from: {instance_name, notify_email},
@@ -3947,4 +3948,84 @@ test "resend account confirmation email", %{conn: conn, user: user} do
       )
     end
   end
+
+  describe "GET /api/v1/suggestions" do
+    setup do
+      user = insert(:user)
+      other_user = insert(:user)
+      config = Config.get(:suggestions)
+      on_exit(fn -> Config.put(:suggestions, config) end)
+
+      host = Config.get([Pleroma.Web.Endpoint, :url, :host])
+      url500 = "http://test500?#{host}&#{user.nickname}"
+      url200 = "http://test200?#{host}&#{user.nickname}"
+
+      mock(fn
+        %{method: :get, url: ^url500} ->
+          %Tesla.Env{status: 500, body: "bad request"}
+
+        %{method: :get, url: ^url200} ->
+          %Tesla.Env{
+            status: 200,
+            body:
+              ~s([{"acct":"yj455","avatar":"https://social.heldscal.la/avatar/201.jpeg","avatar_static":"https://social.heldscal.la/avatar/s/201.jpeg"}, {"acct":"#{
+                other_user.ap_id
+              }","avatar":"https://social.heldscal.la/avatar/202.jpeg","avatar_static":"https://social.heldscal.la/avatar/s/202.jpeg"}])
+          }
+      end)
+
+      [user: user, other_user: other_user]
+    end
+
+    test "returns empty result when suggestions disabled", %{conn: conn, user: user} do
+      Config.put([:suggestions, :enabled], false)
+
+      res =
+        conn
+        |> assign(:user, user)
+        |> get("/api/v1/suggestions")
+        |> json_response(200)
+
+      assert res == []
+    end
+
+    test "returns error", %{conn: conn, user: user} do
+      Config.put([:suggestions, :enabled], true)
+      Config.put([:suggestions, :third_party_engine], "http://test500?{{host}}&{{user}}")
+
+      res =
+        conn
+        |> assign(:user, user)
+        |> get("/api/v1/suggestions")
+        |> json_response(500)
+
+      assert res == "Something went wrong"
+    end
+
+    test "returns suggestions", %{conn: conn, user: user, other_user: other_user} do
+      Config.put([:suggestions, :enabled], true)
+      Config.put([:suggestions, :third_party_engine], "http://test200?{{host}}&{{user}}")
+
+      res =
+        conn
+        |> assign(:user, user)
+        |> get("/api/v1/suggestions")
+        |> json_response(200)
+
+      assert res == [
+               %{
+                 "acct" => "yj455",
+                 "avatar" => "https://social.heldscal.la/avatar/201.jpeg",
+                 "avatar_static" => "https://social.heldscal.la/avatar/s/201.jpeg",
+                 "id" => 0
+               },
+               %{
+                 "acct" => other_user.ap_id,
+                 "avatar" => "https://social.heldscal.la/avatar/202.jpeg",
+                 "avatar_static" => "https://social.heldscal.la/avatar/s/202.jpeg",
+                 "id" => other_user.id
+               }
+             ]
+    end
+  end
 end

test/web/mastodon_api/mastodon_api_test.exs

@@ -0,0 +1,103 @@
+# Pleroma: A lightweight social networking server
+# Copyright © 2017-2019 Pleroma Authors <https://pleroma.social/>
+# SPDX-License-Identifier: AGPL-3.0-only
+
+defmodule Pleroma.Web.MastodonAPI.MastodonAPITest do
+  use Pleroma.Web.ConnCase
+
+  alias Pleroma.Notification
+  alias Pleroma.ScheduledActivity
+  alias Pleroma.User
+  alias Pleroma.Web.MastodonAPI.MastodonAPI
+  alias Pleroma.Web.TwitterAPI.TwitterAPI
+
+  import Pleroma.Factory
+
+  describe "follow/3" do
+    test "returns error when user deactivated" do
+      follower = insert(:user)
+      user = insert(:user, local: true, info: %{deactivated: true})
+      {:error, error} = MastodonAPI.follow(follower, user)
+      assert error == "Could not follow user: You are deactivated."
+    end
+
+    test "following for user" do
+      follower = insert(:user)
+      user = insert(:user)
+      {:ok, follower} = MastodonAPI.follow(follower, user)
+      assert User.following?(follower, user)
+    end
+
+    test "returns ok if user already followed" do
+      follower = insert(:user)
+      user = insert(:user)
+      {:ok, follower} = User.follow(follower, user)
+      {:ok, follower} = MastodonAPI.follow(follower, refresh_record(user))
+      assert User.following?(follower, user)
+    end
+  end
+
+  describe "get_followers/2" do
+    test "returns user followers" do
+      follower1_user = insert(:user)
+      follower2_user = insert(:user)
+      user = insert(:user)
+      {:ok, _follower1_user} = User.follow(follower1_user, user)
+      {:ok, follower2_user} = User.follow(follower2_user, user)
+
+      assert MastodonAPI.get_followers(user, %{"limit" => 1}) == [follower2_user]
+    end
+  end
+
+  describe "get_friends/2" do
+    test "returns user friends" do
+      user = insert(:user)
+      followed_one = insert(:user)
+      followed_two = insert(:user)
+      followed_three = insert(:user)
+
+      {:ok, user} = User.follow(user, followed_one)
+      {:ok, user} = User.follow(user, followed_two)
+      {:ok, user} = User.follow(user, followed_three)
+      res = MastodonAPI.get_friends(user)
+
+      assert length(res) == 3
+      assert Enum.member?(res, refresh_record(followed_three))
+      assert Enum.member?(res, refresh_record(followed_two))
+      assert Enum.member?(res, refresh_record(followed_one))
+    end
+  end
+
+  describe "get_notifications/2" do
+    test "returns notifications for user" do
+      user = insert(:user)
+      subscriber = insert(:user)
+
+      User.subscribe(subscriber, user)
+
+      {:ok, status} = TwitterAPI.create_status(user, %{"status" => "Akariiiin"})
+      {:ok, status1} = TwitterAPI.create_status(user, %{"status" => "Magi"})
+      {:ok, [notification]} = Notification.create_notifications(status)
+      {:ok, [notification1]} = Notification.create_notifications(status1)
+      res = MastodonAPI.get_notifications(subscriber)
+
+      assert Enum.member?(Enum.map(res, & &1.id), notification.id)
+      assert Enum.member?(Enum.map(res, & &1.id), notification1.id)
+    end
+  end
+
+  describe "get_scheduled_activities/2" do
+    test "returns user scheduled activities" do
+      user = insert(:user)
+
+      today =
+        NaiveDateTime.utc_now()
+        |> NaiveDateTime.add(:timer.minutes(6), :millisecond)
+        |> NaiveDateTime.to_iso8601()
+
+      attrs = %{params: %{}, scheduled_at: today}
+      {:ok, schedule} = ScheduledActivity.create(user, attrs)
+      assert MastodonAPI.get_scheduled_activities(user) == [schedule]
+    end
+  end
+end

test/web/node_info_test.exs

@@ -85,6 +85,9 @@ test "it returns the safe_dm_mentions feature if enabled", %{conn: conn} do
   end
 
   test "it shows MRF transparency data if enabled", %{conn: conn} do
+    config = Pleroma.Config.get([:instance, :rewrite_policy])
+    Pleroma.Config.put([:instance, :rewrite_policy], [Pleroma.Web.ActivityPub.MRF.SimplePolicy])
+
     option = Pleroma.Config.get([:instance, :mrf_transparency])
     Pleroma.Config.put([:instance, :mrf_transparency], true)
 
@@ -98,11 +101,15 @@ test "it shows MRF transparency data if enabled", %{conn: conn} do
 
     assert response["metadata"]["federation"]["mrf_simple"] == simple_config
 
+    Pleroma.Config.put([:instance, :rewrite_policy], config)
     Pleroma.Config.put([:instance, :mrf_transparency], option)
     Pleroma.Config.put(:mrf_simple, %{})
   end
 
   test "it performs exclusions from MRF transparency data if configured", %{conn: conn} do
+    config = Pleroma.Config.get([:instance, :rewrite_policy])
+    Pleroma.Config.put([:instance, :rewrite_policy], [Pleroma.Web.ActivityPub.MRF.SimplePolicy])
+
     option = Pleroma.Config.get([:instance, :mrf_transparency])
     Pleroma.Config.put([:instance, :mrf_transparency], true)
 
@@ -122,6 +129,7 @@ test "it performs exclusions from MRF transparency data if configured", %{conn:
     assert response["metadata"]["federation"]["mrf_simple"] == expected_config
     assert response["metadata"]["federation"]["exclusions"] == true
 
+    Pleroma.Config.put([:instance, :rewrite_policy], config)
     Pleroma.Config.put([:instance, :mrf_transparency], option)
     Pleroma.Config.put([:instance, :mrf_transparency_exclusions], exclusions)
     Pleroma.Config.put(:mrf_simple, %{})