Merge branch 'develop' into issue/1383

CHANGELOG.md

@@ -12,8 +12,9 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/).
 
 ### Changed
 - **Breaking:** Pleroma won't start if it detects unapplied migrations
-- **Breaking:** attachments are removed along with statuses when there are no other references to it
+- **Breaking:** attachments are removed along with statuses. Does not affect duplicate files and attachments without status.
 - **Breaking:** Elixir >=1.8 is now required (was >= 1.7)
+- **Breaking:** `Pleroma.Plugs.RemoteIp` and `:rate_limiter` enabled by default. Please ensure your reverse proxy forwards the real IP!
 - **Breaking:** attachment links (`config :pleroma, :instance, no_attachment_links` and `config :pleroma, Pleroma.Upload, link_name`) disabled by default
 - **Breaking:** OAuth: defaulted `[:auth, :enforce_oauth_admin_scope_usage]` setting to `true` which demands `admin` OAuth scope to perform admin actions (in addition to `is_admin` flag on User); make sure to use bundled or newer versions of AdminFE & PleromaFE to access admin / moderator features.
 - **Breaking:** Dynamic configuration has been rearchitected. The `:pleroma, :instance, dynamic_configuration` setting has been replaced with `config :pleroma, configurable_from_database`. Please backup your configuration to a file and run the migration task to ensure consistency with the new schema.
@@ -27,6 +28,7 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/).
 - Deprecated `User.Info` embedded schema (fields moved to `User`)
 - Store status data inside Flag activity
 - Deprecated (reorganized as `UserRelationship` entity) User fields with user AP IDs (`blocks`, `mutes`, `muted_reblogs`, `muted_notifications`, `subscribers`).
+- Rate limiter is now disabled for localhost/socket (unless remoteip plug is enabled)
 - Logger: default log level changed from `warn` to `info`.
 - Config mix task `migrate_to_db` truncates `config` table before migrating the config file.
 <details>
@@ -104,6 +106,7 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/).
 - Mastodon API: Change emoji reaction reply format once more
 - Configuration: `feed.logo` option for tag feed.
 - Tag feed: `/tags/:tag.rss` - list public statuses by hashtag.
+- Mastodon API: Add `reacted` property to `emoji_reactions`
 </details>
 
 ### Fixed

config/config.exs

@@ -586,11 +586,21 @@
 config :http_signatures,
   adapter: Pleroma.Signature
 
-config :pleroma, :rate_limit, authentication: {60_000, 15}
+config :pleroma, :rate_limit,
+  authentication: {60_000, 15},
+  search: [{1000, 10}, {1000, 30}],
+  app_account_creation: {1_800_000, 25},
+  relations_actions: {10_000, 10},
+  relation_id_action: {60_000, 2},
+  statuses_actions: {10_000, 15},
+  status_id_action: {60_000, 3},
+  password_reset: {1_800_000, 5},
+  account_confirmation_resend: {8_640_000, 5},
+  ap_routes: {60_000, 15}
 
 config :pleroma, Pleroma.ActivityExpiration, enabled: true
 
-config :pleroma, Pleroma.Plugs.RemoteIp, enabled: false
+config :pleroma, Pleroma.Plugs.RemoteIp, enabled: true
 
 config :pleroma, :static_fe, enabled: false
 

docs/API/differences_in_mastoapi_responses.md

@@ -29,7 +29,7 @@ Has these additional fields under the `pleroma` object:
 - `spoiler_text`: a map consisting of alternate representations of the `spoiler_text` property with the key being it's mimetype. Currently the only alternate representation supported is `text/plain`
 - `expires_at`: a datetime (iso8601) that states when the post will expire (be deleted automatically), or empty if the post won't expire
 - `thread_muted`: true if the thread the post belongs to is muted
-- `emoji_reactions`: A list with emoji / reaction maps. The format is {emoji: "☕", count: 1}. Contains no information about the reacting users, for that use the `emoji_reactions_by` endpoint.
+- `emoji_reactions`: A list with emoji / reaction maps. The format is `{emoji: "☕", count: 1, reacted: true}`. Contains no information about the reacting users, for that use the `emoji_reactions_by` endpoint.
 
 ## Attachments
 

docs/API/pleroma_api.md

@@ -455,7 +455,7 @@ Emoji reactions work a lot like favourites do. They make it possible to react to
 * Example Response:
 ```json
 [
-  {"emoji": "😀", "count": 2, "accounts": [{"id" => "xyz.."...}, {"id" => "zyx..."}]},
-  {"emoji": "☕", "count": 1, "accounts": [{"id" => "abc..."}]}
+  {"emoji": "😀", "count": 2, "reacted": true, "accounts": [{"id" => "xyz.."...}, {"id" => "zyx..."}]},
+  {"emoji": "☕", "count": 1, "reacted": false, "accounts": [{"id" => "abc..."}]}
 ]
 ```

docs/configuration/cheatsheet.md

@@ -308,16 +308,15 @@ This will make Pleroma listen on `127.0.0.1` port `8080` and generate urls start
 Available options:
 
 * `enabled` - Enable/disable the plug. Defaults to `false`.
-* `headers` - A list of strings naming the `req_headers` to use when deriving the `remote_ip`. Order does not matter. Defaults to `~w[forwarded x-forwarded-for x-client-ip x-real-ip]`.
+* `headers` - A list of strings naming the `req_headers` to use when deriving the `remote_ip`. Order does not matter. Defaults to `["x-forwarded-for"]`.
 * `proxies` - A list of strings in [CIDR](https://en.wikipedia.org/wiki/CIDR) notation specifying the IPs of known proxies. Defaults to `[]`.
 * `reserved` - Defaults to [localhost](https://en.wikipedia.org/wiki/Localhost) and [private network](https://en.wikipedia.org/wiki/Private_network).
 
 
 ### :rate_limit
 
-This is an advanced feature and disabled by default.
-
-If your instance is behind a reverse proxy you must enable and configure [`Pleroma.Plugs.RemoteIp`](#pleroma-plugs-remoteip).
+!!! note
+   If your instance is behind a reverse proxy ensure [`Pleroma.Plugs.RemoteIp`](#pleroma-plugs-remoteip) is enabled (it is enabled by default).
 
 A keyword list of rate limiters where a key is a limiter name and value is the limiter configuration. The basic configuration is a tuple where:
 
@@ -326,14 +325,31 @@ A keyword list of rate limiters where a key is a limiter name and value is the l
 
 It is also possible to have different limits for unauthenticated and authenticated users: the keyword value must be a list of two tuples where the first one is a config for unauthenticated users and the second one is for authenticated.
 
+For example:
+
+```elixir
+config :pleroma, :rate_limit,
+  authentication: {60_000, 15},
+  search: [{1000, 10}, {1000, 30}]
+```
+
+Means that:
+
+1. In 60 seconds, 15 authentication attempts can be performed from the same IP address.
+2. In 1 second, 10 search requests can be performed from the same IP adress by unauthenticated users, while authenticated users can perform 30 search requests per second.
+
 Supported rate limiters:
 
-* `:search` for the search requests (account & status search etc.)
-* `:app_account_creation` for registering user accounts from the same IP address
-* `:relations_actions` for actions on relations with all users (follow, unfollow)
-* `:relation_id_action` for actions on relation with a specific user (follow, unfollow)
-* `:statuses_actions` for create / delete / fav / unfav / reblog / unreblog actions on any statuses
-* `:status_id_action` for fav / unfav or reblog / unreblog actions on the same status by the same user
+* `:search` - Account/Status search.
+* `:app_account_creation` - Account registration from the API.
+* `:relations_actions` - Following/Unfollowing in general.
+* `:relation_id_action` - Following/Unfollowing for a specific user.
+* `:statuses_actions` - Status actions such as: (un)repeating, (un)favouriting, creating, deleting.
+* `:status_id_action` - (un)Repeating/(un)Favouriting a particular status.
+* `:authentication` - Authentication actions, i.e getting an OAuth token.
+* `:password_reset` - Requesting password reset emails.
+* `:account_confirmation_resend` - Requesting resending account confirmation emails.
+* `:ap_routes` - Requesting statuses via ActivityPub.
 
 ### :web_cache_ttl
 

lib/pleroma/application.ex

@@ -33,6 +33,7 @@ def user_agent do
   def start(_type, _args) do
     Pleroma.HTML.compile_scrubbers()
     Pleroma.Config.DeprecationWarnings.warn()
+    Pleroma.Plugs.HTTPSecurityPlug.warn_if_disabled()
     Pleroma.Repo.check_migrations_applied!()
     setup_instrumenters()
     load_custom_modules()

lib/pleroma/plugs/http_security_plug.ex

@@ -6,6 +6,8 @@ defmodule Pleroma.Plugs.HTTPSecurityPlug do
   alias Pleroma.Config
   import Plug.Conn
 
+  require Logger
+
   def init(opts), do: opts
 
   def call(conn, _options) do
@@ -90,6 +92,51 @@ defp csp_string do
     |> Enum.join("; ")
   end
 
+  def warn_if_disabled do
+    unless Config.get([:http_security, :enabled]) do
+      Logger.warn("
+                                 .i;;;;i.
+                               iYcviii;vXY:
+                             .YXi       .i1c.
+                            .YC.     .    in7.
+                           .vc.   ......   ;1c.
+                           i7,   ..        .;1;
+                          i7,   .. ...      .Y1i
+                         ,7v     .6MMM@;     .YX,
+                        .7;.   ..IMMMMMM1     :t7.
+                       .;Y.     ;$MMMMMM9.     :tc.
+                       vY.   .. .nMMM@MMU.      ;1v.
+                      i7i   ...  .#MM@M@C. .....:71i
+                     it:   ....   $MMM@9;.,i;;;i,;tti
+                    :t7.  .....   0MMMWv.,iii:::,,;St.
+                   .nC.   .....   IMMMQ..,::::::,.,czX.
+                  .ct:   ....... .ZMMMI..,:::::::,,:76Y.
+                  c2:   ......,i..Y$M@t..:::::::,,..inZY
+                 vov   ......:ii..c$MBc..,,,,,,,,,,..iI9i
+                i9Y   ......iii:..7@MA,..,,,,,,,,,....;AA:
+               iIS.  ......:ii::..;@MI....,............;Ez.
+              .I9.  ......:i::::...8M1..................C0z.
+             .z9;  ......:i::::,.. .i:...................zWX.
+             vbv  ......,i::::,,.      ................. :AQY
+            c6Y.  .,...,::::,,..:t0@@QY. ................ :8bi
+           :6S. ..,,...,:::,,,..EMMMMMMI. ............... .;bZ,
+          :6o,  .,,,,..:::,,,..i#MMMMMM#v.................  YW2.
+         .n8i ..,,,,,,,::,,,,.. tMMMMM@C:.................. .1Wn
+         7Uc. .:::,,,,,::,,,,..   i1t;,..................... .UEi
+         7C...::::::::::::,,,,..        ....................  vSi.
+         ;1;...,,::::::,.........       ..................    Yz:
+          v97,.........                                     .voC.
+           izAotX7777777777777777777777777777777777777777Y7n92:
+             .;CoIIIIIUAA666666699999ZZZZZZZZZZZZZZZZZZZZ6ov.
+
+HTTP Security is disabled. Please re-enable it to prevent users from attacking
+your instance and your users via malicious posts:
+
+      config :pleroma, :http_security, enabled: true
+      ")
+    end
+  end
+
   defp maybe_send_sts_header(conn, true) do
     max_age_sts = Config.get([:http_security, :sts_max_age])
     max_age_ct = Config.get([:http_security, :ct_max_age])

lib/pleroma/plugs/rate_limiter/rate_limiter.ex

@@ -67,6 +67,8 @@ defmodule Pleroma.Plugs.RateLimiter do
   alias Pleroma.Plugs.RateLimiter.LimiterSupervisor
   alias Pleroma.User
 
+  require Logger
+
   def init(opts) do
     limiter_name = Keyword.get(opts, :name)
 
@@ -89,18 +91,39 @@ def init(opts) do
   def call(conn, nil), do: conn
 
   def call(conn, settings) do
-    settings
-    |> incorporate_conn_info(conn)
-    |> check_rate()
-    |> case do
-      {:ok, _count} ->
+    case disabled?() do
+      true ->
+        if Pleroma.Config.get(:env) == :prod,
+          do: Logger.warn("Rate limiter is disabled for localhost/socket")
+
         conn
 
-      {:error, _count} ->
-        render_throttled_error(conn)
+      false ->
+        settings
+        |> incorporate_conn_info(conn)
+        |> check_rate()
+        |> case do
+          {:ok, _count} ->
+            conn
+
+          {:error, _count} ->
+            render_throttled_error(conn)
+        end
     end
   end
 
+  def disabled? do
+    localhost_or_socket =
+      Pleroma.Config.get([Pleroma.Web.Endpoint, :http, :ip])
+      |> Tuple.to_list()
+      |> Enum.join(".")
+      |> String.match?(~r/^local|^127.0.0.1/)
+
+    remote_ip_disabled = not Pleroma.Config.get([Pleroma.Plugs.RemoteIp, :enabled])
+
+    localhost_or_socket and remote_ip_disabled
+  end
+
   def inspect_bucket(conn, name_root, settings) do
     settings =
       settings

lib/pleroma/plugs/remote_ip.ex

@@ -10,10 +10,7 @@ defmodule Pleroma.Plugs.RemoteIp do
   @behaviour Plug
 
   @headers ~w[
-    forwarded
     x-forwarded-for
-    x-client-ip
-    x-real-ip
   ]
 
   # https://en.wikipedia.org/wiki/Localhost

lib/pleroma/web/activity_pub/activity_pub.ex

@@ -325,12 +325,14 @@ def update(%{to: to, cc: cc, actor: actor, object: object} = params) do
   def react_with_emoji(user, object, emoji, options \\ []) do
     with local <- Keyword.get(options, :local, true),
          activity_id <- Keyword.get(options, :activity_id, nil),
-         Pleroma.Emoji.is_unicode_emoji?(emoji),
+         true <- Pleroma.Emoji.is_unicode_emoji?(emoji),
          reaction_data <- make_emoji_reaction_data(user, object, emoji, activity_id),
          {:ok, activity} <- insert(reaction_data, local),
          {:ok, object} <- add_emoji_reaction_to_object(activity, object),
          :ok <- maybe_federate(activity) do
       {:ok, activity, object}
+    else
+      e -> {:error, e}
     end
   end
 
@@ -345,6 +347,8 @@ def unreact_with_emoji(user, reaction_id, options \\ []) do
          {:ok, object} <- remove_emoji_reaction_from_object(reaction_activity, object),
          :ok <- maybe_federate(activity) do
       {:ok, activity, object}
+    else
+      e -> {:error, e}
     end
   end
 

lib/pleroma/web/mastodon_api/views/status_view.ex

@@ -256,7 +256,11 @@ def render("show.json", %{activity: %{data: %{"object" => _object}} = activity}
     emoji_reactions =
       with %{data: %{"reactions" => emoji_reactions}} <- object do
         Enum.map(emoji_reactions, fn [emoji, users] ->
-          %{emoji: emoji, count: length(users)}
+          %{
+            emoji: emoji,
+            count: length(users),
+            reacted: !!(opts[:for] && opts[:for].ap_id in users)
+          }
         end)
       else
         _ -> []

lib/pleroma/web/pleroma_api/controllers/emoji_api_controller.ex

@@ -573,11 +573,14 @@ def update_file(conn, %{"action" => action}) do
   assumed to be emojis and stored in the new `pack.json` file.
   """
   def import_from_fs(conn, _params) do
-    with {:ok, results} <- File.ls(emoji_dir_path()) do
+    emoji_path = emoji_dir_path()
+
+    with {:ok, %{access: :read_write}} <- File.stat(emoji_path),
+         {:ok, results} <- File.ls(emoji_path) do
       imported_pack_names =
         results
         |> Enum.filter(fn file ->
-          dir_path = Path.join(emoji_dir_path(), file)
+          dir_path = Path.join(emoji_path, file)
           # Find the directories that do NOT have pack.json
           File.dir?(dir_path) and not File.exists?(Path.join(dir_path, "pack.json"))
         end)
@@ -585,6 +588,11 @@ def import_from_fs(conn, _params) do
 
       json(conn, imported_pack_names)
     else
+      {:ok, %{access: _}} ->
+        conn
+        |> put_status(:internal_server_error)
+        |> json(%{error: "Error: emoji pack directory must be writable"})
+
       {:error, _} ->
         conn
         |> put_status(:internal_server_error)

lib/pleroma/web/pleroma_api/controllers/pleroma_api_controller.ex

@@ -47,13 +47,16 @@ def emoji_reactions_by(%{assigns: %{user: user}} = conn, %{"id" => activity_id})
            Object.normalize(activity) do
       reactions =
         emoji_reactions
-        |> Enum.map(fn [emoji, users] ->
-          users = Enum.map(users, &User.get_cached_by_ap_id/1)
+        |> Enum.map(fn [emoji, user_ap_ids] ->
+          users =
+            Enum.map(user_ap_ids, &User.get_cached_by_ap_id/1)
+            |> Enum.filter(& &1)
 
           %{
             emoji: emoji,
             count: length(users),
-            accounts: AccountView.render("index.json", %{users: users, for: user, as: :user})
+            accounts: AccountView.render("index.json", %{users: users, for: user, as: :user}),
+            reacted: !!(user && user.ap_id in user_ap_ids)
           }
         end)
 

lib/pleroma/web/rich_media/parsers/meta_tags_parser.ex

@@ -48,6 +48,6 @@ defp maybe_put_title(meta, html) when meta != %{} do
   defp maybe_put_title(meta, _), do: meta
 
   defp get_page_title(html) do
-    Floki.find(html, "title") |> List.first() |> Floki.text()
+    Floki.find(html, "html head title") |> List.first() |> Floki.text()
   end
 end

lib/pleroma/web/streamer/worker.ex

@@ -138,7 +138,8 @@ defp should_send?(%User{} = user, %Activity{} = item) do
 
     with parent <- Object.normalize(item) || item,
          true <-
-           Enum.all?([blocked_ap_ids, muted_ap_ids, reblog_muted_ap_ids], &(item.actor not in &1)),
+           Enum.all?([blocked_ap_ids, muted_ap_ids], &(item.actor not in &1)),
+         true <- item.data["type"] != "Announce" || item.actor not in reblog_muted_ap_ids,
          true <- Enum.all?([blocked_ap_ids, muted_ap_ids], &(parent.data["actor"] not in &1)),
          true <- MapSet.disjoint?(recipients, recipient_blocks),
          %{host: item_host} <- URI.parse(item.actor),

priv/static/index.html

@@ -1 +1 @@
-<!DOCTYPE html><html lang=en><head><meta charset=utf-8><meta name=viewport content="width=device-width,initial-scale=1,user-scalable=no"><title>Pleroma</title><!--server-generated-meta--><link rel=icon type=image/png href=/favicon.png><link href=/static/css/vendors~app.b2603a50868c68a1c192.css rel=stylesheet><link href=/static/css/app.ae04505b31bb0ee2765e.css rel=stylesheet><link href=/static/fontello.1579102213354.css rel=stylesheet></head><body class=hidden><noscript>To use Pleroma, please enable JavaScript.</noscript><div id=app></div><script type=text/javascript src=/static/js/vendors~app.86bc6d5e06d2e17976c5.js></script><script type=text/javascript src=/static/js/app.a43640742dacfb13b6b0.js></script></body></html>
+<!DOCTYPE html><html lang=en><head><meta charset=utf-8><meta name=viewport content="width=device-width,initial-scale=1,user-scalable=no"><title>Pleroma</title><!--server-generated-meta--><link rel=icon type=image/png href=/favicon.png><link href=/static/css/vendors~app.b2603a50868c68a1c192.css rel=stylesheet><link href=/static/css/app.ae04505b31bb0ee2765e.css rel=stylesheet><link href=/static/fontello.1580232989700.css rel=stylesheet></head><body class=hidden><noscript>To use Pleroma, please enable JavaScript.</noscript><div id=app></div><script type=text/javascript src=/static/js/vendors~app.9ab182239f3a2abee89f.js></script><script type=text/javascript src=/static/js/app.9cfed8f3d06c299128ea.js></script></body></html>

test/fixtures/emoji-reaction-no-emoji.json

@@ -0,0 +1,30 @@
+{
+  "type": "EmojiReaction",
+  "signature": {
+    "type": "RsaSignature2017",
+    "signatureValue": "fdxMfQSMwbC6wP6sh6neS/vM5879K67yQkHTbiT5Npr5wAac0y6+o3Ij+41tN3rL6wfuGTosSBTHOtta6R4GCOOhCaCSLMZKypnp1VltCzLDoyrZELnYQIC8gpUXVmIycZbREk22qWUe/w7DAFaKK4UscBlHDzeDVcA0K3Se5Sluqi9/Zh+ldAnEzj/rSEPDjrtvf5wGNf3fHxbKSRKFt90JvKK6hS+vxKUhlRFDf6/SMETw+EhwJSNW4d10yMUakqUWsFv4Acq5LW7l+HpYMvlYY1FZhNde1+uonnCyuQDyvzkff8zwtEJmAXC4RivO/VVLa17SmqheJZfI8oluVg==",
+    "creator": "http://mastodon.example.org/users/admin#main-key",
+    "created": "2018-02-17T18:57:49Z"
+  },
+  "object": "http://localtesting.pleroma.lol/objects/eb92579d-3417-42a8-8652-2492c2d4f454",
+  "content": "~",
+  "nickname": "lain",
+  "id": "http://mastodon.example.org/users/admin#reactions/2",
+  "actor": "http://mastodon.example.org/users/admin",
+  "@context": [
+    "https://www.w3.org/ns/activitystreams",
+    "https://w3id.org/security/v1",
+    {
+      "toot": "http://joinmastodon.org/ns#",
+      "sensitive": "as:sensitive",
+      "ostatus": "http://ostatus.org#",
+      "movedTo": "as:movedTo",
+      "manuallyApprovesFollowers": "as:manuallyApprovesFollowers",
+      "inReplyToAtomUri": "ostatus:inReplyToAtomUri",
+      "conversation": "ostatus:conversation",
+      "atomUri": "ostatus:atomUri",
+      "Hashtag": "as:Hashtag",
+      "Emoji": "toot:Emoji"
+    }
+  ]
+}

test/fixtures/emoji-reaction-too-long.json

@@ -0,0 +1,30 @@
+{
+  "type": "EmojiReaction",
+  "signature": {
+    "type": "RsaSignature2017",
+    "signatureValue": "fdxMfQSMwbC6wP6sh6neS/vM5879K67yQkHTbiT5Npr5wAac0y6+o3Ij+41tN3rL6wfuGTosSBTHOtta6R4GCOOhCaCSLMZKypnp1VltCzLDoyrZELnYQIC8gpUXVmIycZbREk22qWUe/w7DAFaKK4UscBlHDzeDVcA0K3Se5Sluqi9/Zh+ldAnEzj/rSEPDjrtvf5wGNf3fHxbKSRKFt90JvKK6hS+vxKUhlRFDf6/SMETw+EhwJSNW4d10yMUakqUWsFv4Acq5LW7l+HpYMvlYY1FZhNde1+uonnCyuQDyvzkff8zwtEJmAXC4RivO/VVLa17SmqheJZfI8oluVg==",
+    "creator": "http://mastodon.example.org/users/admin#main-key",
+    "created": "2018-02-17T18:57:49Z"
+  },
+  "object": "http://localtesting.pleroma.lol/objects/eb92579d-3417-42a8-8652-2492c2d4f454",
+  "content": "👌👌",
+  "nickname": "lain",
+  "id": "http://mastodon.example.org/users/admin#reactions/2",
+  "actor": "http://mastodon.example.org/users/admin",
+  "@context": [
+    "https://www.w3.org/ns/activitystreams",
+    "https://w3id.org/security/v1",
+    {
+      "toot": "http://joinmastodon.org/ns#",
+      "sensitive": "as:sensitive",
+      "ostatus": "http://ostatus.org#",
+      "movedTo": "as:movedTo",
+      "manuallyApprovesFollowers": "as:manuallyApprovesFollowers",
+      "inReplyToAtomUri": "ostatus:inReplyToAtomUri",
+      "conversation": "ostatus:conversation",
+      "atomUri": "ostatus:atomUri",
+      "Hashtag": "as:Hashtag",
+      "Emoji": "toot:Emoji"
+    }
+  ]
+}

test/plugs/rate_limiter_test.exs

@@ -16,6 +16,7 @@ defmodule Pleroma.Plugs.RateLimiterTest do
     test "config is required for plug to work" do
       limiter_name = :test_init
       Pleroma.Config.put([:rate_limit, limiter_name], {1, 1})
+      Pleroma.Config.put([Pleroma.Web.Endpoint, :http, :ip], {8, 8, 8, 8})
 
       assert %{limits: {1, 1}, name: :test_init, opts: [name: :test_init]} ==
                RateLimiter.init(name: limiter_name)
@@ -23,11 +24,39 @@ test "config is required for plug to work" do
       assert nil == RateLimiter.init(name: :foo)
     end
 
+    test "it is disabled for localhost" do
+      limiter_name = :test_init
+      Pleroma.Config.put([:rate_limit, limiter_name], {1, 1})
+      Pleroma.Config.put([Pleroma.Web.Endpoint, :http, :ip], {127, 0, 0, 1})
+      Pleroma.Config.put([Pleroma.Plugs.RemoteIp, :enabled], false)
+
+      assert RateLimiter.disabled?() == true
+    end
+
+    test "it is disabled for socket" do
+      limiter_name = :test_init
+      Pleroma.Config.put([:rate_limit, limiter_name], {1, 1})
+      Pleroma.Config.put([Pleroma.Web.Endpoint, :http, :ip], {:local, "/path/to/pleroma.sock"})
+      Pleroma.Config.put([Pleroma.Plugs.RemoteIp, :enabled], false)
+
+      assert RateLimiter.disabled?() == true
+    end
+
+    test "it is enabled for socket when remote ip is enabled" do
+      limiter_name = :test_init
+      Pleroma.Config.put([:rate_limit, limiter_name], {1, 1})
+      Pleroma.Config.put([Pleroma.Web.Endpoint, :http, :ip], {:local, "/path/to/pleroma.sock"})
+      Pleroma.Config.put([Pleroma.Plugs.RemoteIp, :enabled], true)
+
+      assert RateLimiter.disabled?() == false
+    end
+
     test "it restricts based on config values" do
       limiter_name = :test_opts
       scale = 80
       limit = 5
 
+      Pleroma.Config.put([Pleroma.Web.Endpoint, :http, :ip], {8, 8, 8, 8})
       Pleroma.Config.put([:rate_limit, limiter_name], {scale, limit})
 
       opts = RateLimiter.init(name: limiter_name)
@@ -61,6 +90,7 @@ test "`bucket_name` option overrides default bucket name" do
       limiter_name = :test_bucket_name
 
       Pleroma.Config.put([:rate_limit, limiter_name], {1000, 5})
+      Pleroma.Config.put([Pleroma.Web.Endpoint, :http, :ip], {8, 8, 8, 8})
 
       base_bucket_name = "#{limiter_name}:group1"
       opts = RateLimiter.init(name: limiter_name, bucket_name: base_bucket_name)
@@ -75,6 +105,7 @@ test "`bucket_name` option overrides default bucket name" do
     test "`params` option allows different queries to be tracked independently" do
       limiter_name = :test_params
       Pleroma.Config.put([:rate_limit, limiter_name], {1000, 5})
+      Pleroma.Config.put([Pleroma.Web.Endpoint, :http, :ip], {8, 8, 8, 8})
 
       opts = RateLimiter.init(name: limiter_name, params: ["id"])
 
@@ -90,6 +121,7 @@ test "`params` option allows different queries to be tracked independently" do
     test "it supports combination of options modifying bucket name" do
       limiter_name = :test_options_combo
       Pleroma.Config.put([:rate_limit, limiter_name], {1000, 5})
+      Pleroma.Config.put([Pleroma.Web.Endpoint, :http, :ip], {8, 8, 8, 8})
 
       base_bucket_name = "#{limiter_name}:group1"
       opts = RateLimiter.init(name: limiter_name, bucket_name: base_bucket_name, params: ["id"])
@@ -109,6 +141,7 @@ test "it supports combination of options modifying bucket name" do
     test "are restricted based on remote IP" do
       limiter_name = :test_unauthenticated
       Pleroma.Config.put([:rate_limit, limiter_name], [{1000, 5}, {1, 10}])
+      Pleroma.Config.put([Pleroma.Web.Endpoint, :http, :ip], {8, 8, 8, 8})
 
       opts = RateLimiter.init(name: limiter_name)
 
@@ -147,6 +180,7 @@ test "can have limits seperate from unauthenticated connections" do
 
       scale = 50
       limit = 5
+      Pleroma.Config.put([Pleroma.Web.Endpoint, :http, :ip], {8, 8, 8, 8})
       Pleroma.Config.put([:rate_limit, limiter_name], [{1000, 1}, {scale, limit}])
 
       opts = RateLimiter.init(name: limiter_name)
@@ -169,6 +203,7 @@ test "can have limits seperate from unauthenticated connections" do
     test "diffrerent users are counted independently" do
       limiter_name = :test_authenticated
       Pleroma.Config.put([:rate_limit, limiter_name], [{1, 10}, {1000, 5}])
+      Pleroma.Config.put([Pleroma.Web.Endpoint, :http, :ip], {8, 8, 8, 8})
 
       opts = RateLimiter.init(name: limiter_name)
 

test/web/activity_pub/transmogrifier_test.exs

@@ -395,6 +395,25 @@ test "it works for incoming emoji reactions" do
       assert data["content"] == "👌"
     end
 
+    test "it reject invalid emoji reactions" do
+      user = insert(:user)
+      {:ok, activity} = CommonAPI.post(user, %{"status" => "hello"})
+
+      data =
+        File.read!("test/fixtures/emoji-reaction-too-long.json")
+        |> Poison.decode!()
+        |> Map.put("object", activity.data["object"])
+
+      assert :error = Transmogrifier.handle_incoming(data)
+
+      data =
+        File.read!("test/fixtures/emoji-reaction-no-emoji.json")
+        |> Poison.decode!()
+        |> Map.put("object", activity.data["object"])
+
+      assert :error = Transmogrifier.handle_incoming(data)
+    end
+
     test "it works for incoming emoji reaction undos" do
       user = insert(:user)
 

test/web/common_api/common_api_test.exs

@@ -238,7 +238,9 @@ test "reacting to a status with an emoji" do
       assert reaction.data["actor"] == user.ap_id
       assert reaction.data["content"] == "👍"
 
-      # TODO: test error case.
+      {:ok, activity} = CommonAPI.post(other_user, %{"status" => "cofe"})
+
+      {:error, _} = CommonAPI.react_with_emoji(activity.id, user, ".")
     end
 
     test "unreacting to a status with an emoji" do

test/web/mastodon_api/controllers/account_controller_test.exs

@@ -668,6 +668,7 @@ test "returns error when user already registred", %{conn: conn, valid_params: va
     end
 
     test "rate limit", %{conn: conn} do
+      Pleroma.Config.put([Pleroma.Plugs.RemoteIp, :enabled], true)
       app_token = insert(:oauth_token, user: nil)
 
       conn =

test/web/mastodon_api/views/status_view_test.exs

@@ -37,8 +37,15 @@ test "has an emoji reaction list" do
     status = StatusView.render("show.json", activity: activity)
 
     assert status[:pleroma][:emoji_reactions] == [
-             %{emoji: "☕", count: 2},
-             %{emoji: "🍵", count: 1}
+             %{emoji: "☕", count: 2, reacted: false},
+             %{emoji: "🍵", count: 1, reacted: false}
+           ]
+
+    status = StatusView.render("show.json", activity: activity, for: user)
+
+    assert status[:pleroma][:emoji_reactions] == [
+             %{emoji: "☕", count: 2, reacted: true},
+             %{emoji: "🍵", count: 1, reacted: false}
            ]
   end
 

test/web/pleroma_api/controllers/emoji_api_controller_test.exs

@@ -6,7 +6,6 @@ defmodule Pleroma.Web.PleromaAPI.EmojiAPIControllerTest do
   use Pleroma.Web.ConnCase
 
   import Tesla.Mock
-
   import Pleroma.Factory
 
   @emoji_dir_path Path.join(

test/web/pleroma_api/controllers/pleroma_api_controller_test.exs

@@ -25,9 +25,14 @@ test "POST /api/v1/pleroma/statuses/:id/react_with_emoji", %{conn: conn} do
       |> assign(:user, other_user)
       |> assign(:token, insert(:oauth_token, user: other_user, scopes: ["write:statuses"]))
       |> post("/api/v1/pleroma/statuses/#{activity.id}/react_with_emoji", %{"emoji" => "☕"})
+      |> json_response(200)
 
-    assert %{"id" => id} = json_response(result, 200)
+    assert %{"id" => id} = result
     assert to_string(activity.id) == id
+
+    assert result["pleroma"]["emoji_reactions"] == [
+             %{"emoji" => "☕", "count" => 1, "reacted" => true}
+           ]
   end
 
   test "POST /api/v1/pleroma/statuses/:id/unreact_with_emoji", %{conn: conn} do
@@ -54,6 +59,7 @@ test "POST /api/v1/pleroma/statuses/:id/unreact_with_emoji", %{conn: conn} do
   test "GET /api/v1/pleroma/statuses/:id/emoji_reactions_by", %{conn: conn} do
     user = insert(:user)
     other_user = insert(:user)
+    doomed_user = insert(:user)
 
     {:ok, activity} = CommonAPI.post(user, %{"status" => "#cofe"})
 
@@ -65,14 +71,29 @@ test "GET /api/v1/pleroma/statuses/:id/emoji_reactions_by", %{conn: conn} do
     assert result == []
 
     {:ok, _, _} = CommonAPI.react_with_emoji(activity.id, other_user, "🎅")
+    {:ok, _, _} = CommonAPI.react_with_emoji(activity.id, doomed_user, "🎅")
+
+    User.perform(:delete, doomed_user)
 
     result =
       conn
       |> get("/api/v1/pleroma/statuses/#{activity.id}/emoji_reactions_by")
       |> json_response(200)
 
-    [%{"emoji" => "🎅", "count" => 1, "accounts" => [represented_user]}] = result
+    [%{"emoji" => "🎅", "count" => 1, "accounts" => [represented_user], "reacted" => false}] =
+      result
+
     assert represented_user["id"] == other_user.id
+
+    result =
+      conn
+      |> assign(:user, other_user)
+      |> assign(:token, insert(:oauth_token, user: other_user, scopes: ["read:statuses"]))
+      |> get("/api/v1/pleroma/statuses/#{activity.id}/emoji_reactions_by")
+      |> json_response(200)
+
+    assert [%{"emoji" => "🎅", "count" => 1, "accounts" => [_represented_user], "reacted" => true}] =
+             result
   end
 
   test "/api/v1/pleroma/conversations/:id" do

test/web/rich_media/parsers/twitter_card_test.exs

@@ -85,4 +85,19 @@ test "respect only first title tag on the page" do
                 image: image_path
               }}
   end
+
+  test "takes first founded title in html head if there is html markup error" do
+    html = File.read!("test/fixtures/nypd-facial-recognition-children-teenagers4.html")
+
+    assert TwitterCard.parse(html, %{}) ==
+             {:ok,
+              %{
+                site: nil,
+                title:
+                  "She Was Arrested at 14. Then Her Photo Went to a Facial Recognition Database. - The New York Times",
+                "app:id:googleplay": "com.nytimes.android",
+                "app:name:googleplay": "NYTimes",
+                "app:url:googleplay": "nytimes://reader/id/100000006583622"
+              }}
+  end
 end

test/web/streamer/streamer_test.exs

@@ -65,6 +65,9 @@ test "it doesn't send notify to the 'user:notification' stream when a user is bl
       blocked = insert(:user)
       {:ok, _user_relationship} = User.block(user, blocked)
 
+      {:ok, activity} = CommonAPI.post(user, %{"status" => ":("})
+      {:ok, notif, _} = CommonAPI.favorite(activity.id, blocked)
+
       task = Task.async(fn -> refute_receive {:text, _}, @streamer_timeout end)
 
       Streamer.add_socket(
@@ -72,9 +75,6 @@ test "it doesn't send notify to the 'user:notification' stream when a user is bl
         %{transport_pid: task.pid, assigns: %{user: user}}
       )
 
-      {:ok, activity} = CommonAPI.post(user, %{"status" => ":("})
-      {:ok, notif, _} = CommonAPI.favorite(activity.id, blocked)
-
       Streamer.stream("user:notification", notif)
       Task.await(task)
     end
@@ -83,6 +83,11 @@ test "it doesn't send notify to the 'user:notification' stream when a thread is
       user: user
     } do
       user2 = insert(:user)
+
+      {:ok, activity} = CommonAPI.post(user, %{"status" => "super hot take"})
+      {:ok, activity} = CommonAPI.add_mute(user, activity)
+      {:ok, notif, _} = CommonAPI.favorite(activity.id, user2)
+
       task = Task.async(fn -> refute_receive {:text, _}, @streamer_timeout end)
 
       Streamer.add_socket(
@@ -90,9 +95,6 @@ test "it doesn't send notify to the 'user:notification' stream when a thread is
         %{transport_pid: task.pid, assigns: %{user: user}}
       )
 
-      {:ok, activity} = CommonAPI.post(user, %{"status" => "super hot take"})
-      {:ok, activity} = CommonAPI.add_mute(user, activity)
-      {:ok, notif, _} = CommonAPI.favorite(activity.id, user2)
       Streamer.stream("user:notification", notif)
       Task.await(task)
     end
@@ -101,6 +103,11 @@ test "it doesn't send notify to the 'user:notification' stream' when a domain is
       user: user
     } do
       user2 = insert(:user, %{ap_id: "https://hecking-lewd-place.com/user/meanie"})
+
+      {:ok, user} = User.block_domain(user, "hecking-lewd-place.com")
+      {:ok, activity} = CommonAPI.post(user, %{"status" => "super hot take"})
+      {:ok, notif, _} = CommonAPI.favorite(activity.id, user2)
+
       task = Task.async(fn -> refute_receive {:text, _}, @streamer_timeout end)
 
       Streamer.add_socket(
@@ -108,10 +115,6 @@ test "it doesn't send notify to the 'user:notification' stream' when a domain is
         %{transport_pid: task.pid, assigns: %{user: user}}
       )
 
-      {:ok, user} = User.block_domain(user, "hecking-lewd-place.com")
-      {:ok, activity} = CommonAPI.post(user, %{"status" => "super hot take"})
-      {:ok, notif, _} = CommonAPI.favorite(activity.id, user2)
-
       Streamer.stream("user:notification", notif)
       Task.await(task)
     end
@@ -267,6 +270,8 @@ test "it doesn't send messages involving blocked users" do
       blocked_user = insert(:user)
       {:ok, _user_relationship} = User.block(user, blocked_user)
 
+      {:ok, activity} = CommonAPI.post(blocked_user, %{"status" => "Test"})
+
       task =
         Task.async(fn ->
           refute_receive {:text, _}, 1_000
@@ -277,8 +282,6 @@ test "it doesn't send messages involving blocked users" do
         user: user
       }
 
-      {:ok, activity} = CommonAPI.post(blocked_user, %{"status" => "Test"})
-
       topics = %{
         "public" => [fake_socket]
       }
@@ -335,6 +338,12 @@ test "it doesn't send unwanted DMs to list" do
     {:ok, list} = List.create("Test", user_a)
     {:ok, list} = List.follow(list, user_b)
 
+    {:ok, activity} =
+      CommonAPI.post(user_b, %{
+        "status" => "@#{user_c.nickname} Test",
+        "visibility" => "direct"
+      })
+
     task =
       Task.async(fn ->
         refute_receive {:text, _}, 1_000
@@ -345,12 +354,6 @@ test "it doesn't send unwanted DMs to list" do
       user: user_a
     }
 
-    {:ok, activity} =
-      CommonAPI.post(user_b, %{
-        "status" => "@#{user_c.nickname} Test",
-        "visibility" => "direct"
-      })
-
     topics = %{
       "list:#{list.id}" => [fake_socket]
     }
@@ -367,6 +370,12 @@ test "it doesn't send unwanted private posts to list" do
     {:ok, list} = List.create("Test", user_a)
     {:ok, list} = List.follow(list, user_b)
 
+    {:ok, activity} =
+      CommonAPI.post(user_b, %{
+        "status" => "Test",
+        "visibility" => "private"
+      })
+
     task =
       Task.async(fn ->
         refute_receive {:text, _}, 1_000
@@ -377,12 +386,6 @@ test "it doesn't send unwanted private posts to list" do
       user: user_a
     }
 
-    {:ok, activity} =
-      CommonAPI.post(user_b, %{
-        "status" => "Test",
-        "visibility" => "private"
-      })
-
     topics = %{
       "list:#{list.id}" => [fake_socket]
     }
@@ -401,6 +404,12 @@ test "it sends wanted private posts to list" do
     {:ok, list} = List.create("Test", user_a)
     {:ok, list} = List.follow(list, user_b)
 
+    {:ok, activity} =
+      CommonAPI.post(user_b, %{
+        "status" => "Test",
+        "visibility" => "private"
+      })
+
     task =
       Task.async(fn ->
         assert_receive {:text, _}, 1_000
@@ -411,12 +420,6 @@ test "it sends wanted private posts to list" do
       user: user_a
     }
 
-    {:ok, activity} =
-      CommonAPI.post(user_b, %{
-        "status" => "Test",
-        "visibility" => "private"
-      })
-
     Streamer.add_socket(
       "list:#{list.id}",
       fake_socket
@@ -433,6 +436,9 @@ test "it doesn't send muted reblogs" do
     user3 = insert(:user)
     CommonAPI.hide_reblogs(user1, user2)
 
+    {:ok, create_activity} = CommonAPI.post(user3, %{"status" => "I'm kawen"})
+    {:ok, announce_activity, _} = CommonAPI.repeat(create_activity.id, user2)
+
     task =
       Task.async(fn ->
         refute_receive {:text, _}, 1_000
@@ -443,9 +449,6 @@ test "it doesn't send muted reblogs" do
       user: user1
     }
 
-    {:ok, create_activity} = CommonAPI.post(user3, %{"status" => "I'm kawen"})
-    {:ok, announce_activity, _} = CommonAPI.repeat(create_activity.id, user2)
-
     topics = %{
       "public" => [fake_socket]
     }
@@ -455,6 +458,34 @@ test "it doesn't send muted reblogs" do
     Task.await(task)
   end
 
+  test "it does send non-reblog notification for reblog-muted actors" do
+    user1 = insert(:user)
+    user2 = insert(:user)
+    user3 = insert(:user)
+    CommonAPI.hide_reblogs(user1, user2)
+
+    {:ok, create_activity} = CommonAPI.post(user3, %{"status" => "I'm kawen"})
+    {:ok, favorite_activity, _} = CommonAPI.favorite(create_activity.id, user2)
+
+    task =
+      Task.async(fn ->
+        assert_receive {:text, _}, 1_000
+      end)
+
+    fake_socket = %StreamerSocket{
+      transport_pid: task.pid,
+      user: user1
+    }
+
+    topics = %{
+      "public" => [fake_socket]
+    }
+
+    Worker.push_to_socket(topics, "public", favorite_activity)
+
+    Task.await(task)
+  end
+
   test "it doesn't send posts from muted threads" do
     user = insert(:user)
     user2 = insert(:user)