Merge remote-tracking branch 'pleroma/develop' into feature/addressable-lists

.mailmap

@@ -0,0 +1,2 @@
+Ariadne Conill <ariadne@dereferenced.org> <nenolod@dereferenced.org>
+Ariadne Conill <ariadne@dereferenced.org> <nenolod@gmail.com>

CC-BY-NC-ND-4.0

@@ -1,403 +0,0 @@
-Attribution-NonCommercial-NoDerivatives 4.0 International
-
-=======================================================================
-
-Creative Commons Corporation ("Creative Commons") is not a law firm and
-does not provide legal services or legal advice. Distribution of
-Creative Commons public licenses does not create a lawyer-client or
-other relationship. Creative Commons makes its licenses and related
-information available on an "as-is" basis. Creative Commons gives no
-warranties regarding its licenses, any material licensed under their
-terms and conditions, or any related information. Creative Commons
-disclaims all liability for damages resulting from their use to the
-fullest extent possible.
-
-Using Creative Commons Public Licenses
-
-Creative Commons public licenses provide a standard set of terms and
-conditions that creators and other rights holders may use to share
-original works of authorship and other material subject to copyright
-and certain other rights specified in the public license below. The
-following considerations are for informational purposes only, are not
-exhaustive, and do not form part of our licenses.
-
-     Considerations for licensors: Our public licenses are
-     intended for use by those authorized to give the public
-     permission to use material in ways otherwise restricted by
-     copyright and certain other rights. Our licenses are
-     irrevocable. Licensors should read and understand the terms
-     and conditions of the license they choose before applying it.
-     Licensors should also secure all rights necessary before
-     applying our licenses so that the public can reuse the
-     material as expected. Licensors should clearly mark any
-     material not subject to the license. This includes other CC-
-     licensed material, or material used under an exception or
-     limitation to copyright. More considerations for licensors:
-	wiki.creativecommons.org/Considerations_for_licensors
-
-     Considerations for the public: By using one of our public
-     licenses, a licensor grants the public permission to use the
-     licensed material under specified terms and conditions. If
-     the licensor's permission is not necessary for any reason--for
-     example, because of any applicable exception or limitation to
-     copyright--then that use is not regulated by the license. Our
-     licenses grant only permissions under copyright and certain
-     other rights that a licensor has authority to grant. Use of
-     the licensed material may still be restricted for other
-     reasons, including because others have copyright or other
-     rights in the material. A licensor may make special requests,
-     such as asking that all changes be marked or described.
-     Although not required by our licenses, you are encouraged to
-     respect those requests where reasonable. More considerations
-     for the public: 
-	wiki.creativecommons.org/Considerations_for_licensees
-
-=======================================================================
-
-Creative Commons Attribution-NonCommercial-NoDerivatives 4.0
-International Public License
-
-By exercising the Licensed Rights (defined below), You accept and agree
-to be bound by the terms and conditions of this Creative Commons
-Attribution-NonCommercial-NoDerivatives 4.0 International Public
-License ("Public License"). To the extent this Public License may be
-interpreted as a contract, You are granted the Licensed Rights in
-consideration of Your acceptance of these terms and conditions, and the
-Licensor grants You such rights in consideration of benefits the
-Licensor receives from making the Licensed Material available under
-these terms and conditions.
-
-
-Section 1 -- Definitions.
-
-  a. Adapted Material means material subject to Copyright and Similar
-     Rights that is derived from or based upon the Licensed Material
-     and in which the Licensed Material is translated, altered,
-     arranged, transformed, or otherwise modified in a manner requiring
-     permission under the Copyright and Similar Rights held by the
-     Licensor. For purposes of this Public License, where the Licensed
-     Material is a musical work, performance, or sound recording,
-     Adapted Material is always produced where the Licensed Material is
-     synched in timed relation with a moving image.
-
-  b. Copyright and Similar Rights means copyright and/or similar rights
-     closely related to copyright including, without limitation,
-     performance, broadcast, sound recording, and Sui Generis Database
-     Rights, without regard to how the rights are labeled or
-     categorized. For purposes of this Public License, the rights
-     specified in Section 2(b)(1)-(2) are not Copyright and Similar
-     Rights.
-
-  c. Effective Technological Measures means those measures that, in the
-     absence of proper authority, may not be circumvented under laws
-     fulfilling obligations under Article 11 of the WIPO Copyright
-     Treaty adopted on December 20, 1996, and/or similar international
-     agreements.
-
-  d. Exceptions and Limitations means fair use, fair dealing, and/or
-     any other exception or limitation to Copyright and Similar Rights
-     that applies to Your use of the Licensed Material.
-
-  e. Licensed Material means the artistic or literary work, database,
-     or other material to which the Licensor applied this Public
-     License.
-
-  f. Licensed Rights means the rights granted to You subject to the
-     terms and conditions of this Public License, which are limited to
-     all Copyright and Similar Rights that apply to Your use of the
-     Licensed Material and that the Licensor has authority to license.
-
-  g. Licensor means the individual(s) or entity(ies) granting rights
-     under this Public License.
-
-  h. NonCommercial means not primarily intended for or directed towards
-     commercial advantage or monetary compensation. For purposes of
-     this Public License, the exchange of the Licensed Material for
-     other material subject to Copyright and Similar Rights by digital
-     file-sharing or similar means is NonCommercial provided there is
-     no payment of monetary compensation in connection with the
-     exchange.
-
-  i. Share means to provide material to the public by any means or
-     process that requires permission under the Licensed Rights, such
-     as reproduction, public display, public performance, distribution,
-     dissemination, communication, or importation, and to make material
-     available to the public including in ways that members of the
-     public may access the material from a place and at a time
-     individually chosen by them.
-
-  j. Sui Generis Database Rights means rights other than copyright
-     resulting from Directive 96/9/EC of the European Parliament and of
-     the Council of 11 March 1996 on the legal protection of databases,
-     as amended and/or succeeded, as well as other essentially
-     equivalent rights anywhere in the world.
-
-  k. You means the individual or entity exercising the Licensed Rights
-     under this Public License. Your has a corresponding meaning.
-
-
-Section 2 -- Scope.
-
-  a. License grant.
-
-       1. Subject to the terms and conditions of this Public License,
-          the Licensor hereby grants You a worldwide, royalty-free,
-          non-sublicensable, non-exclusive, irrevocable license to
-          exercise the Licensed Rights in the Licensed Material to:
-
-            a. reproduce and Share the Licensed Material, in whole or
-               in part, for NonCommercial purposes only; and
-
-            b. produce and reproduce, but not Share, Adapted Material
-               for NonCommercial purposes only.
-
-       2. Exceptions and Limitations. For the avoidance of doubt, where
-          Exceptions and Limitations apply to Your use, this Public
-          License does not apply, and You do not need to comply with
-          its terms and conditions.
-
-       3. Term. The term of this Public License is specified in Section
-          6(a).
-
-       4. Media and formats; technical modifications allowed. The
-          Licensor authorizes You to exercise the Licensed Rights in
-          all media and formats whether now known or hereafter created,
-          and to make technical modifications necessary to do so. The
-          Licensor waives and/or agrees not to assert any right or
-          authority to forbid You from making technical modifications
-          necessary to exercise the Licensed Rights, including
-          technical modifications necessary to circumvent Effective
-          Technological Measures. For purposes of this Public License,
-          simply making modifications authorized by this Section 2(a)
-          (4) never produces Adapted Material.
-
-       5. Downstream recipients.
-
-            a. Offer from the Licensor -- Licensed Material. Every
-               recipient of the Licensed Material automatically
-               receives an offer from the Licensor to exercise the
-               Licensed Rights under the terms and conditions of this
-               Public License.
-
-            b. No downstream restrictions. You may not offer or impose
-               any additional or different terms or conditions on, or
-               apply any Effective Technological Measures to, the
-               Licensed Material if doing so restricts exercise of the
-               Licensed Rights by any recipient of the Licensed
-               Material.
-
-       6. No endorsement. Nothing in this Public License constitutes or
-          may be construed as permission to assert or imply that You
-          are, or that Your use of the Licensed Material is, connected
-          with, or sponsored, endorsed, or granted official status by,
-          the Licensor or others designated to receive attribution as
-          provided in Section 3(a)(1)(A)(i).
-
-  b. Other rights.
-
-       1. Moral rights, such as the right of integrity, are not
-          licensed under this Public License, nor are publicity,
-          privacy, and/or other similar personality rights; however, to
-          the extent possible, the Licensor waives and/or agrees not to
-          assert any such rights held by the Licensor to the limited
-          extent necessary to allow You to exercise the Licensed
-          Rights, but not otherwise.
-
-       2. Patent and trademark rights are not licensed under this
-          Public License.
-
-       3. To the extent possible, the Licensor waives any right to
-          collect royalties from You for the exercise of the Licensed
-          Rights, whether directly or through a collecting society
-          under any voluntary or waivable statutory or compulsory
-          licensing scheme. In all other cases the Licensor expressly
-          reserves any right to collect such royalties, including when
-          the Licensed Material is used other than for NonCommercial
-          purposes.
-
-
-Section 3 -- License Conditions.
-
-Your exercise of the Licensed Rights is expressly made subject to the
-following conditions.
-
-  a. Attribution.
-
-       1. If You Share the Licensed Material, You must:
-
-            a. retain the following if it is supplied by the Licensor
-               with the Licensed Material:
-
-                 i. identification of the creator(s) of the Licensed
-                    Material and any others designated to receive
-                    attribution, in any reasonable manner requested by
-                    the Licensor (including by pseudonym if
-                    designated);
-
-                ii. a copyright notice;
-
-               iii. a notice that refers to this Public License;
-
-                iv. a notice that refers to the disclaimer of
-                    warranties;
-
-                 v. a URI or hyperlink to the Licensed Material to the
-                    extent reasonably practicable;
-
-            b. indicate if You modified the Licensed Material and
-               retain an indication of any previous modifications; and
-
-            c. indicate the Licensed Material is licensed under this
-               Public License, and include the text of, or the URI or
-               hyperlink to, this Public License.
-
-          For the avoidance of doubt, You do not have permission under
-          this Public License to Share Adapted Material.
-
-       2. You may satisfy the conditions in Section 3(a)(1) in any
-          reasonable manner based on the medium, means, and context in
-          which You Share the Licensed Material. For example, it may be
-          reasonable to satisfy the conditions by providing a URI or
-          hyperlink to a resource that includes the required
-          information.
-
-       3. If requested by the Licensor, You must remove any of the
-          information required by Section 3(a)(1)(A) to the extent
-          reasonably practicable.
-
-
-Section 4 -- Sui Generis Database Rights.
-
-Where the Licensed Rights include Sui Generis Database Rights that
-apply to Your use of the Licensed Material:
-
-  a. for the avoidance of doubt, Section 2(a)(1) grants You the right
-     to extract, reuse, reproduce, and Share all or a substantial
-     portion of the contents of the database for NonCommercial purposes
-     only and provided You do not Share Adapted Material;
-
-  b. if You include all or a substantial portion of the database
-     contents in a database in which You have Sui Generis Database
-     Rights, then the database in which You have Sui Generis Database
-     Rights (but not its individual contents) is Adapted Material; and
-
-  c. You must comply with the conditions in Section 3(a) if You Share
-     all or a substantial portion of the contents of the database.
-
-For the avoidance of doubt, this Section 4 supplements and does not
-replace Your obligations under this Public License where the Licensed
-Rights include other Copyright and Similar Rights.
-
-
-Section 5 -- Disclaimer of Warranties and Limitation of Liability.
-
-  a. UNLESS OTHERWISE SEPARATELY UNDERTAKEN BY THE LICENSOR, TO THE
-     EXTENT POSSIBLE, THE LICENSOR OFFERS THE LICENSED MATERIAL AS-IS
-     AND AS-AVAILABLE, AND MAKES NO REPRESENTATIONS OR WARRANTIES OF
-     ANY KIND CONCERNING THE LICENSED MATERIAL, WHETHER EXPRESS,
-     IMPLIED, STATUTORY, OR OTHER. THIS INCLUDES, WITHOUT LIMITATION,
-     WARRANTIES OF TITLE, MERCHANTABILITY, FITNESS FOR A PARTICULAR
-     PURPOSE, NON-INFRINGEMENT, ABSENCE OF LATENT OR OTHER DEFECTS,
-     ACCURACY, OR THE PRESENCE OR ABSENCE OF ERRORS, WHETHER OR NOT
-     KNOWN OR DISCOVERABLE. WHERE DISCLAIMERS OF WARRANTIES ARE NOT
-     ALLOWED IN FULL OR IN PART, THIS DISCLAIMER MAY NOT APPLY TO YOU.
-
-  b. TO THE EXTENT POSSIBLE, IN NO EVENT WILL THE LICENSOR BE LIABLE
-     TO YOU ON ANY LEGAL THEORY (INCLUDING, WITHOUT LIMITATION,
-     NEGLIGENCE) OR OTHERWISE FOR ANY DIRECT, SPECIAL, INDIRECT,
-     INCIDENTAL, CONSEQUENTIAL, PUNITIVE, EXEMPLARY, OR OTHER LOSSES,
-     COSTS, EXPENSES, OR DAMAGES ARISING OUT OF THIS PUBLIC LICENSE OR
-     USE OF THE LICENSED MATERIAL, EVEN IF THE LICENSOR HAS BEEN
-     ADVISED OF THE POSSIBILITY OF SUCH LOSSES, COSTS, EXPENSES, OR
-     DAMAGES. WHERE A LIMITATION OF LIABILITY IS NOT ALLOWED IN FULL OR
-     IN PART, THIS LIMITATION MAY NOT APPLY TO YOU.
-
-  c. The disclaimer of warranties and limitation of liability provided
-     above shall be interpreted in a manner that, to the extent
-     possible, most closely approximates an absolute disclaimer and
-     waiver of all liability.
-
-
-Section 6 -- Term and Termination.
-
-  a. This Public License applies for the term of the Copyright and
-     Similar Rights licensed here. However, if You fail to comply with
-     this Public License, then Your rights under this Public License
-     terminate automatically.
-
-  b. Where Your right to use the Licensed Material has terminated under
-     Section 6(a), it reinstates:
-
-       1. automatically as of the date the violation is cured, provided
-          it is cured within 30 days of Your discovery of the
-          violation; or
-
-       2. upon express reinstatement by the Licensor.
-
-     For the avoidance of doubt, this Section 6(b) does not affect any
-     right the Licensor may have to seek remedies for Your violations
-     of this Public License.
-
-  c. For the avoidance of doubt, the Licensor may also offer the
-     Licensed Material under separate terms or conditions or stop
-     distributing the Licensed Material at any time; however, doing so
-     will not terminate this Public License.
-
-  d. Sections 1, 5, 6, 7, and 8 survive termination of this Public
-     License.
-
-
-Section 7 -- Other Terms and Conditions.
-
-  a. The Licensor shall not be bound by any additional or different
-     terms or conditions communicated by You unless expressly agreed.
-
-  b. Any arrangements, understandings, or agreements regarding the
-     Licensed Material not stated herein are separate from and
-     independent of the terms and conditions of this Public License.
-
-
-Section 8 -- Interpretation.
-
-  a. For the avoidance of doubt, this Public License does not, and
-     shall not be interpreted to, reduce, limit, restrict, or impose
-     conditions on any use of the Licensed Material that could lawfully
-     be made without permission under this Public License.
-
-  b. To the extent possible, if any provision of this Public License is
-     deemed unenforceable, it shall be automatically reformed to the
-     minimum extent necessary to make it enforceable. If the provision
-     cannot be reformed, it shall be severed from this Public License
-     without affecting the enforceability of the remaining terms and
-     conditions.
-
-  c. No term or condition of this Public License will be waived and no
-     failure to comply consented to unless expressly agreed to by the
-     Licensor.
-
-  d. Nothing in this Public License constitutes or may be interpreted
-     as a limitation upon, or waiver of, any privileges and immunities
-     that apply to the Licensor or You, including from the legal
-     processes of any jurisdiction or authority.
-
-=======================================================================
-
-Creative Commons is not a party to its public
-licenses. Notwithstanding, Creative Commons may elect to apply one of
-its public licenses to material it publishes and in those instances
-will be considered the “Licensor.” The text of the Creative Commons
-public licenses is dedicated to the public domain under the CC0 Public
-Domain Dedication. Except for the limited purpose of indicating that
-material is shared under a Creative Commons public license or as
-otherwise permitted by the Creative Commons policies published at
-creativecommons.org/policies, Creative Commons does not authorize the
-use of the trademark "Creative Commons" or any other trademark or logo
-of Creative Commons without its prior written consent including,
-without limitation, in connection with any unauthorized modifications
-to any of its public licenses or any other arrangements,
-understandings, or agreements concerning use of licensed material. For
-the avoidance of doubt, this paragraph does not form part of the
-public licenses.
-
-Creative Commons may be contacted at creativecommons.org.
-

CHANGELOG.md

@@ -8,28 +8,46 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/).
 - **Breaking:** Configuration: A setting to explicitly disable the mailer was added, defaulting to true, if you are using a mailer add `config :pleroma, Pleroma.Emails.Mailer, enabled: true` to your config
 - Configuration: OpenGraph and TwitterCard providers enabled by default
 - Configuration: Filter.AnonymizeFilename added ability to retain file extension with custom text
+- Federation: Return 403 errors when trying to request pages from a user's follower/following collections if they have `hide_followers`/`hide_follows` set
 - NodeInfo: Return `skipThreadContainment` in `metadata` for the `skip_thread_containment` option
+- Mastodon API: Unsubscribe followers when they unfollow a user
 
 ### Fixed
 - Not being able to pin unlisted posts
 - Metadata rendering errors resulting in the entire page being inaccessible
+- Federation/MediaProxy not working with instances that have wrong certificate order
 - Mastodon API: Handling of search timeouts (`/api/v1/search` and `/api/v2/search`)
 - Mastodon API: Embedded relationships not being properly rendered in the Account entity of Status entity
+- Mastodon API: Add `account_id`, `type`, `offset`, and `limit` to search API (`/api/v1/search` and `/api/v2/search`)
+- ActivityPub C2S: follower/following collection pages being inaccessible even when authentifucated if `hide_followers`/ `hide_follows` was set
 
 ### Added
 - MRF: Support for priming the mediaproxy cache (`Pleroma.Web.ActivityPub.MRF.MediaProxyWarmingPolicy`)
-Configuration: `federation_incoming_replies_max_depth` option
+- MRF: Support for excluding specific domains from Transparency.
+- Configuration: `federation_incoming_replies_max_depth` option
 - Mastodon API: Support for the [`tagged` filter](https://github.com/tootsuite/mastodon/pull/9755) in [`GET /api/v1/accounts/:id/statuses`](https://docs.joinmastodon.org/api/rest/accounts/#get-api-v1-accounts-id-statuses)
 - Mastodon API, streaming: Add support for passing the token in the `Sec-WebSocket-Protocol` header
 - Mastodon API, extension: Ability to reset avatar, profile banner, and background
+- Mastodon API: Add support for categories for custom emojis by reusing the group feature. <https://github.com/tootsuite/mastodon/pull/11196>
+- Mastodon API: Add support for muting/unmuting notifications
 - Admin API: Return users' tags when querying reports
 - Admin API: Return avatar and display name when querying users
 - Admin API: Allow querying user by ID
+- Admin API: Added support for `tuples`.
 - Added synchronization of following/followers counters for external users
 - Configuration: `enabled` option for `Pleroma.Emails.Mailer`, defaulting to `false`.
-- Mastodon API: Add support for categories for custom emojis by reusing the group feature. <https://github.com/tootsuite/mastodon/pull/11196>
+- Configuration: Pleroma.Plugs.RateLimiter `bucket_name`, `params` options.
 - Addressable lists
 
+### Changed
+- Configuration: Filter.AnonymizeFilename added ability to retain file extension with custom text
+- Admin API: changed json structure for saving config settings.
+- RichMedia: parsers and their order are configured in `rich_media` config.
+
+## [1.0.1] - 2019-07-14
+### Security
+- OStatus: fix an object spoofing vulnerability.
+
 ## [1.0.0] - 2019-06-29
 ### Security
 - Mastodon API: Fix display names not being sanitized

config/config.exs

@@ -194,6 +194,8 @@
   send_user_agent: true,
   adapter: [
     ssl_options: [
+      # Workaround for remote server certificate chain issues
+      partial_chain: &:hackney_connect.partial_chain/1,
       # We don't support TLS v1.3 yet
       versions: [:tlsv1, :"tlsv1.1", :"tlsv1.2"]
     ]
@@ -238,6 +240,7 @@
     "text/bbcode"
   ],
   mrf_transparency: true,
+  mrf_transparency_exclusions: [],
   autofollowed_nicknames: [],
   max_pinned_statuses: 1,
   no_attachment_links: false,
@@ -250,13 +253,7 @@
   skip_thread_containment: true,
   limit_to_local_content: :unauthenticated,
   dynamic_configuration: false,
-  external_user_synchronization: [
+  external_user_synchronization: true
-    enabled: false,
-    # every 2 hours
-    interval: 60 * 60 * 2,
-    max_retries: 3,
-    limit: 500
-  ]
 
 config :pleroma, :markup,
   # XXX - unfortunately, inline images must be enabled by default right now, because
@@ -342,7 +339,12 @@
 config :pleroma, :rich_media,
   enabled: true,
   ignore_hosts: [],
-  ignore_tld: ["local", "localdomain", "lan"]
+  ignore_tld: ["local", "localdomain", "lan"],
+  parsers: [
+    Pleroma.Web.RichMedia.Parsers.TwitterCard,
+    Pleroma.Web.RichMedia.Parsers.OGP,
+    Pleroma.Web.RichMedia.Parsers.OEmbed
+  ]
 
 config :pleroma, :media_proxy,
   enabled: false,
@@ -525,7 +527,9 @@
 
 config :pleroma, :rate_limit,
   search: [{1000, 10}, {1000, 30}],
-  app_account_creation: {1_800_000, 25}
+  app_account_creation: {1_800_000, 25},
+  statuses_actions: {10_000, 15},
+  status_id_action: {60_000, 3}
 
 # Import environment specific config. This must remain at the bottom
 # of this file so it overrides the configuration defined above.

config/test.exs

@@ -65,7 +65,9 @@
   total_user_limit: 3,
   enabled: false
 
-config :pleroma, :rate_limit, app_account_creation: {10_000, 5}
+config :pleroma, :rate_limit,
+  search: [{1000, 30}, {1000, 30}],
+  app_account_creation: {10_000, 5}
 
 config :pleroma, :http_security, report_uri: "https://endpoint.com"
 

docs/api/admin_api.md

@@ -573,7 +573,7 @@ Note: Available `:permission_group` is currently moderator and admin. 404 is ret
   configs: [
     {
       "group": string,
-      "key": string,
+      "key": string or string with leading `:` for atoms,
       "value": string or {} or [] or {"tuple": []}
      }
   ]
@@ -583,10 +583,11 @@ Note: Available `:permission_group` is currently moderator and admin. 404 is ret
 ## `/api/pleroma/admin/config`
 ### Update config settings
 Module name can be passed as string, which starts with `Pleroma`, e.g. `"Pleroma.Upload"`.
-Atom or boolean value can be passed with `:` in the beginning, e.g. `":true"`, `":upload"`. For keys it is not needed.
+Atom keys and values can be passed with `:` in the beginning, e.g. `":upload"`.
-Integer with `i:`, e.g. `"i:150"`.
+Tuples can be passed as `{"tuple": ["first_val", Pleroma.Module, []]}`.
-Tuple with more than 2 values with `{"tuple": ["first_val", Pleroma.Module, []]}`.
 `{"tuple": ["some_string", "Pleroma.Some.Module", []]}` will be converted to `{"some_string", Pleroma.Some.Module, []}`.
+Keywords can be passed as lists with 2 child tuples, e.g.
+`[{"tuple": ["first_val", Pleroma.Module]}, {"tuple": ["second_val", true]}]`.
 
 Compile time settings (need instance reboot):
 - all settings by this keys:
@@ -603,7 +604,7 @@ Compile time settings (need instance reboot):
 - Params:
   - `configs` => [
     - `group` (string)
-    - `key` (string)
+    - `key` (string or string with leading `:` for atoms)
     - `value` (string, [], {} or {"tuple": []})
     - `delete` = true (optional, if parameter must be deleted)
   ]
@@ -616,24 +617,25 @@ Compile time settings (need instance reboot):
     {
       "group": "pleroma",
       "key": "Pleroma.Upload",
-      "value": {
+      "value": [
-        "uploader": "Pleroma.Uploaders.Local",
+        {"tuple": [":uploader", "Pleroma.Uploaders.Local"]},
-        "filters": ["Pleroma.Upload.Filter.Dedupe"],
+        {"tuple": [":filters", ["Pleroma.Upload.Filter.Dedupe"]]},
-        "link_name": ":true",
+        {"tuple": [":link_name", true]},
-        "proxy_remote": ":false",
+        {"tuple": [":proxy_remote", false]},
-        "proxy_opts": {
+        {"tuple": [":proxy_opts", [
-          "redirect_on_failure": ":false",
+          {"tuple": [":redirect_on_failure", false]},
-          "max_body_length": "i:1048576",
+          {"tuple": [":max_body_length", 1048576]},
-          "http": {
+          {"tuple": [":http": [
-            "follow_redirect": ":true",
+            {"tuple": [":follow_redirect", true]},
-            "pool": ":upload"
+            {"tuple": [":pool", ":upload"]},
-          }
+          ]]}
-        },
+        ]
-        "dispatch": {
+        ]},
+        {"tuple": [":dispatch", {
           "tuple": ["/api/v1/streaming", "Pleroma.Web.MastodonAPI.WebsocketHandler", []]
-        }
+        }]}
-      }
+      ]
-     }
+    }
   ]
 }
 
@@ -644,7 +646,7 @@ Compile time settings (need instance reboot):
   configs: [
     {
       "group": string,
-      "key": string,
+      "key": string or string with leading `:` for atoms,
       "value": string or {} or [] or {"tuple": []}
      }
   ]

docs/api/differences_in_mastoapi_responses.md

@@ -46,14 +46,6 @@ Has these additional fields under the `pleroma` object:
 - `settings_store`: A generic map of settings for frontends. Opaque to the backend. Only returned in `verify_credentials` and `update_credentials`
 - `chat_token`: The token needed for Pleroma chat. Only returned in `verify_credentials`
 
-### Extensions for PleromaFE
-
-These endpoints added for controlling PleromaFE features over the Mastodon API
-
-- PATCH `/api/v1/accounts/update_avatar`: Set/clear user avatar image
-- PATCH `/api/v1/accounts/update_banner`: Set/clear user banner image
-- PATCH `/api/v1/accounts/update_background`: Set/clear user background image
-
 ### Source
 
 Has these additional fields under the `pleroma` object:

docs/api/pleroma_api.md

@@ -238,6 +238,13 @@ See [Admin-API](Admin-API.md)
 ]
 ```
 
+## `/api/v1/pleroma/accounts/update_*`
+### Set and clear account avatar, banner, and background
+
+- PATCH `/api/v1/pleroma/accounts/update_avatar`: Set/clear user avatar image
+- PATCH `/api/v1/pleroma/accounts/update_banner`: Set/clear user banner image
+- PATCH `/api/v1/pleroma/accounts/update_background`: Set/clear user background image
+
 ## `/api/v1/pleroma/mascot`
 ### Gets user mascot image
 * Method `GET`

docs/config.md

@@ -106,6 +106,7 @@ config :pleroma, Pleroma.Emails.Mailer,
 * `managed_config`: Whenether the config for pleroma-fe is configured in this config or in ``static/config.json``
 * `allowed_post_formats`: MIME-type list of formats allowed to be posted (transformed into HTML)
 * `mrf_transparency`: Make the content of your Message Rewrite Facility settings public (via nodeinfo).
+* `mrf_transparency_exclusions`: Exclude specific instance names from MRF transparency.  The use of the exclusions feature will be disclosed in nodeinfo as a boolean value.
 * `scope_copy`: Copy the scope (private/unlisted/public) in replies to posts by default.
 * `subject_line_behavior`: Allows changing the default behaviour of subject lines in replies. Valid values:
   * "email": Copy and preprend re:, as in email.
@@ -126,11 +127,7 @@ config :pleroma, Pleroma.Emails.Mailer,
 * `skip_thread_containment`: Skip filter out broken threads. The default is `false`.
 * `limit_to_local_content`: Limit unauthenticated users to search for local statutes and users only. Possible values: `:unauthenticated`, `:all` and `false`. The default is `:unauthenticated`.
 * `dynamic_configuration`: Allow transferring configuration to DB with the subsequent customization from Admin api.
-* `external_user_synchronization`: Following/followers counters synchronization settings.
+* `external_user_synchronization`: Enabling following/followers counters synchronization for external users.
-  * `enabled`: Enables synchronization
-  * `interval`: Interval between synchronization.
-  * `max_retries`: Max rettries for host. After exceeding the limit, the check will not be carried out for users from this host.
-  * `limit`: Users batch size for processing in one time.
 
 
 
@@ -428,6 +425,7 @@ This config contains two queues: `federator_incoming` and `federator_outgoing`.
 * `enabled`: if enabled the instance will parse metadata from attached links to generate link previews
 * `ignore_hosts`: list of hosts which will be ignored by the metadata parser. For example `["accounts.google.com", "xss.website"]`, defaults to `[]`.
 * `ignore_tld`: list TLDs (top-level domains) which will ignore for parse metadata. default is ["local", "localdomain", "lan"]
+* `parsers`: list of Rich Media parsers
 
 ## :fetch_initial_posts
 * `enabled`: if enabled, when a new user is federated with, fetch some of their latest posts
@@ -644,3 +642,10 @@ A keyword list of rate limiters where a key is a limiter name and value is the l
 It is also possible to have different limits for unauthenticated and authenticated users: the keyword value must be a list of two tuples where the first one is a config for unauthenticated users and the second one is for authenticated.
 
 See [`Pleroma.Plugs.RateLimiter`](Pleroma.Plugs.RateLimiter.html) documentation for examples.
+
+Supported rate limiters:
+
+* `:search` for the search requests (account & status search etc.)
+* `:app_account_creation` for registering user accounts from the same IP address
+* `:statuses_actions` for create / delete / fav / unfav / reblog / unreblog actions on any statuses
+* `:status_id_action` for fav / unfav or reblog / unreblog actions on the same status by the same user

lib/pleroma/activity.ex

@@ -344,5 +344,5 @@ def restrict_deactivated_users(query) do
     )
   end
 
-  defdelegate search(user, query), to: Pleroma.Activity.Search
+  defdelegate search(user, query, options \\ []), to: Pleroma.Activity.Search
 end

lib/pleroma/activity/search.ex

@@ -5,14 +5,17 @@
 defmodule Pleroma.Activity.Search do
   alias Pleroma.Activity
   alias Pleroma.Object.Fetcher
-  alias Pleroma.Repo
+  alias Pleroma.Pagination
   alias Pleroma.User
   alias Pleroma.Web.ActivityPub.Visibility
 
   import Ecto.Query
 
-  def search(user, search_query) do
+  def search(user, search_query, options \\ []) do
     index_type = if Pleroma.Config.get([:database, :rum_enabled]), do: :rum, else: :gin
+    limit = Enum.min([Keyword.get(options, :limit), 40])
+    offset = Keyword.get(options, :offset, 0)
+    author = Keyword.get(options, :author)
 
     Activity
     |> Activity.with_preloaded_object()
@@ -20,15 +23,23 @@ def search(user, search_query) do
     |> restrict_public()
     |> query_with(index_type, search_query)
     |> maybe_restrict_local(user)
-    |> Repo.all()
+    |> maybe_restrict_author(author)
+    |> Pagination.fetch_paginated(%{"offset" => offset, "limit" => limit}, :offset)
     |> maybe_fetch(user, search_query)
   end
 
+  def maybe_restrict_author(query, %User{} = author) do
+    from([a, o] in query,
+      where: a.actor == ^author.ap_id
+    )
+  end
+
+  def maybe_restrict_author(query, _), do: query
+
   defp restrict_public(q) do
     from([a, o] in q,
       where: fragment("?->>'type' = 'Create'", a.data),
-      where: "https://www.w3.org/ns/activitystreams#Public" in a.recipients,
+      where: "https://www.w3.org/ns/activitystreams#Public" in a.recipients
-      limit: 40
     )
   end
 

lib/pleroma/application.ex

@@ -151,11 +151,7 @@ def start(_type, _args) do
             start: {Pleroma.Web.Endpoint, :start_link, []},
             type: :supervisor
           },
-          %{id: Pleroma.Gopher.Server, start: {Pleroma.Gopher.Server, :start_link, []}},
+          %{id: Pleroma.Gopher.Server, start: {Pleroma.Gopher.Server, :start_link, []}}
-          %{
-            id: Pleroma.User.SynchronizationWorker,
-            start: {Pleroma.User.SynchronizationWorker, :start_link, []}
-          }
         ]
 
     # See http://elixir-lang.org/docs/stable/elixir/Supervisor.html

lib/pleroma/http/connection.ex

@@ -29,7 +29,7 @@ def new(opts \\ []) do
 
   # fetch Hackney options
   #
-  defp hackney_options(opts) do
+  def hackney_options(opts) do
     options = Keyword.get(opts, :adapter, [])
     adapter_options = Pleroma.Config.get([:http, :adapter], [])
     proxy_url = Pleroma.Config.get([:http, :proxy_url], nil)

lib/pleroma/http/http.ex

@@ -65,10 +65,7 @@ defp process_sni_options(options, url) do
   end
 
   def process_request_options(options) do
-    case Pleroma.Config.get([:http, :proxy_url]) do
+    Keyword.merge(Pleroma.HTTP.Connection.hackney_options([]), options)
-      nil -> options
-      proxy -> options ++ [proxy: proxy]
-    end
   end
 
   @doc """

lib/pleroma/notification.ex

@@ -11,7 +11,6 @@ defmodule Pleroma.Notification do
   alias Pleroma.Pagination
   alias Pleroma.Repo
   alias Pleroma.User
-  alias Pleroma.Web.CommonAPI
   alias Pleroma.Web.CommonAPI.Utils
   alias Pleroma.Web.Push
   alias Pleroma.Web.Streamer
@@ -32,31 +31,47 @@ def changeset(%Notification{} = notification, attrs) do
     |> cast(attrs, [:seen])
   end
 
-  def for_user_query(user) do
+  def for_user_query(user, opts) do
-    Notification
+    query =
-    |> where(user_id: ^user.id)
+      Notification
-    |> where(
+      |> where(user_id: ^user.id)
-      [n, a],
+      |> where(
-      fragment(
+        [n, a],
-        "? not in (SELECT ap_id FROM users WHERE info->'deactivated' @> 'true')",
-        a.actor
-      )
-    )
-    |> join(:inner, [n], activity in assoc(n, :activity))
-    |> join(:left, [n, a], object in Object,
-      on:
         fragment(
-          "(?->>'id') = COALESCE((? -> 'object'::text) ->> 'id'::text)",
+          "? not in (SELECT ap_id FROM users WHERE info->'deactivated' @> 'true')",
-          object.data,
+          a.actor
-          a.data
         )
-    )
+      )
-    |> preload([n, a, o], activity: {a, object: o})
+      |> join(:inner, [n], activity in assoc(n, :activity))
+      |> join(:left, [n, a], object in Object,
+        on:
+          fragment(
+            "(?->>'id') = COALESCE((? -> 'object'::text) ->> 'id'::text)",
+            object.data,
+            a.data
+          )
+      )
+      |> preload([n, a, o], activity: {a, object: o})
+
+    if opts[:with_muted] do
+      query
+    else
+      where(query, [n, a], a.actor not in ^user.info.muted_notifications)
+      |> where([n, a], a.actor not in ^user.info.blocks)
+      |> where(
+        [n, a],
+        fragment("substring(? from '.*://([^/]*)')", a.actor) not in ^user.info.domain_blocks
+      )
+      |> join(:left, [n, a], tm in Pleroma.ThreadMute,
+        on: tm.user_id == ^user.id and tm.context == fragment("?->>'context'", a.data)
+      )
+      |> where([n, a, o, tm], is_nil(tm.id))
+    end
   end
 
   def for_user(user, opts \\ %{}) do
     user
-    |> for_user_query()
+    |> for_user_query(opts)
     |> Pagination.fetch_paginated(opts)
   end
 
@@ -179,11 +194,10 @@ def get_notified_from_activity(
 
   def get_notified_from_activity(_, _local_only), do: []
 
+  @spec skip?(Activity.t(), User.t()) :: boolean()
   def skip?(activity, user) do
     [
       :self,
-      :blocked,
-      :muted,
       :followers,
       :follows,
       :non_followers,
@@ -193,21 +207,11 @@ def skip?(activity, user) do
     |> Enum.any?(&skip?(&1, activity, user))
   end
 
+  @spec skip?(atom(), Activity.t(), User.t()) :: boolean()
   def skip?(:self, activity, user) do
     activity.data["actor"] == user.ap_id
   end
 
-  def skip?(:blocked, activity, user) do
-    actor = activity.data["actor"]
-    User.blocks?(user, %{ap_id: actor})
-  end
-
-  def skip?(:muted, activity, user) do
-    actor = activity.data["actor"]
-
-    User.mutes?(user, %{ap_id: actor}) or CommonAPI.thread_muted?(user, activity)
-  end
-
   def skip?(
         :followers,
         activity,

lib/pleroma/object/containment.ex

@@ -48,6 +48,9 @@ def contain_origin(id, %{"actor" => _actor} = params) do
     end
   end
 
+  def contain_origin(id, %{"attributedTo" => actor} = params),
+    do: contain_origin(id, Map.put(params, "actor", actor))
+
   def contain_origin_from_id(_id, %{"id" => nil}), do: :error
 
   def contain_origin_from_id(id, %{"id" => other_id} = _params) do
@@ -60,4 +63,9 @@ def contain_origin_from_id(id, %{"id" => other_id} = _params) do
       :error
     end
   end
+
+  def contain_child(%{"object" => %{"id" => id, "attributedTo" => _} = object}),
+    do: contain_origin(id, object)
+
+  def contain_child(_), do: :ok
 end

lib/pleroma/object/fetcher.ex

@@ -32,33 +32,39 @@ def fetch_object_from_id(id, options \\ []) do
     else
       Logger.info("Fetching #{id} via AP")
 
-      with {:ok, data} <- fetch_and_contain_remote_object_from_id(id),
+      with {:fetch, {:ok, data}} <- {:fetch, fetch_and_contain_remote_object_from_id(id)},
-           nil <- Object.normalize(data, false),
+           {:normalize, nil} <- {:normalize, Object.normalize(data, false)},
            params <- %{
              "type" => "Create",
              "to" => data["to"],
              "cc" => data["cc"],
+             # Should we seriously keep this attributedTo thing?
              "actor" => data["actor"] || data["attributedTo"],
              "object" => data
            },
-           :ok <- Containment.contain_origin(id, params),
+           {:containment, :ok} <- {:containment, Containment.contain_origin(id, params)},
            {:ok, activity} <- Transmogrifier.handle_incoming(params, options),
            {:object, _data, %Object{} = object} <-
              {:object, data, Object.normalize(activity, false)} do
         {:ok, object}
       else
+        {:containment, _} ->
+          {:error, "Object containment failed."}
+
         {:error, {:reject, nil}} ->
           {:reject, nil}
 
         {:object, data, nil} ->
           reinject_object(data)
 
-        object = %Object{} ->
+        {:normalize, object = %Object{}} ->
           {:ok, object}
 
         _e ->
+          # Only fallback when receiving a fetch/normalization error with ActivityPub
           Logger.info("Couldn't get object via AP, trying out OStatus fetching...")
 
+          # FIXME: OStatus Object Containment?
           case OStatus.fetch_activity_from_url(id) do
             {:ok, [activity | _]} -> {:ok, Object.normalize(activity, false)}
             e -> e

lib/pleroma/pagination.ex

@@ -14,16 +14,28 @@ defmodule Pleroma.Pagination do
 
   @default_limit 20
 
-  def fetch_paginated(query, params) do
+  def fetch_paginated(query, params, type \\ :keyset)
+
+  def fetch_paginated(query, params, :keyset) do
     options = cast_params(params)
 
     query
-    |> paginate(options)
+    |> paginate(options, :keyset)
     |> Repo.all()
     |> enforce_order(options)
   end
 
-  def paginate(query, options) do
+  def fetch_paginated(query, params, :offset) do
+    options = cast_params(params)
+
+    query
+    |> paginate(options, :offset)
+    |> Repo.all()
+  end
+
+  def paginate(query, options, method \\ :keyset)
+
+  def paginate(query, options, :keyset) do
     query
     |> restrict(:min_id, options)
     |> restrict(:since_id, options)
@@ -32,11 +44,18 @@ def paginate(query, options) do
     |> restrict(:limit, options)
   end
 
+  def paginate(query, options, :offset) do
+    query
+    |> restrict(:offset, options)
+    |> restrict(:limit, options)
+  end
+
   defp cast_params(params) do
     param_types = %{
       min_id: :string,
       since_id: :string,
       max_id: :string,
+      offset: :integer,
       limit: :integer
     }
 
@@ -70,6 +89,10 @@ defp restrict(query, :order, _options) do
     order_by(query, [u], fragment("? desc nulls last", u.id))
   end
 
+  defp restrict(query, :offset, %{offset: offset}) do
+    offset(query, ^offset)
+  end
+
   defp restrict(query, :limit, options) do
     limit = Map.get(options, :limit, @default_limit)
 

lib/pleroma/plugs/rate_limiter.ex

@@ -31,12 +31,28 @@ defmodule Pleroma.Plugs.RateLimiter do
 
   ## Usage
 
+  AllowedSyntax:
+
+      plug(Pleroma.Plugs.RateLimiter, :limiter_name)
+      plug(Pleroma.Plugs.RateLimiter, {:limiter_name, options})
+
+  Allowed options:
+
+      * `bucket_name` overrides bucket name (e.g. to have a separate limit for a set of actions)
+      * `params` appends values of specified request params (e.g. ["id"]) to bucket name
+
   Inside a controller:
 
       plug(Pleroma.Plugs.RateLimiter, :one when action == :one)
       plug(Pleroma.Plugs.RateLimiter, :two when action in [:two, :three])
 
-  or inside a router pipiline:
+      plug(
+        Pleroma.Plugs.RateLimiter,
+        {:status_id_action, bucket_name: "status_id_action:fav_unfav", params: ["id"]}
+        when action in ~w(fav_status unfav_status)a
+      )
+
+  or inside a router pipeline:
 
       pipeline :api do
         ...
@@ -49,33 +65,56 @@ defmodule Pleroma.Plugs.RateLimiter do
 
   alias Pleroma.User
 
-  def init(limiter_name) do
+  def init(limiter_name) when is_atom(limiter_name) do
+    init({limiter_name, []})
+  end
+
+  def init({limiter_name, opts}) do
     case Pleroma.Config.get([:rate_limit, limiter_name]) do
       nil -> nil
-      config -> {limiter_name, config}
+      config -> {limiter_name, config, opts}
     end
   end
 
-  # do not limit if there is no limiter configuration
+  # Do not limit if there is no limiter configuration
   def call(conn, nil), do: conn
 
-  def call(conn, opts) do
+  def call(conn, settings) do
-    case check_rate(conn, opts) do
+    case check_rate(conn, settings) do
-      {:ok, _count} -> conn
+      {:ok, _count} ->
-      {:error, _count} -> render_throttled_error(conn)
+        conn
+
+      {:error, _count} ->
+        render_throttled_error(conn)
     end
   end
 
-  defp check_rate(%{assigns: %{user: %User{id: user_id}}}, {limiter_name, [_, {scale, limit}]}) do
+  defp bucket_name(conn, limiter_name, opts) do
-    ExRated.check_rate("#{limiter_name}:#{user_id}", scale, limit)
+    bucket_name = opts[:bucket_name] || limiter_name
+
+    if params_names = opts[:params] do
+      params_values = for p <- Enum.sort(params_names), do: conn.params[p]
+      Enum.join([bucket_name] ++ params_values, ":")
+    else
+      bucket_name
+    end
   end
 
-  defp check_rate(conn, {limiter_name, [{scale, limit} | _]}) do
+  defp check_rate(
-    ExRated.check_rate("#{limiter_name}:#{ip(conn)}", scale, limit)
+         %{assigns: %{user: %User{id: user_id}}} = conn,
+         {limiter_name, [_, {scale, limit}], opts}
+       ) do
+    bucket_name = bucket_name(conn, limiter_name, opts)
+    ExRated.check_rate("#{bucket_name}:#{user_id}", scale, limit)
   end
 
-  defp check_rate(conn, {limiter_name, {scale, limit}}) do
+  defp check_rate(conn, {limiter_name, [{scale, limit} | _], opts}) do
-    check_rate(conn, {limiter_name, [{scale, limit}]})
+    bucket_name = bucket_name(conn, limiter_name, opts)
+    ExRated.check_rate("#{bucket_name}:#{ip(conn)}", scale, limit)
+  end
+
+  defp check_rate(conn, {limiter_name, {scale, limit}, opts}) do
+    check_rate(conn, {limiter_name, [{scale, limit}, {scale, limit}], opts})
   end
 
   def ip(%{remote_ip: remote_ip}) do

lib/pleroma/reverse_proxy/reverse_proxy.ex

@@ -61,7 +61,7 @@ defmodule Pleroma.ReverseProxy do
   * `http`: options for [hackney](https://github.com/benoitc/hackney).
 
   """
-  @default_hackney_options []
+  @default_hackney_options [pool: :media]
 
   @inline_content_types [
     "image/gif",
@@ -94,7 +94,8 @@ def call(_conn, _url, _opts \\ [])
 
   def call(conn = %{method: method}, url, opts) when method in @methods do
     hackney_opts =
-      @default_hackney_options
+      Pleroma.HTTP.Connection.hackney_options([])
+      |> Keyword.merge(@default_hackney_options)
       |> Keyword.merge(Keyword.get(opts, :http, []))
       |> HTTP.process_request_options()
 

lib/pleroma/user.ex

@@ -52,6 +52,7 @@ defmodule Pleroma.User do
     field(:avatar, :map)
     field(:local, :boolean, default: true)
     field(:follower_address, :string)
+    field(:following_address, :string)
     field(:search_rank, :float, virtual: true)
     field(:search_type, :integer, virtual: true)
     field(:tags, {:array, :string}, default: [])
@@ -107,6 +108,10 @@ def ap_id(%User{nickname: nickname}) do
   def ap_followers(%User{follower_address: fa}) when is_binary(fa), do: fa
   def ap_followers(%User{} = user), do: "#{ap_id(user)}/followers"
 
+  @spec ap_following(User.t()) :: Sring.t()
+  def ap_following(%User{following_address: fa}) when is_binary(fa), do: fa
+  def ap_following(%User{} = user), do: "#{ap_id(user)}/following"
+
   def user_info(%User{} = user, args \\ %{}) do
     following_count =
       if args[:following_count], do: args[:following_count], else: following_count(user)
@@ -128,6 +133,7 @@ def set_info_cache(user, args) do
     Cachex.put(:user_cache, "user_info:#{user.id}", user_info(user, args))
   end
 
+  @spec restrict_deactivated(Ecto.Query.t()) :: Ecto.Query.t()
   def restrict_deactivated(query) do
     from(u in query,
       where: not fragment("? \\? 'deactivated' AND ?->'deactivated' @> 'true'", u.info, u.info)
@@ -162,9 +168,10 @@ def remote_user_creation(params) do
 
     if changes.valid? do
       case info_cng.changes[:source_data] do
-        %{"followers" => followers} ->
+        %{"followers" => followers, "following" => following} ->
           changes
           |> put_change(:follower_address, followers)
+          |> put_change(:following_address, following)
 
         _ ->
           followers = User.ap_followers(%User{nickname: changes.changes[:nickname]})
@@ -196,7 +203,14 @@ def upgrade_changeset(struct, params \\ %{}) do
       |> User.Info.user_upgrade(params[:info])
 
     struct
-    |> cast(params, [:bio, :name, :follower_address, :avatar, :last_refreshed_at])
+    |> cast(params, [
+      :bio,
+      :name,
+      :follower_address,
+      :following_address,
+      :avatar,
+      :last_refreshed_at
+    ])
     |> unique_constraint(:nickname)
     |> validate_format(:nickname, local_nickname_regex())
     |> validate_length(:bio, max: 5000)
@@ -735,10 +749,13 @@ def get_recipients_from_activity(%Activity{recipients: to}) do
     |> Repo.all()
   end
 
-  def mute(muter, %User{ap_id: ap_id}) do
+  @spec mute(User.t(), User.t(), boolean()) :: {:ok, User.t()} | {:error, String.t()}
+  def mute(muter, %User{ap_id: ap_id}, notifications? \\ true) do
+    info = muter.info
+
     info_cng =
-      muter.info
+      User.Info.add_to_mutes(info, ap_id)
-      |> User.Info.add_to_mutes(ap_id)
+      |> User.Info.add_to_muted_notifications(info, ap_id, notifications?)
 
     cng =
       change(muter)
@@ -748,9 +765,11 @@ def mute(muter, %User{ap_id: ap_id}) do
   end
 
   def unmute(muter, %{ap_id: ap_id}) do
+    info = muter.info
+
     info_cng =
-      muter.info
+      User.Info.remove_from_mutes(info, ap_id)
-      |> User.Info.remove_from_mutes(ap_id)
+      |> User.Info.remove_from_muted_notifications(info, ap_id)
 
     cng =
       change(muter)
@@ -846,6 +865,12 @@ def unblock(blocker, %{ap_id: ap_id}) do
   def mutes?(nil, _), do: false
   def mutes?(user, %{ap_id: ap_id}), do: Enum.member?(user.info.mutes, ap_id)
 
+  @spec muted_notifications?(User.t() | nil, User.t() | map()) :: boolean()
+  def muted_notifications?(nil, _), do: false
+
+  def muted_notifications?(user, %{ap_id: ap_id}),
+    do: Enum.member?(user.info.muted_notifications, ap_id)
+
   def blocks?(%User{info: info} = _user, %{ap_id: ap_id}) do
     blocks = info.blocks
     domain_blocks = info.domain_blocks
@@ -1012,42 +1037,20 @@ def perform(:follow_import, %User{} = follower, followed_identifiers)
     )
   end
 
-  @spec sync_follow_counter() :: :ok
+  @spec external_users_query() :: Ecto.Query.t()
-  def sync_follow_counter,
+  def external_users_query do
-    do: PleromaJobQueue.enqueue(:background, __MODULE__, [:sync_follow_counters])
+    User.Query.build(%{
-
+      external: true,
-  @spec perform(:sync_follow_counters) :: :ok
+      active: true,
-  def perform(:sync_follow_counters) do
+      order_by: :id
-    {:ok, _pid} = Agent.start_link(fn -> %{} end, name: :domain_errors)
+    })
-    config = Pleroma.Config.get([:instance, :external_user_synchronization])
-
-    :ok = sync_follow_counters(config)
-    Agent.stop(:domain_errors)
-  end
-
-  @spec sync_follow_counters(keyword()) :: :ok
-  def sync_follow_counters(opts \\ []) do
-    users = external_users(opts)
-
-    if length(users) > 0 do
-      errors = Agent.get(:domain_errors, fn state -> state end)
-      {last, updated_errors} = User.Synchronization.call(users, errors, opts)
-      Agent.update(:domain_errors, fn _state -> updated_errors end)
-      sync_follow_counters(max_id: last.id, limit: opts[:limit])
-    else
-      :ok
-    end
   end
 
   @spec external_users(keyword()) :: [User.t()]
   def external_users(opts \\ []) do
     query =
-      User.Query.build(%{
+      external_users_query()
-        external: true,
+      |> select([u], struct(u, [:id, :ap_id, :info]))
-        active: true,
-        order_by: :id,
-        select: [:id, :ap_id, :info]
-      })
 
     query =
       if opts[:max_id],

lib/pleroma/user/info.ex

@@ -24,6 +24,7 @@ defmodule Pleroma.User.Info do
     field(:domain_blocks, {:array, :string}, default: [])
     field(:mutes, {:array, :string}, default: [])
     field(:muted_reblogs, {:array, :string}, default: [])
+    field(:muted_notifications, {:array, :string}, default: [])
     field(:subscribers, {:array, :string}, default: [])
     field(:deactivated, :boolean, default: false)
     field(:no_rich_text, :boolean, default: false)
@@ -120,6 +121,16 @@ def set_mutes(info, mutes) do
     |> validate_required([:mutes])
   end
 
+  @spec set_notification_mutes(Changeset.t(), [String.t()], boolean()) :: Changeset.t()
+  def set_notification_mutes(changeset, muted_notifications, notifications?) do
+    if notifications? do
+      put_change(changeset, :muted_notifications, muted_notifications)
+      |> validate_required([:muted_notifications])
+    else
+      changeset
+    end
+  end
+
   def set_blocks(info, blocks) do
     params = %{blocks: blocks}
 
@@ -136,14 +147,31 @@ def set_subscribers(info, subscribers) do
     |> validate_required([:subscribers])
   end
 
+  @spec add_to_mutes(Info.t(), String.t()) :: Changeset.t()
   def add_to_mutes(info, muted) do
     set_mutes(info, Enum.uniq([muted | info.mutes]))
   end
 
+  @spec add_to_muted_notifications(Changeset.t(), Info.t(), String.t(), boolean()) ::
+          Changeset.t()
+  def add_to_muted_notifications(changeset, info, muted, notifications?) do
+    set_notification_mutes(
+      changeset,
+      Enum.uniq([muted | info.muted_notifications]),
+      notifications?
+    )
+  end
+
+  @spec remove_from_mutes(Info.t(), String.t()) :: Changeset.t()
   def remove_from_mutes(info, muted) do
     set_mutes(info, List.delete(info.mutes, muted))
   end
 
+  @spec remove_from_muted_notifications(Changeset.t(), Info.t(), String.t()) :: Changeset.t()
+  def remove_from_muted_notifications(changeset, info, muted) do
+    set_notification_mutes(changeset, List.delete(info.muted_notifications, muted), true)
+  end
+
   def add_to_block(info, blocked) do
     set_blocks(info, Enum.uniq([blocked | info.blocks]))
   end

lib/pleroma/user/search.ex

@@ -3,6 +3,7 @@
 # SPDX-License-Identifier: AGPL-3.0-only
 
 defmodule Pleroma.User.Search do
+  alias Pleroma.Pagination
   alias Pleroma.Repo
   alias Pleroma.User
   import Ecto.Query
@@ -32,8 +33,7 @@ def search(query_string, opts \\ []) do
 
         query_string
         |> search_query(for_user, following)
-        |> paginate(result_limit, offset)
+        |> Pagination.fetch_paginated(%{"offset" => offset, "limit" => result_limit}, :offset)
-        |> Repo.all()
       end)
 
     results
@@ -87,10 +87,6 @@ defp filter_blocked_domains(query, %User{info: %{domain_blocks: domain_blocks}})
 
   defp filter_blocked_domains(query, _), do: query
 
-  defp paginate(query, limit, offset) do
-    from(q in query, limit: ^limit, offset: ^offset)
-  end
-
   defp union_subqueries({fts_subquery, trigram_subquery}) do
     from(s in trigram_subquery, union_all: ^fts_subquery)
   end

lib/pleroma/user/synchronization.ex

@@ -1,60 +0,0 @@
-# Pleroma: A lightweight social networking server
-# Copyright © 2017-2018 Pleroma Authors <https://pleroma.social/>
-# SPDX-License-Identifier: AGPL-3.0-only
-
-defmodule Pleroma.User.Synchronization do
-  alias Pleroma.HTTP
-  alias Pleroma.User
-
-  @spec call([User.t()], map(), keyword()) :: {User.t(), map()}
-  def call(users, errors, opts \\ []) do
-    do_call(users, errors, opts)
-  end
-
-  defp do_call([user | []], errors, opts) do
-    updated = fetch_counters(user, errors, opts)
-    {user, updated}
-  end
-
-  defp do_call([user | others], errors, opts) do
-    updated = fetch_counters(user, errors, opts)
-    do_call(others, updated, opts)
-  end
-
-  defp fetch_counters(user, errors, opts) do
-    %{host: host} = URI.parse(user.ap_id)
-
-    info = %{}
-    {following, errors} = fetch_counter(user.ap_id <> "/following", host, errors, opts)
-    info = if following, do: Map.put(info, :following_count, following), else: info
-
-    {followers, errors} = fetch_counter(user.ap_id <> "/followers", host, errors, opts)
-    info = if followers, do: Map.put(info, :follower_count, followers), else: info
-
-    User.set_info_cache(user, info)
-    errors
-  end
-
-  defp available_domain?(domain, errors, opts) do
-    max_retries = Keyword.get(opts, :max_retries, 3)
-    not (Map.has_key?(errors, domain) && errors[domain] >= max_retries)
-  end
-
-  defp fetch_counter(url, host, errors, opts) do
-    with true <- available_domain?(host, errors, opts),
-         {:ok, %{body: body, status: code}} when code in 200..299 <-
-           HTTP.get(
-             url,
-             [{:Accept, "application/activity+json"}]
-           ),
-         {:ok, data} <- Jason.decode(body) do
-      {data["totalItems"], errors}
-    else
-      false ->
-        {nil, errors}
-
-      _ ->
-        {nil, Map.update(errors, host, 1, &(&1 + 1))}
-    end
-  end
-end

lib/pleroma/user/synchronization_worker.ex

@@ -1,32 +0,0 @@
-# Pleroma: A lightweight social networking server
-# Copyright © 2017-2018 Pleroma Authors <https://pleroma.social/>
-# SPDX-License-Identifier: AGPL-3.0-onl
-
-defmodule Pleroma.User.SynchronizationWorker do
-  use GenServer
-
-  def start_link do
-    config = Pleroma.Config.get([:instance, :external_user_synchronization])
-
-    if config[:enabled] do
-      GenServer.start_link(__MODULE__, interval: config[:interval])
-    else
-      :ignore
-    end
-  end
-
-  def init(opts) do
-    schedule_next(opts)
-    {:ok, opts}
-  end
-
-  def handle_info(:sync_follow_counters, opts) do
-    Pleroma.User.sync_follow_counter()
-    schedule_next(opts)
-    {:noreply, opts}
-  end
-
-  defp schedule_next(opts) do
-    Process.send_after(self(), :sync_follow_counters, opts[:interval])
-  end
-end

lib/pleroma/web/activity_pub/activity_pub.ex

@@ -8,6 +8,7 @@ defmodule Pleroma.Web.ActivityPub.ActivityPub do
   alias Pleroma.Conversation
   alias Pleroma.Notification
   alias Pleroma.Object
+  alias Pleroma.Object.Containment
   alias Pleroma.Object.Fetcher
   alias Pleroma.Pagination
   alias Pleroma.Repo
@@ -125,6 +126,7 @@ def insert(map, local \\ true, fake \\ false) when is_map(map) do
          {:ok, map} <- MRF.filter(map),
          {recipients, _, _} = get_recipients(map),
          {:fake, false, map, recipients} <- {:fake, fake, map, recipients},
+         :ok <- Containment.contain_child(map),
          {:ok, map, object} <- insert_full_object(map) do
       {:ok, activity} =
         Repo.insert(%Activity{
@@ -1011,6 +1013,7 @@ defp object_to_user_data(data) do
       avatar: avatar,
       name: data["name"],
       follower_address: data["followers"],
+      following_address: data["following"],
       bio: data["summary"]
     }
 

lib/pleroma/web/activity_pub/activity_pub_controller.ex

@@ -103,43 +103,57 @@ def activity(conn, %{"uuid" => uuid}) do
     end
   end
 
-  def following(conn, %{"nickname" => nickname, "page" => page}) do
+  def following(%{assigns: %{user: for_user}} = conn, %{"nickname" => nickname, "page" => page}) do
     with %User{} = user <- User.get_cached_by_nickname(nickname),
-         {:ok, user} <- User.ensure_keys_present(user) do
+         {user, for_user} <- ensure_user_keys_present_and_maybe_refresh_for_user(user, for_user),
+         {:show_follows, true} <-
+           {:show_follows, (for_user && for_user == user) || !user.info.hide_follows} do
       {page, _} = Integer.parse(page)
 
       conn
       |> put_resp_header("content-type", "application/activity+json")
-      |> json(UserView.render("following.json", %{user: user, page: page}))
+      |> json(UserView.render("following.json", %{user: user, page: page, for: for_user}))
+    else
+      {:show_follows, _} ->
+        conn
+        |> put_resp_header("content-type", "application/activity+json")
+        |> send_resp(403, "")
     end
   end
 
-  def following(conn, %{"nickname" => nickname}) do
+  def following(%{assigns: %{user: for_user}} = conn, %{"nickname" => nickname}) do
     with %User{} = user <- User.get_cached_by_nickname(nickname),
-         {:ok, user} <- User.ensure_keys_present(user) do
+         {user, for_user} <- ensure_user_keys_present_and_maybe_refresh_for_user(user, for_user) do
       conn
       |> put_resp_header("content-type", "application/activity+json")
-      |> json(UserView.render("following.json", %{user: user}))
+      |> json(UserView.render("following.json", %{user: user, for: for_user}))
     end
   end
 
-  def followers(conn, %{"nickname" => nickname, "page" => page}) do
+  def followers(%{assigns: %{user: for_user}} = conn, %{"nickname" => nickname, "page" => page}) do
     with %User{} = user <- User.get_cached_by_nickname(nickname),
-         {:ok, user} <- User.ensure_keys_present(user) do
+         {user, for_user} <- ensure_user_keys_present_and_maybe_refresh_for_user(user, for_user),
+         {:show_followers, true} <-
+           {:show_followers, (for_user && for_user == user) || !user.info.hide_followers} do
       {page, _} = Integer.parse(page)
 
       conn
       |> put_resp_header("content-type", "application/activity+json")
-      |> json(UserView.render("followers.json", %{user: user, page: page}))
+      |> json(UserView.render("followers.json", %{user: user, page: page, for: for_user}))
+    else
+      {:show_followers, _} ->
+        conn
+        |> put_resp_header("content-type", "application/activity+json")
+        |> send_resp(403, "")
     end
   end
 
-  def followers(conn, %{"nickname" => nickname}) do
+  def followers(%{assigns: %{user: for_user}} = conn, %{"nickname" => nickname}) do
     with %User{} = user <- User.get_cached_by_nickname(nickname),
-         {:ok, user} <- User.ensure_keys_present(user) do
+         {user, for_user} <- ensure_user_keys_present_and_maybe_refresh_for_user(user, for_user) do
       conn
       |> put_resp_header("content-type", "application/activity+json")
-      |> json(UserView.render("followers.json", %{user: user}))
+      |> json(UserView.render("followers.json", %{user: user, for: for_user}))
     end
   end
 
@@ -325,4 +339,17 @@ defp set_requester_reachable(%Plug.Conn{} = conn, _) do
 
     conn
   end
+
+  defp ensure_user_keys_present_and_maybe_refresh_for_user(user, for_user) do
+    {:ok, new_user} = User.ensure_keys_present(user)
+
+    for_user =
+      if new_user != user and match?(%User{}, for_user) do
+        User.get_cached_by_nickname(for_user.nickname)
+      else
+        for_user
+      end
+
+    {new_user, for_user}
+  end
 end

lib/pleroma/web/activity_pub/transmogrifier.ex

@@ -1090,6 +1090,10 @@ def upgrade_user_from_ap_id(ap_id) do
         PleromaJobQueue.enqueue(:transmogrifier, __MODULE__, [:user_upgrade, user])
       end
 
+      if Pleroma.Config.get([:instance, :external_user_synchronization]) do
+        update_following_followers_counters(user)
+      end
+
       {:ok, user}
     else
       %User{} = user -> {:ok, user}
@@ -1122,4 +1126,27 @@ def maybe_fix_user_object(data) do
     data
     |> maybe_fix_user_url
   end
+
+  def update_following_followers_counters(user) do
+    info = %{}
+
+    following = fetch_counter(user.following_address)
+    info = if following, do: Map.put(info, :following_count, following), else: info
+
+    followers = fetch_counter(user.follower_address)
+    info = if followers, do: Map.put(info, :follower_count, followers), else: info
+
+    User.set_info_cache(user, info)
+  end
+
+  defp fetch_counter(url) do
+    with {:ok, %{body: body, status: code}} when code in 200..299 <-
+           Pleroma.HTTP.get(
+             url,
+             [{:Accept, "application/activity+json"}]
+           ),
+         {:ok, data} <- Jason.decode(body) do
+      data["totalItems"]
+    end
+  end
 end

lib/pleroma/web/activity_pub/views/user_view.ex

@@ -98,29 +98,31 @@ def render("user.json", %{user: user}) do
     |> Map.merge(Utils.make_json_ld_header())
   end
 
-  def render("following.json", %{user: user, page: page}) do
+  def render("following.json", %{user: user, page: page} = opts) do
+    showing = (opts[:for] && opts[:for] == user) || !user.info.hide_follows
     query = User.get_friends_query(user)
     query = from(user in query, select: [:ap_id])
     following = Repo.all(query)
 
     total =
-      if !user.info.hide_follows do
+      if showing do
         length(following)
       else
         0
       end
 
-    collection(following, "#{user.ap_id}/following", page, !user.info.hide_follows, total)
+    collection(following, "#{user.ap_id}/following", page, showing, total)
     |> Map.merge(Utils.make_json_ld_header())
   end
 
-  def render("following.json", %{user: user}) do
+  def render("following.json", %{user: user} = opts) do
+    showing = (opts[:for] && opts[:for] == user) || !user.info.hide_follows
     query = User.get_friends_query(user)
     query = from(user in query, select: [:ap_id])
     following = Repo.all(query)
 
     total =
-      if !user.info.hide_follows do
+      if showing do
         length(following)
       else
         0
@@ -130,34 +132,43 @@ def render("following.json", %{user: user}) do
       "id" => "#{user.ap_id}/following",
       "type" => "OrderedCollection",
       "totalItems" => total,
-      "first" => collection(following, "#{user.ap_id}/following", 1, !user.info.hide_follows)
+      "first" =>
+        if showing do
+          collection(following, "#{user.ap_id}/following", 1, !user.info.hide_follows)
+        else
+          "#{user.ap_id}/following?page=1"
+        end
     }
     |> Map.merge(Utils.make_json_ld_header())
   end
 
-  def render("followers.json", %{user: user, page: page}) do
+  def render("followers.json", %{user: user, page: page} = opts) do
+    showing = (opts[:for] && opts[:for] == user) || !user.info.hide_followers
+
     query = User.get_followers_query(user)
     query = from(user in query, select: [:ap_id])
     followers = Repo.all(query)
 
     total =
-      if !user.info.hide_followers do
+      if showing do
         length(followers)
       else
         0
       end
 
-    collection(followers, "#{user.ap_id}/followers", page, !user.info.hide_followers, total)
+    collection(followers, "#{user.ap_id}/followers", page, showing, total)
     |> Map.merge(Utils.make_json_ld_header())
   end
 
-  def render("followers.json", %{user: user}) do
+  def render("followers.json", %{user: user} = opts) do
+    showing = (opts[:for] && opts[:for] == user) || !user.info.hide_followers
+
     query = User.get_followers_query(user)
     query = from(user in query, select: [:ap_id])
     followers = Repo.all(query)
 
     total =
-      if !user.info.hide_followers do
+      if showing do
         length(followers)
       else
         0
@@ -168,7 +179,11 @@ def render("followers.json", %{user: user}) do
       "type" => "OrderedCollection",
       "totalItems" => total,
       "first" =>
-        collection(followers, "#{user.ap_id}/followers", 1, !user.info.hide_followers, total)
+        if showing do
+          collection(followers, "#{user.ap_id}/followers", 1, showing, total)
+        else
+          "#{user.ap_id}/followers?page=1"
+        end
     }
     |> Map.merge(Utils.make_json_ld_header())
   end

lib/pleroma/web/admin_api/admin_api_controller.ex

@@ -371,13 +371,13 @@ def config_update(conn, %{"configs" => configs}) do
       if Pleroma.Config.get([:instance, :dynamic_configuration]) do
         updated =
           Enum.map(configs, fn
-            %{"group" => group, "key" => key, "value" => value} ->
-              {:ok, config} = Config.update_or_create(%{group: group, key: key, value: value})
-              config
-
             %{"group" => group, "key" => key, "delete" => "true"} ->
               {:ok, _} = Config.delete(%{group: group, key: key})
               nil
+
+            %{"group" => group, "key" => key, "value" => value} ->
+              {:ok, config} = Config.update_or_create(%{group: group, key: key, value: value})
+              config
           end)
           |> Enum.reject(&is_nil(&1))
 

lib/pleroma/web/admin_api/config.ex

@@ -67,99 +67,86 @@ def delete(params) do
   end
 
   @spec from_binary(binary()) :: term()
-  def from_binary(value), do: :erlang.binary_to_term(value)
+  def from_binary(binary), do: :erlang.binary_to_term(binary)
 
-  @spec from_binary_to_map(binary()) :: any()
+  @spec from_binary_with_convert(binary()) :: any()
-  def from_binary_to_map(binary) do
+  def from_binary_with_convert(binary) do
     from_binary(binary)
     |> do_convert()
   end
 
-  defp do_convert([{k, v}] = value) when is_list(value) and length(value) == 1,
+  defp do_convert(entity) when is_list(entity) do
-    do: %{k => do_convert(v)}
+    for v <- entity, into: [], do: do_convert(v)
+  end
 
-  defp do_convert(values) when is_list(values), do: for(val <- values, do: do_convert(val))
+  defp do_convert(entity) when is_map(entity) do
+    for {k, v} <- entity, into: %{}, do: {do_convert(k), do_convert(v)}
+  end
 
-  defp do_convert({k, v} = value) when is_tuple(value),
+  defp do_convert({:dispatch, [entity]}), do: %{"tuple" => [":dispatch", [inspect(entity)]]}
-    do: %{k => do_convert(v)}
 
-  defp do_convert(value) when is_tuple(value), do: %{"tuple" => do_convert(Tuple.to_list(value))}
+  defp do_convert(entity) when is_tuple(entity),
+    do: %{"tuple" => do_convert(Tuple.to_list(entity))}
 
-  defp do_convert(value) when is_binary(value) or is_map(value) or is_number(value), do: value
+  defp do_convert(entity) when is_boolean(entity) or is_number(entity) or is_nil(entity),
+    do: entity
 
-  defp do_convert(value) when is_atom(value) do
+  defp do_convert(entity) when is_atom(entity) do
-    string = to_string(value)
+    string = to_string(entity)
 
     if String.starts_with?(string, "Elixir."),
-      do: String.trim_leading(string, "Elixir."),
+      do: do_convert(string),
-      else: value
+      else: ":" <> string
   end
 
+  defp do_convert("Elixir." <> module_name), do: module_name
+
+  defp do_convert(entity) when is_binary(entity), do: entity
+
   @spec transform(any()) :: binary()
-  def transform(%{"tuple" => _} = entity), do: :erlang.term_to_binary(do_transform(entity))
+  def transform(entity) when is_binary(entity) or is_map(entity) or is_list(entity) do
-
+    :erlang.term_to_binary(do_transform(entity))
-  def transform(entity) when is_map(entity) do
-    tuples =
-      for {k, v} <- entity,
-          into: [],
-          do: {if(is_atom(k), do: k, else: String.to_atom(k)), do_transform(v)}
-
-    Enum.reject(tuples, fn {_k, v} -> is_nil(v) end)
-    |> Enum.sort()
-    |> :erlang.term_to_binary()
-  end
-
-  def transform(entity) when is_list(entity) do
-    list = Enum.map(entity, &do_transform(&1))
-    :erlang.term_to_binary(list)
   end
 
   def transform(entity), do: :erlang.term_to_binary(entity)
 
-  defp do_transform(%Regex{} = value) when is_map(value), do: value
+  defp do_transform(%Regex{} = entity) when is_map(entity), do: entity
 
-  defp do_transform(%{"tuple" => [k, values] = entity}) when length(entity) == 2 do
+  defp do_transform(%{"tuple" => [":dispatch", [entity]]}) do
-    {do_transform(k), do_transform(values)}
+    cleaned_string = String.replace(entity, ~r/[^\w|^{:,[|^,|^[|^\]^}|^\/|^\.|^"]^\s/, "")
+    {dispatch_settings, []} = Code.eval_string(cleaned_string, [], requires: [], macros: [])
+    {:dispatch, [dispatch_settings]}
   end
 
-  defp do_transform(%{"tuple" => values}) do
+  defp do_transform(%{"tuple" => entity}) do
-    Enum.reduce(values, {}, fn val, acc -> Tuple.append(acc, do_transform(val)) end)
+    Enum.reduce(entity, {}, fn val, acc -> Tuple.append(acc, do_transform(val)) end)
   end
 
-  defp do_transform(value) when is_map(value) do
+  defp do_transform(entity) when is_map(entity) do
-    values = for {key, val} <- value, into: [], do: {String.to_atom(key), do_transform(val)}
+    for {k, v} <- entity, into: %{}, do: {do_transform(k), do_transform(v)}
-
-    Enum.sort(values)
   end
 
-  defp do_transform(value) when is_list(value) do
+  defp do_transform(entity) when is_list(entity) do
-    Enum.map(value, &do_transform(&1))
+    for v <- entity, into: [], do: do_transform(v)
   end
 
-  defp do_transform(entity) when is_list(entity) and length(entity) == 1, do: hd(entity)
+  defp do_transform(entity) when is_binary(entity) do
-
+    String.trim(entity)
-  defp do_transform(value) when is_binary(value) do
-    String.trim(value)
     |> do_transform_string()
   end
 
-  defp do_transform(value), do: value
+  defp do_transform(entity), do: entity
 
-  defp do_transform_string(value) when byte_size(value) == 0, do: nil
+  defp do_transform_string("~r/" <> pattern) do
+    pattern = String.trim_trailing(pattern, "/")
+    ~r/#{pattern}/
+  end
+
+  defp do_transform_string(":" <> atom), do: String.to_atom(atom)
 
   defp do_transform_string(value) do
-    cond do
+    if String.starts_with?(value, "Pleroma") or String.starts_with?(value, "Phoenix"),
-      String.starts_with?(value, "Pleroma") or String.starts_with?(value, "Phoenix") ->
+      do: String.to_existing_atom("Elixir." <> value),
-        String.to_existing_atom("Elixir." <> value)
+      else: value
-
-      String.starts_with?(value, ":") ->
-        String.replace(value, ":", "") |> String.to_existing_atom()
-
-      String.starts_with?(value, "i:") ->
-        String.replace(value, "i:", "") |> String.to_integer()
-
-      true ->
-        value
-    end
   end
 end

lib/pleroma/web/admin_api/views/config_view.ex

@@ -15,7 +15,7 @@ def render("show.json", %{config: config}) do
     %{
       key: config.key,
       group: config.group,
-      value: Pleroma.Web.AdminAPI.Config.from_binary_to_map(config.value)
+      value: Pleroma.Web.AdminAPI.Config.from_binary_with_convert(config.value)
     }
   end
 end

lib/pleroma/web/common_api/common_api.ex

@@ -31,7 +31,8 @@ def follow(follower, followed) do
 
   def unfollow(follower, unfollowed) do
     with {:ok, follower, _follow_activity} <- User.unfollow(follower, unfollowed),
-         {:ok, _activity} <- ActivityPub.unfollow(follower, unfollowed) do
+         {:ok, _activity} <- ActivityPub.unfollow(follower, unfollowed),
+         {:ok, _unfollowed} <- User.unsubscribe(follower, unfollowed) do
       {:ok, follower}
     end
   end

lib/pleroma/web/mastodon_api/mastodon_api.ex

@@ -53,7 +53,7 @@ def get_notifications(user, params \\ %{}) do
     options = cast_params(params)
 
     user
-    |> Notification.for_user_query()
+    |> Notification.for_user_query(options)
     |> restrict(:exclude_types, options)
     |> Pagination.fetch_paginated(params)
   end
@@ -67,7 +67,8 @@ def get_scheduled_activities(user, params \\ %{}) do
   defp cast_params(params) do
     param_types = %{
       exclude_types: {:array, :string},
-      reblogs: :boolean
+      reblogs: :boolean,
+      with_muted: :boolean
     }
 
     changeset = cast({%{}, param_types}, params, Map.keys(param_types))

lib/pleroma/web/mastodon_api/mastodon_api_controller.ex

@@ -15,6 +15,7 @@ defmodule Pleroma.Web.MastodonAPI.MastodonAPIController do
   alias Pleroma.Notification
   alias Pleroma.Object
   alias Pleroma.Pagination
+  alias Pleroma.Plugs.RateLimiter
   alias Pleroma.Repo
   alias Pleroma.ScheduledActivity
   alias Pleroma.Stats
@@ -46,8 +47,24 @@ defmodule Pleroma.Web.MastodonAPI.MastodonAPIController do
 
   require Logger
 
-  plug(Pleroma.Plugs.RateLimiter, :app_account_creation when action == :account_register)
+  @rate_limited_status_actions ~w(reblog_status unreblog_status fav_status unfav_status
-  plug(Pleroma.Plugs.RateLimiter, :search when action in [:search, :search2, :account_search])
+    post_status delete_status)a
+
+  plug(
+    RateLimiter,
+    {:status_id_action, bucket_name: "status_id_action:reblog_unreblog", params: ["id"]}
+    when action in ~w(reblog_status unreblog_status)a
+  )
+
+  plug(
+    RateLimiter,
+    {:status_id_action, bucket_name: "status_id_action:fav_unfav", params: ["id"]}
+    when action in ~w(fav_status unfav_status)a
+  )
+
+  plug(RateLimiter, :statuses_actions when action in @rate_limited_status_actions)
+  plug(RateLimiter, :app_account_creation when action == :account_register)
+  plug(RateLimiter, :search when action in [:search, :search2, :account_search])
 
   @local_mastodon_name "Mastodon-Local"
 
@@ -1051,9 +1068,14 @@ def unfollow(%{assigns: %{user: follower}} = conn, %{"id" => id}) do
     end
   end
 
-  def mute(%{assigns: %{user: muter}} = conn, %{"id" => id}) do
+  def mute(%{assigns: %{user: muter}} = conn, %{"id" => id} = params) do
+    notifications =
+      if Map.has_key?(params, "notifications"),
+        do: params["notifications"] in [true, "True", "true", "1"],
+        else: true
+
     with %User{} = muted <- User.get_cached_by_id(id),
-         {:ok, muter} <- User.mute(muter, muted) do
+         {:ok, muter} <- User.mute(muter, muted, notifications) do
       conn
       |> put_view(AccountView)
       |> render("relationship.json", %{user: muter, target: muted})

lib/pleroma/web/mastodon_api/search_controller.ex

@@ -7,6 +7,7 @@ defmodule Pleroma.Web.MastodonAPI.SearchController do
 
   alias Pleroma.Activity
   alias Pleroma.Plugs.RateLimiter
+  alias Pleroma.Repo
   alias Pleroma.User
   alias Pleroma.Web
   alias Pleroma.Web.ControllerHelper
@@ -16,43 +17,6 @@ defmodule Pleroma.Web.MastodonAPI.SearchController do
   require Logger
   plug(RateLimiter, :search when action in [:search, :search2, :account_search])
 
-  def search2(%{assigns: %{user: user}} = conn, %{"q" => query} = params) do
-    accounts = with_fallback(fn -> User.search(query, search_options(params, user)) end, [])
-    statuses = with_fallback(fn -> Activity.search(user, query) end, [])
-
-    tags_path = Web.base_url() <> "/tag/"
-
-    tags =
-      query
-      |> prepare_tags
-      |> Enum.map(fn tag -> %{name: tag, url: tags_path <> tag} end)
-
-    res = %{
-      "accounts" => AccountView.render("accounts.json", users: accounts, for: user, as: :user),
-      "statuses" =>
-        StatusView.render("index.json", activities: statuses, for: user, as: :activity),
-      "hashtags" => tags
-    }
-
-    json(conn, res)
-  end
-
-  def search(%{assigns: %{user: user}} = conn, %{"q" => query} = params) do
-    accounts = with_fallback(fn -> User.search(query, search_options(params, user)) end)
-    statuses = with_fallback(fn -> Activity.search(user, query) end)
-
-    tags = prepare_tags(query)
-
-    res = %{
-      "accounts" => AccountView.render("accounts.json", users: accounts, for: user, as: :user),
-      "statuses" =>
-        StatusView.render("index.json", activities: statuses, for: user, as: :activity),
-      "hashtags" => tags
-    }
-
-    json(conn, res)
-  end
-
   def account_search(%{assigns: %{user: user}} = conn, %{"q" => query} = params) do
     accounts = User.search(query, search_options(params, user))
     res = AccountView.render("accounts.json", users: accounts, for: user, as: :user)
@@ -60,12 +24,36 @@ def account_search(%{assigns: %{user: user}} = conn, %{"q" => query} = params) d
     json(conn, res)
   end
 
-  defp prepare_tags(query) do
+  def search2(conn, params), do: do_search(:v2, conn, params)
-    query
+  def search(conn, params), do: do_search(:v1, conn, params)
-    |> String.split()
+
-    |> Enum.uniq()
+  defp do_search(version, %{assigns: %{user: user}} = conn, %{"q" => query} = params) do
-    |> Enum.filter(fn tag -> String.starts_with?(tag, "#") end)
+    options = search_options(params, user)
-    |> Enum.map(fn tag -> String.slice(tag, 1..-1) end)
+    timeout = Keyword.get(Repo.config(), :timeout, 15_000)
+    default_values = %{"statuses" => [], "accounts" => [], "hashtags" => []}
+
+    result =
+      default_values
+      |> Enum.map(fn {resource, default_value} ->
+        if params["type"] == nil or params["type"] == resource do
+          {resource, fn -> resource_search(version, resource, query, options) end}
+        else
+          {resource, fn -> default_value end}
+        end
+      end)
+      |> Task.async_stream(fn {resource, f} -> {resource, with_fallback(f)} end,
+        timeout: timeout,
+        on_timeout: :kill_task
+      )
+      |> Enum.reduce(default_values, fn
+        {:ok, {resource, result}}, acc ->
+          Map.put(acc, resource, result)
+
+        _error, acc ->
+          acc
+      end)
+
+    json(conn, result)
   end
 
   defp search_options(params, user) do
@@ -74,8 +62,45 @@ defp search_options(params, user) do
       following: params["following"] == "true",
       limit: ControllerHelper.fetch_integer_param(params, "limit"),
       offset: ControllerHelper.fetch_integer_param(params, "offset"),
+      type: params["type"],
+      author: get_author(params),
       for_user: user
     ]
+    |> Enum.filter(&elem(&1, 1))
+  end
+
+  defp resource_search(_, "accounts", query, options) do
+    accounts = with_fallback(fn -> User.search(query, options) end)
+    AccountView.render("accounts.json", users: accounts, for: options[:for_user], as: :user)
+  end
+
+  defp resource_search(_, "statuses", query, options) do
+    statuses = with_fallback(fn -> Activity.search(options[:for_user], query, options) end)
+    StatusView.render("index.json", activities: statuses, for: options[:for_user], as: :activity)
+  end
+
+  defp resource_search(:v2, "hashtags", query, _options) do
+    tags_path = Web.base_url() <> "/tag/"
+
+    query
+    |> prepare_tags()
+    |> Enum.map(fn tag ->
+      tag = String.trim_leading(tag, "#")
+      %{name: tag, url: tags_path <> tag}
+    end)
+  end
+
+  defp resource_search(:v1, "hashtags", query, _options) do
+    query
+    |> prepare_tags()
+    |> Enum.map(fn tag -> String.trim_leading(tag, "#") end)
+  end
+
+  defp prepare_tags(query) do
+    query
+    |> String.split()
+    |> Enum.uniq()
+    |> Enum.filter(fn tag -> String.starts_with?(tag, "#") end)
   end
 
   defp with_fallback(f, fallback \\ []) do
@@ -87,4 +112,9 @@ defp with_fallback(f, fallback \\ []) do
         fallback
     end
   end
+
+  defp get_author(%{"account_id" => account_id}) when is_binary(account_id),
+    do: User.get_cached_by_id(account_id)
+
+  defp get_author(_params), do: nil
 end

lib/pleroma/web/mastodon_api/views/account_view.ex

@@ -52,7 +52,7 @@ def render("relationship.json", %{user: %User{} = user, target: %User{} = target
       followed_by: User.following?(target, user),
       blocking: User.blocks?(user, target),
       muting: User.mutes?(user, target),
-      muting_notifications: false,
+      muting_notifications: User.muted_notifications?(user, target),
       subscribing: User.subscribed_to?(user, target),
       requested: requested,
       domain_blocking: false,

lib/pleroma/web/media_proxy/media_proxy.ex

@@ -3,68 +3,71 @@
 # SPDX-License-Identifier: AGPL-3.0-only
 
 defmodule Pleroma.Web.MediaProxy do
+  alias Pleroma.Config
+  alias Pleroma.Web
+
   @base64_opts [padding: false]
 
-  def url(nil), do: nil
+  def url(url) when is_nil(url) or url == "", do: nil
-
-  def url(""), do: nil
-
   def url("/" <> _ = url), do: url
 
   def url(url) do
-    if !enabled?() or local?(url) or whitelisted?(url) do
+    if disabled?() or local?(url) or whitelisted?(url) do
       url
     else
       encode_url(url)
     end
   end
 
-  defp enabled?, do: Pleroma.Config.get([:media_proxy, :enabled], false)
+  defp disabled?, do: !Config.get([:media_proxy, :enabled], false)
 
   defp local?(url), do: String.starts_with?(url, Pleroma.Web.base_url())
 
   defp whitelisted?(url) do
     %{host: domain} = URI.parse(url)
 
-    Enum.any?(Pleroma.Config.get([:media_proxy, :whitelist]), fn pattern ->
+    Enum.any?(Config.get([:media_proxy, :whitelist]), fn pattern ->
       String.equivalent?(domain, pattern)
     end)
   end
 
   def encode_url(url) do
-    secret = Pleroma.Config.get([Pleroma.Web.Endpoint, :secret_key_base])
     base64 = Base.url_encode64(url, @base64_opts)
-    sig = :crypto.hmac(:sha, secret, base64)
+
-    sig64 = sig |> Base.url_encode64(@base64_opts)
+    sig64 =
+      base64
+      |> signed_url
+      |> Base.url_encode64(@base64_opts)
 
     build_url(sig64, base64, filename(url))
   end
 
   def decode_url(sig, url) do
-    secret = Pleroma.Config.get([Pleroma.Web.Endpoint, :secret_key_base])
+    with {:ok, sig} <- Base.url_decode64(sig, @base64_opts),
-    sig = Base.url_decode64!(sig, @base64_opts)
+         signature when signature == sig <- signed_url(url) do
-    local_sig = :crypto.hmac(:sha, secret, url)
-
-    if local_sig == sig do
       {:ok, Base.url_decode64!(url, @base64_opts)}
     else
-      {:error, :invalid_signature}
+      _ -> {:error, :invalid_signature}
     end
   end
 
+  defp signed_url(url) do
+    :crypto.hmac(:sha, Config.get([Web.Endpoint, :secret_key_base]), url)
+  end
+
   def filename(url_or_path) do
     if path = URI.parse(url_or_path).path, do: Path.basename(path)
   end
 
   def build_url(sig_base64, url_base64, filename \\ nil) do
     [
-      Pleroma.Config.get([:media_proxy, :base_url], Pleroma.Web.base_url()),
+      Pleroma.Config.get([:media_proxy, :base_url], Web.base_url()),
       "proxy",
       sig_base64,
       url_base64,
       filename
     ]
-    |> Enum.filter(fn value -> value end)
+    |> Enum.filter(& &1)
     |> Path.join()
   end
 end

lib/pleroma/web/media_proxy/media_proxy_controller.ex

@@ -13,7 +13,7 @@ def remote(conn, %{"sig" => sig64, "url" => url64} = params) do
     with config <- Pleroma.Config.get([:media_proxy], []),
          true <- Keyword.get(config, :enabled, false),
          {:ok, url} <- MediaProxy.decode_url(sig64, url64),
-         :ok <- filename_matches(Map.has_key?(params, "filename"), conn.request_path, url) do
+         :ok <- filename_matches(params, conn.request_path, url) do
       ReverseProxy.call(conn, url, Keyword.get(config, :proxy_opts, @default_proxy_opts))
     else
       false ->
@@ -27,18 +27,15 @@ def remote(conn, %{"sig" => sig64, "url" => url64} = params) do
     end
   end
 
-  def filename_matches(has_filename, path, url) do
+  def filename_matches(%{"filename" => _} = _, path, url) do
-    filename =
+    filename = MediaProxy.filename(url)
-      url
-      |> MediaProxy.filename()
-      |> URI.decode()
 
-    path = URI.decode(path)
+    if filename && Path.basename(path) != filename do
-
-    if has_filename && filename && Path.basename(path) != filename do
       {:wrong_filename, filename}
     else
       :ok
     end
   end
+
+  def filename_matches(_, _, _), do: :ok
 end

lib/pleroma/web/metadata/opengraph.ex

@@ -9,6 +9,7 @@ defmodule Pleroma.Web.Metadata.Providers.OpenGraph do
   alias Pleroma.Web.Metadata.Utils
 
   @behaviour Provider
+  @media_types ["image", "audio", "video"]
 
   @impl Provider
   def build_tags(%{
@@ -81,26 +82,19 @@ defp build_attachments(%{data: %{"attachment" => attachments}}) do
     Enum.reduce(attachments, [], fn attachment, acc ->
       rendered_tags =
         Enum.reduce(attachment["url"], [], fn url, acc ->
-          media_type =
-            Enum.find(["image", "audio", "video"], fn media_type ->
-              String.starts_with?(url["mediaType"], media_type)
-            end)
-
           # TODO: Add additional properties to objects when we have the data available.
           # Also, Whatsapp only wants JPEG or PNGs. It seems that if we add a second og:image
           # object when a Video or GIF is attached it will display that in Whatsapp Rich Preview.
-          case media_type do
+          case Utils.fetch_media_type(@media_types, url["mediaType"]) do
             "audio" ->
               [
-                {:meta,
+                {:meta, [property: "og:audio", content: Utils.attachment_url(url["href"])], []}
-                 [property: "og:" <> media_type, content: Utils.attachment_url(url["href"])], []}
                 | acc
               ]
 
             "image" ->
               [
-                {:meta,
+                {:meta, [property: "og:image", content: Utils.attachment_url(url["href"])], []},
-                 [property: "og:" <> media_type, content: Utils.attachment_url(url["href"])], []},
                 {:meta, [property: "og:image:width", content: 150], []},
                 {:meta, [property: "og:image:height", content: 150], []}
                 | acc
@@ -108,8 +102,7 @@ defp build_attachments(%{data: %{"attachment" => attachments}}) do
 
             "video" ->
               [
-                {:meta,
+                {:meta, [property: "og:video", content: Utils.attachment_url(url["href"])], []}
-                 [property: "og:" <> media_type, content: Utils.attachment_url(url["href"])], []}
                 | acc
               ]
 

lib/pleroma/web/metadata/twitter_card.ex

@@ -1,4 +1,5 @@
 # Pleroma: A lightweight social networking server
+
 # Copyright © 2017-2019 Pleroma Authors <https://pleroma.social/>
 # SPDX-License-Identifier: AGPL-3.0-only
 
@@ -9,13 +10,10 @@ defmodule Pleroma.Web.Metadata.Providers.TwitterCard do
   alias Pleroma.Web.Metadata.Utils
 
   @behaviour Provider
+  @media_types ["image", "audio", "video"]
 
   @impl Provider
-  def build_tags(%{
+  def build_tags(%{activity_id: id, object: object, user: user}) do
-        activity_id: id,
-        object: object,
-        user: user
-      }) do
     attachments = build_attachments(id, object)
     scrubbed_content = Utils.scrub_html_and_truncate(object)
     # Zero width space
@@ -27,21 +25,12 @@ def build_tags(%{
       end
 
     [
-      {:meta,
+      title_tag(user),
-       [
+      {:meta, [property: "twitter:description", content: content], []}
-         property: "twitter:title",
-         content: Utils.user_name_string(user)
-       ], []},
-      {:meta,
-       [
-         property: "twitter:description",
-         content: content
-       ], []}
     ] ++
       if attachments == [] or Metadata.activity_nsfw?(object) do
         [
-          {:meta,
+          image_tag(user),
-           [property: "twitter:image", content: Utils.attachment_url(User.avatar_url(user))], []},
           {:meta, [property: "twitter:card", content: "summary_large_image"], []}
         ]
       else
@@ -53,30 +42,28 @@ def build_tags(%{
   def build_tags(%{user: user}) do
     with truncated_bio = Utils.scrub_html_and_truncate(user.bio || "") do
       [
-        {:meta,
+        title_tag(user),
-         [
-           property: "twitter:title",
-           content: Utils.user_name_string(user)
-         ], []},
         {:meta, [property: "twitter:description", content: truncated_bio], []},
-        {:meta, [property: "twitter:image", content: Utils.attachment_url(User.avatar_url(user))],
+        image_tag(user),
-         []},
         {:meta, [property: "twitter:card", content: "summary"], []}
       ]
     end
   end
 
+  defp title_tag(user) do
+    {:meta, [property: "twitter:title", content: Utils.user_name_string(user)], []}
+  end
+
+  def image_tag(user) do
+    {:meta, [property: "twitter:image", content: Utils.attachment_url(User.avatar_url(user))], []}
+  end
+
   defp build_attachments(id, %{data: %{"attachment" => attachments}}) do
     Enum.reduce(attachments, [], fn attachment, acc ->
       rendered_tags =
         Enum.reduce(attachment["url"], [], fn url, acc ->
-          media_type =
-            Enum.find(["image", "audio", "video"], fn media_type ->
-              String.starts_with?(url["mediaType"], media_type)
-            end)
-
           # TODO: Add additional properties to objects when we have the data available.
-          case media_type do
+          case Utils.fetch_media_type(@media_types, url["mediaType"]) do
             "audio" ->
               [
                 {:meta, [property: "twitter:card", content: "player"], []},

lib/pleroma/web/metadata/utils.ex

@@ -39,4 +39,11 @@ def user_name_string(user) do
         "(@#{user.nickname})"
       end
   end
+
+  @spec fetch_media_type(list(String.t()), String.t()) :: String.t() | nil
+  def fetch_media_type(supported_types, media_type) do
+    Enum.find(supported_types, fn support_type ->
+      String.starts_with?(media_type, support_type)
+    end)
+  end
 end

lib/pleroma/web/nodeinfo/nodeinfo_controller.ex

@@ -34,8 +34,11 @@ def schemas(conn, _params) do
   def raw_nodeinfo do
     stats = Stats.get_stats()
 
+    exclusions = Config.get([:instance, :mrf_transparency_exclusions])
+
     mrf_simple =
       Config.get(:mrf_simple)
+      |> Enum.map(fn {k, v} -> {k, Enum.reject(v, fn v -> v in exclusions end)} end)
       |> Enum.into(%{})
 
     # This horror is needed to convert regex sigils to strings
@@ -86,7 +89,8 @@ def raw_nodeinfo do
           mrf_simple: mrf_simple,
           mrf_keyword: mrf_keyword,
           mrf_user_allowlist: mrf_user_allowlist,
-          quarantined_instances: quarantined
+          quarantined_instances: quarantined,
+          exclusions: length(exclusions) > 0
         }
       else
         %{}

lib/pleroma/web/rich_media/parser.ex

@@ -3,12 +3,6 @@
 # SPDX-License-Identifier: AGPL-3.0-only
 
 defmodule Pleroma.Web.RichMedia.Parser do
-  @parsers [
-    Pleroma.Web.RichMedia.Parsers.OGP,
-    Pleroma.Web.RichMedia.Parsers.TwitterCard,
-    Pleroma.Web.RichMedia.Parsers.OEmbed
-  ]
-
   @hackney_options [
     pool: :media,
     recv_timeout: 2_000,
@@ -16,6 +10,10 @@ defmodule Pleroma.Web.RichMedia.Parser do
     with_body: true
   ]
 
+  defp parsers do
+    Pleroma.Config.get([:rich_media, :parsers])
+  end
+
   def parse(nil), do: {:error, "No URL provided"}
 
   if Pleroma.Config.get(:env) == :test do
@@ -48,7 +46,7 @@ defp parse_url(url) do
   end
 
   defp maybe_parse(html) do
-    Enum.reduce_while(@parsers, %{}, fn parser, acc ->
+    Enum.reduce_while(parsers(), %{}, fn parser, acc ->
       case parser.parse(html, acc) do
         {:ok, data} -> {:halt, data}
         {:error, _msg} -> {:cont, acc}

lib/pleroma/web/router.ex

@@ -322,10 +322,6 @@ defmodule Pleroma.Web.Router do
 
       patch("/accounts/update_credentials", MastodonAPIController, :update_credentials)
 
-      patch("/accounts/update_avatar", MastodonAPIController, :update_avatar)
-      patch("/accounts/update_banner", MastodonAPIController, :update_banner)
-      patch("/accounts/update_background", MastodonAPIController, :update_background)
-
       post("/statuses", MastodonAPIController, :post_status)
       delete("/statuses/:id", MastodonAPIController, :delete_status)
 
@@ -360,6 +356,10 @@ defmodule Pleroma.Web.Router do
       put("/filters/:id", MastodonAPIController, :update_filter)
       delete("/filters/:id", MastodonAPIController, :delete_filter)
 
+      patch("/pleroma/accounts/update_avatar", MastodonAPIController, :update_avatar)
+      patch("/pleroma/accounts/update_banner", MastodonAPIController, :update_banner)
+      patch("/pleroma/accounts/update_background", MastodonAPIController, :update_background)
+
       get("/pleroma/mascot", MastodonAPIController, :get_mascot)
       put("/pleroma/mascot", MastodonAPIController, :set_mascot)
 
@@ -623,8 +623,6 @@ defmodule Pleroma.Web.Router do
     # XXX: not really ostatus
     pipe_through(:ostatus)
 
-    get("/users/:nickname/followers", ActivityPubController, :followers)
-    get("/users/:nickname/following", ActivityPubController, :following)
     get("/users/:nickname/outbox", ActivityPubController, :outbox)
     get("/objects/:uuid/likes", ActivityPubController, :object_likes)
   end
@@ -656,6 +654,12 @@ defmodule Pleroma.Web.Router do
       pipe_through(:oauth_write)
       post("/users/:nickname/outbox", ActivityPubController, :update_outbox)
     end
+
+    scope [] do
+      pipe_through(:oauth_read_or_public)
+      get("/users/:nickname/followers", ActivityPubController, :followers)
+      get("/users/:nickname/following", ActivityPubController, :following)
+    end
   end
 
   scope "/relay", Pleroma.Web.ActivityPub do

lib/pleroma/web/twitter_api/twitter_api_controller.ex

@@ -192,6 +192,13 @@ def dm_timeline(%{assigns: %{user: user}} = conn, params) do
   end
 
   def notifications(%{assigns: %{user: user}} = conn, params) do
+    params =
+      if Map.has_key?(params, "with_muted") do
+        Map.put(params, :with_muted, params["with_muted"] in [true, "True", "true", "1"])
+      else
+        params
+      end
+
     notifications = Notification.for_user(user, params)
 
     conn

mix.exs

@@ -14,7 +14,7 @@ def project do
       aliases: aliases(),
       deps: deps(),
       test_coverage: [tool: ExCoveralls],
-
+      preferred_cli_env: ["coveralls.html": :test],
       # Docs
       name: "Pleroma",
       homepage_url: "https://pleroma.social/",

priv/repo/migrations/20190710115833_add_following_address_to_user.exs

@@ -0,0 +1,9 @@
+defmodule Pleroma.Repo.Migrations.AddFollowingAddressToUser do
+  use Ecto.Migration
+
+  def change do
+    alter table(:users) do
+      add(:following_address, :string, unique: true)
+    end
+  end
+end

priv/repo/migrations/20190710125051_add_following_address_index_to_user.exs

@@ -0,0 +1,8 @@
+defmodule Pleroma.Repo.Migrations.AddFollowingAddressIndexToUser do
+  use Ecto.Migration
+
+  @disable_ddl_transaction true
+  def change do
+    create(index(:users, [:following_address], concurrently: true))
+  end
+end

priv/repo/migrations/20190710125158_add_following_address_from_source_data.exs

@@ -0,0 +1,20 @@
+defmodule Pleroma.Repo.Migrations.AddFollowingAddressFromSourceData do
+  use Ecto.Migration
+  import Ecto.Query
+  alias Pleroma.User
+
+  def change do
+    query =
+      User.external_users_query()
+      |> select([u], struct(u, [:id, :ap_id, :info]))
+
+    Pleroma.Repo.stream(query)
+    |> Enum.each(fn
+      %{info: %{source_data: source_data}} = user ->
+        Ecto.Changeset.cast(user, %{following_address: source_data["following"]}, [
+          :following_address
+        ])
+        |> Pleroma.Repo.update()
+    end)
+  end
+end

priv/repo/migrations/20190711042024_copy_muted_to_muted_notifications.exs

@@ -0,0 +1,24 @@
+defmodule Pleroma.Repo.Migrations.CopyMutedToMutedNotifications do
+  use Ecto.Migration
+  alias Pleroma.User
+
+  def change do
+    query =
+      User.Query.build(%{
+        local: true,
+        active: true,
+        order_by: :id
+      })
+
+    Pleroma.Repo.stream(query)
+    |> Enum.each(fn
+      %{info: %{mutes: mutes} = info} = user ->
+        info_cng =
+          Ecto.Changeset.cast(info, %{muted_notifications: mutes}, [:muted_notifications])
+
+        Ecto.Changeset.change(user)
+        |> Ecto.Changeset.put_embed(:info, info_cng)
+        |> Pleroma.Repo.update()
+    end)
+  end
+end

test/notification_test.exs

@@ -74,26 +74,37 @@ test "it creates a notification for user and send to the 'user' and the 'user:no
       Task.await(task_user_notification)
     end
 
-    test "it doesn't create a notification for user if the user blocks the activity author" do
+    test "it creates a notification for user if the user blocks the activity author" do
       activity = insert(:note_activity)
       author = User.get_cached_by_ap_id(activity.data["actor"])
       user = insert(:user)
       {:ok, user} = User.block(user, author)
 
-      refute Notification.create_notification(activity, user)
+      assert Notification.create_notification(activity, user)
     end
 
-    test "it doesn't create a notificatin for the user if the user mutes the activity author" do
+    test "it creates a notificatin for the user if the user mutes the activity author" do
       muter = insert(:user)
       muted = insert(:user)
       {:ok, _} = User.mute(muter, muted)
       muter = Repo.get(User, muter.id)
       {:ok, activity} = CommonAPI.post(muted, %{"status" => "Hi @#{muter.nickname}"})
 
-      refute Notification.create_notification(activity, muter)
+      assert Notification.create_notification(activity, muter)
     end
 
-    test "it doesn't create a notification for an activity from a muted thread" do
+    test "notification created if user is muted without notifications" do
+      muter = insert(:user)
+      muted = insert(:user)
+
+      {:ok, muter} = User.mute(muter, muted, false)
+
+      {:ok, activity} = CommonAPI.post(muted, %{"status" => "Hi @#{muter.nickname}"})
+
+      assert Notification.create_notification(activity, muter)
+    end
+
+    test "it creates a notification for an activity from a muted thread" do
       muter = insert(:user)
       other_user = insert(:user)
       {:ok, activity} = CommonAPI.post(muter, %{"status" => "hey"})
@@ -105,7 +116,7 @@ test "it doesn't create a notification for an activity from a muted thread" do
           "in_reply_to_status_id" => activity.id
         })
 
-      refute Notification.create_notification(activity, muter)
+      assert Notification.create_notification(activity, muter)
     end
 
     test "it disables notifications from followers" do
@@ -532,4 +543,98 @@ test "replying to a deleted post without tagging does not generate a notificatio
       assert Enum.empty?(Notification.for_user(user))
     end
   end
+
+  describe "for_user" do
+    test "it returns notifications for muted user without notifications" do
+      user = insert(:user)
+      muted = insert(:user)
+      {:ok, user} = User.mute(user, muted, false)
+
+      {:ok, _activity} = TwitterAPI.create_status(muted, %{"status" => "hey @#{user.nickname}"})
+
+      assert length(Notification.for_user(user)) == 1
+    end
+
+    test "it doesn't return notifications for muted user with notifications" do
+      user = insert(:user)
+      muted = insert(:user)
+      {:ok, user} = User.mute(user, muted)
+
+      {:ok, _activity} = TwitterAPI.create_status(muted, %{"status" => "hey @#{user.nickname}"})
+
+      assert Notification.for_user(user) == []
+    end
+
+    test "it doesn't return notifications for blocked user" do
+      user = insert(:user)
+      blocked = insert(:user)
+      {:ok, user} = User.block(user, blocked)
+
+      {:ok, _activity} = TwitterAPI.create_status(blocked, %{"status" => "hey @#{user.nickname}"})
+
+      assert Notification.for_user(user) == []
+    end
+
+    test "it doesn't return notificatitons for blocked domain" do
+      user = insert(:user)
+      blocked = insert(:user, ap_id: "http://some-domain.com")
+      {:ok, user} = User.block_domain(user, "some-domain.com")
+
+      {:ok, _activity} = TwitterAPI.create_status(blocked, %{"status" => "hey @#{user.nickname}"})
+
+      assert Notification.for_user(user) == []
+    end
+
+    test "it doesn't return notifications for muted thread" do
+      user = insert(:user)
+      another_user = insert(:user)
+
+      {:ok, activity} =
+        TwitterAPI.create_status(another_user, %{"status" => "hey @#{user.nickname}"})
+
+      {:ok, _} = Pleroma.ThreadMute.add_mute(user.id, activity.data["context"])
+      assert Notification.for_user(user) == []
+    end
+
+    test "it returns notifications for muted user with notifications and with_muted parameter" do
+      user = insert(:user)
+      muted = insert(:user)
+      {:ok, user} = User.mute(user, muted)
+
+      {:ok, _activity} = TwitterAPI.create_status(muted, %{"status" => "hey @#{user.nickname}"})
+
+      assert length(Notification.for_user(user, %{with_muted: true})) == 1
+    end
+
+    test "it returns notifications for blocked user and with_muted parameter" do
+      user = insert(:user)
+      blocked = insert(:user)
+      {:ok, user} = User.block(user, blocked)
+
+      {:ok, _activity} = TwitterAPI.create_status(blocked, %{"status" => "hey @#{user.nickname}"})
+
+      assert length(Notification.for_user(user, %{with_muted: true})) == 1
+    end
+
+    test "it returns notificatitons for blocked domain and with_muted parameter" do
+      user = insert(:user)
+      blocked = insert(:user, ap_id: "http://some-domain.com")
+      {:ok, user} = User.block_domain(user, "some-domain.com")
+
+      {:ok, _activity} = TwitterAPI.create_status(blocked, %{"status" => "hey @#{user.nickname}"})
+
+      assert length(Notification.for_user(user, %{with_muted: true})) == 1
+    end
+
+    test "it returns notifications for muted thread with_muted parameter" do
+      user = insert(:user)
+      another_user = insert(:user)
+
+      {:ok, activity} =
+        TwitterAPI.create_status(another_user, %{"status" => "hey @#{user.nickname}"})
+
+      {:ok, _} = Pleroma.ThreadMute.add_mute(user.id, activity.data["context"])
+      assert length(Notification.for_user(user, %{with_muted: true})) == 1
+    end
+  end
 end

test/object/containment_test.exs

@@ -68,4 +68,34 @@ test "users cannot be collided through fake direction spoofing attempts" do
                "[error] Could not decode user at fetch https://n1u.moe/users/rye, {:error, :error}"
     end
   end
+
+  describe "containment of children" do
+    test "contain_child() catches spoofing attempts" do
+      data = %{
+        "id" => "http://example.com/whatever",
+        "type" => "Create",
+        "object" => %{
+          "id" => "http://example.net/~alyssa/activities/1234",
+          "attributedTo" => "http://example.org/~alyssa"
+        },
+        "actor" => "http://example.com/~bob"
+      }
+
+      :error = Containment.contain_child(data)
+    end
+
+    test "contain_child() allows correct origins" do
+      data = %{
+        "id" => "http://example.org/~alyssa/activities/5678",
+        "type" => "Create",
+        "object" => %{
+          "id" => "http://example.org/~alyssa/activities/1234",
+          "attributedTo" => "http://example.org/~alyssa"
+        },
+        "actor" => "http://example.org/~alyssa"
+      }
+
+      :ok = Containment.contain_child(data)
+    end
+  end
 end

test/object/fetcher_test.exs

@@ -9,6 +9,7 @@ defmodule Pleroma.Object.FetcherTest do
   alias Pleroma.Object
   alias Pleroma.Object.Fetcher
   import Tesla.Mock
+  import Mock
 
   setup do
     mock(fn
@@ -26,16 +27,31 @@ defmodule Pleroma.Object.FetcherTest do
   end
 
   describe "actor origin containment" do
-    test "it rejects objects with a bogus origin" do
+    test_with_mock "it rejects objects with a bogus origin",
+                   Pleroma.Web.OStatus,
+                   [:passthrough],
+                   [] do
       {:error, _} = Fetcher.fetch_object_from_id("https://info.pleroma.site/activity.json")
+
+      refute called(Pleroma.Web.OStatus.fetch_activity_from_url(:_))
     end
 
-    test "it rejects objects when attributedTo is wrong (variant 1)" do
+    test_with_mock "it rejects objects when attributedTo is wrong (variant 1)",
+                   Pleroma.Web.OStatus,
+                   [:passthrough],
+                   [] do
       {:error, _} = Fetcher.fetch_object_from_id("https://info.pleroma.site/activity2.json")
+
+      refute called(Pleroma.Web.OStatus.fetch_activity_from_url(:_))
     end
 
-    test "it rejects objects when attributedTo is wrong (variant 2)" do
+    test_with_mock "it rejects objects when attributedTo is wrong (variant 2)",
+                   Pleroma.Web.OStatus,
+                   [:passthrough],
+                   [] do
       {:error, _} = Fetcher.fetch_object_from_id("https://info.pleroma.site/activity3.json")
+
+      refute called(Pleroma.Web.OStatus.fetch_activity_from_url(:_))
     end
   end
 

test/plugs/rate_limiter_test.exs

@@ -10,12 +10,13 @@ defmodule Pleroma.Plugs.RateLimiterTest do
 
   import Pleroma.Factory
 
-  @limiter_name :testing
+  # Note: each example must work with separate buckets in order to prevent concurrency issues
 
   test "init/1" do
-    Pleroma.Config.put([:rate_limit, @limiter_name], {1, 1})
+    limiter_name = :test_init
+    Pleroma.Config.put([:rate_limit, limiter_name], {1, 1})
 
-    assert {@limiter_name, {1, 1}} == RateLimiter.init(@limiter_name)
+    assert {limiter_name, {1, 1}, []} == RateLimiter.init(limiter_name)
     assert nil == RateLimiter.init(:foo)
   end
 
@@ -24,14 +25,15 @@ test "ip/1" do
   end
 
   test "it restricts by opts" do
+    limiter_name = :test_opts
     scale = 1000
     limit = 5
 
-    Pleroma.Config.put([:rate_limit, @limiter_name], {scale, limit})
+    Pleroma.Config.put([:rate_limit, limiter_name], {scale, limit})
 
-    opts = RateLimiter.init(@limiter_name)
+    opts = RateLimiter.init(limiter_name)
     conn = conn(:get, "/")
-    bucket_name = "#{@limiter_name}:#{RateLimiter.ip(conn)}"
+    bucket_name = "#{limiter_name}:#{RateLimiter.ip(conn)}"
 
     conn = RateLimiter.call(conn, opts)
     assert {1, 4, _, _, _} = ExRated.inspect_bucket(bucket_name, scale, limit)
@@ -65,18 +67,78 @@ test "it restricts by opts" do
     refute conn.halted
   end
 
+  test "`bucket_name` option overrides default bucket name" do
+    limiter_name = :test_bucket_name
+    scale = 1000
+    limit = 5
+
+    Pleroma.Config.put([:rate_limit, limiter_name], {scale, limit})
+    base_bucket_name = "#{limiter_name}:group1"
+    opts = RateLimiter.init({limiter_name, bucket_name: base_bucket_name})
+
+    conn = conn(:get, "/")
+    default_bucket_name = "#{limiter_name}:#{RateLimiter.ip(conn)}"
+    customized_bucket_name = "#{base_bucket_name}:#{RateLimiter.ip(conn)}"
+
+    RateLimiter.call(conn, opts)
+    assert {1, 4, _, _, _} = ExRated.inspect_bucket(customized_bucket_name, scale, limit)
+    assert {0, 5, _, _, _} = ExRated.inspect_bucket(default_bucket_name, scale, limit)
+  end
+
+  test "`params` option appends specified params' values to bucket name" do
+    limiter_name = :test_params
+    scale = 1000
+    limit = 5
+
+    Pleroma.Config.put([:rate_limit, limiter_name], {scale, limit})
+    opts = RateLimiter.init({limiter_name, params: ["id"]})
+    id = "1"
+
+    conn = conn(:get, "/?id=#{id}")
+    conn = Plug.Conn.fetch_query_params(conn)
+
+    default_bucket_name = "#{limiter_name}:#{RateLimiter.ip(conn)}"
+    parametrized_bucket_name = "#{limiter_name}:#{id}:#{RateLimiter.ip(conn)}"
+
+    RateLimiter.call(conn, opts)
+    assert {1, 4, _, _, _} = ExRated.inspect_bucket(parametrized_bucket_name, scale, limit)
+    assert {0, 5, _, _, _} = ExRated.inspect_bucket(default_bucket_name, scale, limit)
+  end
+
+  test "it supports combination of options modifying bucket name" do
+    limiter_name = :test_options_combo
+    scale = 1000
+    limit = 5
+
+    Pleroma.Config.put([:rate_limit, limiter_name], {scale, limit})
+    base_bucket_name = "#{limiter_name}:group1"
+    opts = RateLimiter.init({limiter_name, bucket_name: base_bucket_name, params: ["id"]})
+    id = "100"
+
+    conn = conn(:get, "/?id=#{id}")
+    conn = Plug.Conn.fetch_query_params(conn)
+
+    default_bucket_name = "#{limiter_name}:#{RateLimiter.ip(conn)}"
+    parametrized_bucket_name = "#{base_bucket_name}:#{id}:#{RateLimiter.ip(conn)}"
+
+    RateLimiter.call(conn, opts)
+    assert {1, 4, _, _, _} = ExRated.inspect_bucket(parametrized_bucket_name, scale, limit)
+    assert {0, 5, _, _, _} = ExRated.inspect_bucket(default_bucket_name, scale, limit)
+  end
+
   test "optional limits for authenticated users" do
+    limiter_name = :test_authenticated
     Ecto.Adapters.SQL.Sandbox.checkout(Pleroma.Repo)
 
     scale = 1000
     limit = 5
-    Pleroma.Config.put([:rate_limit, @limiter_name], [{1, 10}, {scale, limit}])
+    Pleroma.Config.put([:rate_limit, limiter_name], [{1, 10}, {scale, limit}])
 
-    opts = RateLimiter.init(@limiter_name)
+    opts = RateLimiter.init(limiter_name)
 
     user = insert(:user)
     conn = conn(:get, "/") |> assign(:user, user)
-    bucket_name = "#{@limiter_name}:#{user.id}"
+    bucket_name = "#{limiter_name}:#{user.id}"
 
     conn = RateLimiter.call(conn, opts)
     assert {1, 4, _, _, _} = ExRated.inspect_bucket(bucket_name, scale, limit)

test/support/factory.ex

@@ -38,6 +38,7 @@ def user_factory do
       user
       | ap_id: User.ap_id(user),
         follower_address: User.ap_followers(user),
+        following_address: User.ap_following(user),
         following: [User.ap_id(user)]
     }
   end

test/support/http_request_mock.ex

@@ -879,6 +879,42 @@ def get(
      }}
   end
 
+  def get("https://info.pleroma.site/activity.json", _, _, Accept: "application/activity+json") do
+    {:ok,
+     %Tesla.Env{
+       status: 200,
+       body: File.read!("test/fixtures/tesla_mock/https__info.pleroma.site_activity.json")
+     }}
+  end
+
+  def get("https://info.pleroma.site/activity.json", _, _, _) do
+    {:ok, %Tesla.Env{status: 404, body: ""}}
+  end
+
+  def get("https://info.pleroma.site/activity2.json", _, _, Accept: "application/activity+json") do
+    {:ok,
+     %Tesla.Env{
+       status: 200,
+       body: File.read!("test/fixtures/tesla_mock/https__info.pleroma.site_activity2.json")
+     }}
+  end
+
+  def get("https://info.pleroma.site/activity2.json", _, _, _) do
+    {:ok, %Tesla.Env{status: 404, body: ""}}
+  end
+
+  def get("https://info.pleroma.site/activity3.json", _, _, Accept: "application/activity+json") do
+    {:ok,
+     %Tesla.Env{
+       status: 200,
+       body: File.read!("test/fixtures/tesla_mock/https__info.pleroma.site_activity3.json")
+     }}
+  end
+
+  def get("https://info.pleroma.site/activity3.json", _, _, _) do
+    {:ok, %Tesla.Env{status: 404, body: ""}}
+  end
+
   def get(url, query, body, headers) do
     {:error,
      "Not implemented the mock response for get #{inspect(url)}, #{query}, #{inspect(body)}, #{

test/tasks/robots_txt_test.exs

@@ -3,7 +3,7 @@
 # SPDX-License-Identifier: AGPL-3.0-only
 
 defmodule Mix.Tasks.Pleroma.RobotsTxtTest do
-  use ExUnit.Case, async: true
+  use ExUnit.Case
   alias Mix.Tasks.Pleroma.RobotsTxt
 
   test "creates new dir" do

test/user/synchronization_test.exs

@@ -1,104 +0,0 @@
-# Pleroma: A lightweight social networking server
-# Copyright © 2017-2018 Pleroma Authors <https://pleroma.social/>
-# SPDX-License-Identifier: AGPL-3.0-only
-
-defmodule Pleroma.User.SynchronizationTest do
-  use Pleroma.DataCase
-  import Pleroma.Factory
-  alias Pleroma.User
-  alias Pleroma.User.Synchronization
-
-  setup do
-    Tesla.Mock.mock(fn env -> apply(HttpRequestMock, :request, [env]) end)
-    :ok
-  end
-
-  test "update following/followers counters" do
-    user1 =
-      insert(:user,
-        local: false,
-        ap_id: "http://localhost:4001/users/masto_closed"
-      )
-
-    user2 = insert(:user, local: false, ap_id: "http://localhost:4001/users/fuser2")
-
-    users = User.external_users()
-    assert length(users) == 2
-    {user, %{}} = Synchronization.call(users, %{})
-    assert user == List.last(users)
-
-    %{follower_count: followers, following_count: following} = User.get_cached_user_info(user1)
-    assert followers == 437
-    assert following == 152
-
-    %{follower_count: followers, following_count: following} = User.get_cached_user_info(user2)
-
-    assert followers == 527
-    assert following == 267
-  end
-
-  test "don't check host if errors exist" do
-    user1 = insert(:user, local: false, ap_id: "http://domain-with-errors:4001/users/fuser1")
-
-    user2 = insert(:user, local: false, ap_id: "http://domain-with-errors:4001/users/fuser2")
-
-    users = User.external_users()
-    assert length(users) == 2
-
-    {user, %{"domain-with-errors" => 2}} =
-      Synchronization.call(users, %{"domain-with-errors" => 2}, max_retries: 2)
-
-    assert user == List.last(users)
-
-    %{follower_count: followers, following_count: following} = User.get_cached_user_info(user1)
-    assert followers == 0
-    assert following == 0
-
-    %{follower_count: followers, following_count: following} = User.get_cached_user_info(user2)
-
-    assert followers == 0
-    assert following == 0
-  end
-
-  test "don't check host if errors appeared" do
-    user1 = insert(:user, local: false, ap_id: "http://domain-with-errors:4001/users/fuser1")
-
-    user2 = insert(:user, local: false, ap_id: "http://domain-with-errors:4001/users/fuser2")
-
-    users = User.external_users()
-    assert length(users) == 2
-
-    {user, %{"domain-with-errors" => 2}} = Synchronization.call(users, %{}, max_retries: 2)
-
-    assert user == List.last(users)
-
-    %{follower_count: followers, following_count: following} = User.get_cached_user_info(user1)
-    assert followers == 0
-    assert following == 0
-
-    %{follower_count: followers, following_count: following} = User.get_cached_user_info(user2)
-
-    assert followers == 0
-    assert following == 0
-  end
-
-  test "other users after error appeared" do
-    user1 = insert(:user, local: false, ap_id: "http://domain-with-errors:4001/users/fuser1")
-    user2 = insert(:user, local: false, ap_id: "http://localhost:4001/users/fuser2")
-
-    users = User.external_users()
-    assert length(users) == 2
-
-    {user, %{"domain-with-errors" => 2}} = Synchronization.call(users, %{}, max_retries: 2)
-    assert user == List.last(users)
-
-    %{follower_count: followers, following_count: following} = User.get_cached_user_info(user1)
-    assert followers == 0
-    assert following == 0
-
-    %{follower_count: followers, following_count: following} = User.get_cached_user_info(user2)
-
-    assert followers == 527
-    assert following == 267
-  end
-end

test/user/synchronization_worker_test.exs

@@ -1,49 +0,0 @@
-# Pleroma: A lightweight social networking server
-# Copyright © 2017-2018 Pleroma Authors <https://pleroma.social/>
-# SPDX-License-Identifier: AGPL-3.0-only
-
-defmodule Pleroma.User.SynchronizationWorkerTest do
-  use Pleroma.DataCase
-  import Pleroma.Factory
-
-  setup do
-    Tesla.Mock.mock_global(fn env -> apply(HttpRequestMock, :request, [env]) end)
-
-    config = Pleroma.Config.get([:instance, :external_user_synchronization])
-
-    for_update = [enabled: true, interval: 1000]
-
-    Pleroma.Config.put([:instance, :external_user_synchronization], for_update)
-
-    on_exit(fn ->
-      Pleroma.Config.put([:instance, :external_user_synchronization], config)
-    end)
-
-    :ok
-  end
-
-  test "sync follow counters" do
-    user1 =
-      insert(:user,
-        local: false,
-        ap_id: "http://localhost:4001/users/masto_closed"
-      )
-
-    user2 = insert(:user, local: false, ap_id: "http://localhost:4001/users/fuser2")
-
-    {:ok, _} = Pleroma.User.SynchronizationWorker.start_link()
-    :timer.sleep(1500)
-
-    %{follower_count: followers, following_count: following} =
-      Pleroma.User.get_cached_user_info(user1)
-
-    assert followers == 437
-    assert following == 152
-
-    %{follower_count: followers, following_count: following} =
-      Pleroma.User.get_cached_user_info(user2)
-
-    assert followers == 527
-    assert following == 267
-  end
-end

test/user_test.exs

@@ -54,6 +54,14 @@ test "ap_followers returns the followers collection for the user" do
     assert expected_followers_collection == User.ap_followers(user)
   end
 
+  test "ap_following returns the following collection for the user" do
+    user = UserBuilder.build()
+
+    expected_followers_collection = "#{User.ap_id(user)}/following"
+
+    assert expected_followers_collection == User.ap_following(user)
+  end
+
   test "returns all pending follow requests" do
     unlocked = insert(:user)
     locked = insert(:user, %{info: %{locked: true}})
@@ -679,10 +687,12 @@ test "it mutes people" do
       muted_user = insert(:user)
 
       refute User.mutes?(user, muted_user)
+      refute User.muted_notifications?(user, muted_user)
 
       {:ok, user} = User.mute(user, muted_user)
 
       assert User.mutes?(user, muted_user)
+      assert User.muted_notifications?(user, muted_user)
     end
 
     test "it unmutes users" do
@@ -693,6 +703,20 @@ test "it unmutes users" do
       {:ok, user} = User.unmute(user, muted_user)
 
       refute User.mutes?(user, muted_user)
+      refute User.muted_notifications?(user, muted_user)
+    end
+
+    test "it mutes user without notifications" do
+      user = insert(:user)
+      muted_user = insert(:user)
+
+      refute User.mutes?(user, muted_user)
+      refute User.muted_notifications?(user, muted_user)
+
+      {:ok, user} = User.mute(user, muted_user, false)
+
+      assert User.mutes?(user, muted_user)
+      refute User.muted_notifications?(user, muted_user)
     end
   end
 
@@ -1240,52 +1264,6 @@ test "external_users/1 external active users with limit", %{user1: user1, user2:
 
       assert User.external_users(max_id: fdb_user2.id, limit: 1) == []
     end
-
-    test "sync_follow_counters/1", %{user1: user1, user2: user2} do
-      {:ok, _pid} = Agent.start_link(fn -> %{} end, name: :domain_errors)
-
-      :ok = User.sync_follow_counters()
-
-      %{follower_count: followers, following_count: following} = User.get_cached_user_info(user1)
-      assert followers == 437
-      assert following == 152
-
-      %{follower_count: followers, following_count: following} = User.get_cached_user_info(user2)
-
-      assert followers == 527
-      assert following == 267
-
-      Agent.stop(:domain_errors)
-    end
-
-    test "sync_follow_counters/1 in separate batches", %{user1: user1, user2: user2} do
-      {:ok, _pid} = Agent.start_link(fn -> %{} end, name: :domain_errors)
-
-      :ok = User.sync_follow_counters(limit: 1)
-
-      %{follower_count: followers, following_count: following} = User.get_cached_user_info(user1)
-      assert followers == 437
-      assert following == 152
-
-      %{follower_count: followers, following_count: following} = User.get_cached_user_info(user2)
-
-      assert followers == 527
-      assert following == 267
-
-      Agent.stop(:domain_errors)
-    end
-
-    test "perform/1 with :sync_follow_counters", %{user1: user1, user2: user2} do
-      :ok = User.perform(:sync_follow_counters)
-      %{follower_count: followers, following_count: following} = User.get_cached_user_info(user1)
-      assert followers == 437
-      assert following == 152
-
-      %{follower_count: followers, following_count: following} = User.get_cached_user_info(user2)
-
-      assert followers == 527
-      assert following == 267
-    end
   end
 
   describe "set_info_cache/2" do

test/web/activity_pub/activity_pub_controller_test.exs

@@ -12,6 +12,7 @@ defmodule Pleroma.Web.ActivityPub.ActivityPubControllerTest do
   alias Pleroma.Web.ActivityPub.ObjectView
   alias Pleroma.Web.ActivityPub.UserView
   alias Pleroma.Web.ActivityPub.Utils
+  alias Pleroma.Web.CommonAPI
 
   setup_all do
     Tesla.Mock.mock_global(fn env -> apply(HttpRequestMock, :request, [env]) end)
@@ -551,7 +552,7 @@ test "it returns the followers in a collection", %{conn: conn} do
       assert result["first"]["orderedItems"] == [user.ap_id]
     end
 
-    test "it returns returns empty if the user has 'hide_followers' set", %{conn: conn} do
+    test "it returns returns a uri if the user has 'hide_followers' set", %{conn: conn} do
       user = insert(:user)
       user_two = insert(:user, %{info: %{hide_followers: true}})
       User.follow(user, user_two)
@@ -561,8 +562,35 @@ test "it returns returns empty if the user has 'hide_followers' set", %{conn: co
         |> get("/users/#{user_two.nickname}/followers")
         |> json_response(200)
 
-      assert result["first"]["orderedItems"] == []
+      assert is_binary(result["first"])
-      assert result["totalItems"] == 0
+    end
+
+    test "it returns a 403 error on pages, if the user has 'hide_followers' set and the request is not authenticated",
+         %{conn: conn} do
+      user = insert(:user, %{info: %{hide_followers: true}})
+
+      result =
+        conn
+        |> get("/users/#{user.nickname}/followers?page=1")
+
+      assert result.status == 403
+      assert result.resp_body == ""
+    end
+
+    test "it renders the page, if the user has 'hide_followers' set and the request is authenticated with the same user",
+         %{conn: conn} do
+      user = insert(:user, %{info: %{hide_followers: true}})
+      other_user = insert(:user)
+      {:ok, _other_user, user, _activity} = CommonAPI.follow(other_user, user)
+
+      result =
+        conn
+        |> assign(:user, user)
+        |> get("/users/#{user.nickname}/followers?page=1")
+        |> json_response(200)
+
+      assert result["totalItems"] == 1
+      assert result["orderedItems"] == [other_user.ap_id]
     end
 
     test "it works for more than 10 users", %{conn: conn} do
@@ -606,7 +634,7 @@ test "it returns the following in a collection", %{conn: conn} do
       assert result["first"]["orderedItems"] == [user_two.ap_id]
     end
 
-    test "it returns returns empty if the user has 'hide_follows' set", %{conn: conn} do
+    test "it returns a uri if the user has 'hide_follows' set", %{conn: conn} do
       user = insert(:user, %{info: %{hide_follows: true}})
       user_two = insert(:user)
       User.follow(user, user_two)
@@ -616,8 +644,35 @@ test "it returns returns empty if the user has 'hide_follows' set", %{conn: conn
         |> get("/users/#{user.nickname}/following")
         |> json_response(200)
 
-      assert result["first"]["orderedItems"] == []
+      assert is_binary(result["first"])
-      assert result["totalItems"] == 0
+    end
+
+    test "it returns a 403 error on pages, if the user has 'hide_follows' set and the request is not authenticated",
+         %{conn: conn} do
+      user = insert(:user, %{info: %{hide_follows: true}})
+
+      result =
+        conn
+        |> get("/users/#{user.nickname}/following?page=1")
+
+      assert result.status == 403
+      assert result.resp_body == ""
+    end
+
+    test "it renders the page, if the user has 'hide_follows' set and the request is authenticated with the same user",
+         %{conn: conn} do
+      user = insert(:user, %{info: %{hide_follows: true}})
+      other_user = insert(:user)
+      {:ok, user, _other_user, _activity} = CommonAPI.follow(user, other_user)
+
+      result =
+        conn
+        |> assign(:user, user)
+        |> get("/users/#{user.nickname}/following?page=1")
+        |> json_response(200)
+
+      assert result["totalItems"] == 1
+      assert result["orderedItems"] == [other_user.ap_id]
     end
 
     test "it works for more than 10 users", %{conn: conn} do

test/web/activity_pub/transmogrifier_test.exs

@@ -416,6 +416,7 @@ test "it ensures that as:Public activities make it to their followers collection
         |> Map.put("attributedTo", user.ap_id)
         |> Map.put("to", ["https://www.w3.org/ns/activitystreams#Public"])
         |> Map.put("cc", [])
+        |> Map.put("id", user.ap_id <> "/activities/12345678")
 
       data = Map.put(data, "object", object)
 
@@ -439,6 +440,7 @@ test "it ensures that address fields become lists" do
         |> Map.put("attributedTo", user.ap_id)
         |> Map.put("to", nil)
         |> Map.put("cc", nil)
+        |> Map.put("id", user.ap_id <> "/activities/12345678")
 
       data = Map.put(data, "object", object)
 
@@ -1133,6 +1135,7 @@ test "it upgrades a user to activitypub" do
       assert user.info.ap_enabled
       assert user.info.note_count == 1
       assert user.follower_address == "https://niu.moe/users/rye/followers"
+      assert user.following_address == "https://niu.moe/users/rye/following"
 
       user = User.get_cached_by_id(user.id)
       assert user.info.note_count == 1
@@ -1370,4 +1373,32 @@ test "removes recipient's follower collection from cc", %{user: user} do
       refute recipient.follower_address in fixed_object["to"]
     end
   end
+
+  test "update_following_followers_counters/1" do
+    user1 =
+      insert(:user,
+        local: false,
+        follower_address: "http://localhost:4001/users/masto_closed/followers",
+        following_address: "http://localhost:4001/users/masto_closed/following"
+      )
+
+    user2 =
+      insert(:user,
+        local: false,
+        follower_address: "http://localhost:4001/users/fuser2/followers",
+        following_address: "http://localhost:4001/users/fuser2/following"
+      )
+
+    Transmogrifier.update_following_followers_counters(user1)
+    Transmogrifier.update_following_followers_counters(user2)
+
+    %{follower_count: followers, following_count: following} = User.get_cached_user_info(user1)
+    assert followers == 437
+    assert following == 152
+
+    %{follower_count: followers, following_count: following} = User.get_cached_user_info(user2)
+
+    assert followers == 527
+    assert following == 267
+  end
 end

test/web/activity_pub/views/user_view_test.exs

@@ -8,6 +8,7 @@ defmodule Pleroma.Web.ActivityPub.UserViewTest do
 
   alias Pleroma.User
   alias Pleroma.Web.ActivityPub.UserView
+  alias Pleroma.Web.CommonAPI
 
   test "Renders a user, including the public key" do
     user = insert(:user)
@@ -82,4 +83,28 @@ test "instance users do not expose oAuth endpoints" do
       refute result["endpoints"]["oauthTokenEndpoint"]
     end
   end
+
+  describe "followers" do
+    test "sets totalItems to zero when followers are hidden" do
+      user = insert(:user)
+      other_user = insert(:user)
+      {:ok, _other_user, user, _activity} = CommonAPI.follow(other_user, user)
+      assert %{"totalItems" => 1} = UserView.render("followers.json", %{user: user})
+      info = Map.put(user.info, :hide_followers, true)
+      user = Map.put(user, :info, info)
+      assert %{"totalItems" => 0} = UserView.render("followers.json", %{user: user})
+    end
+  end
+
+  describe "following" do
+    test "sets totalItems to zero when follows are hidden" do
+      user = insert(:user)
+      other_user = insert(:user)
+      {:ok, user, _other_user, _activity} = CommonAPI.follow(user, other_user)
+      assert %{"totalItems" => 1} = UserView.render("following.json", %{user: user})
+      info = Map.put(user.info, :hide_follows, true)
+      user = Map.put(user, :info, info)
+      assert %{"totalItems" => 0} = UserView.render("following.json", %{user: user})
+    end
+  end
 end

test/web/admin_api/admin_api_controller_test.exs

@@ -1407,14 +1407,19 @@ test "create new config setting in db", %{conn: conn} do
         post(conn, "/api/pleroma/admin/config", %{
           configs: [
             %{group: "pleroma", key: "key1", value: "value1"},
+            %{
+              group: "ueberauth",
+              key: "Ueberauth.Strategy.Twitter.OAuth",
+              value: [%{"tuple" => [":consumer_secret", "aaaa"]}]
+            },
             %{
               group: "pleroma",
               key: "key2",
               value: %{
-                "nested_1" => "nested_value1",
+                ":nested_1" => "nested_value1",
-                "nested_2" => [
+                ":nested_2" => [
-                  %{"nested_22" => "nested_value222"},
+                  %{":nested_22" => "nested_value222"},
-                  %{"nested_33" => %{"nested_44" => "nested_444"}}
+                  %{":nested_33" => %{":nested_44" => "nested_444"}}
                 ]
               }
             },
@@ -1423,13 +1428,13 @@ test "create new config setting in db", %{conn: conn} do
               key: "key3",
               value: [
                 %{"nested_3" => ":nested_3", "nested_33" => "nested_33"},
-                %{"nested_4" => ":true"}
+                %{"nested_4" => true}
               ]
             },
             %{
               group: "pleroma",
               key: "key4",
-              value: %{"nested_5" => ":upload", "endpoint" => "https://example.com"}
+              value: %{":nested_5" => ":upload", "endpoint" => "https://example.com"}
             },
             %{
               group: "idna",
@@ -1446,31 +1451,34 @@ test "create new config setting in db", %{conn: conn} do
                    "key" => "key1",
                    "value" => "value1"
                  },
+                 %{
+                   "group" => "ueberauth",
+                   "key" => "Ueberauth.Strategy.Twitter.OAuth",
+                   "value" => [%{"tuple" => [":consumer_secret", "aaaa"]}]
+                 },
                  %{
                    "group" => "pleroma",
                    "key" => "key2",
-                   "value" => [
+                   "value" => %{
-                     %{"nested_1" => "nested_value1"},
+                     ":nested_1" => "nested_value1",
-                     %{
+                     ":nested_2" => [
-                       "nested_2" => [
+                       %{":nested_22" => "nested_value222"},
-                         %{"nested_22" => "nested_value222"},
+                       %{":nested_33" => %{":nested_44" => "nested_444"}}
-                         %{"nested_33" => %{"nested_44" => "nested_444"}}
+                     ]
-                       ]
+                   }
-                     }
-                   ]
                  },
                  %{
                    "group" => "pleroma",
                    "key" => "key3",
                    "value" => [
-                     [%{"nested_3" => "nested_3"}, %{"nested_33" => "nested_33"}],
+                     %{"nested_3" => ":nested_3", "nested_33" => "nested_33"},
                      %{"nested_4" => true}
                    ]
                  },
                  %{
                    "group" => "pleroma",
                    "key" => "key4",
-                   "value" => [%{"endpoint" => "https://example.com"}, %{"nested_5" => "upload"}]
+                   "value" => %{"endpoint" => "https://example.com", ":nested_5" => ":upload"}
                  },
                  %{
                    "group" => "idna",
@@ -1482,23 +1490,23 @@ test "create new config setting in db", %{conn: conn} do
 
       assert Application.get_env(:pleroma, :key1) == "value1"
 
-      assert Application.get_env(:pleroma, :key2) == [
+      assert Application.get_env(:pleroma, :key2) == %{
                nested_1: "nested_value1",
                nested_2: [
-                 [nested_22: "nested_value222"],
+                 %{nested_22: "nested_value222"},
-                 [nested_33: [nested_44: "nested_444"]]
+                 %{nested_33: %{nested_44: "nested_444"}}
                ]
-             ]
+             }
 
       assert Application.get_env(:pleroma, :key3) == [
-               [nested_3: :nested_3, nested_33: "nested_33"],
+               %{"nested_3" => :nested_3, "nested_33" => "nested_33"},
-               [nested_4: true]
+               %{"nested_4" => true}
              ]
 
-      assert Application.get_env(:pleroma, :key4) == [
+      assert Application.get_env(:pleroma, :key4) == %{
-               endpoint: "https://example.com",
+               "endpoint" => "https://example.com",
                nested_5: :upload
-             ]
+             }
 
       assert Application.get_env(:idna, :key5) == {"string", Pleroma.Captcha.NotReal, []}
     end
@@ -1507,11 +1515,22 @@ test "update config setting & delete", %{conn: conn} do
       config1 = insert(:config, key: "keyaa1")
       config2 = insert(:config, key: "keyaa2")
 
+      insert(:config,
+        group: "ueberauth",
+        key: "Ueberauth.Strategy.Microsoft.OAuth",
+        value: :erlang.term_to_binary([])
+      )
+
       conn =
         post(conn, "/api/pleroma/admin/config", %{
           configs: [
             %{group: config1.group, key: config1.key, value: "another_value"},
-            %{group: config2.group, key: config2.key, delete: "true"}
+            %{group: config2.group, key: config2.key, delete: "true"},
+            %{
+              group: "ueberauth",
+              key: "Ueberauth.Strategy.Microsoft.OAuth",
+              delete: "true"
+            }
           ]
         })
 
@@ -1536,11 +1555,13 @@ test "common config example", %{conn: conn} do
             %{
               "group" => "pleroma",
               "key" => "Pleroma.Captcha.NotReal",
-              "value" => %{
+              "value" => [
-                "enabled" => ":false",
+                %{"tuple" => [":enabled", false]},
-                "method" => "Pleroma.Captcha.Kocaptcha",
+                %{"tuple" => [":method", "Pleroma.Captcha.Kocaptcha"]},
-                "seconds_valid" => "i:60"
+                %{"tuple" => [":seconds_valid", 60]},
-              }
+                %{"tuple" => [":path", ""]},
+                %{"tuple" => [":key1", nil]}
+              ]
             }
           ]
         })
@@ -1551,9 +1572,11 @@ test "common config example", %{conn: conn} do
                    "group" => "pleroma",
                    "key" => "Pleroma.Captcha.NotReal",
                    "value" => [
-                     %{"enabled" => false},
+                     %{"tuple" => [":enabled", false]},
-                     %{"method" => "Pleroma.Captcha.Kocaptcha"},
+                     %{"tuple" => [":method", "Pleroma.Captcha.Kocaptcha"]},
-                     %{"seconds_valid" => 60}
+                     %{"tuple" => [":seconds_valid", 60]},
+                     %{"tuple" => [":path", ""]},
+                     %{"tuple" => [":key1", nil]}
                    ]
                  }
                ]
@@ -1569,51 +1592,57 @@ test "tuples with more than two values", %{conn: conn} do
               "key" => "Pleroma.Web.Endpoint.NotReal",
               "value" => [
                 %{
-                  "http" => %{
+                  "tuple" => [
-                    "dispatch" => [
+                    ":http",
+                    [
                       %{
                         "tuple" => [
-                          ":_",
+                          ":key2",
                           [
-                            %{
-                              "tuple" => [
-                                "/api/v1/streaming",
-                                "Pleroma.Web.MastodonAPI.WebsocketHandler",
-                                []
-                              ]
-                            },
-                            %{
-                              "tuple" => [
-                                "/websocket",
-                                "Phoenix.Endpoint.CowboyWebSocket",
-                                %{
-                                  "tuple" => [
-                                    "Phoenix.Transports.WebSocket",
-                                    %{
-                                      "tuple" => [
-                                        "Pleroma.Web.Endpoint",
-                                        "Pleroma.Web.UserSocket",
-                                        []
-                                      ]
-                                    }
-                                  ]
-                                }
-                              ]
-                            },
                             %{
                               "tuple" => [
                                 ":_",
-                                "Phoenix.Endpoint.Cowboy2Handler",
+                                [
-                                %{
+                                  %{
-                                  "tuple" => ["Pleroma.Web.Endpoint", []]
+                                    "tuple" => [
-                                }
+                                      "/api/v1/streaming",
+                                      "Pleroma.Web.MastodonAPI.WebsocketHandler",
+                                      []
+                                    ]
+                                  },
+                                  %{
+                                    "tuple" => [
+                                      "/websocket",
+                                      "Phoenix.Endpoint.CowboyWebSocket",
+                                      %{
+                                        "tuple" => [
+                                          "Phoenix.Transports.WebSocket",
+                                          %{
+                                            "tuple" => [
+                                              "Pleroma.Web.Endpoint",
+                                              "Pleroma.Web.UserSocket",
+                                              []
+                                            ]
+                                          }
+                                        ]
+                                      }
+                                    ]
+                                  },
+                                  %{
+                                    "tuple" => [
+                                      ":_",
+                                      "Phoenix.Endpoint.Cowboy2Handler",
+                                      %{"tuple" => ["Pleroma.Web.Endpoint", []]}
+                                    ]
+                                  }
+                                ]
                               ]
                             }
                           ]
                         ]
                       }
                     ]
-                  }
+                  ]
                 }
               ]
             }
@@ -1627,41 +1656,206 @@ test "tuples with more than two values", %{conn: conn} do
                    "key" => "Pleroma.Web.Endpoint.NotReal",
                    "value" => [
                      %{
-                       "http" => %{
+                       "tuple" => [
-                         "dispatch" => %{
+                         ":http",
-                           "_" => [
+                         [
-                             %{
+                           %{
-                               "tuple" => [
+                             "tuple" => [
-                                 "/api/v1/streaming",
+                               ":key2",
-                                 "Pleroma.Web.MastodonAPI.WebsocketHandler",
+                               [
-                                 []
-                               ]
-                             },
-                             %{
-                               "tuple" => [
-                                 "/websocket",
-                                 "Phoenix.Endpoint.CowboyWebSocket",
                                  %{
-                                   "Elixir.Phoenix.Transports.WebSocket" => %{
+                                   "tuple" => [
-                                     "tuple" => [
+                                     ":_",
-                                       "Pleroma.Web.Endpoint",
+                                     [
-                                       "Pleroma.Web.UserSocket",
+                                       %{
-                                       []
+                                         "tuple" => [
+                                           "/api/v1/streaming",
+                                           "Pleroma.Web.MastodonAPI.WebsocketHandler",
+                                           []
+                                         ]
+                                       },
+                                       %{
+                                         "tuple" => [
+                                           "/websocket",
+                                           "Phoenix.Endpoint.CowboyWebSocket",
+                                           %{
+                                             "tuple" => [
+                                               "Phoenix.Transports.WebSocket",
+                                               %{
+                                                 "tuple" => [
+                                                   "Pleroma.Web.Endpoint",
+                                                   "Pleroma.Web.UserSocket",
+                                                   []
+                                                 ]
+                                               }
+                                             ]
+                                           }
+                                         ]
+                                       },
+                                       %{
+                                         "tuple" => [
+                                           ":_",
+                                           "Phoenix.Endpoint.Cowboy2Handler",
+                                           %{"tuple" => ["Pleroma.Web.Endpoint", []]}
+                                         ]
+                                       }
                                      ]
-                                   }
+                                   ]
                                  }
                                ]
-                             },
+                             ]
-                             %{
+                           }
-                               "tuple" => [
+                         ]
-                                 "_",
+                       ]
-                                 "Phoenix.Endpoint.Cowboy2Handler",
+                     }
-                                 %{"Elixir.Pleroma.Web.Endpoint" => []}
+                   ]
-                               ]
+                 }
+               ]
+             }
+    end
+
+    test "settings with nesting map", %{conn: conn} do
+      conn =
+        post(conn, "/api/pleroma/admin/config", %{
+          configs: [
+            %{
+              "group" => "pleroma",
+              "key" => "key1",
+              "value" => [
+                %{"tuple" => [":key2", "some_val"]},
+                %{
+                  "tuple" => [
+                    ":key3",
+                    %{
+                      ":max_options" => 20,
+                      ":max_option_chars" => 200,
+                      ":min_expiration" => 0,
+                      ":max_expiration" => 31_536_000,
+                      "nested" => %{
+                        ":max_options" => 20,
+                        ":max_option_chars" => 200,
+                        ":min_expiration" => 0,
+                        ":max_expiration" => 31_536_000
+                      }
+                    }
+                  ]
+                }
+              ]
+            }
+          ]
+        })
+
+      assert json_response(conn, 200) ==
+               %{
+                 "configs" => [
+                   %{
+                     "group" => "pleroma",
+                     "key" => "key1",
+                     "value" => [
+                       %{"tuple" => [":key2", "some_val"]},
+                       %{
+                         "tuple" => [
+                           ":key3",
+                           %{
+                             ":max_expiration" => 31_536_000,
+                             ":max_option_chars" => 200,
+                             ":max_options" => 20,
+                             ":min_expiration" => 0,
+                             "nested" => %{
+                               ":max_expiration" => 31_536_000,
+                               ":max_option_chars" => 200,
+                               ":max_options" => 20,
+                               ":min_expiration" => 0
                              }
-                           ]
+                           }
-                         }
+                         ]
                        }
+                     ]
+                   }
+                 ]
+               }
+    end
+
+    test "value as map", %{conn: conn} do
+      conn =
+        post(conn, "/api/pleroma/admin/config", %{
+          configs: [
+            %{
+              "group" => "pleroma",
+              "key" => "key1",
+              "value" => %{"key" => "some_val"}
+            }
+          ]
+        })
+
+      assert json_response(conn, 200) ==
+               %{
+                 "configs" => [
+                   %{
+                     "group" => "pleroma",
+                     "key" => "key1",
+                     "value" => %{"key" => "some_val"}
+                   }
+                 ]
+               }
+    end
+
+    test "dispatch setting", %{conn: conn} do
+      conn =
+        post(conn, "/api/pleroma/admin/config", %{
+          configs: [
+            %{
+              "group" => "pleroma",
+              "key" => "Pleroma.Web.Endpoint.NotReal",
+              "value" => [
+                %{
+                  "tuple" => [
+                    ":http",
+                    [
+                      %{"tuple" => [":ip", %{"tuple" => [127, 0, 0, 1]}]},
+                      %{"tuple" => [":dispatch", ["{:_,
+       [
+         {\"/api/v1/streaming\", Pleroma.Web.MastodonAPI.WebsocketHandler, []},
+         {\"/websocket\", Phoenix.Endpoint.CowboyWebSocket,
+          {Phoenix.Transports.WebSocket,
+           {Pleroma.Web.Endpoint, Pleroma.Web.UserSocket, [path: \"/websocket\"]}}},
+         {:_, Phoenix.Endpoint.Cowboy2Handler, {Pleroma.Web.Endpoint, []}}
+       ]}"]]}
+                    ]
+                  ]
+                }
+              ]
+            }
+          ]
+        })
+
+      dispatch_string =
+        "{:_, [{\"/api/v1/streaming\", Pleroma.Web.MastodonAPI.WebsocketHandler, []}, " <>
+          "{\"/websocket\", Phoenix.Endpoint.CowboyWebSocket, {Phoenix.Transports.WebSocket, " <>
+          "{Pleroma.Web.Endpoint, Pleroma.Web.UserSocket, [path: \"/websocket\"]}}}, " <>
+          "{:_, Phoenix.Endpoint.Cowboy2Handler, {Pleroma.Web.Endpoint, []}}]}"
+
+      assert json_response(conn, 200) == %{
+               "configs" => [
+                 %{
+                   "group" => "pleroma",
+                   "key" => "Pleroma.Web.Endpoint.NotReal",
+                   "value" => [
+                     %{
+                       "tuple" => [
+                         ":http",
+                         [
+                           %{"tuple" => [":ip", %{"tuple" => [127, 0, 0, 1]}]},
+                           %{
+                             "tuple" => [
+                               ":dispatch",
+                               [
+                                 dispatch_string
+                               ]
+                             ]
+                           }
+                         ]
+                       ]
                      }
                    ]
                  }

test/web/admin_api/config_test.exs

@@ -61,117 +61,306 @@ test "string" do
       assert Config.from_binary(binary) == "value as string"
     end
 
+    test "boolean" do
+      binary = Config.transform(false)
+      assert binary == :erlang.term_to_binary(false)
+      assert Config.from_binary(binary) == false
+    end
+
+    test "nil" do
+      binary = Config.transform(nil)
+      assert binary == :erlang.term_to_binary(nil)
+      assert Config.from_binary(binary) == nil
+    end
+
+    test "integer" do
+      binary = Config.transform(150)
+      assert binary == :erlang.term_to_binary(150)
+      assert Config.from_binary(binary) == 150
+    end
+
+    test "atom" do
+      binary = Config.transform(":atom")
+      assert binary == :erlang.term_to_binary(:atom)
+      assert Config.from_binary(binary) == :atom
+    end
+
+    test "pleroma module" do
+      binary = Config.transform("Pleroma.Bookmark")
+      assert binary == :erlang.term_to_binary(Pleroma.Bookmark)
+      assert Config.from_binary(binary) == Pleroma.Bookmark
+    end
+
+    test "phoenix module" do
+      binary = Config.transform("Phoenix.Socket.V1.JSONSerializer")
+      assert binary == :erlang.term_to_binary(Phoenix.Socket.V1.JSONSerializer)
+      assert Config.from_binary(binary) == Phoenix.Socket.V1.JSONSerializer
+    end
+
+    test "sigil" do
+      binary = Config.transform("~r/comp[lL][aA][iI][nN]er/")
+      assert binary == :erlang.term_to_binary(~r/comp[lL][aA][iI][nN]er/)
+      assert Config.from_binary(binary) == ~r/comp[lL][aA][iI][nN]er/
+    end
+
+    test "2 child tuple" do
+      binary = Config.transform(%{"tuple" => ["v1", ":v2"]})
+      assert binary == :erlang.term_to_binary({"v1", :v2})
+      assert Config.from_binary(binary) == {"v1", :v2}
+    end
+
+    test "tuple with n childs" do
+      binary =
+        Config.transform(%{
+          "tuple" => [
+            "v1",
+            ":v2",
+            "Pleroma.Bookmark",
+            150,
+            false,
+            "Phoenix.Socket.V1.JSONSerializer"
+          ]
+        })
+
+      assert binary ==
+               :erlang.term_to_binary(
+                 {"v1", :v2, Pleroma.Bookmark, 150, false, Phoenix.Socket.V1.JSONSerializer}
+               )
+
+      assert Config.from_binary(binary) ==
+               {"v1", :v2, Pleroma.Bookmark, 150, false, Phoenix.Socket.V1.JSONSerializer}
+    end
+
+    test "tuple with dispatch key" do
+      binary = Config.transform(%{"tuple" => [":dispatch", ["{:_,
+       [
+         {\"/api/v1/streaming\", Pleroma.Web.MastodonAPI.WebsocketHandler, []},
+         {\"/websocket\", Phoenix.Endpoint.CowboyWebSocket,
+          {Phoenix.Transports.WebSocket,
+           {Pleroma.Web.Endpoint, Pleroma.Web.UserSocket, [path: \"/websocket\"]}}},
+         {:_, Phoenix.Endpoint.Cowboy2Handler, {Pleroma.Web.Endpoint, []}}
+       ]}"]]})
+
+      assert binary ==
+               :erlang.term_to_binary(
+                 {:dispatch,
+                  [
+                    {:_,
+                     [
+                       {"/api/v1/streaming", Pleroma.Web.MastodonAPI.WebsocketHandler, []},
+                       {"/websocket", Phoenix.Endpoint.CowboyWebSocket,
+                        {Phoenix.Transports.WebSocket,
+                         {Pleroma.Web.Endpoint, Pleroma.Web.UserSocket, [path: "/websocket"]}}},
+                       {:_, Phoenix.Endpoint.Cowboy2Handler, {Pleroma.Web.Endpoint, []}}
+                     ]}
+                  ]}
+               )
+
+      assert Config.from_binary(binary) ==
+               {:dispatch,
+                [
+                  {:_,
+                   [
+                     {"/api/v1/streaming", Pleroma.Web.MastodonAPI.WebsocketHandler, []},
+                     {"/websocket", Phoenix.Endpoint.CowboyWebSocket,
+                      {Phoenix.Transports.WebSocket,
+                       {Pleroma.Web.Endpoint, Pleroma.Web.UserSocket, [path: "/websocket"]}}},
+                     {:_, Phoenix.Endpoint.Cowboy2Handler, {Pleroma.Web.Endpoint, []}}
+                   ]}
+                ]}
+    end
+
+    test "map with string key" do
+      binary = Config.transform(%{"key" => "value"})
+      assert binary == :erlang.term_to_binary(%{"key" => "value"})
+      assert Config.from_binary(binary) == %{"key" => "value"}
+    end
+
+    test "map with atom key" do
+      binary = Config.transform(%{":key" => "value"})
+      assert binary == :erlang.term_to_binary(%{key: "value"})
+      assert Config.from_binary(binary) == %{key: "value"}
+    end
+
+    test "list of strings" do
+      binary = Config.transform(["v1", "v2", "v3"])
+      assert binary == :erlang.term_to_binary(["v1", "v2", "v3"])
+      assert Config.from_binary(binary) == ["v1", "v2", "v3"]
+    end
+
     test "list of modules" do
       binary = Config.transform(["Pleroma.Repo", "Pleroma.Activity"])
       assert binary == :erlang.term_to_binary([Pleroma.Repo, Pleroma.Activity])
       assert Config.from_binary(binary) == [Pleroma.Repo, Pleroma.Activity]
     end
 
-    test "list of strings" do
+    test "list of atoms" do
-      binary = Config.transform(["string1", "string2"])
+      binary = Config.transform([":v1", ":v2", ":v3"])
-      assert binary == :erlang.term_to_binary(["string1", "string2"])
+      assert binary == :erlang.term_to_binary([:v1, :v2, :v3])
-      assert Config.from_binary(binary) == ["string1", "string2"]
+      assert Config.from_binary(binary) == [:v1, :v2, :v3]
     end
 
-    test "map" do
+    test "list of mixed values" do
       binary =
-        Config.transform(%{
+        Config.transform([
-          "types" => "Pleroma.PostgresTypes",
+          "v1",
-          "telemetry_event" => ["Pleroma.Repo.Instrumenter"],
+          ":v2",
-          "migration_lock" => ""
+          "Pleroma.Repo",
-        })
+          "Phoenix.Socket.V1.JSONSerializer",
+          15,
+          false
+        ])
 
       assert binary ==
-               :erlang.term_to_binary(
+               :erlang.term_to_binary([
-                 telemetry_event: [Pleroma.Repo.Instrumenter],
+                 "v1",
-                 types: Pleroma.PostgresTypes
+                 :v2,
-               )
+                 Pleroma.Repo,
+                 Phoenix.Socket.V1.JSONSerializer,
+                 15,
+                 false
+               ])
 
       assert Config.from_binary(binary) == [
-               telemetry_event: [Pleroma.Repo.Instrumenter],
+               "v1",
-               types: Pleroma.PostgresTypes
+               :v2,
+               Pleroma.Repo,
+               Phoenix.Socket.V1.JSONSerializer,
+               15,
+               false
              ]
     end
 
-    test "complex map with nested integers, lists and atoms" do
+    test "simple keyword" do
-      binary =
+      binary = Config.transform([%{"tuple" => [":key", "value"]}])
-        Config.transform(%{
+      assert binary == :erlang.term_to_binary([{:key, "value"}])
-          "uploader" => "Pleroma.Uploaders.Local",
+      assert Config.from_binary(binary) == [{:key, "value"}]
-          "filters" => ["Pleroma.Upload.Filter.Dedupe"],
+      assert Config.from_binary(binary) == [key: "value"]
-          "link_name" => ":true",
-          "proxy_remote" => ":false",
-          "proxy_opts" => %{
-            "redirect_on_failure" => ":false",
-            "max_body_length" => "i:1048576",
-            "http" => %{
-              "follow_redirect" => ":true",
-              "pool" => ":upload"
-            }
-          }
-        })
-
-      assert binary ==
-               :erlang.term_to_binary(
-                 filters: [Pleroma.Upload.Filter.Dedupe],
-                 link_name: true,
-                 proxy_opts: [
-                   http: [
-                     follow_redirect: true,
-                     pool: :upload
-                   ],
-                   max_body_length: 1_048_576,
-                   redirect_on_failure: false
-                 ],
-                 proxy_remote: false,
-                 uploader: Pleroma.Uploaders.Local
-               )
-
-      assert Config.from_binary(binary) ==
-               [
-                 filters: [Pleroma.Upload.Filter.Dedupe],
-                 link_name: true,
-                 proxy_opts: [
-                   http: [
-                     follow_redirect: true,
-                     pool: :upload
-                   ],
-                   max_body_length: 1_048_576,
-                   redirect_on_failure: false
-                 ],
-                 proxy_remote: false,
-                 uploader: Pleroma.Uploaders.Local
-               ]
     end
 
     test "keyword" do
       binary =
-        Config.transform(%{
+        Config.transform([
-          "level" => ":warn",
+          %{"tuple" => [":types", "Pleroma.PostgresTypes"]},
-          "meta" => [":all"],
+          %{"tuple" => [":telemetry_event", ["Pleroma.Repo.Instrumenter"]]},
-          "webhook_url" => "https://hooks.slack.com/services/YOUR-KEY-HERE"
+          %{"tuple" => [":migration_lock", nil]},
-        })
+          %{"tuple" => [":key1", 150]},
+          %{"tuple" => [":key2", "string"]}
+        ])
+
+      assert binary ==
+               :erlang.term_to_binary(
+                 types: Pleroma.PostgresTypes,
+                 telemetry_event: [Pleroma.Repo.Instrumenter],
+                 migration_lock: nil,
+                 key1: 150,
+                 key2: "string"
+               )
+
+      assert Config.from_binary(binary) == [
+               types: Pleroma.PostgresTypes,
+               telemetry_event: [Pleroma.Repo.Instrumenter],
+               migration_lock: nil,
+               key1: 150,
+               key2: "string"
+             ]
+    end
+
+    test "complex keyword with nested mixed childs" do
+      binary =
+        Config.transform([
+          %{"tuple" => [":uploader", "Pleroma.Uploaders.Local"]},
+          %{"tuple" => [":filters", ["Pleroma.Upload.Filter.Dedupe"]]},
+          %{"tuple" => [":link_name", true]},
+          %{"tuple" => [":proxy_remote", false]},
+          %{"tuple" => [":common_map", %{":key" => "value"}]},
+          %{
+            "tuple" => [
+              ":proxy_opts",
+              [
+                %{"tuple" => [":redirect_on_failure", false]},
+                %{"tuple" => [":max_body_length", 1_048_576]},
+                %{
+                  "tuple" => [
+                    ":http",
+                    [%{"tuple" => [":follow_redirect", true]}, %{"tuple" => [":pool", ":upload"]}]
+                  ]
+                }
+              ]
+            ]
+          }
+        ])
+
+      assert binary ==
+               :erlang.term_to_binary(
+                 uploader: Pleroma.Uploaders.Local,
+                 filters: [Pleroma.Upload.Filter.Dedupe],
+                 link_name: true,
+                 proxy_remote: false,
+                 common_map: %{key: "value"},
+                 proxy_opts: [
+                   redirect_on_failure: false,
+                   max_body_length: 1_048_576,
+                   http: [
+                     follow_redirect: true,
+                     pool: :upload
+                   ]
+                 ]
+               )
+
+      assert Config.from_binary(binary) ==
+               [
+                 uploader: Pleroma.Uploaders.Local,
+                 filters: [Pleroma.Upload.Filter.Dedupe],
+                 link_name: true,
+                 proxy_remote: false,
+                 common_map: %{key: "value"},
+                 proxy_opts: [
+                   redirect_on_failure: false,
+                   max_body_length: 1_048_576,
+                   http: [
+                     follow_redirect: true,
+                     pool: :upload
+                   ]
+                 ]
+               ]
+    end
+
+    test "common keyword" do
+      binary =
+        Config.transform([
+          %{"tuple" => [":level", ":warn"]},
+          %{"tuple" => [":meta", [":all"]]},
+          %{"tuple" => [":path", ""]},
+          %{"tuple" => [":val", nil]},
+          %{"tuple" => [":webhook_url", "https://hooks.slack.com/services/YOUR-KEY-HERE"]}
+        ])
 
       assert binary ==
                :erlang.term_to_binary(
                  level: :warn,
                  meta: [:all],
+                 path: "",
+                 val: nil,
                  webhook_url: "https://hooks.slack.com/services/YOUR-KEY-HERE"
                )
 
       assert Config.from_binary(binary) == [
                level: :warn,
                meta: [:all],
+               path: "",
+               val: nil,
                webhook_url: "https://hooks.slack.com/services/YOUR-KEY-HERE"
              ]
     end
 
-    test "complex map with sigil" do
+    test "complex keyword with sigil" do
       binary =
-        Config.transform(%{
+        Config.transform([
-          federated_timeline_removal: [],
+          %{"tuple" => [":federated_timeline_removal", []]},
-          reject: [~r/comp[lL][aA][iI][nN]er/],
+          %{"tuple" => [":reject", ["~r/comp[lL][aA][iI][nN]er/"]]},
-          replace: []
+          %{"tuple" => [":replace", []]}
-        })
+        ])
 
       assert binary ==
                :erlang.term_to_binary(
@@ -184,54 +373,68 @@ test "complex map with sigil" do
                [federated_timeline_removal: [], reject: [~r/comp[lL][aA][iI][nN]er/], replace: []]
     end
 
-    test "complex map with tuples with more than 2 values" do
+    test "complex keyword with tuples with more than 2 values" do
       binary =
-        Config.transform(%{
+        Config.transform([
-          "http" => %{
+          %{
-            "dispatch" => [
+            "tuple" => [
-              %{
+              ":http",
-                "tuple" => [
+              [
-                  ":_",
+                %{
-                  [
+                  "tuple" => [
-                    %{
+                    ":key1",
-                      "tuple" => [
+                    [
-                        "/api/v1/streaming",
+                      %{
-                        "Pleroma.Web.MastodonAPI.WebsocketHandler",
+                        "tuple" => [
-                        []
+                          ":_",
-                      ]
+                          [
-                    },
+                            %{
-                    %{
+                              "tuple" => [
-                      "tuple" => [
+                                "/api/v1/streaming",
-                        "/websocket",
+                                "Pleroma.Web.MastodonAPI.WebsocketHandler",
-                        "Phoenix.Endpoint.CowboyWebSocket",
+                                []
-                        %{
+                              ]
-                          "tuple" => [
+                            },
-                            "Phoenix.Transports.WebSocket",
+                            %{
-                            %{"tuple" => ["Pleroma.Web.Endpoint", "Pleroma.Web.UserSocket", []]}
+                              "tuple" => [
+                                "/websocket",
+                                "Phoenix.Endpoint.CowboyWebSocket",
+                                %{
+                                  "tuple" => [
+                                    "Phoenix.Transports.WebSocket",
+                                    %{
+                                      "tuple" => [
+                                        "Pleroma.Web.Endpoint",
+                                        "Pleroma.Web.UserSocket",
+                                        []
+                                      ]
+                                    }
+                                  ]
+                                }
+                              ]
+                            },
+                            %{
+                              "tuple" => [
+                                ":_",
+                                "Phoenix.Endpoint.Cowboy2Handler",
+                                %{"tuple" => ["Pleroma.Web.Endpoint", []]}
+                              ]
+                            }
                           ]
-                        }
+                        ]
-                      ]
+                      }
-                    },
+                    ]
-                    %{
-                      "tuple" => [
-                        ":_",
-                        "Phoenix.Endpoint.Cowboy2Handler",
-                        %{
-                          "tuple" => ["Pleroma.Web.Endpoint", []]
-                        }
-                      ]
-                    }
                   ]
-                ]
+                }
-              }
+              ]
             ]
           }
-        })
+        ])
 
       assert binary ==
                :erlang.term_to_binary(
                  http: [
-                   dispatch: [
+                   key1: [
                      _: [
                        {"/api/v1/streaming", Pleroma.Web.MastodonAPI.WebsocketHandler, []},
                        {"/websocket", Phoenix.Endpoint.CowboyWebSocket,
@@ -245,7 +448,7 @@ test "complex map with tuples with more than 2 values" do
 
       assert Config.from_binary(binary) == [
                http: [
-                 dispatch: [
+                 key1: [
                    {:_,
                     [
                       {"/api/v1/streaming", Pleroma.Web.MastodonAPI.WebsocketHandler, []},

test/web/common_api/common_api_test.exs

@@ -358,6 +358,20 @@ test "remove a reblog mute", %{muter: muter, muted: muted} do
     end
   end
 
+  describe "unfollow/2" do
+    test "also unsubscribes a user" do
+      [follower, followed] = insert_pair(:user)
+      {:ok, follower, followed, _} = CommonAPI.follow(follower, followed)
+      {:ok, followed} = User.subscribe(follower, followed)
+
+      assert User.subscribed_to?(follower, followed)
+
+      {:ok, follower} = CommonAPI.unfollow(follower, followed)
+
+      refute User.subscribed_to?(follower, followed)
+    end
+  end
+
   describe "accept_follow_request/2" do
     test "after acceptance, it sets all existing pending follow request states to 'accept'" do
       user = insert(:user, info: %{locked: true})

test/web/mastodon_api/mastodon_api_controller_test.exs

@@ -593,7 +593,7 @@ test "user avatar can be set", %{conn: conn} do
     conn =
       conn
       |> assign(:user, user)
-      |> patch("/api/v1/accounts/update_avatar", %{img: avatar_image})
+      |> patch("/api/v1/pleroma/accounts/update_avatar", %{img: avatar_image})
 
     user = refresh_record(user)
 
@@ -618,7 +618,7 @@ test "user avatar can be reset", %{conn: conn} do
     conn =
       conn
       |> assign(:user, user)
-      |> patch("/api/v1/accounts/update_avatar", %{img: ""})
+      |> patch("/api/v1/pleroma/accounts/update_avatar", %{img: ""})
 
     user = User.get_cached_by_id(user.id)
 
@@ -633,7 +633,7 @@ test "can set profile banner", %{conn: conn} do
     conn =
       conn
       |> assign(:user, user)
-      |> patch("/api/v1/accounts/update_banner", %{"banner" => @image})
+      |> patch("/api/v1/pleroma/accounts/update_banner", %{"banner" => @image})
 
     user = refresh_record(user)
     assert user.info.banner["type"] == "Image"
@@ -647,7 +647,7 @@ test "can reset profile banner", %{conn: conn} do
     conn =
       conn
       |> assign(:user, user)
-      |> patch("/api/v1/accounts/update_banner", %{"banner" => ""})
+      |> patch("/api/v1/pleroma/accounts/update_banner", %{"banner" => ""})
 
     user = refresh_record(user)
     assert user.info.banner == %{}
@@ -661,7 +661,7 @@ test "background image can be set", %{conn: conn} do
     conn =
       conn
       |> assign(:user, user)
-      |> patch("/api/v1/accounts/update_background", %{"img" => @image})
+      |> patch("/api/v1/pleroma/accounts/update_background", %{"img" => @image})
 
     user = refresh_record(user)
     assert user.info.background["type"] == "Image"
@@ -674,7 +674,7 @@ test "background image can be reset", %{conn: conn} do
     conn =
       conn
       |> assign(:user, user)
-      |> patch("/api/v1/accounts/update_background", %{"img" => ""})
+      |> patch("/api/v1/pleroma/accounts/update_background", %{"img" => ""})
 
     user = refresh_record(user)
     assert user.info.background == %{}
@@ -1274,6 +1274,71 @@ test "destroy multiple", %{conn: conn} do
       result = json_response(conn_res, 200)
       assert [%{"id" => ^notification4_id}, %{"id" => ^notification3_id}] = result
     end
+
+    test "doesn't see notifications after muting user with notifications", %{conn: conn} do
+      user = insert(:user)
+      user2 = insert(:user)
+
+      {:ok, _, _, _} = CommonAPI.follow(user, user2)
+      {:ok, _} = CommonAPI.post(user2, %{"status" => "hey @#{user.nickname}"})
+
+      conn = assign(conn, :user, user)
+
+      conn = get(conn, "/api/v1/notifications")
+
+      assert length(json_response(conn, 200)) == 1
+
+      {:ok, user} = User.mute(user, user2)
+
+      conn = assign(build_conn(), :user, user)
+      conn = get(conn, "/api/v1/notifications")
+
+      assert json_response(conn, 200) == []
+    end
+
+    test "see notifications after muting user without notifications", %{conn: conn} do
+      user = insert(:user)
+      user2 = insert(:user)
+
+      {:ok, _, _, _} = CommonAPI.follow(user, user2)
+      {:ok, _} = CommonAPI.post(user2, %{"status" => "hey @#{user.nickname}"})
+
+      conn = assign(conn, :user, user)
+
+      conn = get(conn, "/api/v1/notifications")
+
+      assert length(json_response(conn, 200)) == 1
+
+      {:ok, user} = User.mute(user, user2, false)
+
+      conn = assign(build_conn(), :user, user)
+      conn = get(conn, "/api/v1/notifications")
+
+      assert length(json_response(conn, 200)) == 1
+    end
+
+    test "see notifications after muting user with notifications and with_muted parameter", %{
+      conn: conn
+    } do
+      user = insert(:user)
+      user2 = insert(:user)
+
+      {:ok, _, _, _} = CommonAPI.follow(user, user2)
+      {:ok, _} = CommonAPI.post(user2, %{"status" => "hey @#{user.nickname}"})
+
+      conn = assign(conn, :user, user)
+
+      conn = get(conn, "/api/v1/notifications")
+
+      assert length(json_response(conn, 200)) == 1
+
+      {:ok, user} = User.mute(user, user2)
+
+      conn = assign(build_conn(), :user, user)
+      conn = get(conn, "/api/v1/notifications", %{"with_muted" => "true"})
+
+      assert length(json_response(conn, 200)) == 1
+    end
   end
 
   describe "reblogging" do
@@ -2105,25 +2170,52 @@ test "following / unfollowing errors" do
     assert %{"error" => "Record not found"} = json_response(conn_res, 404)
   end
 
-  test "muting / unmuting a user", %{conn: conn} do
+  describe "mute/unmute" do
-    user = insert(:user)
+    test "with notifications", %{conn: conn} do
-    other_user = insert(:user)
+      user = insert(:user)
+      other_user = insert(:user)
 
-    conn =
+      conn =
-      conn
+        conn
-      |> assign(:user, user)
+        |> assign(:user, user)
-      |> post("/api/v1/accounts/#{other_user.id}/mute")
+        |> post("/api/v1/accounts/#{other_user.id}/mute")
 
-    assert %{"id" => _id, "muting" => true} = json_response(conn, 200)
+      response = json_response(conn, 200)
 
-    user = User.get_cached_by_id(user.id)
+      assert %{"id" => _id, "muting" => true, "muting_notifications" => true} = response
+      user = User.get_cached_by_id(user.id)
 
-    conn =
+      conn =
-      build_conn()
+        build_conn()
-      |> assign(:user, user)
+        |> assign(:user, user)
-      |> post("/api/v1/accounts/#{other_user.id}/unmute")
+        |> post("/api/v1/accounts/#{other_user.id}/unmute")
 
-    assert %{"id" => _id, "muting" => false} = json_response(conn, 200)
+      response = json_response(conn, 200)
+      assert %{"id" => _id, "muting" => false, "muting_notifications" => false} = response
+    end
+
+    test "without notifications", %{conn: conn} do
+      user = insert(:user)
+      other_user = insert(:user)
+
+      conn =
+        conn
+        |> assign(:user, user)
+        |> post("/api/v1/accounts/#{other_user.id}/mute", %{"notifications" => "false"})
+
+      response = json_response(conn, 200)
+
+      assert %{"id" => _id, "muting" => true, "muting_notifications" => false} = response
+      user = User.get_cached_by_id(user.id)
+
+      conn =
+        build_conn()
+        |> assign(:user, user)
+        |> post("/api/v1/accounts/#{other_user.id}/unmute")
+
+      response = json_response(conn, 200)
+      assert %{"id" => _id, "muting" => false, "muting_notifications" => false} = response
+    end
   end
 
   test "subscribing / unsubscribing to a user", %{conn: conn} do

test/web/mastodon_api/search_controller_test.exs

@@ -22,7 +22,7 @@ defmodule Pleroma.Web.MastodonAPI.SearchControllerTest do
     test "it returns empty result if user or status search return undefined error", %{conn: conn} do
       with_mocks [
         {Pleroma.User, [], [search: fn _q, _o -> raise "Oops" end]},
-        {Pleroma.Activity, [], [search: fn _u, _q -> raise "Oops" end]}
+        {Pleroma.Activity, [], [search: fn _u, _q, _o -> raise "Oops" end]}
       ] do
         conn = get(conn, "/api/v2/search", %{"q" => "2hu"})
 
@@ -51,7 +51,6 @@ test "search", %{conn: conn} do
       conn = get(conn, "/api/v2/search", %{"q" => "2hu #private"})
 
       assert results = json_response(conn, 200)
-      # IO.inspect results
 
       [account | _] = results["accounts"]
       assert account["id"] == to_string(user_three.id)
@@ -98,7 +97,7 @@ test "account search", %{conn: conn} do
     test "it returns empty result if user or status search return undefined error", %{conn: conn} do
       with_mocks [
         {Pleroma.User, [], [search: fn _q, _o -> raise "Oops" end]},
-        {Pleroma.Activity, [], [search: fn _u, _q -> raise "Oops" end]}
+        {Pleroma.Activity, [], [search: fn _u, _q, _o -> raise "Oops" end]}
       ] do
         conn =
           conn
@@ -195,5 +194,74 @@ test "search doesn't fetch remote accounts if resolve is false", %{conn: conn} d
       assert results = json_response(conn, 200)
       assert [] == results["accounts"]
     end
+
+    test "search with limit and offset", %{conn: conn} do
+      user = insert(:user)
+      _user_two = insert(:user, %{nickname: "shp@shitposter.club"})
+      _user_three = insert(:user, %{nickname: "shp@heldscal.la", name: "I love 2hu"})
+
+      {:ok, _activity1} = CommonAPI.post(user, %{"status" => "This is about 2hu"})
+      {:ok, _activity2} = CommonAPI.post(user, %{"status" => "This is also about 2hu"})
+
+      result =
+        conn
+        |> get("/api/v1/search", %{"q" => "2hu", "limit" => 1})
+
+      assert results = json_response(result, 200)
+      assert [%{"id" => activity_id1}] = results["statuses"]
+      assert [_] = results["accounts"]
+
+      results =
+        conn
+        |> get("/api/v1/search", %{"q" => "2hu", "limit" => 1, "offset" => 1})
+        |> json_response(200)
+
+      assert [%{"id" => activity_id2}] = results["statuses"]
+      assert [] = results["accounts"]
+
+      assert activity_id1 != activity_id2
+    end
+
+    test "search returns results only for the given type", %{conn: conn} do
+      user = insert(:user)
+      _user_two = insert(:user, %{nickname: "shp@heldscal.la", name: "I love 2hu"})
+
+      {:ok, _activity} = CommonAPI.post(user, %{"status" => "This is about 2hu"})
+
+      assert %{"statuses" => [_activity], "accounts" => [], "hashtags" => []} =
+               conn
+               |> get("/api/v1/search", %{"q" => "2hu", "type" => "statuses"})
+               |> json_response(200)
+
+      assert %{"statuses" => [], "accounts" => [_user_two], "hashtags" => []} =
+               conn
+               |> get("/api/v1/search", %{"q" => "2hu", "type" => "accounts"})
+               |> json_response(200)
+    end
+
+    test "search uses account_id to filter statuses by the author", %{conn: conn} do
+      user = insert(:user, %{nickname: "shp@shitposter.club"})
+      user_two = insert(:user, %{nickname: "shp@heldscal.la", name: "I love 2hu"})
+
+      {:ok, activity1} = CommonAPI.post(user, %{"status" => "This is about 2hu"})
+      {:ok, activity2} = CommonAPI.post(user_two, %{"status" => "This is also about 2hu"})
+
+      results =
+        conn
+        |> get("/api/v1/search", %{"q" => "2hu", "account_id" => user.id})
+        |> json_response(200)
+
+      assert [%{"id" => activity_id1}] = results["statuses"]
+      assert activity_id1 == activity1.id
+      assert [_] = results["accounts"]
+
+      results =
+        conn
+        |> get("/api/v1/search", %{"q" => "2hu", "account_id" => user_two.id})
+        |> json_response(200)
+
+      assert [%{"id" => activity_id2}] = results["statuses"]
+      assert activity_id2 == activity2.id
+    end
   end
 end

test/web/media_proxy/media_proxy_controller_test.exs

@@ -0,0 +1,73 @@
+# Pleroma: A lightweight social networking server
+# Copyright © 2017-2018 Pleroma Authors <https://pleroma.social/>
+# SPDX-License-Identifier: AGPL-3.0-only
+
+defmodule Pleroma.Web.MediaProxy.MediaProxyControllerTest do
+  use Pleroma.Web.ConnCase
+  import Mock
+  alias Pleroma.Config
+
+  setup do
+    media_proxy_config = Config.get([:media_proxy]) || []
+    on_exit(fn -> Config.put([:media_proxy], media_proxy_config) end)
+    :ok
+  end
+
+  test "it returns 404 when MediaProxy disabled", %{conn: conn} do
+    Config.put([:media_proxy, :enabled], false)
+
+    assert %Plug.Conn{
+             status: 404,
+             resp_body: "Not Found"
+           } = get(conn, "/proxy/hhgfh/eeeee")
+
+    assert %Plug.Conn{
+             status: 404,
+             resp_body: "Not Found"
+           } = get(conn, "/proxy/hhgfh/eeee/fff")
+  end
+
+  test "it returns 403 when signature invalidated", %{conn: conn} do
+    Config.put([:media_proxy, :enabled], true)
+    Config.put([Pleroma.Web.Endpoint, :secret_key_base], "00000000000")
+    path = URI.parse(Pleroma.Web.MediaProxy.encode_url("https://google.fn")).path
+    Config.put([Pleroma.Web.Endpoint, :secret_key_base], "000")
+
+    assert %Plug.Conn{
+             status: 403,
+             resp_body: "Forbidden"
+           } = get(conn, path)
+
+    assert %Plug.Conn{
+             status: 403,
+             resp_body: "Forbidden"
+           } = get(conn, "/proxy/hhgfh/eeee")
+
+    assert %Plug.Conn{
+             status: 403,
+             resp_body: "Forbidden"
+           } = get(conn, "/proxy/hhgfh/eeee/fff")
+  end
+
+  test "redirects on valid url when filename invalidated", %{conn: conn} do
+    Config.put([:media_proxy, :enabled], true)
+    Config.put([Pleroma.Web.Endpoint, :secret_key_base], "00000000000")
+    url = Pleroma.Web.MediaProxy.encode_url("https://google.fn/test.png")
+    invalid_url = String.replace(url, "test.png", "test-file.png")
+    response = get(conn, invalid_url)
+    html = "<html><body>You are being <a href=\"#{url}\">redirected</a>.</body></html>"
+    assert response.status == 302
+    assert response.resp_body == html
+  end
+
+  test "it performs ReverseProxy.call when signature valid", %{conn: conn} do
+    Config.put([:media_proxy, :enabled], true)
+    Config.put([Pleroma.Web.Endpoint, :secret_key_base], "00000000000")
+    url = Pleroma.Web.MediaProxy.encode_url("https://google.fn/test.png")
+
+    with_mock Pleroma.ReverseProxy,
+      call: fn _conn, _url, _opts -> %Plug.Conn{status: :success} end do
+      assert %Plug.Conn{status: :success} = get(conn, url)
+    end
+  end
+end

test/web/media_proxy/media_proxy_test.exs

@@ -2,7 +2,7 @@
 # Copyright © 2017-2018 Pleroma Authors <https://pleroma.social/>
 # SPDX-License-Identifier: AGPL-3.0-only
 
-defmodule Pleroma.MediaProxyTest do
+defmodule Pleroma.Web.MediaProxyTest do
   use ExUnit.Case
   import Pleroma.Web.MediaProxy
   alias Pleroma.Web.MediaProxy.MediaProxyController
@@ -88,32 +88,30 @@ test "validates signature" do
       assert decode_url(sig, base64) == {:error, :invalid_signature}
     end
 
-    test "filename_matches matches url encoded paths" do
+    test "filename_matches preserves the encoded or decoded path" do
       assert MediaProxyController.filename_matches(
-               true,
+               %{"filename" => "/Hello world.jpg"},
-               "/Hello%20world.jpg",
-               "http://pleroma.social/Hello world.jpg"
-             ) == :ok
-
-      assert MediaProxyController.filename_matches(
-               true,
-               "/Hello%20world.jpg",
-               "http://pleroma.social/Hello%20world.jpg"
-             ) == :ok
-    end
-
-    test "filename_matches matches non-url encoded paths" do
-      assert MediaProxyController.filename_matches(
-               true,
-               "/Hello world.jpg",
-               "http://pleroma.social/Hello%20world.jpg"
-             ) == :ok
-
-      assert MediaProxyController.filename_matches(
-               true,
                "/Hello world.jpg",
                "http://pleroma.social/Hello world.jpg"
              ) == :ok
+
+      assert MediaProxyController.filename_matches(
+               %{"filename" => "/Hello%20world.jpg"},
+               "/Hello%20world.jpg",
+               "http://pleroma.social/Hello%20world.jpg"
+             ) == :ok
+
+      assert MediaProxyController.filename_matches(
+               %{"filename" => "/my%2Flong%2Furl%2F2019%2F07%2FS.jpg"},
+               "/my%2Flong%2Furl%2F2019%2F07%2FS.jpg",
+               "http://pleroma.social/my%2Flong%2Furl%2F2019%2F07%2FS.jpg"
+             ) == :ok
+
+      assert MediaProxyController.filename_matches(
+               %{"filename" => "/my%2Flong%2Furl%2F2019%2F07%2FS.jp"},
+               "/my%2Flong%2Furl%2F2019%2F07%2FS.jp",
+               "http://pleroma.social/my%2Flong%2Furl%2F2019%2F07%2FS.jpg"
+             ) == {:wrong_filename, "my%2Flong%2Furl%2F2019%2F07%2FS.jpg"}
     end
 
     test "uses the configured base_url" do

test/web/metadata/player_view_test.exs

@@ -0,0 +1,33 @@
+# Pleroma: A lightweight social networking server
+# Copyright © 2017-2019 Pleroma Authors <https://pleroma.social/>
+# SPDX-License-Identifier: AGPL-3.0-only
+
+defmodule Pleroma.Web.Metadata.PlayerViewTest do
+  use Pleroma.DataCase
+
+  alias Pleroma.Web.Metadata.PlayerView
+
+  test "it renders audio tag" do
+    res =
+      PlayerView.render(
+        "player.html",
+        %{"mediaType" => "audio", "href" => "test-href"}
+      )
+      |> Phoenix.HTML.safe_to_string()
+
+    assert res ==
+             "<audio controls><source src=\"test-href\" type=\"audio\">Your browser does not support audio playback.</audio>"
+  end
+
+  test "it renders videos tag" do
+    res =
+      PlayerView.render(
+        "player.html",
+        %{"mediaType" => "video", "href" => "test-href"}
+      )
+      |> Phoenix.HTML.safe_to_string()
+
+    assert res ==
+             "<video controls loop><source src=\"test-href\" type=\"video\">Your browser does not support video playback.</video>"
+  end
+end

test/web/metadata/twitter_card_test.exs

@@ -0,0 +1,123 @@
+# Pleroma: A lightweight social networking server
+# Copyright © 2017-2019 Pleroma Authors <https://pleroma.social/>
+# SPDX-License-Identifier: AGPL-3.0-only
+
+defmodule Pleroma.Web.Metadata.Providers.TwitterCardTest do
+  use Pleroma.DataCase
+  import Pleroma.Factory
+
+  alias Pleroma.User
+  alias Pleroma.Web.CommonAPI
+  alias Pleroma.Web.Endpoint
+  alias Pleroma.Web.Metadata.Providers.TwitterCard
+  alias Pleroma.Web.Metadata.Utils
+  alias Pleroma.Web.Router
+
+  test "it renders twitter card for user info" do
+    user = insert(:user, name: "Jimmy Hendriks", bio: "born 19 March 1994")
+    avatar_url = Utils.attachment_url(User.avatar_url(user))
+    res = TwitterCard.build_tags(%{user: user})
+
+    assert res == [
+             {:meta, [property: "twitter:title", content: Utils.user_name_string(user)], []},
+             {:meta, [property: "twitter:description", content: "born 19 March 1994"], []},
+             {:meta, [property: "twitter:image", content: avatar_url], []},
+             {:meta, [property: "twitter:card", content: "summary"], []}
+           ]
+  end
+
+  test "it does not render attachments if post is nsfw" do
+    Pleroma.Config.put([Pleroma.Web.Metadata, :unfurl_nsfw], false)
+    user = insert(:user, name: "Jimmy Hendriks", bio: "born 19 March 1994")
+    {:ok, activity} = CommonAPI.post(user, %{"status" => "HI"})
+
+    note =
+      insert(:note, %{
+        data: %{
+          "actor" => user.ap_id,
+          "tag" => [],
+          "id" => "https://pleroma.gov/objects/whatever",
+          "content" => "pleroma in a nutshell",
+          "sensitive" => true,
+          "attachment" => [
+            %{
+              "url" => [%{"mediaType" => "image/png", "href" => "https://pleroma.gov/tenshi.png"}]
+            },
+            %{
+              "url" => [
+                %{
+                  "mediaType" => "application/octet-stream",
+                  "href" => "https://pleroma.gov/fqa/badapple.sfc"
+                }
+              ]
+            },
+            %{
+              "url" => [
+                %{"mediaType" => "video/webm", "href" => "https://pleroma.gov/about/juche.webm"}
+              ]
+            }
+          ]
+        }
+      })
+
+    result = TwitterCard.build_tags(%{object: note, user: user, activity_id: activity.id})
+
+    assert [
+             {:meta, [property: "twitter:title", content: Utils.user_name_string(user)], []},
+             {:meta, [property: "twitter:description", content: "“pleroma in a nutshell”"], []},
+             {:meta, [property: "twitter:image", content: "http://localhost:4001/images/avi.png"],
+              []},
+             {:meta, [property: "twitter:card", content: "summary_large_image"], []}
+           ] == result
+  end
+
+  test "it renders supported types of attachments and skips unknown types" do
+    user = insert(:user, name: "Jimmy Hendriks", bio: "born 19 March 1994")
+    {:ok, activity} = CommonAPI.post(user, %{"status" => "HI"})
+
+    note =
+      insert(:note, %{
+        data: %{
+          "actor" => user.ap_id,
+          "tag" => [],
+          "id" => "https://pleroma.gov/objects/whatever",
+          "content" => "pleroma in a nutshell",
+          "attachment" => [
+            %{
+              "url" => [%{"mediaType" => "image/png", "href" => "https://pleroma.gov/tenshi.png"}]
+            },
+            %{
+              "url" => [
+                %{
+                  "mediaType" => "application/octet-stream",
+                  "href" => "https://pleroma.gov/fqa/badapple.sfc"
+                }
+              ]
+            },
+            %{
+              "url" => [
+                %{"mediaType" => "video/webm", "href" => "https://pleroma.gov/about/juche.webm"}
+              ]
+            }
+          ]
+        }
+      })
+
+    result = TwitterCard.build_tags(%{object: note, user: user, activity_id: activity.id})
+
+    assert [
+             {:meta, [property: "twitter:title", content: Utils.user_name_string(user)], []},
+             {:meta, [property: "twitter:description", content: "“pleroma in a nutshell”"], []},
+             {:meta, [property: "twitter:card", content: "summary_large_image"], []},
+             {:meta, [property: "twitter:player", content: "https://pleroma.gov/tenshi.png"], []},
+             {:meta, [property: "twitter:card", content: "player"], []},
+             {:meta,
+              [
+                property: "twitter:player",
+                content: Router.Helpers.o_status_url(Endpoint, :notice_player, activity.id)
+              ], []},
+             {:meta, [property: "twitter:player:width", content: "480"], []},
+             {:meta, [property: "twitter:player:height", content: "480"], []}
+           ] == result
+  end
+end

test/web/node_info_test.exs

@@ -83,4 +83,47 @@ test "it returns the safe_dm_mentions feature if enabled", %{conn: conn} do
 
     Pleroma.Config.put([:instance, :safe_dm_mentions], option)
   end
+
+  test "it shows MRF transparency data if enabled", %{conn: conn} do
+    option = Pleroma.Config.get([:instance, :mrf_transparency])
+    Pleroma.Config.put([:instance, :mrf_transparency], true)
+
+    simple_config = %{"reject" => ["example.com"]}
+    Pleroma.Config.put(:mrf_simple, simple_config)
+
+    response =
+      conn
+      |> get("/nodeinfo/2.1.json")
+      |> json_response(:ok)
+
+    assert response["metadata"]["federation"]["mrf_simple"] == simple_config
+
+    Pleroma.Config.put([:instance, :mrf_transparency], option)
+    Pleroma.Config.put(:mrf_simple, %{})
+  end
+
+  test "it performs exclusions from MRF transparency data if configured", %{conn: conn} do
+    option = Pleroma.Config.get([:instance, :mrf_transparency])
+    Pleroma.Config.put([:instance, :mrf_transparency], true)
+
+    exclusions = Pleroma.Config.get([:instance, :mrf_transparency_exclusions])
+    Pleroma.Config.put([:instance, :mrf_transparency_exclusions], ["other.site"])
+
+    simple_config = %{"reject" => ["example.com", "other.site"]}
+    expected_config = %{"reject" => ["example.com"]}
+
+    Pleroma.Config.put(:mrf_simple, simple_config)
+
+    response =
+      conn
+      |> get("/nodeinfo/2.1.json")
+      |> json_response(:ok)
+
+    assert response["metadata"]["federation"]["mrf_simple"] == expected_config
+    assert response["metadata"]["federation"]["exclusions"] == true
+
+    Pleroma.Config.put([:instance, :mrf_transparency], option)
+    Pleroma.Config.put([:instance, :mrf_transparency_exclusions], exclusions)
+    Pleroma.Config.put(:mrf_simple, %{})
+  end
 end

test/web/twitter_api/twitter_api_controller_test.exs

@@ -521,6 +521,38 @@ test "with credentials", %{conn: conn, user: current_user} do
                  for: current_user
                })
     end
+
+    test "muted user", %{conn: conn, user: current_user} do
+      other_user = insert(:user)
+
+      {:ok, current_user} = User.mute(current_user, other_user)
+
+      {:ok, _activity} =
+        ActivityBuilder.insert(%{"to" => [current_user.ap_id]}, %{user: other_user})
+
+      conn =
+        conn
+        |> with_credentials(current_user.nickname, "test")
+        |> get("/api/qvitter/statuses/notifications.json")
+
+      assert json_response(conn, 200) == []
+    end
+
+    test "muted user with with_muted parameter", %{conn: conn, user: current_user} do
+      other_user = insert(:user)
+
+      {:ok, current_user} = User.mute(current_user, other_user)
+
+      {:ok, _activity} =
+        ActivityBuilder.insert(%{"to" => [current_user.ap_id]}, %{user: other_user})
+
+      conn =
+        conn
+        |> with_credentials(current_user.nickname, "test")
+        |> get("/api/qvitter/statuses/notifications.json", %{"with_muted" => "true"})
+
+      assert length(json_response(conn, 200)) == 1
+    end
   end
 
   describe "POST /api/qvitter/statuses/notifications/read" do