diff --git a/.dockerignore b/.dockerignore new file mode 100644 index 000000000..c5ef89b86 --- /dev/null +++ b/.dockerignore @@ -0,0 +1,12 @@ +.* +*.md +AGPL-3 +CC-BY-SA-4.0 +COPYING +*file +elixir_buildpack.config +docs/ +test/ + +# Required to get version +!.git diff --git a/CHANGELOG.md b/CHANGELOG.md index f64506637..949577842 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -4,27 +4,33 @@ All notable changes to this project will be documented in this file. The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/). ## [Unreleased] -### Added -- Expiring/ephemeral activites. All activities can have expires_on value set, which controls when they should be deleted automatically. -- Mastodon API: in post_status, the expires_in parameter lets you set the number of minutes until an activity expires. It must be at least one hour. -- Mastodon API: all status JSON responses contain a `pleroma.expires_on` item which states when an activity will expire. The value is only shown to the user who created the activity. To everyone else it's empty. -- Configuration: `ActivityExpiration.enabled` controls whether expired activites will get deleted at the appropriate time. Enabled by default. +### Security +- OStatus: eliminate the possibility of a protocol downgrade attack. +- OStatus: prevent following locked accounts, bypassing the approval process. ### Changed - **Breaking:** Configuration: A setting to explicitly disable the mailer was added, defaulting to true, if you are using a mailer add `config :pleroma, Pleroma.Emails.Mailer, enabled: true` to your config +- **Breaking:** Configuration: `/media/` is now removed when `base_url` is configured, append `/media/` to your `base_url` config to keep the old behaviour if desired - Configuration: OpenGraph and TwitterCard providers enabled by default - Configuration: Filter.AnonymizeFilename added ability to retain file extension with custom text - Federation: Return 403 errors when trying to request pages from a user's follower/following collections if they have `hide_followers`/`hide_follows` set - NodeInfo: Return `skipThreadContainment` in `metadata` for the `skip_thread_containment` option +- NodeInfo: Return `mailerEnabled` in `metadata` - Mastodon API: Unsubscribe followers when they unfollow a user - AdminAPI: Add "godmode" while fetching user statuses (i.e. admin can see private statuses) +- Improve digest email template ### Fixed - Not being able to pin unlisted posts +- Objects being re-embedded to activities after being updated (e.g faved/reposted). Running 'mix pleroma.database prune_objects' again is advised. +- Favorites timeline doing database-intensive queries - Metadata rendering errors resulting in the entire page being inaccessible +- `federation_incoming_replies_max_depth` option being ignored in certain cases - Federation/MediaProxy not working with instances that have wrong certificate order - Mastodon API: Handling of search timeouts (`/api/v1/search` and `/api/v2/search`) - Mastodon API: Embedded relationships not being properly rendered in the Account entity of Status entity +- Mastodon API: follower/following counters not being nullified, when `hide_follows`/`hide_followers` is set +- Mastodon API: `muted` in the Status entity, using author's account to determine if the tread was muted - Mastodon API: Add `account_id`, `type`, `offset`, and `limit` to search API (`/api/v1/search` and `/api/v2/search`) - Mastodon API, streaming: Fix filtering of notifications based on blocks/mutes/thread mutes - ActivityPub C2S: follower/following collection pages being inaccessible even when authentifucated if `hide_followers`/ `hide_follows` was set @@ -32,11 +38,29 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/). - Rich Media: Parser failing when no TTL can be found by image TTL setters - Rich Media: The crawled URL is now spliced into the rich media data. - ActivityPub S2S: sharedInbox usage has been mostly aligned with the rules in the AP specification. +- ActivityPub S2S: remote user deletions now work the same as local user deletions. +- ActivityPub S2S: POST requests are now signed with `(request-target)` pseudo-header. +- Not being able to access the Mastodon FE login page on private instances +- Invalid SemVer version generation, when the current branch does not have commits ahead of tag/checked out on a tag +- Pleroma.Upload base_url was not automatically whitelisted by MediaProxy. Now your custom CDN or file hosting will be accessed directly as expected. +- Report email not being sent to admins when the reporter is a remote user +- MRF: ensure that subdomain_match calls are case-insensitive +- Reverse Proxy limiting `max_body_length` was incorrectly defined and only checked `Content-Length` headers which may not be sufficient in some circumstances +- MRF: fix use of unserializable keyword lists in describe() implementations +- ActivityPub: Deactivated user deletion ### Added +- Expiring/ephemeral activites. All activities can have expires_on value set, which controls when they should be deleted automatically. +- Mastodon API: in post_status, the expires_in parameter lets you set the number of minutes until an activity expires. It must be at least one hour. +- Mastodon API: all status JSON responses contain a `pleroma.expires_on` item which states when an activity will expire. The value is only shown to the user who created the activity. To everyone else it's empty. +- Configuration: `ActivityExpiration.enabled` controls whether expired activites will get deleted at the appropriate time. Enabled by default. +- Conversations: Add Pleroma-specific conversation endpoints and status posting extensions. Run the `bump_all_conversations` task again to create the necessary data. +- **Breaking:** MRF describe API, which adds support for exposing configuration information about MRF policies to NodeInfo. + Custom modules will need to be updated by adding, at the very least, `def describe, do: {:ok, %{}}` to the MRF policy modules. - MRF: Support for priming the mediaproxy cache (`Pleroma.Web.ActivityPub.MRF.MediaProxyWarmingPolicy`) - MRF: Support for excluding specific domains from Transparency. - MRF: Support for filtering posts based on who they mention (`Pleroma.Web.ActivityPub.MRF.MentionPolicy`) +- MRF: Support for filtering posts based on ActivityStreams vocabulary (`Pleroma.Web.ActivityPub.MRF.VocabularyPolicy`) - MRF (Simple Policy): Support for wildcard domains. - Support for wildcard domains in user domain blocks setting. - Configuration: `quarantined_instances` support wildcard domains. @@ -47,21 +71,29 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/). - Mastodon API: Add support for categories for custom emojis by reusing the group feature. - Mastodon API: Add support for muting/unmuting notifications - Mastodon API: Add support for the `blocked_by` attribute in the relationship API (`GET /api/v1/accounts/relationships`). +- Mastodon API: Add support for the `domain_blocking` attribute in the relationship API (`GET /api/v1/accounts/relationships`). - Mastodon API: Add `pleroma.deactivated` to the Account entity - Mastodon API: added `/auth/password` endpoint for password reset with rate limit. - Mastodon API: /api/v1/accounts/:id/statuses now supports nicknames or user id +- Mastodon API: Improve support for the user profile custom fields - Admin API: Return users' tags when querying reports - Admin API: Return avatar and display name when querying users - Admin API: Allow querying user by ID - Admin API: Added support for `tuples`. +- Admin API: Added endpoints to run mix tasks pleroma.config migrate_to_db & pleroma.config migrate_from_db - Added synchronization of following/followers counters for external users - Configuration: `enabled` option for `Pleroma.Emails.Mailer`, defaulting to `false`. - Configuration: Pleroma.Plugs.RateLimiter `bucket_name`, `params` options. +- Configuration: `user_bio_length` and `user_name_length` options. - Addressable lists - Twitter API: added rate limit for `/api/account/password_reset` endpoint. - ActivityPub: Add an internal service actor for fetching ActivityPub objects. - ActivityPub: Optional signing of ActivityPub object fetches. - Admin API: Endpoint for fetching latest user's statuses +- Pleroma API: Add `/api/v1/pleroma/accounts/confirmation_resend?email=` for resending account confirmation. +- Relays: Added a task to list relay subscriptions. +- Mix Tasks: `mix pleroma.database fix_likes_collections` +- Federation: Remove `likes` from objects. ### Changed - Configuration: Filter.AnonymizeFilename added ability to retain file extension with custom text @@ -69,6 +101,11 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/). - RichMedia: parsers and their order are configured in `rich_media` config. - RichMedia: add the rich media ttl based on image expiration time. +### Removed +- Emoji: Remove longfox emojis. +- Remove `Reply-To` header from report emails for admins. +- ActivityPub: The `accept_blocks` configuration setting. + ## [1.0.1] - 2019-07-14 ### Security - OStatus: fix an object spoofing vulnerability. @@ -79,6 +116,7 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/). - Rich media: Do not crawl private IP ranges ### Added +- Digest email for inactive users - Add a generic settings store for frontends / clients to use. - Explicit addressing option for posting. - Optional SSH access mode. (Needs `erlang-ssh` package on some distributions). @@ -105,6 +143,7 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/). - Configuration: `notify_email` option - Configuration: Media proxy `whitelist` option - Configuration: `report_uri` option +- Configuration: `email_notifications` option - Configuration: `limit_to_local_content` option - Pleroma API: User subscriptions - Pleroma API: Healthcheck endpoint diff --git a/Dockerfile b/Dockerfile new file mode 100644 index 000000000..268ec61dc --- /dev/null +++ b/Dockerfile @@ -0,0 +1,39 @@ +FROM rinpatch/elixir:1.9.0-rc.0-alpine as build + +COPY . . + +ENV MIX_ENV=prod + +RUN apk add git gcc g++ musl-dev make &&\ + echo "import Mix.Config" > config/prod.secret.exs &&\ + mix local.hex --force &&\ + mix local.rebar --force &&\ + mix deps.get --only prod &&\ + mkdir release &&\ + mix release --path release + +FROM alpine:latest + +ARG HOME=/opt/pleroma +ARG DATA=/var/lib/pleroma + +RUN echo "http://nl.alpinelinux.org/alpine/latest-stable/community" >> /etc/apk/repositories &&\ + apk update &&\ + apk add ncurses postgresql-client &&\ + adduser --system --shell /bin/false --home ${HOME} pleroma &&\ + mkdir -p ${DATA}/uploads &&\ + mkdir -p ${DATA}/static &&\ + chown -R pleroma ${DATA} &&\ + mkdir -p /etc/pleroma &&\ + chown -R pleroma /etc/pleroma + +USER pleroma + +COPY --from=build --chown=pleroma:0 /release ${HOME} + +COPY ./config/docker.exs /etc/pleroma/config.exs +COPY ./docker-entrypoint.sh ${HOME} + +EXPOSE 4000 + +ENTRYPOINT ["/opt/pleroma/docker-entrypoint.sh"] diff --git a/README.md b/README.md index 41d454a03..5aad34ccc 100644 --- a/README.md +++ b/README.md @@ -21,7 +21,7 @@ If you want to run your own server, feel free to contact us at @lain@pleroma.soy Currently Pleroma is not packaged by any OS/Distros, but feel free to reach out to us at [#pleroma-dev on freenode](https://webchat.freenode.net/?channels=%23pleroma-dev) or via matrix at for assistance. If you want to change default options in your Pleroma package, please **discuss it with us first**. ### Docker -While we don’t provide docker files, other people have written very good ones. Take a look at or . +While we don’t provide docker files, other people have written very good ones. Take a look at or . ### Dependencies diff --git a/config/config.exs b/config/config.exs index 2887353fb..e58454d68 100644 --- a/config/config.exs +++ b/config/config.exs @@ -253,6 +253,12 @@ skip_thread_containment: true, limit_to_local_content: :unauthenticated, dynamic_configuration: false, + user_bio_length: 5000, + user_name_length: 100, + max_account_fields: 10, + max_remote_account_fields: 20, + account_field_name_length: 512, + account_field_value_length: 512, external_user_synchronization: true config :pleroma, :markup, @@ -302,7 +308,6 @@ default_mascot: :pleroma_fox_tan config :pleroma, :activitypub, - accept_blocks: true, unfollow_blocked: true, outgoing_blocks: true, follow_handshake_timeout: 500, @@ -337,6 +342,10 @@ config :pleroma, :mrf_subchain, match_actor: %{} +config :pleroma, :mrf_vocabulary, + accept: [], + reject: [] + config :pleroma, :rich_media, enabled: true, ignore_hosts: [], @@ -508,6 +517,17 @@ config :pleroma, Pleroma.Emails.Mailer, adapter: Swoosh.Adapters.Sendmail, enabled: false +config :pleroma, Pleroma.Emails.UserEmail, + logo: nil, + styling: %{ + link_color: "#d8a070", + background_color: "#2C3645", + content_background_color: "#1B2635", + header_color: "#d8a070", + text_color: "#b9b9ba", + text_muted_color: "#b9b9ba" + } + config :prometheus, Pleroma.Web.Endpoint.MetricsExporter, path: "/api/pleroma/app_metrics" config :pleroma, Pleroma.ScheduledActivity, @@ -515,6 +535,14 @@ total_user_limit: 300, enabled: true +config :pleroma, :email_notifications, + digest: %{ + active: false, + schedule: "0 0 * * 0", + interval: 7, + inactivity_threshold: 7 + } + config :pleroma, :oauth2, token_expires_in: 600, issue_new_refresh_token: true, @@ -535,7 +563,9 @@ relation_id_action: {60_000, 2}, statuses_actions: {10_000, 15}, status_id_action: {60_000, 3}, - password_reset: {1_800_000, 5} + password_reset: {1_800_000, 5}, + account_confirmation_resend: {8_640_000, 5}, + ap_routes: {60_000, 15} config :pleroma, Pleroma.ActivityExpiration, enabled: true diff --git a/config/docker.exs b/config/docker.exs new file mode 100644 index 000000000..63ab4cdee --- /dev/null +++ b/config/docker.exs @@ -0,0 +1,68 @@ +import Config + +config :pleroma, Pleroma.Web.Endpoint, + url: [host: System.get_env("DOMAIN", "localhost"), scheme: "https", port: 443], + http: [ip: {0, 0, 0, 0}, port: 4000] + +config :pleroma, :instance, + name: System.get_env("INSTANCE_NAME", "Pleroma"), + email: System.get_env("ADMIN_EMAIL"), + notify_email: System.get_env("NOTIFY_EMAIL"), + limit: 5000, + registrations_open: false, + dynamic_configuration: true + +config :pleroma, Pleroma.Repo, + adapter: Ecto.Adapters.Postgres, + username: System.get_env("DB_USER", "pleroma"), + password: System.fetch_env!("DB_PASS"), + database: System.get_env("DB_NAME", "pleroma"), + hostname: System.get_env("DB_HOST", "db"), + pool_size: 10 + +# Configure web push notifications +config :web_push_encryption, :vapid_details, subject: "mailto:#{System.get_env("NOTIFY_EMAIL")}" + +config :pleroma, :database, rum_enabled: false +config :pleroma, :instance, static_dir: "/var/lib/pleroma/static" +config :pleroma, Pleroma.Uploaders.Local, uploads: "/var/lib/pleroma/uploads" + +# We can't store the secrets in this file, since this is baked into the docker image +if not File.exists?("/var/lib/pleroma/secret.exs") do + secret = :crypto.strong_rand_bytes(64) |> Base.encode64() |> binary_part(0, 64) + signing_salt = :crypto.strong_rand_bytes(8) |> Base.encode64() |> binary_part(0, 8) + {web_push_public_key, web_push_private_key} = :crypto.generate_key(:ecdh, :prime256v1) + + secret_file = + EEx.eval_string( + """ + import Config + + config :pleroma, Pleroma.Web.Endpoint, + secret_key_base: "<%= secret %>", + signing_salt: "<%= signing_salt %>" + + config :web_push_encryption, :vapid_details, + public_key: "<%= web_push_public_key %>", + private_key: "<%= web_push_private_key %>" + """, + secret: secret, + signing_salt: signing_salt, + web_push_public_key: Base.url_encode64(web_push_public_key, padding: false), + web_push_private_key: Base.url_encode64(web_push_private_key, padding: false) + ) + + File.write("/var/lib/pleroma/secret.exs", secret_file) +end + +import_config("/var/lib/pleroma/secret.exs") + +# For additional user config +if File.exists?("/var/lib/pleroma/config.exs"), + do: import_config("/var/lib/pleroma/config.exs"), + else: + File.write("/var/lib/pleroma/config.exs", """ + import Config + + # For additional configuration outside of environmental variables + """) diff --git a/config/emoji.txt b/config/emoji.txt index 79246f239..200768ad1 100644 --- a/config/emoji.txt +++ b/config/emoji.txt @@ -1,30 +1,2 @@ firefox, /emoji/Firefox.gif, Gif,Fun blank, /emoji/blank.png, Fun -f_00b, /emoji/f_00b.png -f_00b11b, /emoji/f_00b11b.png -f_00b33b, /emoji/f_00b33b.png -f_00h, /emoji/f_00h.png -f_00t, /emoji/f_00t.png -f_01b, /emoji/f_01b.png -f_03b, /emoji/f_03b.png -f_10b, /emoji/f_10b.png -f_11b, /emoji/f_11b.png -f_11b00b, /emoji/f_11b00b.png -f_11b22b, /emoji/f_11b22b.png -f_11h, /emoji/f_11h.png -f_11t, /emoji/f_11t.png -f_12b, /emoji/f_12b.png -f_21b, /emoji/f_21b.png -f_22b, /emoji/f_22b.png -f_22b11b, /emoji/f_22b11b.png -f_22b33b, /emoji/f_22b33b.png -f_22h, /emoji/f_22h.png -f_22t, /emoji/f_22t.png -f_23b, /emoji/f_23b.png -f_30b, /emoji/f_30b.png -f_32b, /emoji/f_32b.png -f_33b, /emoji/f_33b.png -f_33b00b, /emoji/f_33b00b.png -f_33b22b, /emoji/f_33b22b.png -f_33h, /emoji/f_33h.png -f_33t, /emoji/f_33t.png diff --git a/config/test.exs b/config/test.exs index 3f606aa81..567780987 100644 --- a/config/test.exs +++ b/config/test.exs @@ -29,7 +29,8 @@ email: "admin@example.com", notify_email: "noreply@example.com", skip_thread_containment: false, - federating: false + federating: false, + external_user_synchronization: false config :pleroma, :activitypub, sign_object_fetches: false @@ -70,7 +71,8 @@ config :pleroma, :rate_limit, search: [{1000, 30}, {1000, 30}], app_account_creation: {10_000, 5}, - password_reset: {1000, 30} + password_reset: {1000, 30}, + ap_routes: nil config :pleroma, :http_security, report_uri: "https://endpoint.com" @@ -80,6 +82,8 @@ config :pleroma, :database, rum_enabled: rum_enabled IO.puts("RUM enabled: #{rum_enabled}") +config :joken, default_signer: "yU8uHKq+yyAkZ11Hx//jcdacWc8yQ1bxAAGrplzB0Zwwjkp35v0RK9SO8WTPr6QZ" + config :pleroma, Pleroma.ReverseProxy.Client, Pleroma.ReverseProxy.ClientMock if File.exists?("./config/test.secret.exs") do diff --git a/docker-entrypoint.sh b/docker-entrypoint.sh new file mode 100755 index 000000000..f56f8c50a --- /dev/null +++ b/docker-entrypoint.sh @@ -0,0 +1,14 @@ +#!/bin/ash + +set -e + +echo "-- Waiting for database..." +while ! pg_isready -U ${DB_USER:-pleroma} -d postgres://${DB_HOST:-db}:5432/${DB_NAME:-pleroma} -t 1; do + sleep 1s +done + +echo "-- Running migrations..." +$HOME/bin/pleroma_ctl migrate + +echo "-- Starting!" +exec $HOME/bin/pleroma start diff --git a/docs/api/admin_api.md b/docs/api/admin_api.md index ca9303227..7ccb90836 100644 --- a/docs/api/admin_api.md +++ b/docs/api/admin_api.md @@ -575,6 +575,29 @@ Note: Available `:permission_group` is currently moderator and admin. 404 is ret - 404 Not Found `"Not found"` - On success: 200 OK `{}` + +## `/api/pleroma/admin/config/migrate_to_db` +### Run mix task pleroma.config migrate_to_db +Copy settings on key `:pleroma` to DB. +- Method `GET` +- Params: none +- Response: + +```json +{} +``` + +## `/api/pleroma/admin/config/migrate_from_db` +### Run mix task pleroma.config migrate_from_db +Copy all settings from DB to `config/prod.exported_from_db.secret.exs` with deletion from DB. +- Method `GET` +- Params: none +- Response: + +```json +{} +``` + ## `/api/pleroma/admin/config` ### List config settings List config settings only works with `:pleroma => :instance => :dynamic_configuration` setting to `true`. @@ -604,6 +627,9 @@ Tuples can be passed as `{"tuple": ["first_val", Pleroma.Module, []]}`. Keywords can be passed as lists with 2 child tuples, e.g. `[{"tuple": ["first_val", Pleroma.Module]}, {"tuple": ["second_val", true]}]`. +If value contains list of settings `[subkey: val1, subkey2: val2, subkey3: val3]`, it's possible to remove only subkeys instead of all settings passing `subkeys` parameter. E.g.: +{"group": "pleroma", "key": "some_key", "delete": "true", "subkeys": [":subkey", ":subkey3"]}. + Compile time settings (need instance reboot): - all settings by this keys: - `:hackney_pools` @@ -622,6 +648,7 @@ Compile time settings (need instance reboot): - `key` (string or string with leading `:` for atoms) - `value` (string, [], {} or {"tuple": []}) - `delete` = true (optional, if parameter must be deleted) + - `subkeys` [(string with leading `:` for atoms)] (optional, works only if `delete=true` parameter is passed, otherwise will be ignored) ] - Request (example): diff --git a/docs/api/differences_in_mastoapi_responses.md b/docs/api/differences_in_mastoapi_responses.md index 168a13f4e..197c465d8 100644 --- a/docs/api/differences_in_mastoapi_responses.md +++ b/docs/api/differences_in_mastoapi_responses.md @@ -60,12 +60,19 @@ Has these additional fields under the `pleroma` object: - `show_role`: boolean, nullable, true when the user wants his role (e.g admin, moderator) to be shown - `no_rich_text` - boolean, nullable, true when html tags are stripped from all statuses requested from the API +## Conversations + +Has an additional field under the `pleroma` object: + +- `recipients`: The list of the recipients of this Conversation. These will be addressed when replying to this conversation. + ## Account Search Behavior has changed: - `/api/v1/accounts/search`: Does not require authentication + ## Notifications Has these additional fields under the `pleroma` object: @@ -81,6 +88,7 @@ Additional parameters can be added to the JSON body/Form data: - `to`: A list of nicknames (like `lain@soykaf.club` or `lain` on the local server) that will be used to determine who is going to be addressed by this post. Using this will disable the implicit addressing by mentioned names in the `status` body, only the people in the `to` list will be addressed. The normal rules for for post visibility are not affected by this and will still apply. - `visibility`: string, besides standard MastoAPI values (`direct`, `private`, `unlisted` or `public`) it can be used to address a List by setting it to `list:LIST_ID`. - `expires_on`: datetime (iso8601), sets when the posted activity should expire. When a posted activity expires it will be deleted from the server, and a delete request for it will be federated. +- `in_reply_to_conversation_id`: Will reply to a given conversation, addressing only the people who are part of the recipient set of that conversation. Sets the visibility to `direct`. ## PATCH `/api/v1/update_credentials` diff --git a/docs/api/pleroma_api.md b/docs/api/pleroma_api.md index d83ebd734..b134b31a8 100644 --- a/docs/api/pleroma_api.md +++ b/docs/api/pleroma_api.md @@ -245,6 +245,14 @@ See [Admin-API](Admin-API.md) - PATCH `/api/v1/pleroma/accounts/update_banner`: Set/clear user banner image - PATCH `/api/v1/pleroma/accounts/update_background`: Set/clear user background image +## `/api/v1/pleroma/accounts/confirmation_resend` +### Resend confirmation email +* Method `POST` +* Params: + * `email`: email of that needs to be verified +* Authentication: not required +* Response: 204 No Content + ## `/api/v1/pleroma/mascot` ### Gets user mascot image * Method `GET` @@ -311,3 +319,38 @@ See [Admin-API](Admin-API.md) "healthy": true # Instance state } ``` + +# Pleroma Conversations + +Pleroma Conversations have the same general structure that Mastodon Conversations have. The behavior differs in the following ways when using these endpoints: + +1. Pleroma Conversations never add or remove recipients, unless explicitly changed by the user. +2. Pleroma Conversations statuses can be requested by Conversation id. +3. Pleroma Conversations can be replied to. + +Conversations have the additional field "recipients" under the "pleroma" key. This holds a list of all the accounts that will receive a message in this conversation. + +The status posting endpoint takes an additional parameter, `in_reply_to_conversation_id`, which, when set, will set the visiblity to direct and address only the people who are the recipients of that Conversation. + + +## `GET /api/v1/pleroma/conversations/:id/statuses` +### Timeline for a given conversation +* Method `GET` +* Authentication: required +* Params: Like other timelines +* Response: JSON, statuses (200 - healthy, 503 unhealthy). + +## `GET /api/v1/pleroma/conversations/:id` +### The conversation with the given ID. +* Method `GET` +* Authentication: required +* Params: None +* Response: JSON, statuses (200 - healthy, 503 unhealthy). + +## `PATCH /api/v1/pleroma/conversations/:id` +### Update a conversation. Used to change the set of recipients. +* Method `PATCH` +* Authentication: required +* Params: + * `recipients`: A list of ids of users that should receive posts to this conversation. This will replace the current list of recipients, so submit the full list. The owner of owner of the conversation will always be part of the set of recipients, though. +* Response: JSON, statuses (200 - healthy, 503 unhealthy) diff --git a/docs/config.md b/docs/config.md index a20ed704f..414b54660 100644 --- a/docs/config.md +++ b/docs/config.md @@ -18,6 +18,7 @@ Note: `strip_exif` has been replaced by `Pleroma.Upload.Filter.Mogrify`. ## Pleroma.Uploaders.S3 * `bucket`: S3 bucket name +* `bucket_namespace`: S3 bucket namespace * `public_endpoint`: S3 endpoint that the user finally accesses(ex. "https://s3.dualstack.ap-northeast-1.amazonaws.com") * `truncated_namespace`: If you use S3 compatible service such as Digital Ocean Spaces or CDN, set folder name or "" etc. For example, when using CDN to S3 virtual host format, set "". @@ -25,7 +26,7 @@ At this time, write CNAME to CDN in public_endpoint. ## Pleroma.Upload.Filter.Mogrify -* `args`: List of actions for the `mogrify` command like `"strip"` or `["strip", "auto-orient", {"impode", "1"}]`. +* `args`: List of actions for the `mogrify` command like `"strip"` or `["strip", "auto-orient", {"implode", "1"}]`. ## Pleroma.Upload.Filter.Dedupe @@ -102,6 +103,7 @@ config :pleroma, Pleroma.Emails.Mailer, * `Pleroma.Web.ActivityPub.MRF.AntiLinkSpamPolicy`: Rejects posts from likely spambots by rejecting posts from new users that contain links. * `Pleroma.Web.ActivityPub.MRF.MediaProxyWarmingPolicy`: Crawls attachments using their MediaProxy URLs so that the MediaProxy cache is primed. * `Pleroma.Web.ActivityPub.MRF.MentionPolicy`: Drops posts mentioning configurable users. (see `:mrf_mention` section) + * `Pleroma.Web.ActivityPub.MRF.VocabularyPolicy`: Restricts activities to a configured set of vocabulary. (see `:mrf_vocabulary` section) * `public`: Makes the client API in authentificated mode-only except for user-profiles. Useful for disabling the Local Timeline and The Whole Known Network. * `quarantined_instances`: List of ActivityPub instances where private(DMs, followers-only) activities will not be send. * `managed_config`: Whenether the config for pleroma-fe is configured in this config or in ``static/config.json`` @@ -125,9 +127,15 @@ config :pleroma, Pleroma.Emails.Mailer, * `safe_dm_mentions`: If set to true, only mentions at the beginning of a post will be used to address people in direct messages. This is to prevent accidental mentioning of people when talking about them (e.g. "@friend hey i really don't like @enemy"). Default: `false`. * `healthcheck`: If set to true, system data will be shown on ``/api/pleroma/healthcheck``. * `remote_post_retention_days`: The default amount of days to retain remote posts when pruning the database. +* `user_bio_length`: A user bio maximum length (default: `5000`) +* `user_name_length`: A user name maximum length (default: `100`) * `skip_thread_containment`: Skip filter out broken threads. The default is `false`. * `limit_to_local_content`: Limit unauthenticated users to search for local statutes and users only. Possible values: `:unauthenticated`, `:all` and `false`. The default is `:unauthenticated`. * `dynamic_configuration`: Allow transferring configuration to DB with the subsequent customization from Admin api. +* `max_account_fields`: The maximum number of custom fields in the user profile (default: `10`) +* `max_remote_account_fields`: The maximum number of custom fields in the remote user profile (default: `20`) +* `account_field_name_length`: An account field name maximum length (default: `512`) +* `account_field_value_length`: An account field value maximum length (default: `512`) * `external_user_synchronization`: Enabling following/followers counters synchronization for external users. @@ -275,6 +283,10 @@ config :pleroma, :mrf_subchain, ## :mrf_mention * `actors`: A list of actors, for which to drop any posts mentioning. +## :mrf_vocabulary +* `accept`: A list of ActivityStreams terms to accept. If empty, all supported messages are accepted. +* `reject`: A list of ActivityStreams terms to reject. If empty, no messages are rejected. + ## :media_proxy * `enabled`: Enables proxying of remote media to the instance’s proxy * `base_url`: The base URL to access a user-uploaded file. Useful when you want to proxy the media files via another host/CDN fronts. @@ -328,7 +340,6 @@ config :pleroma, Pleroma.Web.Endpoint, This will make Pleroma listen on `127.0.0.1` port `8080` and generate urls starting with `https://example.com:2020` ## :activitypub -* ``accept_blocks``: Whether to accept incoming block activities from other instances * ``unfollow_blocked``: Whether blocks result in people getting unfollowed * ``outgoing_blocks``: Whether to federate blocks to other instances * ``deny_follow_blocked``: Whether to disallow following an account that has blocked the user in question @@ -540,6 +551,23 @@ Authentication / authorization settings. * `oauth_consumer_template`: OAuth consumer mode authentication form template. By default it's `consumer.html` which corresponds to `lib/pleroma/web/templates/o_auth/o_auth/consumer.html.eex`. * `oauth_consumer_strategies`: the list of enabled OAuth consumer strategies; by default it's set by `OAUTH_CONSUMER_STRATEGIES` environment variable. Each entry in this space-delimited string should be of format `` or `:` (e.g. `twitter` or `keycloak:ueberauth_keycloak_strategy` in case dependency is named differently than `ueberauth_`). +## :email_notifications + +Email notifications settings. + + - digest - emails of "what you've missed" for users who have been + inactive for a while. + - active: globally enable or disable digest emails + - schedule: When to send digest email, in [crontab format](https://en.wikipedia.org/wiki/Cron). + "0 0 * * 0" is the default, meaning "once a week at midnight on Sunday morning" + - interval: Minimum interval between digest emails to one user + - inactivity_threshold: Minimum user inactivity threshold + +## Pleroma.Emails.UserEmail + +- `:logo` - a path to a custom logo. Set it to `nil` to use the default Pleroma logo. +- `:styling` - a map with color settings for email templates. + ## OAuth consumer mode OAuth consumer mode allows sign in / sign up via external OAuth providers (e.g. Twitter, Facebook, Google, Microsoft, etc.). diff --git a/docs/config/howto_mediaproxy.md b/docs/config/howto_mediaproxy.md index ed70c3ed4..16c40c5db 100644 --- a/docs/config/howto_mediaproxy.md +++ b/docs/config/howto_mediaproxy.md @@ -1,8 +1,8 @@ # How to activate mediaproxy ## Explanation -Without the `mediaproxy` function, Pleroma don't store any remote content like pictures, video etc. locally. So every time you open Pleroma, the content is loaded from the source server, from where the post is coming. This can result in slowly loading content or/and increased bandwidth usage on the source server. -With the `mediaproxy` function you can use the cache ability of nginx, to cache these content, so user can access it faster, cause it's loaded from your server. +Without the `mediaproxy` function, Pleroma doesn't store any remote content like pictures, video etc. locally. So every time you open Pleroma, the content is loaded from the source server, from where the post is coming. This can result in slowly loading content or/and increased bandwidth usage on the source server. +With the `mediaproxy` function you can use nginx to cache this content, so users can access it faster, because it's loaded from your server. ## Activate it diff --git a/lib/mix/tasks/pleroma/benchmark.ex b/lib/mix/tasks/pleroma/benchmark.ex index 5222cce80..4cc634727 100644 --- a/lib/mix/tasks/pleroma/benchmark.ex +++ b/lib/mix/tasks/pleroma/benchmark.ex @@ -26,4 +26,48 @@ def run(["tag"]) do end }) end + + def run(["render_timeline", nickname]) do + start_pleroma() + user = Pleroma.User.get_by_nickname(nickname) + + activities = + %{} + |> Map.put("type", ["Create", "Announce"]) + |> Map.put("blocking_user", user) + |> Map.put("muting_user", user) + |> Map.put("user", user) + |> Map.put("limit", 80) + |> Pleroma.Web.ActivityPub.ActivityPub.fetch_public_activities() + |> Enum.reverse() + + inputs = %{ + "One activity" => Enum.take_random(activities, 1), + "Ten activities" => Enum.take_random(activities, 10), + "Twenty activities" => Enum.take_random(activities, 20), + "Forty activities" => Enum.take_random(activities, 40), + "Eighty activities" => Enum.take_random(activities, 80) + } + + Benchee.run( + %{ + "Parallel rendering" => fn activities -> + Pleroma.Web.MastodonAPI.StatusView.render("index.json", %{ + activities: activities, + for: user, + as: :activity + }) + end, + "Standart rendering" => fn activities -> + Pleroma.Web.MastodonAPI.StatusView.render("index.json", %{ + activities: activities, + for: user, + as: :activity, + parallel: false + }) + end + }, + inputs: inputs + ) + end end diff --git a/lib/mix/tasks/pleroma/config.ex b/lib/mix/tasks/pleroma/config.ex index a7d0fac5d..462940e7e 100644 --- a/lib/mix/tasks/pleroma/config.ex +++ b/lib/mix/tasks/pleroma/config.ex @@ -15,7 +15,7 @@ defmodule Mix.Tasks.Pleroma.Config do mix pleroma.config migrate_to_db - ## Transfers config from DB to file. + ## Transfers config from DB to file `config/env.exported_from_db.secret.exs` mix pleroma.config migrate_from_db ENV """ diff --git a/lib/mix/tasks/pleroma/database.ex b/lib/mix/tasks/pleroma/database.ex index e91fb31d1..bcc2052d6 100644 --- a/lib/mix/tasks/pleroma/database.ex +++ b/lib/mix/tasks/pleroma/database.ex @@ -8,6 +8,7 @@ defmodule Mix.Tasks.Pleroma.Database do alias Pleroma.Repo alias Pleroma.User require Logger + require Pleroma.Constants import Mix.Pleroma use Mix.Task @@ -35,6 +36,10 @@ defmodule Mix.Tasks.Pleroma.Database do ## Remove duplicated items from following and update followers count for all users mix pleroma.database update_users_following_followers_counts + + ## Fix the pre-existing "likes" collections for all objects + + mix pleroma.database fix_likes_collections """ def run(["remove_embedded_objects" | args]) do {options, [], []} = @@ -99,10 +104,15 @@ def run(["prune_objects" | args]) do NaiveDateTime.utc_now() |> NaiveDateTime.add(-(deadline * 86_400)) - public = "https://www.w3.org/ns/activitystreams#Public" - from(o in Object, - where: fragment("?->'to' \\? ? OR ?->'cc' \\? ?", o.data, ^public, o.data, ^public), + where: + fragment( + "?->'to' \\? ? OR ?->'cc' \\? ?", + o.data, + ^Pleroma.Constants.as_public(), + o.data, + ^Pleroma.Constants.as_public() + ), where: o.inserted_at < ^time_deadline, where: fragment("split_part(?->>'actor', '/', 3) != ?", o.data, ^Pleroma.Web.Endpoint.host()) @@ -119,4 +129,36 @@ def run(["prune_objects" | args]) do ) end end + + def run(["fix_likes_collections"]) do + import Ecto.Query + + start_pleroma() + + from(object in Object, + where: fragment("(?)->>'likes' is not null", object.data), + select: %{id: object.id, likes: fragment("(?)->>'likes'", object.data)} + ) + |> Pleroma.RepoStreamer.chunk_stream(100) + |> Stream.each(fn objects -> + ids = + objects + |> Enum.filter(fn object -> object.likes |> Jason.decode!() |> is_map() end) + |> Enum.map(& &1.id) + + Object + |> where([object], object.id in ^ids) + |> update([object], + set: [ + data: + fragment( + "jsonb_set(?, '{likes}', '[]'::jsonb, true)", + object.data + ) + ] + ) + |> Repo.update_all([], timeout: :infinity) + end) + |> Stream.run() + end end diff --git a/lib/mix/tasks/pleroma/digest.ex b/lib/mix/tasks/pleroma/digest.ex new file mode 100644 index 000000000..430116a50 --- /dev/null +++ b/lib/mix/tasks/pleroma/digest.ex @@ -0,0 +1,41 @@ +defmodule Mix.Tasks.Pleroma.Digest do + use Mix.Task + + @shortdoc "Manages digest emails" + @moduledoc """ + Manages digest emails + + ## Send digest email since given date (user registration date by default) + ignoring user activity status. + + ``mix pleroma.digest test `` + + Example: ``mix pleroma.digest test donaldtheduck 2019-05-20`` + """ + def run(["test", nickname | opts]) do + Mix.Pleroma.start_pleroma() + + user = Pleroma.User.get_by_nickname(nickname) + + last_digest_emailed_at = + with [date] <- opts, + {:ok, datetime} <- Timex.parse(date, "{YYYY}-{0M}-{0D}") do + datetime + else + _ -> user.inserted_at + end + + patched_user = %{user | last_digest_emailed_at: last_digest_emailed_at} + + with %Swoosh.Email{} = email <- Pleroma.Emails.UserEmail.digest_email(patched_user) do + {:ok, _} = Pleroma.Emails.Mailer.deliver(email) + + Mix.shell().info("Digest email have been sent to #{nickname} (#{user.email})") + else + _ -> + Mix.shell().info( + "Cound't find any mentions for #{nickname} since #{last_digest_emailed_at}" + ) + end + end +end diff --git a/lib/mix/tasks/pleroma/instance.ex b/lib/mix/tasks/pleroma/instance.ex index 9080adb52..b9b1991c2 100644 --- a/lib/mix/tasks/pleroma/instance.ex +++ b/lib/mix/tasks/pleroma/instance.ex @@ -183,6 +183,7 @@ def run(["gen" | rest]) do ) secret = :crypto.strong_rand_bytes(64) |> Base.encode64() |> binary_part(0, 64) + jwt_secret = :crypto.strong_rand_bytes(64) |> Base.encode64() |> binary_part(0, 64) signing_salt = :crypto.strong_rand_bytes(8) |> Base.encode64() |> binary_part(0, 8) {web_push_public_key, web_push_private_key} = :crypto.generate_key(:ecdh, :prime256v1) template_dir = Application.app_dir(:pleroma, "priv") <> "/templates" @@ -200,6 +201,7 @@ def run(["gen" | rest]) do dbuser: dbuser, dbpass: dbpass, secret: secret, + jwt_secret: jwt_secret, signing_salt: signing_salt, web_push_public_key: Base.url_encode64(web_push_public_key, padding: false), web_push_private_key: Base.url_encode64(web_push_private_key, padding: false), diff --git a/lib/mix/tasks/pleroma/relay.ex b/lib/mix/tasks/pleroma/relay.ex index 83ed0ed02..a738fae75 100644 --- a/lib/mix/tasks/pleroma/relay.ex +++ b/lib/mix/tasks/pleroma/relay.ex @@ -5,6 +5,7 @@ defmodule Mix.Tasks.Pleroma.Relay do use Mix.Task import Mix.Pleroma + alias Pleroma.User alias Pleroma.Web.ActivityPub.Relay @shortdoc "Manages remote relays" @@ -22,6 +23,10 @@ defmodule Mix.Tasks.Pleroma.Relay do ``mix pleroma.relay unfollow `` Example: ``mix pleroma.relay unfollow https://example.org/relay`` + + ## List relay subscriptions + + ``mix pleroma.relay list`` """ def run(["follow", target]) do start_pleroma() @@ -44,4 +49,17 @@ def run(["unfollow", target]) do {:error, e} -> shell_error("Error while following #{target}: #{inspect(e)}") end end + + def run(["list"]) do + start_pleroma() + + with %User{following: following} = _user <- Relay.get_actor() do + following + |> Enum.map(fn entry -> URI.parse(entry).host end) + |> Enum.uniq() + |> Enum.each(&shell_info(&1)) + else + e -> shell_error("Error while fetching relay subscription list: #{inspect(e)}") + end + end end diff --git a/lib/mix/tasks/pleroma/user.ex b/lib/mix/tasks/pleroma/user.ex index c9b84b8f9..a3f8bc945 100644 --- a/lib/mix/tasks/pleroma/user.ex +++ b/lib/mix/tasks/pleroma/user.ex @@ -31,8 +31,8 @@ defmodule Mix.Tasks.Pleroma.User do mix pleroma.user invite [OPTION...] Options: - - `--expires_at DATE` - last day on which token is active (e.g. "2019-04-05") - - `--max_use NUMBER` - maximum numbers of token uses + - `--expires-at DATE` - last day on which token is active (e.g. "2019-04-05") + - `--max-use NUMBER` - maximum numbers of token uses ## List generated invites diff --git a/lib/pleroma/activity.ex b/lib/pleroma/activity.ex index be4850560..2d4e9da0c 100644 --- a/lib/pleroma/activity.ex +++ b/lib/pleroma/activity.ex @@ -99,6 +99,7 @@ def with_set_thread_muted_field(query, %User{} = user) do from([a] in query, left_join: tm in ThreadMute, on: tm.user_id == ^user.id and tm.context == fragment("?->>'context'", a.data), + as: :thread_mute, select: %Activity{a | thread_muted?: not is_nil(tm.id)} ) end @@ -227,6 +228,29 @@ def get_create_by_object_ap_id(ap_id) when is_binary(ap_id) do def get_create_by_object_ap_id(_), do: nil + def create_by_object_ap_id_with_object(ap_ids) when is_list(ap_ids) do + from( + activity in Activity, + where: + fragment( + "coalesce((?)->'object'->>'id', (?)->>'object') = ANY(?)", + activity.data, + activity.data, + ^ap_ids + ), + where: fragment("(?)->>'type' = 'Create'", activity.data), + inner_join: o in Object, + on: + fragment( + "(?->>'id') = COALESCE(?->'object'->>'id', ?->>'object')", + o.data, + activity.data, + activity.data + ), + preload: [object: o] + ) + end + def create_by_object_ap_id_with_object(ap_id) when is_binary(ap_id) do from( activity in Activity, @@ -266,8 +290,8 @@ defp get_in_reply_to_activity_from_object(%Object{data: %{"inReplyTo" => ap_id}} defp get_in_reply_to_activity_from_object(_), do: nil - def get_in_reply_to_activity(%Activity{data: %{"object" => object}}) do - get_in_reply_to_activity_from_object(Object.normalize(object)) + def get_in_reply_to_activity(%Activity{} = activity) do + get_in_reply_to_activity_from_object(Object.normalize(activity)) end def normalize(obj) when is_map(obj), do: get_by_ap_id_with_object(obj["id"]) diff --git a/lib/pleroma/activity/search.ex b/lib/pleroma/activity/search.ex index 0cc3770a7..f847ac238 100644 --- a/lib/pleroma/activity/search.ex +++ b/lib/pleroma/activity/search.ex @@ -9,6 +9,8 @@ defmodule Pleroma.Activity.Search do alias Pleroma.User alias Pleroma.Web.ActivityPub.Visibility + require Pleroma.Constants + import Ecto.Query def search(user, search_query, options \\ []) do @@ -39,7 +41,7 @@ def maybe_restrict_author(query, _), do: query defp restrict_public(q) do from([a, o] in q, where: fragment("?->>'type' = 'Create'", a.data), - where: "https://www.w3.org/ns/activitystreams#Public" in a.recipients + where: ^Pleroma.Constants.as_public() in a.recipients ) end diff --git a/lib/pleroma/application.ex b/lib/pleroma/application.ex index 42e4a1dfa..1e4de272c 100644 --- a/lib/pleroma/application.ex +++ b/lib/pleroma/application.ex @@ -3,11 +3,14 @@ # SPDX-License-Identifier: AGPL-3.0-only defmodule Pleroma.Application do + import Cachex.Spec use Application @name Mix.Project.config()[:name] @version Mix.Project.config()[:version] @repository Mix.Project.config()[:source_url] + @env Mix.env() + def name, do: @name def version, do: @version def named_version, do: @name <> " " <> @version @@ -21,120 +24,25 @@ def user_agent do # See http://elixir-lang.org/docs/stable/elixir/Application.html # for more information on OTP Applications def start(_type, _args) do - import Cachex.Spec - Pleroma.Config.DeprecationWarnings.warn() setup_instrumenters() # Define workers and child supervisors to be supervised children = [ - # Start the Ecto repository - %{id: Pleroma.Repo, start: {Pleroma.Repo, :start_link, []}, type: :supervisor}, - %{id: Pleroma.Config.TransferTask, start: {Pleroma.Config.TransferTask, :start_link, []}}, - %{id: Pleroma.Emoji, start: {Pleroma.Emoji, :start_link, []}}, - %{id: Pleroma.Captcha, start: {Pleroma.Captcha, :start_link, []}}, - %{ - id: :cachex_used_captcha_cache, - start: - {Cachex, :start_link, - [ - :used_captcha_cache, - [ - ttl_interval: - :timer.seconds(Pleroma.Config.get!([Pleroma.Captcha, :seconds_valid])) - ] - ]} - }, - %{ - id: :cachex_user, - start: - {Cachex, :start_link, - [ - :user_cache, - [ - default_ttl: 25_000, - ttl_interval: 1000, - limit: 2500 - ] - ]} - }, - %{ - id: :cachex_object, - start: - {Cachex, :start_link, - [ - :object_cache, - [ - default_ttl: 25_000, - ttl_interval: 1000, - limit: 2500 - ] - ]} - }, - %{ - id: :cachex_rich_media, - start: - {Cachex, :start_link, - [ - :rich_media_cache, - [ - default_ttl: :timer.minutes(120), - limit: 5000 - ] - ]} - }, - %{ - id: :cachex_scrubber, - start: - {Cachex, :start_link, - [ - :scrubber_cache, - [ - limit: 2500 - ] - ]} - }, - %{ - id: :cachex_idem, - start: - {Cachex, :start_link, - [ - :idempotency_cache, - [ - expiration: - expiration( - default: :timer.seconds(6 * 60 * 60), - interval: :timer.seconds(60) - ), - limit: 2500 - ] - ]} - }, - %{id: Pleroma.FlakeId, start: {Pleroma.FlakeId, :start_link, []}}, - %{ - id: Pleroma.ScheduledActivityWorker, - start: {Pleroma.ScheduledActivityWorker, :start_link, []} - }, - %{ - id: Pleroma.ActivityExpirationWorker, - start: {Pleroma.ActivityExpirationWorker, :start_link, []} - } + Pleroma.Repo, + Pleroma.Config.TransferTask, + Pleroma.Emoji, + Pleroma.Captcha, + Pleroma.FlakeId, + Pleroma.ScheduledActivityWorker, + Pleroma.ActiviyExpirationWorker ] ++ + cachex_children() ++ hackney_pool_children() ++ [ - %{ - id: Pleroma.Web.Federator.RetryQueue, - start: {Pleroma.Web.Federator.RetryQueue, :start_link, []} - }, - %{ - id: Pleroma.Web.OAuth.Token.CleanWorker, - start: {Pleroma.Web.OAuth.Token.CleanWorker, :start_link, []} - }, - %{ - id: Pleroma.Stats, - start: {Pleroma.Stats, :start_link, []} - }, + Pleroma.Web.Federator.RetryQueue, + Pleroma.Stats, %{ id: :web_push_init, start: {Task, :start_link, [&Pleroma.Web.Push.init/0]}, @@ -151,22 +59,20 @@ def start(_type, _args) do restart: :temporary } ] ++ - streamer_child() ++ - chat_child() ++ + oauth_cleanup_child(oauth_cleanup_enabled?()) ++ + streamer_child(@env) ++ + chat_child(@env, chat_enabled?()) ++ [ - # Start the endpoint when the application starts - %{ - id: Pleroma.Web.Endpoint, - start: {Pleroma.Web.Endpoint, :start_link, []}, - type: :supervisor - }, - %{id: Pleroma.Gopher.Server, start: {Pleroma.Gopher.Server, :start_link, []}} + Pleroma.Web.Endpoint, + Pleroma.Gopher.Server ] # See http://elixir-lang.org/docs/stable/elixir/Supervisor.html # for other strategies and supported options opts = [strategy: :one_for_one, name: Pleroma.Supervisor] - Supervisor.start_link(children, opts) + result = Supervisor.start_link(children, opts) + :ok = after_supervisor_start() + result end defp setup_instrumenters do @@ -203,32 +109,71 @@ def enabled_hackney_pools do end end - if Pleroma.Config.get(:env) == :test do - defp streamer_child, do: [] - defp chat_child, do: [] - else - defp streamer_child do - [%{id: Pleroma.Web.Streamer, start: {Pleroma.Web.Streamer, :start_link, []}}] - end - - defp chat_child do - if Pleroma.Config.get([:chat, :enabled]) do - [ - %{ - id: Pleroma.Web.ChatChannel.ChatChannelState, - start: {Pleroma.Web.ChatChannel.ChatChannelState, :start_link, []} - } - ] - else - [] - end - end + defp cachex_children do + [ + build_cachex("used_captcha", ttl_interval: seconds_valid_interval()), + build_cachex("user", default_ttl: 25_000, ttl_interval: 1000, limit: 2500), + build_cachex("object", default_ttl: 25_000, ttl_interval: 1000, limit: 2500), + build_cachex("rich_media", default_ttl: :timer.minutes(120), limit: 5000), + build_cachex("scrubber", limit: 2500), + build_cachex("idempotency", expiration: idempotency_expiration(), limit: 2500) + ] end + defp idempotency_expiration, + do: expiration(default: :timer.seconds(6 * 60 * 60), interval: :timer.seconds(60)) + + defp seconds_valid_interval, + do: :timer.seconds(Pleroma.Config.get!([Pleroma.Captcha, :seconds_valid])) + + defp build_cachex(type, opts), + do: %{ + id: String.to_atom("cachex_" <> type), + start: {Cachex, :start_link, [String.to_atom(type <> "_cache"), opts]}, + type: :worker + } + + defp chat_enabled?, do: Pleroma.Config.get([:chat, :enabled]) + + defp oauth_cleanup_enabled?, + do: Pleroma.Config.get([:oauth2, :clean_expired_tokens], false) + + defp streamer_child(:test), do: [] + + defp streamer_child(_) do + [Pleroma.Web.Streamer] + end + + defp oauth_cleanup_child(true), + do: [Pleroma.Web.OAuth.Token.CleanWorker] + + defp oauth_cleanup_child(_), do: [] + + defp chat_child(:test, _), do: [] + + defp chat_child(_env, true) do + [Pleroma.Web.ChatChannel.ChatChannelState] + end + + defp chat_child(_, _), do: [] + defp hackney_pool_children do for pool <- enabled_hackney_pools() do options = Pleroma.Config.get([:hackney_pools, pool]) :hackney_pool.child_spec(pool, options) end end + + defp after_supervisor_start do + with digest_config <- Application.get_env(:pleroma, :email_notifications)[:digest], + true <- digest_config[:active] do + PleromaJobQueue.schedule( + digest_config[:schedule], + :digest_emails, + Pleroma.DigestEmailWorker + ) + end + + :ok + end end diff --git a/lib/pleroma/captcha/captcha.ex b/lib/pleroma/captcha/captcha.ex index a73b87251..c2765a5b8 100644 --- a/lib/pleroma/captcha/captcha.ex +++ b/lib/pleroma/captcha/captcha.ex @@ -12,7 +12,7 @@ defmodule Pleroma.Captcha do use GenServer @doc false - def start_link do + def start_link(_) do GenServer.start_link(__MODULE__, [], name: __MODULE__) end diff --git a/lib/pleroma/config/transfer_task.ex b/lib/pleroma/config/transfer_task.ex index 7799b2a78..3214c9951 100644 --- a/lib/pleroma/config/transfer_task.ex +++ b/lib/pleroma/config/transfer_task.ex @@ -6,7 +6,7 @@ defmodule Pleroma.Config.TransferTask do use Task alias Pleroma.Web.AdminAPI.Config - def start_link do + def start_link(_) do load_and_update_env() if Pleroma.Config.get(:env) == :test, do: Ecto.Adapters.SQL.Sandbox.checkin(Pleroma.Repo) :ignore diff --git a/lib/pleroma/constants.ex b/lib/pleroma/constants.ex new file mode 100644 index 000000000..ef1418543 --- /dev/null +++ b/lib/pleroma/constants.ex @@ -0,0 +1,9 @@ +# Pleroma: A lightweight social networking server +# Copyright © 2017-2019 Pleroma Authors +# SPDX-License-Identifier: AGPL-3.0-only + +defmodule Pleroma.Constants do + use Const + + const(as_public, do: "https://www.w3.org/ns/activitystreams#Public") +end diff --git a/lib/pleroma/conversation.ex b/lib/pleroma/conversation.ex index bc97b39ca..be5821ad7 100644 --- a/lib/pleroma/conversation.ex +++ b/lib/pleroma/conversation.ex @@ -4,6 +4,7 @@ defmodule Pleroma.Conversation do alias Pleroma.Conversation.Participation + alias Pleroma.Conversation.Participation.RecipientShip alias Pleroma.Repo alias Pleroma.User use Ecto.Schema @@ -39,6 +40,15 @@ def get_for_ap_id(ap_id) do Repo.get_by(__MODULE__, ap_id: ap_id) end + def maybe_create_recipientships(participation, activity) do + participation = Repo.preload(participation, :recipients) + + if participation.recipients |> Enum.empty?() do + recipients = User.get_all_by_ap_id(activity.recipients) + RecipientShip.create(recipients, participation) + end + end + @doc """ This will 1. Create a conversation if there isn't one already @@ -60,6 +70,7 @@ def create_or_bump_for(activity, opts \\ []) do {:ok, participation} = Participation.create_for_user_and_conversation(user, conversation, opts) + maybe_create_recipientships(participation, activity) participation end) diff --git a/lib/pleroma/conversation/participation.ex b/lib/pleroma/conversation/participation.ex index 5883e4183..ea5b9fe17 100644 --- a/lib/pleroma/conversation/participation.ex +++ b/lib/pleroma/conversation/participation.ex @@ -5,6 +5,7 @@ defmodule Pleroma.Conversation.Participation do use Ecto.Schema alias Pleroma.Conversation + alias Pleroma.Conversation.Participation.RecipientShip alias Pleroma.Repo alias Pleroma.User alias Pleroma.Web.ActivityPub.ActivityPub @@ -17,6 +18,9 @@ defmodule Pleroma.Conversation.Participation do field(:read, :boolean, default: false) field(:last_activity_id, Pleroma.FlakeId, virtual: true) + has_many(:recipient_ships, RecipientShip) + has_many(:recipients, through: [:recipient_ships, :user]) + timestamps() end @@ -65,6 +69,14 @@ def for_user(user, params \\ %{}) do |> Pleroma.Pagination.fetch_paginated(params) end + def for_user_and_conversation(user, conversation) do + from(p in __MODULE__, + where: p.user_id == ^user.id, + where: p.conversation_id == ^conversation.id + ) + |> Repo.one() + end + def for_user_with_last_activity_id(user, params \\ %{}) do for_user(user, params) |> Enum.map(fn participation -> @@ -81,4 +93,46 @@ def for_user_with_last_activity_id(user, params \\ %{}) do end) |> Enum.filter(& &1.last_activity_id) end + + def get(_, _ \\ []) + def get(nil, _), do: nil + + def get(id, params) do + query = + if preload = params[:preload] do + from(p in __MODULE__, + preload: ^preload + ) + else + __MODULE__ + end + + Repo.get(query, id) + end + + def set_recipients(participation, user_ids) do + user_ids = + [participation.user_id | user_ids] + |> Enum.uniq() + + Repo.transaction(fn -> + query = + from(r in RecipientShip, + where: r.participation_id == ^participation.id + ) + + Repo.delete_all(query) + + users = + from(u in User, + where: u.id in ^user_ids + ) + |> Repo.all() + + RecipientShip.create(users, participation) + :ok + end) + + {:ok, Repo.preload(participation, :recipients, force: true)} + end end diff --git a/lib/pleroma/conversation/participation_recipient_ship.ex b/lib/pleroma/conversation/participation_recipient_ship.ex new file mode 100644 index 000000000..932cbd04c --- /dev/null +++ b/lib/pleroma/conversation/participation_recipient_ship.ex @@ -0,0 +1,34 @@ +# Pleroma: A lightweight social networking server +# Copyright © 2017-2019 Pleroma Authors +# SPDX-License-Identifier: AGPL-3.0-only + +defmodule Pleroma.Conversation.Participation.RecipientShip do + use Ecto.Schema + + alias Pleroma.Conversation.Participation + alias Pleroma.Repo + alias Pleroma.User + + import Ecto.Changeset + + schema "conversation_participation_recipient_ships" do + belongs_to(:user, User, type: Pleroma.FlakeId) + belongs_to(:participation, Participation) + end + + def creation_cng(struct, params) do + struct + |> cast(params, [:user_id, :participation_id]) + |> validate_required([:user_id, :participation_id]) + end + + def create(%User{} = user, participation), do: create([user], participation) + + def create(users, participation) do + Enum.each(users, fn user -> + %__MODULE__{} + |> creation_cng(%{user_id: user.id, participation_id: participation.id}) + |> Repo.insert!() + end) + end +end diff --git a/lib/pleroma/digest_email_worker.ex b/lib/pleroma/digest_email_worker.ex new file mode 100644 index 000000000..5644d6a67 --- /dev/null +++ b/lib/pleroma/digest_email_worker.ex @@ -0,0 +1,39 @@ +# Pleroma: A lightweight social networking server +# Copyright © 2017-2019 Pleroma Authors +# SPDX-License-Identifier: AGPL-3.0-only + +defmodule Pleroma.DigestEmailWorker do + import Ecto.Query + + @queue_name :digest_emails + + def perform do + config = Pleroma.Config.get([:email_notifications, :digest]) + negative_interval = -Map.fetch!(config, :interval) + inactivity_threshold = Map.fetch!(config, :inactivity_threshold) + inactive_users_query = Pleroma.User.list_inactive_users_query(inactivity_threshold) + + now = NaiveDateTime.truncate(NaiveDateTime.utc_now(), :second) + + from(u in inactive_users_query, + where: fragment(~s(? #> '{"email_notifications","digest"}' @> 'true'), u.info), + where: u.last_digest_emailed_at < datetime_add(^now, ^negative_interval, "day"), + select: u + ) + |> Pleroma.Repo.all() + |> Enum.each(&PleromaJobQueue.enqueue(@queue_name, __MODULE__, [&1])) + end + + @doc """ + Send digest email to the given user. + Updates `last_digest_emailed_at` field for the user and returns the updated user. + """ + @spec perform(Pleroma.User.t()) :: Pleroma.User.t() + def perform(user) do + with %Swoosh.Email{} = email <- Pleroma.Emails.UserEmail.digest_email(user) do + Pleroma.Emails.Mailer.deliver_async(email) + end + + Pleroma.User.touch_last_digest_emailed_at(user) + end +end diff --git a/lib/pleroma/emails/admin_email.ex b/lib/pleroma/emails/admin_email.ex index d0e254362..c14be02dd 100644 --- a/lib/pleroma/emails/admin_email.ex +++ b/lib/pleroma/emails/admin_email.ex @@ -63,7 +63,6 @@ def report(to, reporter, account, statuses, comment) do new() |> to({to.name, to.email}) |> from({instance_name(), instance_notify_email()}) - |> reply_to({reporter.name, reporter.email}) |> subject("#{instance_name()} Report") |> html_body(html_body) end diff --git a/lib/pleroma/emails/user_email.ex b/lib/pleroma/emails/user_email.ex index 934620765..40b67ff56 100644 --- a/lib/pleroma/emails/user_email.ex +++ b/lib/pleroma/emails/user_email.ex @@ -5,23 +5,23 @@ defmodule Pleroma.Emails.UserEmail do @moduledoc "User emails" - import Swoosh.Email + use Phoenix.Swoosh, view: Pleroma.Web.EmailView, layout: {Pleroma.Web.LayoutView, :email} + alias Pleroma.Config + alias Pleroma.User alias Pleroma.Web.Endpoint alias Pleroma.Web.Router - defp instance_config, do: Pleroma.Config.get(:instance) - - defp instance_name, do: instance_config()[:name] + defp instance_name, do: Config.get([:instance, :name]) defp sender do - email = Keyword.get(instance_config(), :notify_email, instance_config()[:email]) + email = Config.get([:instance, :notify_email]) || Config.get([:instance, :email]) {instance_name(), email} end defp recipient(email, nil), do: email defp recipient(email, name), do: {name, email} - defp recipient(%Pleroma.User{} = user), do: recipient(user.email, user.name) + defp recipient(%User{} = user), do: recipient(user.email, user.name) def password_reset_email(user, token) when is_binary(token) do password_reset_url = Router.Helpers.reset_password_url(Endpoint, :reset, token) @@ -87,4 +87,92 @@ def account_confirmation_email(user) do |> subject("#{instance_name()} account confirmation") |> html_body(html_body) end + + @doc """ + Email used in digest email notifications + Includes Mentions and New Followers data + If there are no mentions (even when new followers exist), the function will return nil + """ + @spec digest_email(User.t()) :: Swoosh.Email.t() | nil + def digest_email(user) do + notifications = Pleroma.Notification.for_user_since(user, user.last_digest_emailed_at) + + mentions = + notifications + |> Enum.filter(&(&1.activity.data["type"] == "Create")) + |> Enum.map(fn notification -> + object = Pleroma.Object.normalize(notification.activity) + object = update_in(object.data["content"], &format_links/1) + + %{ + data: notification, + object: object, + from: User.get_by_ap_id(notification.activity.actor) + } + end) + + followers = + notifications + |> Enum.filter(&(&1.activity.data["type"] == "Follow")) + |> Enum.map(fn notification -> + %{ + data: notification, + object: Pleroma.Object.normalize(notification.activity), + from: User.get_by_ap_id(notification.activity.actor) + } + end) + + unless Enum.empty?(mentions) do + styling = Config.get([__MODULE__, :styling]) + logo = Config.get([__MODULE__, :logo]) + + html_data = %{ + instance: instance_name(), + user: user, + mentions: mentions, + followers: followers, + unsubscribe_link: unsubscribe_url(user, "digest"), + styling: styling + } + + logo_path = + if is_nil(logo) do + Path.join(:code.priv_dir(:pleroma), "static/static/logo.png") + else + Path.join(Config.get([:instance, :static_dir]), logo) + end + + new() + |> to(recipient(user)) + |> from(sender()) + |> subject("Your digest from #{instance_name()}") + |> put_layout(false) + |> render_body("digest.html", html_data) + |> attachment(Swoosh.Attachment.new(logo_path, filename: "logo.png", type: :inline)) + end + end + + defp format_links(str) do + re = ~r//iU + %{link_color: color} = Config.get([__MODULE__, :styling]) + + Regex.replace(re, str, fn link -> + String.replace(link, " user.id, "act" => %{"unsubscribe" => notifications_type}, "exp" => false} + |> Pleroma.JWT.generate_and_sign!() + |> Base.encode64() + + Router.Helpers.subscription_url(Endpoint, :unsubscribe, token) + end end diff --git a/lib/pleroma/emoji.ex b/lib/pleroma/emoji.ex index 052501642..66e20f0e4 100644 --- a/lib/pleroma/emoji.ex +++ b/lib/pleroma/emoji.ex @@ -24,7 +24,7 @@ defmodule Pleroma.Emoji do @ets_options [:ordered_set, :protected, :named_table, {:read_concurrency, true}] @doc false - def start_link do + def start_link(_) do GenServer.start_link(__MODULE__, [], name: __MODULE__) end diff --git a/lib/pleroma/flake_id.ex b/lib/pleroma/flake_id.ex index 58ab3650d..47d61ca5f 100644 --- a/lib/pleroma/flake_id.ex +++ b/lib/pleroma/flake_id.ex @@ -66,6 +66,16 @@ def from_integer(integer) do @spec get :: binary def get, do: to_string(:gen_server.call(:flake, :get)) + # checks that ID is is valid FlakeID + # + @spec is_flake_id?(String.t()) :: boolean + def is_flake_id?(id), do: is_flake_id?(String.to_charlist(id), true) + defp is_flake_id?([c | cs], true) when c >= ?0 and c <= ?9, do: is_flake_id?(cs, true) + defp is_flake_id?([c | cs], true) when c >= ?A and c <= ?Z, do: is_flake_id?(cs, true) + defp is_flake_id?([c | cs], true) when c >= ?a and c <= ?z, do: is_flake_id?(cs, true) + defp is_flake_id?([], true), do: true + defp is_flake_id?(_, _), do: false + # -- Ecto.Type API @impl Ecto.Type def type, do: :uuid @@ -88,7 +98,7 @@ def dump(value) do def autogenerate, do: get() # -- GenServer API - def start_link do + def start_link(_) do :gen_server.start_link({:local, :flake}, __MODULE__, [], []) end diff --git a/lib/pleroma/gopher/server.ex b/lib/pleroma/gopher/server.ex index b3319e137..d4e4f3e55 100644 --- a/lib/pleroma/gopher/server.ex +++ b/lib/pleroma/gopher/server.ex @@ -6,7 +6,7 @@ defmodule Pleroma.Gopher.Server do use GenServer require Logger - def start_link do + def start_link(_) do config = Pleroma.Config.get(:gopher, []) ip = Keyword.get(config, :ip, {0, 0, 0, 0}) port = Keyword.get(config, :port, 1234) diff --git a/lib/pleroma/html.ex b/lib/pleroma/html.ex index 2fae7281c..3951f0f51 100644 --- a/lib/pleroma/html.ex +++ b/lib/pleroma/html.ex @@ -203,6 +203,8 @@ defmodule Pleroma.HTML.Scrubber.Default do Meta.allow_tag_with_these_attributes("p", []) Meta.allow_tag_with_these_attributes("pre", []) Meta.allow_tag_with_these_attributes("strong", []) + Meta.allow_tag_with_these_attributes("sub", []) + Meta.allow_tag_with_these_attributes("sup", []) Meta.allow_tag_with_these_attributes("u", []) Meta.allow_tag_with_these_attributes("ul", []) @@ -280,3 +282,31 @@ def scrub({tag, attributes, children}), do: {tag, attributes, children} def scrub({_tag, children}), do: children def scrub(text), do: text end + +defmodule Pleroma.HTML.Scrubber.LinksOnly do + @moduledoc """ + An HTML scrubbing policy which limits to links only. + """ + + @valid_schemes Pleroma.Config.get([:uri_schemes, :valid_schemes], []) + + require HtmlSanitizeEx.Scrubber.Meta + alias HtmlSanitizeEx.Scrubber.Meta + + Meta.remove_cdata_sections_before_scrub() + Meta.strip_comments() + + # links + Meta.allow_tag_with_uri_attributes("a", ["href"], @valid_schemes) + + Meta.allow_tag_with_this_attribute_values("a", "rel", [ + "tag", + "nofollow", + "noopener", + "noreferrer", + "me" + ]) + + Meta.allow_tag_with_these_attributes("a", ["name", "title"]) + Meta.strip_everything_not_covered() +end diff --git a/lib/pleroma/http/connection.ex b/lib/pleroma/http/connection.ex index a1460d303..7e2c6f5e8 100644 --- a/lib/pleroma/http/connection.ex +++ b/lib/pleroma/http/connection.ex @@ -11,6 +11,7 @@ defmodule Pleroma.HTTP.Connection do connect_timeout: 10_000, recv_timeout: 20_000, follow_redirect: true, + force_redirect: true, pool: :federation ] @adapter Application.get_env(:tesla, :adapter) diff --git a/lib/pleroma/jwt.ex b/lib/pleroma/jwt.ex new file mode 100644 index 000000000..10102ff5d --- /dev/null +++ b/lib/pleroma/jwt.ex @@ -0,0 +1,9 @@ +defmodule Pleroma.JWT do + use Joken.Config + + @impl true + def token_config do + default_claims(skip: [:aud]) + |> add_claim("aud", &Pleroma.Web.Endpoint.url/0, &(&1 == Pleroma.Web.Endpoint.url())) + end +end diff --git a/lib/pleroma/notification.ex b/lib/pleroma/notification.ex index d47229258..5d29af853 100644 --- a/lib/pleroma/notification.ex +++ b/lib/pleroma/notification.ex @@ -18,6 +18,8 @@ defmodule Pleroma.Notification do import Ecto.Query import Ecto.Changeset + @type t :: %__MODULE__{} + schema "notifications" do field(:seen, :boolean, default: false) belongs_to(:user, User, type: Pleroma.FlakeId) @@ -31,7 +33,7 @@ def changeset(%Notification{} = notification, attrs) do |> cast(attrs, [:seen]) end - def for_user_query(user, opts) do + def for_user_query(user, opts \\ []) do query = Notification |> where(user_id: ^user.id) @@ -75,6 +77,25 @@ def for_user(user, opts \\ %{}) do |> Pagination.fetch_paginated(opts) end + @doc """ + Returns notifications for user received since given date. + + ## Examples + + iex> Pleroma.Notification.for_user_since(%Pleroma.User{}, ~N[2019-04-13 11:22:33]) + [%Pleroma.Notification{}, %Pleroma.Notification{}] + + iex> Pleroma.Notification.for_user_since(%Pleroma.User{}, ~N[2019-04-15 11:22:33]) + [] + """ + @spec for_user_since(Pleroma.User.t(), NaiveDateTime.t()) :: [t()] + def for_user_since(user, date) do + from(n in for_user_query(user), + where: n.updated_at > ^date + ) + |> Repo.all() + end + def set_read_up_to(%{id: user_id} = _user, id) do query = from( @@ -82,7 +103,10 @@ def set_read_up_to(%{id: user_id} = _user, id) do where: n.user_id == ^user_id, where: n.id <= ^id, update: [ - set: [seen: true] + set: [ + seen: true, + updated_at: ^NaiveDateTime.utc_now() + ] ] ) diff --git a/lib/pleroma/object/fetcher.ex b/lib/pleroma/object/fetcher.ex index 305ce8357..8d79ddb1f 100644 --- a/lib/pleroma/object/fetcher.ex +++ b/lib/pleroma/object/fetcher.ex @@ -114,7 +114,7 @@ defp maybe_date_fetch(headers, date) do end end - def fetch_and_contain_remote_object_from_id(id) do + def fetch_and_contain_remote_object_from_id(id) when is_binary(id) do Logger.info("Fetching object #{id} via AP") date = @@ -141,4 +141,9 @@ def fetch_and_contain_remote_object_from_id(id) do {:error, e} end end + + def fetch_and_contain_remote_object_from_id(%{"id" => id}), + do: fetch_and_contain_remote_object_from_id(id) + + def fetch_and_contain_remote_object_from_id(_id), do: {:error, "id must be a string"} end diff --git a/lib/pleroma/plugs/set_format_plug.ex b/lib/pleroma/plugs/set_format_plug.ex new file mode 100644 index 000000000..5ca741c64 --- /dev/null +++ b/lib/pleroma/plugs/set_format_plug.ex @@ -0,0 +1,24 @@ +# Pleroma: A lightweight social networking server +# Copyright © 2017-2019 Pleroma Authors +# SPDX-License-Identifier: AGPL-3.0-only + +defmodule Pleroma.Plugs.SetFormatPlug do + import Plug.Conn, only: [assign: 3, fetch_query_params: 1] + + def init(_), do: nil + + def call(conn, _) do + case get_format(conn) do + nil -> conn + format -> assign(conn, :format, format) + end + end + + defp get_format(conn) do + conn.private[:phoenix_format] || + case fetch_query_params(conn) do + %{query_params: %{"_format" => format}} -> format + _ -> nil + end + end +end diff --git a/lib/pleroma/reverse_proxy/reverse_proxy.ex b/lib/pleroma/reverse_proxy/reverse_proxy.ex index 1f98f215c..03efad30a 100644 --- a/lib/pleroma/reverse_proxy/reverse_proxy.ex +++ b/lib/pleroma/reverse_proxy/reverse_proxy.ex @@ -109,7 +109,11 @@ def call(conn = %{method: method}, url, opts) when method in @methods do end with {:ok, code, headers, client} <- request(method, url, req_headers, hackney_opts), - :ok <- header_length_constraint(headers, Keyword.get(opts, :max_body_length)) do + :ok <- + header_length_constraint( + headers, + Keyword.get(opts, :max_body_length, @max_body_length) + ) do response(conn, client, url, code, headers, opts) else {:ok, code, headers} -> @@ -200,7 +204,11 @@ defp chunk_reply(conn, client, opts, sent_so_far, duration) do {:ok, data} <- client().stream_body(client), {:ok, duration} <- increase_read_duration(duration), sent_so_far = sent_so_far + byte_size(data), - :ok <- body_size_constraint(sent_so_far, Keyword.get(opts, :max_body_size)), + :ok <- + body_size_constraint( + sent_so_far, + Keyword.get(opts, :max_body_length, @max_body_length) + ), {:ok, conn} <- chunk(conn, data) do chunk_reply(conn, client, opts, sent_so_far, duration) else diff --git a/lib/pleroma/scheduled_activity_worker.ex b/lib/pleroma/scheduled_activity_worker.ex index 65b38622f..8578cab5e 100644 --- a/lib/pleroma/scheduled_activity_worker.ex +++ b/lib/pleroma/scheduled_activity_worker.ex @@ -16,7 +16,7 @@ defmodule Pleroma.ScheduledActivityWorker do @schedule_interval :timer.minutes(1) - def start_link do + def start_link(_) do GenServer.start_link(__MODULE__, nil) end diff --git a/lib/pleroma/signature.ex b/lib/pleroma/signature.ex index 0bf49fd7c..15bf3c317 100644 --- a/lib/pleroma/signature.ex +++ b/lib/pleroma/signature.ex @@ -15,7 +15,7 @@ def key_id_to_actor_id(key_id) do |> Map.put(:fragment, nil) uri = - if String.ends_with?(uri.path, "/publickey") do + if not is_nil(uri.path) and String.ends_with?(uri.path, "/publickey") do Map.put(uri, :path, String.replace(uri.path, "/publickey", "")) else uri diff --git a/lib/pleroma/stats.ex b/lib/pleroma/stats.ex index 5b242927b..df80fbaa4 100644 --- a/lib/pleroma/stats.ex +++ b/lib/pleroma/stats.ex @@ -7,31 +7,56 @@ defmodule Pleroma.Stats do alias Pleroma.Repo alias Pleroma.User - def start_link do - agent = Agent.start_link(fn -> {[], %{}} end, name: __MODULE__) - spawn(fn -> schedule_update() end) - agent + use GenServer + + @interval 1000 * 60 * 60 + + def start_link(_) do + GenServer.start_link(__MODULE__, initial_data(), name: __MODULE__) + end + + def force_update do + GenServer.call(__MODULE__, :force_update) end def get_stats do - Agent.get(__MODULE__, fn {_, stats} -> stats end) + %{stats: stats} = GenServer.call(__MODULE__, :get_state) + + stats end def get_peers do - Agent.get(__MODULE__, fn {peers, _} -> peers end) + %{peers: peers} = GenServer.call(__MODULE__, :get_state) + + peers end - def schedule_update do - spawn(fn -> - # 1 hour - Process.sleep(1000 * 60 * 60) - schedule_update() - end) - - update_stats() + def init(args) do + Process.send(self(), :run_update, []) + {:ok, args} end - def update_stats do + def handle_call(:force_update, _from, _state) do + new_stats = get_stat_data() + {:reply, new_stats, new_stats} + end + + def handle_call(:get_state, _from, state) do + {:reply, state, state} + end + + def handle_info(:run_update, _state) do + new_stats = get_stat_data() + + Process.send_after(self(), :run_update, @interval) + {:noreply, new_stats} + end + + defp initial_data do + %{peers: [], stats: %{}} + end + + defp get_stat_data do peers = from( u in User, @@ -52,8 +77,9 @@ def update_stats do user_count = Repo.aggregate(User.Query.build(%{local: true, active: true}), :count, :id) - Agent.update(__MODULE__, fn _ -> - {peers, %{domain_count: domain_count, status_count: status_count, user_count: user_count}} - end) + %{ + peers: peers, + stats: %{domain_count: domain_count, status_count: status_count, user_count: user_count} + } end end diff --git a/lib/pleroma/upload.ex b/lib/pleroma/upload.ex index c47d65241..9f0adde5b 100644 --- a/lib/pleroma/upload.ex +++ b/lib/pleroma/upload.ex @@ -228,7 +228,14 @@ defp url_from_spec(%__MODULE__{name: name}, base_url, {:file, path}) do "" end - [base_url, "media", path] + prefix = + if is_nil(Pleroma.Config.get([__MODULE__, :base_url])) do + "media" + else + "" + end + + [base_url, prefix, path] |> Path.join() end diff --git a/lib/pleroma/uploaders/local.ex b/lib/pleroma/uploaders/local.ex index fc533da23..36b3c35ec 100644 --- a/lib/pleroma/uploaders/local.ex +++ b/lib/pleroma/uploaders/local.ex @@ -11,7 +11,7 @@ def get_file(_) do def put_file(upload) do {local_path, file} = - case Enum.reverse(String.split(upload.path, "/", trim: true)) do + case Enum.reverse(Path.split(upload.path)) do [file] -> {upload_path(), file} @@ -23,7 +23,7 @@ def put_file(upload) do result_file = Path.join(local_path, file) - unless File.exists?(result_file) do + if not File.exists?(result_file) do File.cp!(upload.tempfile, result_file) end diff --git a/lib/pleroma/uploaders/mdii.ex b/lib/pleroma/uploaders/mdii.ex index 237544337..c36f3d61d 100644 --- a/lib/pleroma/uploaders/mdii.ex +++ b/lib/pleroma/uploaders/mdii.ex @@ -3,6 +3,8 @@ # SPDX-License-Identifier: AGPL-3.0-only defmodule Pleroma.Uploaders.MDII do + @moduledoc "Represents uploader for https://github.com/hakaba-hitoyo/minimal-digital-image-infrastructure" + alias Pleroma.Config alias Pleroma.HTTP diff --git a/lib/pleroma/uploaders/s3.ex b/lib/pleroma/uploaders/s3.ex index 521daa93b..8c353bed3 100644 --- a/lib/pleroma/uploaders/s3.ex +++ b/lib/pleroma/uploaders/s3.ex @@ -6,10 +6,12 @@ defmodule Pleroma.Uploaders.S3 do @behaviour Pleroma.Uploaders.Uploader require Logger + alias Pleroma.Config + # The file name is re-encoded with S3's constraints here to comply with previous # links with less strict filenames def get_file(file) do - config = Pleroma.Config.get([__MODULE__]) + config = Config.get([__MODULE__]) bucket = Keyword.fetch!(config, :bucket) bucket_with_namespace = @@ -34,15 +36,15 @@ def get_file(file) do end def put_file(%Pleroma.Upload{} = upload) do - config = Pleroma.Config.get([__MODULE__]) + config = Config.get([__MODULE__]) bucket = Keyword.get(config, :bucket) - {:ok, file_data} = File.read(upload.tempfile) - s3_name = strict_encode(upload.path) op = - ExAws.S3.put_object(bucket, s3_name, file_data, [ + upload.tempfile + |> ExAws.S3.Upload.stream_file() + |> ExAws.S3.upload(bucket, s3_name, [ {:acl, :public_read}, {:content_type, upload.content_type} ]) diff --git a/lib/pleroma/user.ex b/lib/pleroma/user.ex index 982ca8bc1..134b8bb6c 100644 --- a/lib/pleroma/user.ex +++ b/lib/pleroma/user.ex @@ -21,6 +21,7 @@ defmodule Pleroma.User do alias Pleroma.Web alias Pleroma.Web.ActivityPub.ActivityPub alias Pleroma.Web.ActivityPub.Utils + alias Pleroma.Web.CommonAPI alias Pleroma.Web.CommonAPI.Utils, as: CommonUtils alias Pleroma.Web.OAuth alias Pleroma.Web.OStatus @@ -57,6 +58,7 @@ defmodule Pleroma.User do field(:search_type, :integer, virtual: true) field(:tags, {:array, :string}, default: []) field(:last_refreshed_at, :naive_datetime_usec) + field(:last_digest_emailed_at, :naive_datetime) has_many(:notifications, Notification) has_many(:registrations, Registration) embeds_one(:info, User.Info) @@ -114,7 +116,9 @@ def ap_following(%User{} = user), do: "#{ap_id(user)}/following" def user_info(%User{} = user, args \\ %{}) do following_count = - if args[:following_count], do: args[:following_count], else: following_count(user) + if args[:following_count], + do: args[:following_count], + else: user.info.following_count || following_count(user) follower_count = if args[:follower_count], do: args[:follower_count], else: user.info.follower_count @@ -129,6 +133,28 @@ def user_info(%User{} = user, args \\ %{}) do |> Map.put(:follower_count, follower_count) end + def follow_state(%User{} = user, %User{} = target) do + follow_activity = Utils.fetch_latest_follow(user, target) + + if follow_activity, + do: follow_activity.data["state"], + # Ideally this would be nil, but then Cachex does not commit the value + else: false + end + + def get_cached_follow_state(user, target) do + key = "follow_state:#{user.ap_id}|#{target.ap_id}" + Cachex.fetch!(:user_cache, key, fn _ -> {:commit, follow_state(user, target)} end) + end + + def set_follow_state_cache(user_ap_id, target_ap_id, state) do + Cachex.put( + :user_cache, + "follow_state:#{user_ap_id}|#{target_ap_id}", + state + ) + end + def set_info_cache(user, args) do Cachex.put(:user_cache, "user_info:#{user.id}", user_info(user, args)) end @@ -149,10 +175,10 @@ def following_count(%User{} = user) do end def remote_user_creation(params) do - params = - params - |> Map.put(:info, params[:info] || %{}) + bio_limit = Pleroma.Config.get([:instance, :user_bio_length], 5000) + name_limit = Pleroma.Config.get([:instance, :user_name_length], 100) + params = Map.put(params, :info, params[:info] || %{}) info_cng = User.Info.remote_user_creation(%User.Info{}, params[:info]) changes = @@ -161,8 +187,8 @@ def remote_user_creation(params) do |> validate_required([:name, :ap_id]) |> unique_constraint(:nickname) |> validate_format(:nickname, @email_regex) - |> validate_length(:bio, max: 5000) - |> validate_length(:name, max: 100) + |> validate_length(:bio, max: bio_limit) + |> validate_length(:name, max: name_limit) |> put_change(:local, false) |> put_embed(:info, info_cng) @@ -185,22 +211,23 @@ def remote_user_creation(params) do end def update_changeset(struct, params \\ %{}) do + bio_limit = Pleroma.Config.get([:instance, :user_bio_length], 5000) + name_limit = Pleroma.Config.get([:instance, :user_name_length], 100) + struct |> cast(params, [:bio, :name, :avatar, :following]) |> unique_constraint(:nickname) |> validate_format(:nickname, local_nickname_regex()) - |> validate_length(:bio, max: 5000) - |> validate_length(:name, min: 1, max: 100) + |> validate_length(:bio, max: bio_limit) + |> validate_length(:name, min: 1, max: name_limit) end - def upgrade_changeset(struct, params \\ %{}) do - params = - params - |> Map.put(:last_refreshed_at, NaiveDateTime.utc_now()) + def upgrade_changeset(struct, params \\ %{}, remote? \\ false) do + bio_limit = Pleroma.Config.get([:instance, :user_bio_length], 5000) + name_limit = Pleroma.Config.get([:instance, :user_name_length], 100) - info_cng = - struct.info - |> User.Info.user_upgrade(params[:info]) + params = Map.put(params, :last_refreshed_at, NaiveDateTime.utc_now()) + info_cng = User.Info.user_upgrade(struct.info, params[:info], remote?) struct |> cast(params, [ @@ -213,8 +240,8 @@ def upgrade_changeset(struct, params \\ %{}) do ]) |> unique_constraint(:nickname) |> validate_format(:nickname, local_nickname_regex()) - |> validate_length(:bio, max: 5000) - |> validate_length(:name, max: 100) + |> validate_length(:bio, max: bio_limit) + |> validate_length(:name, max: name_limit) |> put_embed(:info, info_cng) end @@ -226,6 +253,7 @@ def password_update_changeset(struct, params) do |> put_password_hash end + @spec reset_password(User.t(), map) :: {:ok, User.t()} | {:error, Ecto.Changeset.t()} def reset_password(%User{id: user_id} = user, data) do multi = Multi.new() @@ -240,6 +268,9 @@ def reset_password(%User{id: user_id} = user, data) do end def register_changeset(struct, params \\ %{}, opts \\ []) do + bio_limit = Pleroma.Config.get([:instance, :user_bio_length], 5000) + name_limit = Pleroma.Config.get([:instance, :user_name_length], 100) + need_confirmation? = if is_nil(opts[:need_confirmation]) do Pleroma.Config.get([:instance, :account_activation_required]) @@ -260,8 +291,8 @@ def register_changeset(struct, params \\ %{}, opts \\ []) do |> validate_exclusion(:nickname, Pleroma.Config.get([User, :restricted_nicknames])) |> validate_format(:nickname, local_nickname_regex()) |> validate_format(:email, @email_regex) - |> validate_length(:bio, max: 1000) - |> validate_length(:name, min: 1, max: 100) + |> validate_length(:bio, max: bio_limit) + |> validate_length(:name, min: 1, max: name_limit) |> put_change(:info, info_change) changeset = @@ -330,6 +361,7 @@ def needs_update?(%User{local: false} = user) do def needs_update?(_), do: true + @spec maybe_direct_follow(User.t(), User.t()) :: {:ok, User.t()} | {:error, String.t()} def maybe_direct_follow(%User{} = follower, %User{local: true, info: %{locked: true}}) do {:ok, follower} end @@ -404,6 +436,8 @@ def follow(%User{} = follower, %User{info: info} = followed) do {1, [follower]} = Repo.update_all(q, []) + follower = maybe_update_following_count(follower) + {:ok, _} = update_follower_count(followed) set_cache(follower) @@ -423,6 +457,8 @@ def unfollow(%User{} = follower, %User{} = followed) do {1, [follower]} = Repo.update_all(q, []) + follower = maybe_update_following_count(follower) + {:ok, followed} = update_follower_count(followed) set_cache(follower) @@ -450,6 +486,13 @@ def get_by_ap_id(ap_id) do Repo.get_by(User, ap_id: ap_id) end + def get_all_by_ap_id(ap_ids) do + from(u in __MODULE__, + where: u.ap_id in ^ap_ids + ) + |> Repo.all() + end + # This is mostly an SPC migration fix. This guesses the user nickname by taking the last part # of the ap_id and the domain and tries to get that user def get_by_guessed_nickname(ap_id) do @@ -471,7 +514,7 @@ def set_cache(%User{} = user) do end def update_and_set_cache(changeset) do - with {:ok, user} <- Repo.update(changeset) do + with {:ok, user} <- Repo.update(changeset, stale_error_field: :id) do set_cache(user) else e -> e @@ -707,32 +750,75 @@ def update_note_count(%User{} = user) do |> update_and_set_cache() end - def update_follower_count(%User{} = user) do - follower_count_query = - User.Query.build(%{followers: user, deactivated: false}) - |> select([u], %{count: count(u.id)}) + @spec maybe_fetch_follow_information(User.t()) :: User.t() + def maybe_fetch_follow_information(user) do + with {:ok, user} <- fetch_follow_information(user) do + user + else + e -> + Logger.error("Follower/Following counter update for #{user.ap_id} failed.\n#{inspect(e)}") - User - |> where(id: ^user.id) - |> join(:inner, [u], s in subquery(follower_count_query)) - |> update([u, s], - set: [ - info: - fragment( - "jsonb_set(?, '{follower_count}', ?::varchar::jsonb, true)", - u.info, - s.count - ) - ] - ) - |> select([u], u) - |> Repo.update_all([]) - |> case do - {1, [user]} -> set_cache(user) - _ -> {:error, user} + user end end + def fetch_follow_information(user) do + with {:ok, info} <- ActivityPub.fetch_follow_information_for_user(user) do + info_cng = User.Info.follow_information_update(user.info, info) + + changeset = + user + |> change() + |> put_embed(:info, info_cng) + + update_and_set_cache(changeset) + else + {:error, _} = e -> e + e -> {:error, e} + end + end + + def update_follower_count(%User{} = user) do + if user.local or !Pleroma.Config.get([:instance, :external_user_synchronization]) do + follower_count_query = + User.Query.build(%{followers: user, deactivated: false}) + |> select([u], %{count: count(u.id)}) + + User + |> where(id: ^user.id) + |> join(:inner, [u], s in subquery(follower_count_query)) + |> update([u, s], + set: [ + info: + fragment( + "jsonb_set(?, '{follower_count}', ?::varchar::jsonb, true)", + u.info, + s.count + ) + ] + ) + |> select([u], u) + |> Repo.update_all([]) + |> case do + {1, [user]} -> set_cache(user) + _ -> {:error, user} + end + else + {:ok, maybe_fetch_follow_information(user)} + end + end + + @spec maybe_update_following_count(User.t()) :: User.t() + def maybe_update_following_count(%User{local: false} = user) do + if Pleroma.Config.get([:instance, :external_user_synchronization]) do + maybe_fetch_follow_information(user) + else + user + end + end + + def maybe_update_following_count(user), do: user + def remove_duplicated_following(%User{following: following} = user) do uniq_following = Enum.uniq(following) @@ -831,6 +917,13 @@ def block(blocker, %User{ap_id: ap_id} = blocked) do blocker end + # clear any requested follows as well + blocked = + case CommonAPI.reject_follow_request(blocked, blocker) do + {:ok, %User{} = updated_blocked} -> updated_blocked + nil -> blocked + end + blocker = if subscribed_to?(blocked, blocker) do {:ok, blocker} = unsubscribe(blocked, blocker) @@ -882,19 +975,26 @@ def muted_notifications?(nil, _), do: false def muted_notifications?(user, %{ap_id: ap_id}), do: Enum.member?(user.info.muted_notifications, ap_id) - def blocks?(%User{info: info} = _user, %{ap_id: ap_id}) do - blocks = info.blocks - - domain_blocks = Pleroma.Web.ActivityPub.MRF.subdomains_regex(info.domain_blocks) - - %{host: host} = URI.parse(ap_id) - - Enum.member?(blocks, ap_id) || - Pleroma.Web.ActivityPub.MRF.subdomain_match?(domain_blocks, host) + def blocks?(%User{} = user, %User{} = target) do + blocks_ap_id?(user, target) || blocks_domain?(user, target) end def blocks?(nil, _), do: false + def blocks_ap_id?(%User{} = user, %User{} = target) do + Enum.member?(user.info.blocks, target.ap_id) + end + + def blocks_ap_id?(_, _), do: false + + def blocks_domain?(%User{} = user, %User{} = target) do + domain_blocks = Pleroma.Web.ActivityPub.MRF.subdomains_regex(user.info.domain_blocks) + %{host: host} = URI.parse(target.ap_id) + Pleroma.Web.ActivityPub.MRF.subdomain_match?(domain_blocks, host) + end + + def blocks_domain?(_, _), do: false + def subscribed_to?(user, %{ap_id: ap_id}) do with %User{} = target <- get_cached_by_ap_id(ap_id) do Enum.member?(target.info.subscribers, user.ap_id) @@ -1363,6 +1463,80 @@ def showing_reblogs?(%User{} = user, %User{} = target) do target.ap_id not in user.info.muted_reblogs end + @doc """ + The function returns a query to get users with no activity for given interval of days. + Inactive users are those who didn't read any notification, or had any activity where + the user is the activity's actor, during `inactivity_threshold` days. + Deactivated users will not appear in this list. + + ## Examples + + iex> Pleroma.User.list_inactive_users() + %Ecto.Query{} + """ + @spec list_inactive_users_query(integer()) :: Ecto.Query.t() + def list_inactive_users_query(inactivity_threshold \\ 7) do + negative_inactivity_threshold = -inactivity_threshold + now = NaiveDateTime.truncate(NaiveDateTime.utc_now(), :second) + # Subqueries are not supported in `where` clauses, join gets too complicated. + has_read_notifications = + from(n in Pleroma.Notification, + where: n.seen == true, + group_by: n.id, + having: max(n.updated_at) > datetime_add(^now, ^negative_inactivity_threshold, "day"), + select: n.user_id + ) + |> Pleroma.Repo.all() + + from(u in Pleroma.User, + left_join: a in Pleroma.Activity, + on: u.ap_id == a.actor, + where: not is_nil(u.nickname), + where: fragment("not (?->'deactivated' @> 'true')", u.info), + where: u.id not in ^has_read_notifications, + group_by: u.id, + having: + max(a.inserted_at) < datetime_add(^now, ^negative_inactivity_threshold, "day") or + is_nil(max(a.inserted_at)) + ) + end + + @doc """ + Enable or disable email notifications for user + + ## Examples + + iex> Pleroma.User.switch_email_notifications(Pleroma.User{info: %{email_notifications: %{"digest" => false}}}, "digest", true) + Pleroma.User{info: %{email_notifications: %{"digest" => true}}} + + iex> Pleroma.User.switch_email_notifications(Pleroma.User{info: %{email_notifications: %{"digest" => true}}}, "digest", false) + Pleroma.User{info: %{email_notifications: %{"digest" => false}}} + """ + @spec switch_email_notifications(t(), String.t(), boolean()) :: + {:ok, t()} | {:error, Ecto.Changeset.t()} + def switch_email_notifications(user, type, status) do + info = Pleroma.User.Info.update_email_notifications(user.info, %{type => status}) + + change(user) + |> put_embed(:info, info) + |> update_and_set_cache() + end + + @doc """ + Set `last_digest_emailed_at` value for the user to current time + """ + @spec touch_last_digest_emailed_at(t()) :: t() + def touch_last_digest_emailed_at(user) do + now = NaiveDateTime.truncate(NaiveDateTime.utc_now(), :second) + + {:ok, updated_user} = + user + |> change(%{last_digest_emailed_at: now}) + |> update_and_set_cache() + + updated_user + end + @spec toggle_confirmation(User.t()) :: {:ok, User.t()} | {:error, Changeset.t()} def toggle_confirmation(%User{} = user) do need_confirmation? = !user.info.confirmation_pending diff --git a/lib/pleroma/user/info.ex b/lib/pleroma/user/info.ex index 9beb3ddbd..45a39924b 100644 --- a/lib/pleroma/user/info.ex +++ b/lib/pleroma/user/info.ex @@ -16,6 +16,8 @@ defmodule Pleroma.User.Info do field(:source_data, :map, default: %{}) field(:note_count, :integer, default: 0) field(:follower_count, :integer, default: 0) + # Should be filled in only for remote users + field(:following_count, :integer, default: nil) field(:locked, :boolean, default: false) field(:confirmation_pending, :boolean, default: false) field(:confirmation_token, :string, default: nil) @@ -43,9 +45,12 @@ defmodule Pleroma.User.Info do field(:hide_follows, :boolean, default: false) field(:hide_favorites, :boolean, default: true) field(:pinned_activities, {:array, :string}, default: []) + field(:email_notifications, :map, default: %{"digest" => false}) field(:mascot, :map, default: nil) field(:emoji, {:array, :map}, default: []) field(:pleroma_settings_store, :map, default: %{}) + field(:fields, {:array, :map}, default: []) + field(:raw_fields, {:array, :map}, default: []) field(:notification_settings, :map, default: %{ @@ -93,6 +98,30 @@ def update_notification_settings(info, settings) do |> validate_required([:notification_settings]) end + @doc """ + Update email notifications in the given User.Info struct. + + Examples: + + iex> update_email_notifications(%Pleroma.User.Info{email_notifications: %{"digest" => false}}, %{"digest" => true}) + %Pleroma.User.Info{email_notifications: %{"digest" => true}} + + """ + @spec update_email_notifications(t(), map()) :: Ecto.Changeset.t() + def update_email_notifications(info, settings) do + email_notifications = + info.email_notifications + |> Map.merge(settings) + |> Map.take(["digest"]) + + params = %{email_notifications: email_notifications} + fields = [:email_notifications] + + info + |> cast(params, fields) + |> validate_required(fields) + end + def add_to_note_count(info, number) do set_note_count(info, info.note_count + number) end @@ -223,19 +252,31 @@ def remote_user_creation(info, params) do :uri, :hub, :topic, - :salmon + :salmon, + :hide_followers, + :hide_follows, + :follower_count, + :fields, + :following_count ]) + |> validate_fields(true) end - def user_upgrade(info, params) do + def user_upgrade(info, params, remote? \\ false) do info |> cast(params, [ :ap_enabled, :source_data, :banner, :locked, - :magic_key + :magic_key, + :follower_count, + :following_count, + :hide_follows, + :fields, + :hide_followers ]) + |> validate_fields(remote?) end def profile_update(info, params) do @@ -251,10 +292,40 @@ def profile_update(info, params) do :background, :show_role, :skip_thread_containment, + :fields, + :raw_fields, :pleroma_settings_store ]) + |> validate_fields() end + def validate_fields(changeset, remote? \\ false) do + limit_name = if remote?, do: :max_remote_account_fields, else: :max_account_fields + limit = Pleroma.Config.get([:instance, limit_name], 0) + + changeset + |> validate_length(:fields, max: limit) + |> validate_change(:fields, fn :fields, fields -> + if Enum.all?(fields, &valid_field?/1) do + [] + else + [fields: "invalid"] + end + end) + end + + defp valid_field?(%{"name" => name, "value" => value}) do + name_limit = Pleroma.Config.get([:instance, :account_field_name_length], 255) + value_limit = Pleroma.Config.get([:instance, :account_field_value_length], 255) + + is_binary(name) && + is_binary(value) && + String.length(name) <= name_limit && + String.length(value) <= value_limit + end + + defp valid_field?(_), do: false + @spec confirmation_changeset(Info.t(), keyword()) :: Changeset.t() def confirmation_changeset(info, opts) do need_confirmation? = Keyword.get(opts, :need_confirmation) @@ -348,4 +419,27 @@ def remove_reblog_mute(info, ap_id) do cast(info, params, [:muted_reblogs]) end + + # ``fields`` is an array of mastodon profile field, containing ``{"name": "…", "value": "…"}``. + # For example: [{"name": "Pronoun", "value": "she/her"}, …] + def fields(%{fields: [], source_data: %{"attachment" => attachment}}) do + limit = Pleroma.Config.get([:instance, :max_remote_account_fields], 0) + + attachment + |> Enum.filter(fn %{"type" => t} -> t == "PropertyValue" end) + |> Enum.map(fn fields -> Map.take(fields, ["name", "value"]) end) + |> Enum.take(limit) + end + + def fields(%{fields: fields}), do: fields + + def follow_information_update(info, params) do + info + |> cast(params, [ + :hide_followers, + :hide_follows, + :follower_count, + :following_count + ]) + end end diff --git a/lib/pleroma/user/search.ex b/lib/pleroma/user/search.ex index 46620b89a..6fb2c2352 100644 --- a/lib/pleroma/user/search.ex +++ b/lib/pleroma/user/search.ex @@ -44,7 +44,7 @@ defp format_query(query_string) do query_string = String.trim_leading(query_string, "@") with [name, domain] <- String.split(query_string, "@"), - formatted_domain <- String.replace(domain, ~r/[!-\-|@|[-`|{-~|\/|:]+/, "") do + formatted_domain <- String.replace(domain, ~r/[!-\-|@|[-`|{-~|\/|:|\s]+/, "") do name <> "@" <> to_string(:idna.encode(formatted_domain)) else _ -> query_string diff --git a/lib/pleroma/web/activity_pub/activity_pub.ex b/lib/pleroma/web/activity_pub/activity_pub.ex index a42c50875..172c952d4 100644 --- a/lib/pleroma/web/activity_pub/activity_pub.ex +++ b/lib/pleroma/web/activity_pub/activity_pub.ex @@ -23,6 +23,7 @@ defmodule Pleroma.Web.ActivityPub.ActivityPub do import Pleroma.Web.ActivityPub.Visibility require Logger + require Pleroma.Constants # For Announce activities, we filter the recipients based on following status for any actors # that match actual users. See issue #164 for more information about why this is necessary. @@ -64,12 +65,12 @@ defp check_actor_is_active(actor) do if not is_nil(actor) do with user <- User.get_cached_by_ap_id(actor), false <- user.info.deactivated do - :ok + true else - _e -> :reject + _e -> false end else - :ok + true end end @@ -118,10 +119,10 @@ def increase_poll_votes_if_vote(%{ def increase_poll_votes_if_vote(_create_data), do: :noop - def insert(map, local \\ true, fake \\ false) when is_map(map) do + def insert(map, local \\ true, fake \\ false, bypass_actor_check \\ false) when is_map(map) do with nil <- Activity.normalize(map), map <- lazy_put_activity_defaults(map, fake), - :ok <- check_actor_is_active(map["actor"]), + true <- bypass_actor_check || check_actor_is_active(map["actor"]), {_, true} <- {:remote_limit_error, check_remote_limit(map)}, {:ok, map} <- MRF.filter(map), {recipients, _, _} = get_recipients(map), @@ -207,8 +208,6 @@ def stream_out_participations(%Object{data: %{"context" => context}}, user) do def stream_out_participations(_, _), do: :noop def stream_out(activity) do - public = "https://www.w3.org/ns/activitystreams#Public" - if activity.data["type"] in ["Create", "Announce", "Delete"] do object = Object.normalize(activity) # Do not stream out poll replies @@ -216,7 +215,7 @@ def stream_out(activity) do Pleroma.Web.Streamer.stream("user", activity) Pleroma.Web.Streamer.stream("list", activity) - if Enum.member?(activity.data["to"], public) do + if get_visibility(activity) == "public" do Pleroma.Web.Streamer.stream("public", activity) if activity.local do @@ -238,13 +237,8 @@ def stream_out(activity) do end end else - # TODO: Write test, replace with visibility test - if !Enum.member?(activity.data["cc"] || [], public) && - !Enum.member?( - activity.data["to"], - User.get_cached_by_ap_id(activity.data["actor"]).follower_address - ), - do: Pleroma.Web.Streamer.stream("direct", activity) + if get_visibility(activity) == "direct", + do: Pleroma.Web.Streamer.stream("direct", activity) end end end @@ -273,6 +267,9 @@ def create(%{to: to, actor: actor, context: context, object: object} = params, f else {:fake, true, activity} -> {:ok, activity} + + {:error, message} -> + {:error, message} end end @@ -391,7 +388,8 @@ def unannounce( def follow(follower, followed, activity_id \\ nil, local \\ true) do with data <- make_follow_data(follower, followed, activity_id), {:ok, activity} <- insert(data, local), - :ok <- maybe_federate(activity) do + :ok <- maybe_federate(activity), + _ <- User.set_follow_state_cache(follower.ap_id, followed.ap_id, activity.data["state"]) do {:ok, activity} end end @@ -413,7 +411,7 @@ def delete(%User{ap_id: ap_id, follower_address: follower_address} = user) do "actor" => ap_id, "object" => %{"type" => "Person", "id" => ap_id} }, - {:ok, activity} <- insert(data, true, true), + {:ok, activity} <- insert(data, true, true, true), :ok <- maybe_federate(activity) do {:ok, user} end @@ -514,13 +512,15 @@ def flag( end defp fetch_activities_for_context_query(context, opts) do - public = ["https://www.w3.org/ns/activitystreams#Public"] + public = [Pleroma.Constants.as_public()] recipients = if opts["user"], do: [opts["user"].ap_id | opts["user"].following] ++ public, else: public from(activity in Activity) |> maybe_preload_objects(opts) + |> maybe_preload_bookmarks(opts) + |> maybe_set_thread_muted_field(opts) |> restrict_blocked(opts) |> restrict_recipients(recipients, opts["user"]) |> where( @@ -534,6 +534,7 @@ defp fetch_activities_for_context_query(context, opts) do ) ) |> exclude_poll_votes(opts) + |> exclude_id(opts) |> order_by([activity], desc: activity.id) end @@ -555,7 +556,7 @@ def fetch_latest_activity_id_for_context(context, opts \\ %{}) do end def fetch_public_activities(opts \\ %{}) do - q = fetch_activities_query(["https://www.w3.org/ns/activitystreams#Public"], opts) + q = fetch_activities_query([Pleroma.Constants.as_public()], opts) q |> restrict_unlisted() @@ -626,6 +627,7 @@ def fetch_user_activities(user, reading_user, params \\ %{}) do params = params |> Map.put("type", ["Create", "Announce"]) + |> Map.put("user", reading_user) |> Map.put("actor_id", user.ap_id) |> Map.put("whole_db", true) |> Map.put("pinned_activity_ids", user.info.pinned_activities) @@ -646,10 +648,9 @@ defp user_activities_recipients(%{"godmode" => true}) do defp user_activities_recipients(%{"reading_user" => reading_user}) do if reading_user do - ["https://www.w3.org/ns/activitystreams#Public"] ++ - [reading_user.ap_id | reading_user.following] + [Pleroma.Constants.as_public()] ++ [reading_user.ap_id | reading_user.following] else - ["https://www.w3.org/ns/activitystreams#Public"] + [Pleroma.Constants.as_public()] end end @@ -753,8 +754,8 @@ defp restrict_state(query, _), do: query defp restrict_favorited_by(query, %{"favorited_by" => ap_id}) do from( - activity in query, - where: fragment(~s(? <@ (? #> '{"object","likes"}'\)), ^ap_id, activity.data) + [_activity, object] in query, + where: fragment("(?)->'likes' \\? (?)", object.data, ^ap_id) ) end @@ -790,14 +791,20 @@ defp restrict_reblogs(query, _), do: query defp restrict_muted(query, %{"with_muted" => val}) when val in [true, "true", "1"], do: query - defp restrict_muted(query, %{"muting_user" => %User{info: info}}) do + defp restrict_muted(query, %{"muting_user" => %User{info: info}} = opts) do mutes = info.mutes - from( - activity in query, - where: fragment("not (? = ANY(?))", activity.actor, ^mutes), - where: fragment("not (?->'to' \\?| ?)", activity.data, ^mutes) - ) + query = + from([activity] in query, + where: fragment("not (? = ANY(?))", activity.actor, ^mutes), + where: fragment("not (?->'to' \\?| ?)", activity.data, ^mutes) + ) + + unless opts["skip_preload"] do + from([thread_mute: tm] in query, where: is_nil(tm)) + else + query + end end defp restrict_muted(query, _), do: query @@ -834,7 +841,7 @@ defp restrict_unlisted(query) do fragment( "not (coalesce(?->'cc', '{}'::jsonb) \\?| ?)", activity.data, - ^["https://www.w3.org/ns/activitystreams#Public"] + ^[Pleroma.Constants.as_public()] ) ) end @@ -874,6 +881,12 @@ defp exclude_poll_votes(query, _) do end end + defp exclude_id(query, %{"exclude_id" => id}) when is_binary(id) do + from(activity in query, where: activity.id != ^id) + end + + defp exclude_id(query, _), do: query + defp maybe_preload_objects(query, %{"skip_preload" => true}), do: query defp maybe_preload_objects(query, _) do @@ -892,7 +905,7 @@ defp maybe_set_thread_muted_field(query, %{"skip_preload" => true}), do: query defp maybe_set_thread_muted_field(query, opts) do query - |> Activity.with_set_thread_muted_field(opts["user"]) + |> Activity.with_set_thread_muted_field(opts["muting_user"] || opts["user"]) end defp maybe_order(query, %{order: :desc}) do @@ -971,7 +984,7 @@ def fetch_activities_bounded_query(query, recipients, recipients_with_public) do where: fragment("? && ?", activity.recipients, ^recipients) or (fragment("? && ?", activity.recipients, ^recipients_with_public) and - "https://www.w3.org/ns/activitystreams#Public" in activity.recipients) + ^Pleroma.Constants.as_public() in activity.recipients) ) end @@ -1010,16 +1023,23 @@ defp object_to_user_data(data) do "url" => [%{"href" => data["image"]["url"]}] } + fields = + data + |> Map.get("attachment", []) + |> Enum.filter(fn %{"type" => t} -> t == "PropertyValue" end) + |> Enum.map(fn fields -> Map.take(fields, ["name", "value"]) end) + locked = data["manuallyApprovesFollowers"] || false data = Transmogrifier.maybe_fix_user_object(data) user_data = %{ ap_id: data["id"], info: %{ - "ap_enabled" => true, - "source_data" => data, - "banner" => banner, - "locked" => locked + ap_enabled: true, + source_data: data, + banner: banner, + fields: fields, + locked: locked }, avatar: avatar, name: data["name"], @@ -1043,6 +1063,71 @@ defp object_to_user_data(data) do {:ok, user_data} end + def fetch_follow_information_for_user(user) do + with {:ok, following_data} <- + Fetcher.fetch_and_contain_remote_object_from_id(user.following_address), + following_count when is_integer(following_count) <- following_data["totalItems"], + {:ok, hide_follows} <- collection_private(following_data), + {:ok, followers_data} <- + Fetcher.fetch_and_contain_remote_object_from_id(user.follower_address), + followers_count when is_integer(followers_count) <- followers_data["totalItems"], + {:ok, hide_followers} <- collection_private(followers_data) do + {:ok, + %{ + hide_follows: hide_follows, + follower_count: followers_count, + following_count: following_count, + hide_followers: hide_followers + }} + else + {:error, _} = e -> + e + + e -> + {:error, e} + end + end + + defp maybe_update_follow_information(data) do + with {:enabled, true} <- + {:enabled, Pleroma.Config.get([:instance, :external_user_synchronization])}, + {:ok, info} <- fetch_follow_information_for_user(data) do + info = Map.merge(data.info, info) + Map.put(data, :info, info) + else + {:enabled, false} -> + data + + e -> + Logger.error( + "Follower/Following counter update for #{data.ap_id} failed.\n" <> inspect(e) + ) + + data + end + end + + defp collection_private(data) do + if is_map(data["first"]) and + data["first"]["type"] in ["CollectionPage", "OrderedCollectionPage"] do + {:ok, false} + else + with {:ok, %{"type" => type}} when type in ["CollectionPage", "OrderedCollectionPage"] <- + Fetcher.fetch_and_contain_remote_object_from_id(data["first"]) do + {:ok, false} + else + {:error, {:ok, %{status: code}}} when code in [401, 403] -> + {:ok, true} + + {:error, _} = e -> + e + + e -> + {:error, e} + end + end + end + def user_data_from_user_object(data) do with {:ok, data} <- MRF.filter(data), {:ok, data} <- object_to_user_data(data) do @@ -1054,7 +1139,8 @@ def user_data_from_user_object(data) do def fetch_and_prepare_user_from_ap_id(ap_id) do with {:ok, data} <- Fetcher.fetch_and_contain_remote_object_from_id(ap_id), - {:ok, data} <- user_data_from_user_object(data) do + {:ok, data} <- user_data_from_user_object(data), + data <- maybe_update_follow_information(data) do {:ok, data} else e -> Logger.error("Could not decode user at fetch #{ap_id}, #{inspect(e)}") diff --git a/lib/pleroma/web/activity_pub/mrf.ex b/lib/pleroma/web/activity_pub/mrf.ex index dd204b21c..263ed11af 100644 --- a/lib/pleroma/web/activity_pub/mrf.ex +++ b/lib/pleroma/web/activity_pub/mrf.ex @@ -28,11 +28,43 @@ defp get_policies(_), do: [] @spec subdomains_regex([String.t()]) :: [Regex.t()] def subdomains_regex(domains) when is_list(domains) do - for domain <- domains, do: ~r(^#{String.replace(domain, "*.", "(.*\\.)*")}$) + for domain <- domains, do: ~r(^#{String.replace(domain, "*.", "(.*\\.)*")}$)i end @spec subdomain_match?([Regex.t()], String.t()) :: boolean() def subdomain_match?(domains, host) do Enum.any?(domains, fn domain -> Regex.match?(domain, host) end) end + + @callback describe() :: {:ok | :error, Map.t()} + + def describe(policies) do + {:ok, policy_configs} = + policies + |> Enum.reduce({:ok, %{}}, fn + policy, {:ok, data} -> + {:ok, policy_data} = policy.describe() + {:ok, Map.merge(data, policy_data)} + + _, error -> + error + end) + + mrf_policies = + get_policies() + |> Enum.map(fn policy -> to_string(policy) |> String.split(".") |> List.last() end) + + exclusions = Pleroma.Config.get([:instance, :mrf_transparency_exclusions]) + + base = + %{ + mrf_policies: mrf_policies, + exclusions: length(exclusions) > 0 + } + |> Map.merge(policy_configs) + + {:ok, base} + end + + def describe, do: get_policies() |> describe() end diff --git a/lib/pleroma/web/activity_pub/mrf/anti_followbot_policy.ex b/lib/pleroma/web/activity_pub/mrf/anti_followbot_policy.ex index 87fa514c3..de1eb4aa5 100644 --- a/lib/pleroma/web/activity_pub/mrf/anti_followbot_policy.ex +++ b/lib/pleroma/web/activity_pub/mrf/anti_followbot_policy.ex @@ -62,4 +62,7 @@ def filter(%{"type" => "Follow", "actor" => actor_id} = message) do @impl true def filter(message), do: {:ok, message} + + @impl true + def describe, do: {:ok, %{}} end diff --git a/lib/pleroma/web/activity_pub/mrf/anti_link_spam_policy.ex b/lib/pleroma/web/activity_pub/mrf/anti_link_spam_policy.ex index 2da3eac2f..b90193ca0 100644 --- a/lib/pleroma/web/activity_pub/mrf/anti_link_spam_policy.ex +++ b/lib/pleroma/web/activity_pub/mrf/anti_link_spam_policy.ex @@ -5,6 +5,8 @@ defmodule Pleroma.Web.ActivityPub.MRF.AntiLinkSpamPolicy do alias Pleroma.User + @behaviour Pleroma.Web.ActivityPub.MRF + require Logger # has the user successfully posted before? @@ -22,6 +24,7 @@ defp contains_links?(%{"content" => content} = _object) do defp contains_links?(_), do: false + @impl true def filter(%{"type" => "Create", "actor" => actor, "object" => object} = message) do with {:ok, %User{} = u} <- User.get_or_fetch_by_ap_id(actor), {:contains_links, true} <- {:contains_links, contains_links?(object)}, @@ -45,4 +48,7 @@ def filter(%{"type" => "Create", "actor" => actor, "object" => object} = message # in all other cases, pass through def filter(message), do: {:ok, message} + + @impl true + def describe, do: {:ok, %{}} end diff --git a/lib/pleroma/web/activity_pub/mrf/drop_policy.ex b/lib/pleroma/web/activity_pub/mrf/drop_policy.ex index b8d38aae6..f7831bc3e 100644 --- a/lib/pleroma/web/activity_pub/mrf/drop_policy.ex +++ b/lib/pleroma/web/activity_pub/mrf/drop_policy.ex @@ -12,4 +12,7 @@ def filter(object) do Logger.info("REJECTING #{inspect(object)}") {:reject, object} end + + @impl true + def describe, do: {:ok, %{}} end diff --git a/lib/pleroma/web/activity_pub/mrf/ensure_re_prepended.ex b/lib/pleroma/web/activity_pub/mrf/ensure_re_prepended.ex index 2d03df68a..3a3e72910 100644 --- a/lib/pleroma/web/activity_pub/mrf/ensure_re_prepended.ex +++ b/lib/pleroma/web/activity_pub/mrf/ensure_re_prepended.ex @@ -39,4 +39,6 @@ def filter(%{"type" => "Create", "object" => child_object} = object) do end def filter(object), do: {:ok, object} + + def describe, do: {:ok, %{}} end diff --git a/lib/pleroma/web/activity_pub/mrf/hellthread_policy.ex b/lib/pleroma/web/activity_pub/mrf/hellthread_policy.ex index a699f6a7e..b3c742954 100644 --- a/lib/pleroma/web/activity_pub/mrf/hellthread_policy.ex +++ b/lib/pleroma/web/activity_pub/mrf/hellthread_policy.ex @@ -4,6 +4,9 @@ defmodule Pleroma.Web.ActivityPub.MRF.HellthreadPolicy do alias Pleroma.User + + require Pleroma.Constants + @moduledoc "Block messages with too much mentions (configurable)" @behaviour Pleroma.Web.ActivityPub.MRF @@ -19,12 +22,12 @@ defp delist_message(message, threshold) when threshold > 0 do when follower_collection? and recipients > threshold -> message |> Map.put("to", [follower_collection]) - |> Map.put("cc", ["https://www.w3.org/ns/activitystreams#Public"]) + |> Map.put("cc", [Pleroma.Constants.as_public()]) {:public, recipients} when recipients > threshold -> message |> Map.put("to", []) - |> Map.put("cc", ["https://www.w3.org/ns/activitystreams#Public"]) + |> Map.put("cc", [Pleroma.Constants.as_public()]) _ -> message @@ -51,10 +54,10 @@ defp get_recipient_count(message) do recipients = (message["to"] || []) ++ (message["cc"] || []) follower_collection = User.get_cached_by_ap_id(message["actor"]).follower_address - if Enum.member?(recipients, "https://www.w3.org/ns/activitystreams#Public") do + if Enum.member?(recipients, Pleroma.Constants.as_public()) do recipients = recipients - |> List.delete("https://www.w3.org/ns/activitystreams#Public") + |> List.delete(Pleroma.Constants.as_public()) |> List.delete(follower_collection) {:public, length(recipients)} @@ -87,4 +90,8 @@ def filter(%{"type" => "Create"} = message) do @impl true def filter(message), do: {:ok, message} + + @impl true + def describe, + do: {:ok, %{mrf_hellthread: Pleroma.Config.get(:mrf_hellthread) |> Enum.into(%{})}} end diff --git a/lib/pleroma/web/activity_pub/mrf/keyword_policy.ex b/lib/pleroma/web/activity_pub/mrf/keyword_policy.ex index d5c341433..d6d1396bc 100644 --- a/lib/pleroma/web/activity_pub/mrf/keyword_policy.ex +++ b/lib/pleroma/web/activity_pub/mrf/keyword_policy.ex @@ -3,6 +3,8 @@ # SPDX-License-Identifier: AGPL-3.0-only defmodule Pleroma.Web.ActivityPub.MRF.KeywordPolicy do + require Pleroma.Constants + @moduledoc "Reject or Word-Replace messages with a keyword or regex" @behaviour Pleroma.Web.ActivityPub.MRF @@ -31,12 +33,12 @@ defp check_reject(%{"object" => %{"content" => content, "summary" => summary}} = defp check_ftl_removal( %{"to" => to, "object" => %{"content" => content, "summary" => summary}} = message ) do - if "https://www.w3.org/ns/activitystreams#Public" in to and + if Pleroma.Constants.as_public() in to and Enum.any?(Pleroma.Config.get([:mrf_keyword, :federated_timeline_removal]), fn pattern -> string_matches?(content, pattern) or string_matches?(summary, pattern) end) do - to = List.delete(to, "https://www.w3.org/ns/activitystreams#Public") - cc = ["https://www.w3.org/ns/activitystreams#Public" | message["cc"] || []] + to = List.delete(to, Pleroma.Constants.as_public()) + cc = [Pleroma.Constants.as_public() | message["cc"] || []] message = message @@ -94,4 +96,36 @@ def filter(%{"type" => "Create", "object" => %{"content" => _content}} = message @impl true def filter(message), do: {:ok, message} + + @impl true + def describe do + # This horror is needed to convert regex sigils to strings + mrf_keyword = + Pleroma.Config.get(:mrf_keyword, []) + |> Enum.map(fn {key, value} -> + {key, + Enum.map(value, fn + {pattern, replacement} -> + %{ + "pattern" => + if not is_binary(pattern) do + inspect(pattern) + else + pattern + end, + "replacement" => replacement + } + + pattern -> + if not is_binary(pattern) do + inspect(pattern) + else + pattern + end + end)} + end) + |> Enum.into(%{}) + + {:ok, %{mrf_keyword: mrf_keyword}} + end end diff --git a/lib/pleroma/web/activity_pub/mrf/mediaproxy_warming_policy.ex b/lib/pleroma/web/activity_pub/mrf/mediaproxy_warming_policy.ex index 01d21a299..a179dd54d 100644 --- a/lib/pleroma/web/activity_pub/mrf/mediaproxy_warming_policy.ex +++ b/lib/pleroma/web/activity_pub/mrf/mediaproxy_warming_policy.ex @@ -53,4 +53,7 @@ def filter( @impl true def filter(message), do: {:ok, message} + + @impl true + def describe, do: {:ok, %{}} end diff --git a/lib/pleroma/web/activity_pub/mrf/mention_policy.ex b/lib/pleroma/web/activity_pub/mrf/mention_policy.ex index 1842e1aeb..ce8bc4580 100644 --- a/lib/pleroma/web/activity_pub/mrf/mention_policy.ex +++ b/lib/pleroma/web/activity_pub/mrf/mention_policy.ex @@ -21,4 +21,7 @@ def filter(%{"type" => "Create"} = message) do @impl true def filter(message), do: {:ok, message} + + @impl true + def describe, do: {:ok, %{}} end diff --git a/lib/pleroma/web/activity_pub/mrf/no_placeholder_text_policy.ex b/lib/pleroma/web/activity_pub/mrf/no_placeholder_text_policy.ex index 86a48bda5..f67f48ab6 100644 --- a/lib/pleroma/web/activity_pub/mrf/no_placeholder_text_policy.ex +++ b/lib/pleroma/web/activity_pub/mrf/no_placeholder_text_policy.ex @@ -19,4 +19,7 @@ def filter( @impl true def filter(object), do: {:ok, object} + + @impl true + def describe, do: {:ok, %{}} end diff --git a/lib/pleroma/web/activity_pub/mrf/noop_policy.ex b/lib/pleroma/web/activity_pub/mrf/noop_policy.ex index c47cb3298..878c57925 100644 --- a/lib/pleroma/web/activity_pub/mrf/noop_policy.ex +++ b/lib/pleroma/web/activity_pub/mrf/noop_policy.ex @@ -10,4 +10,7 @@ defmodule Pleroma.Web.ActivityPub.MRF.NoOpPolicy do def filter(object) do {:ok, object} end + + @impl true + def describe, do: {:ok, %{}} end diff --git a/lib/pleroma/web/activity_pub/mrf/normalize_markup.ex b/lib/pleroma/web/activity_pub/mrf/normalize_markup.ex index c269d0f89..daa4c88ad 100644 --- a/lib/pleroma/web/activity_pub/mrf/normalize_markup.ex +++ b/lib/pleroma/web/activity_pub/mrf/normalize_markup.ex @@ -21,4 +21,6 @@ def filter(%{"type" => "Create", "object" => child_object} = object) do end def filter(object), do: {:ok, object} + + def describe, do: {:ok, %{}} end diff --git a/lib/pleroma/web/activity_pub/mrf/reject_non_public.ex b/lib/pleroma/web/activity_pub/mrf/reject_non_public.ex index da13fd7c7..5a809a321 100644 --- a/lib/pleroma/web/activity_pub/mrf/reject_non_public.ex +++ b/lib/pleroma/web/activity_pub/mrf/reject_non_public.ex @@ -10,7 +10,7 @@ defmodule Pleroma.Web.ActivityPub.MRF.RejectNonPublic do @behaviour Pleroma.Web.ActivityPub.MRF - @public "https://www.w3.org/ns/activitystreams#Public" + require Pleroma.Constants @impl true def filter(%{"type" => "Create"} = object) do @@ -19,8 +19,8 @@ def filter(%{"type" => "Create"} = object) do # Determine visibility visibility = cond do - @public in object["to"] -> "public" - @public in object["cc"] -> "unlisted" + Pleroma.Constants.as_public() in object["to"] -> "public" + Pleroma.Constants.as_public() in object["cc"] -> "unlisted" user.follower_address in object["to"] -> "followers" true -> "direct" end @@ -44,4 +44,8 @@ def filter(%{"type" => "Create"} = object) do @impl true def filter(object), do: {:ok, object} + + @impl true + def describe, + do: {:ok, %{mrf_rejectnonpublic: Pleroma.Config.get(:mrf_rejectnonpublic) |> Enum.into(%{})}} end diff --git a/lib/pleroma/web/activity_pub/mrf/simple_policy.ex b/lib/pleroma/web/activity_pub/mrf/simple_policy.ex index 2cf63d3db..8aa6852f0 100644 --- a/lib/pleroma/web/activity_pub/mrf/simple_policy.ex +++ b/lib/pleroma/web/activity_pub/mrf/simple_policy.ex @@ -8,6 +8,8 @@ defmodule Pleroma.Web.ActivityPub.MRF.SimplePolicy do @moduledoc "Filter activities depending on their origin instance" @behaviour MRF + require Pleroma.Constants + defp check_accept(%{host: actor_host} = _actor_info, object) do accepts = Pleroma.Config.get([:mrf_simple, :accept]) @@ -89,14 +91,10 @@ defp check_ftl_removal(%{host: actor_host} = _actor_info, object) do object = with true <- MRF.subdomain_match?(timeline_removal, actor_host), user <- User.get_cached_by_ap_id(object["actor"]), - true <- "https://www.w3.org/ns/activitystreams#Public" in object["to"] do - to = - List.delete(object["to"], "https://www.w3.org/ns/activitystreams#Public") ++ - [user.follower_address] + true <- Pleroma.Constants.as_public() in object["to"] do + to = List.delete(object["to"], Pleroma.Constants.as_public()) ++ [user.follower_address] - cc = - List.delete(object["cc"], user.follower_address) ++ - ["https://www.w3.org/ns/activitystreams#Public"] + cc = List.delete(object["cc"], user.follower_address) ++ [Pleroma.Constants.as_public()] object |> Map.put("to", to) @@ -179,4 +177,16 @@ def filter(%{"id" => actor, "type" => obj_type} = object) end def filter(object), do: {:ok, object} + + @impl true + def describe do + exclusions = Pleroma.Config.get([:instance, :mrf_transparency_exclusions]) + + mrf_simple = + Pleroma.Config.get(:mrf_simple) + |> Enum.map(fn {k, v} -> {k, Enum.reject(v, fn v -> v in exclusions end)} end) + |> Enum.into(%{}) + + {:ok, %{mrf_simple: mrf_simple}} + end end diff --git a/lib/pleroma/web/activity_pub/mrf/subchain_policy.ex b/lib/pleroma/web/activity_pub/mrf/subchain_policy.ex index 765704389..566c1e191 100644 --- a/lib/pleroma/web/activity_pub/mrf/subchain_policy.ex +++ b/lib/pleroma/web/activity_pub/mrf/subchain_policy.ex @@ -37,4 +37,7 @@ def filter(%{"actor" => actor} = message) do @impl true def filter(message), do: {:ok, message} + + @impl true + def describe, do: {:ok, %{}} end diff --git a/lib/pleroma/web/activity_pub/mrf/tag_policy.ex b/lib/pleroma/web/activity_pub/mrf/tag_policy.ex index b42c4ed76..c1801d2ec 100644 --- a/lib/pleroma/web/activity_pub/mrf/tag_policy.ex +++ b/lib/pleroma/web/activity_pub/mrf/tag_policy.ex @@ -19,7 +19,7 @@ defmodule Pleroma.Web.ActivityPub.MRF.TagPolicy do - `mrf_tag:disable-any-subscription`: Reject any follow requests """ - @public "https://www.w3.org/ns/activitystreams#Public" + require Pleroma.Constants defp get_tags(%User{tags: tags}) when is_list(tags), do: tags defp get_tags(_), do: [] @@ -70,9 +70,9 @@ defp process_tag( ) do user = User.get_cached_by_ap_id(actor) - if Enum.member?(to, @public) do - to = List.delete(to, @public) ++ [user.follower_address] - cc = List.delete(cc, user.follower_address) ++ [@public] + if Enum.member?(to, Pleroma.Constants.as_public()) do + to = List.delete(to, Pleroma.Constants.as_public()) ++ [user.follower_address] + cc = List.delete(cc, user.follower_address) ++ [Pleroma.Constants.as_public()] object = object @@ -103,9 +103,10 @@ defp process_tag( ) do user = User.get_cached_by_ap_id(actor) - if Enum.member?(to, @public) or Enum.member?(cc, @public) do - to = List.delete(to, @public) ++ [user.follower_address] - cc = List.delete(cc, @public) + if Enum.member?(to, Pleroma.Constants.as_public()) or + Enum.member?(cc, Pleroma.Constants.as_public()) do + to = List.delete(to, Pleroma.Constants.as_public()) ++ [user.follower_address] + cc = List.delete(cc, Pleroma.Constants.as_public()) object = object @@ -164,4 +165,7 @@ def filter(%{"actor" => actor, "type" => "Create"} = message), @impl true def filter(message), do: {:ok, message} + + @impl true + def describe, do: {:ok, %{}} end diff --git a/lib/pleroma/web/activity_pub/mrf/user_allowlist_policy.ex b/lib/pleroma/web/activity_pub/mrf/user_allowlist_policy.ex index e35d2c422..7389d6a96 100644 --- a/lib/pleroma/web/activity_pub/mrf/user_allowlist_policy.ex +++ b/lib/pleroma/web/activity_pub/mrf/user_allowlist_policy.ex @@ -32,4 +32,13 @@ def filter(%{"actor" => actor} = object) do end def filter(object), do: {:ok, object} + + @impl true + def describe do + mrf_user_allowlist = + Config.get([:mrf_user_allowlist], []) + |> Enum.into(%{}, fn {k, v} -> {k, length(v)} end) + + {:ok, %{mrf_user_allowlist: mrf_user_allowlist}} + end end diff --git a/lib/pleroma/web/activity_pub/mrf/vocabulary_policy.ex b/lib/pleroma/web/activity_pub/mrf/vocabulary_policy.ex new file mode 100644 index 000000000..4eaea00d8 --- /dev/null +++ b/lib/pleroma/web/activity_pub/mrf/vocabulary_policy.ex @@ -0,0 +1,37 @@ +# Pleroma: A lightweight social networking server +# Copyright © 2017-2019 Pleroma Authors +# SPDX-License-Identifier: AGPL-3.0-only + +defmodule Pleroma.Web.ActivityPub.MRF.VocabularyPolicy do + @moduledoc "Filter messages which belong to certain activity vocabularies" + + @behaviour Pleroma.Web.ActivityPub.MRF + + def filter(%{"type" => "Undo", "object" => child_message} = message) do + with {:ok, _} <- filter(child_message) do + {:ok, message} + else + {:reject, nil} -> + {:reject, nil} + end + end + + def filter(%{"type" => message_type} = message) do + with accepted_vocabulary <- Pleroma.Config.get([:mrf_vocabulary, :accept]), + rejected_vocabulary <- Pleroma.Config.get([:mrf_vocabulary, :reject]), + true <- + length(accepted_vocabulary) == 0 || Enum.member?(accepted_vocabulary, message_type), + false <- + length(rejected_vocabulary) > 0 && Enum.member?(rejected_vocabulary, message_type), + {:ok, _} <- filter(message["object"]) do + {:ok, message} + else + _ -> {:reject, nil} + end + end + + def filter(message), do: {:ok, message} + + def describe, + do: {:ok, %{mrf_vocabulary: Pleroma.Config.get(:mrf_vocabulary) |> Enum.into(%{})}} +end diff --git a/lib/pleroma/web/activity_pub/publisher.ex b/lib/pleroma/web/activity_pub/publisher.ex index 016d78216..262529b84 100644 --- a/lib/pleroma/web/activity_pub/publisher.ex +++ b/lib/pleroma/web/activity_pub/publisher.ex @@ -11,6 +11,8 @@ defmodule Pleroma.Web.ActivityPub.Publisher do alias Pleroma.Web.ActivityPub.Relay alias Pleroma.Web.ActivityPub.Transmogrifier + require Pleroma.Constants + import Pleroma.Web.ActivityPub.Visibility @behaviour Pleroma.Web.Federator.Publisher @@ -44,7 +46,7 @@ def is_representable?(%Activity{} = activity) do """ def publish_one(%{inbox: inbox, json: json, actor: %User{} = actor, id: id} = params) do Logger.info("Federating #{id} to #{inbox}") - host = URI.parse(inbox).host + %{host: host, path: path} = URI.parse(inbox) digest = "SHA-256=" <> (:crypto.hash(:sha256, json) |> Base.encode64()) @@ -54,6 +56,7 @@ def publish_one(%{inbox: inbox, json: json, actor: %User{} = actor, id: id} = pa signature = Pleroma.Signature.sign(actor, %{ + "(request-target)": "post #{path}", host: host, "content-length": byte_size(json), digest: digest, @@ -117,8 +120,6 @@ defp get_cc_ap_ids(ap_id, recipients) do |> Enum.map(& &1.ap_id) end - @as_public "https://www.w3.org/ns/activitystreams#Public" - defp maybe_use_sharedinbox(%User{info: %{source_data: data}}), do: (is_map(data["endpoints"]) && Map.get(data["endpoints"], "sharedInbox")) || data["inbox"] @@ -145,7 +146,7 @@ def determine_inbox( type == "Delete" -> maybe_use_sharedinbox(user) - @as_public in to || @as_public in cc -> + Pleroma.Constants.as_public() in to || Pleroma.Constants.as_public() in cc -> maybe_use_sharedinbox(user) length(to) + length(cc) > 1 -> diff --git a/lib/pleroma/web/activity_pub/relay.ex b/lib/pleroma/web/activity_pub/relay.ex index 1ebfcdd86..5f18cc64a 100644 --- a/lib/pleroma/web/activity_pub/relay.ex +++ b/lib/pleroma/web/activity_pub/relay.ex @@ -14,6 +14,7 @@ def get_actor do |> User.get_or_create_service_actor_by_ap_id() end + @spec follow(String.t()) :: {:ok, Activity.t()} | {:error, any()} def follow(target_instance) do with %User{} = local_user <- get_actor(), {:ok, %User{} = target_user} <- User.get_or_fetch_by_ap_id(target_instance), @@ -21,12 +22,17 @@ def follow(target_instance) do Logger.info("relay: followed instance: #{target_instance}; id=#{activity.data["id"]}") {:ok, activity} else + {:error, _} = error -> + Logger.error("error: #{inspect(error)}") + error + e -> Logger.error("error: #{inspect(e)}") {:error, e} end end + @spec unfollow(String.t()) :: {:ok, Activity.t()} | {:error, any()} def unfollow(target_instance) do with %User{} = local_user <- get_actor(), {:ok, %User{} = target_user} <- User.get_or_fetch_by_ap_id(target_instance), @@ -34,20 +40,27 @@ def unfollow(target_instance) do Logger.info("relay: unfollowed instance: #{target_instance}: id=#{activity.data["id"]}") {:ok, activity} else + {:error, _} = error -> + Logger.error("error: #{inspect(error)}") + error + e -> Logger.error("error: #{inspect(e)}") {:error, e} end end + @spec publish(any()) :: {:ok, Activity.t(), Object.t()} | {:error, any()} def publish(%Activity{data: %{"type" => "Create"}} = activity) do with %User{} = user <- get_actor(), %Object{} = object <- Object.normalize(activity) do ActivityPub.announce(user, object, nil, true, false) else - e -> Logger.error("error: #{inspect(e)}") + e -> + Logger.error("error: #{inspect(e)}") + {:error, inspect(e)} end end - def publish(_), do: nil + def publish(_), do: {:error, "Not implemented"} end diff --git a/lib/pleroma/web/activity_pub/transmogrifier.ex b/lib/pleroma/web/activity_pub/transmogrifier.ex index 602ae48e1..36340a3a1 100644 --- a/lib/pleroma/web/activity_pub/transmogrifier.ex +++ b/lib/pleroma/web/activity_pub/transmogrifier.ex @@ -19,12 +19,14 @@ defmodule Pleroma.Web.ActivityPub.Transmogrifier do import Ecto.Query require Logger + require Pleroma.Constants @doc """ Modifies an incoming AP object (mastodon format) to our internal format. """ def fix_object(object, options \\ []) do object + |> strip_internal_fields |> fix_actor |> fix_url |> fix_attachments @@ -33,7 +35,6 @@ def fix_object(object, options \\ []) do |> fix_emoji |> fix_tag |> fix_content_map - |> fix_likes |> fix_addressing |> fix_summary |> fix_type(options) @@ -102,8 +103,7 @@ def fix_explicit_addressing(object) do follower_collection = User.get_cached_by_ap_id(Containment.get_actor(object)).follower_address - explicit_mentions = - explicit_mentions ++ ["https://www.w3.org/ns/activitystreams#Public", follower_collection] + explicit_mentions = explicit_mentions ++ [Pleroma.Constants.as_public(), follower_collection] fix_explicit_addressing(object, explicit_mentions, follower_collection) end @@ -115,11 +115,11 @@ def fix_implicit_addressing(%{"to" => to, "cc" => cc} = object, followers_collec if followers_collection not in recipients do cond do - "https://www.w3.org/ns/activitystreams#Public" in cc -> + Pleroma.Constants.as_public() in cc -> to = to ++ [followers_collection] Map.put(object, "to", to) - "https://www.w3.org/ns/activitystreams#Public" in to -> + Pleroma.Constants.as_public() in to -> cc = cc ++ [followers_collection] Map.put(object, "cc", cc) @@ -151,20 +151,6 @@ def fix_actor(%{"attributedTo" => actor} = object) do |> Map.put("actor", Containment.get_actor(%{"actor" => actor})) end - # Check for standardisation - # This is what Peertube does - # curl -H 'Accept: application/activity+json' $likes | jq .totalItems - # Prismo returns only an integer (count) as "likes" - def fix_likes(%{"likes" => likes} = object) when not is_map(likes) do - object - |> Map.put("likes", []) - |> Map.put("like_count", 0) - end - - def fix_likes(object) do - object - end - def fix_in_reply_to(object, options \\ []) def fix_in_reply_to(%{"inReplyTo" => in_reply_to} = object, options) @@ -347,13 +333,15 @@ def fix_content_map(object), do: object def fix_type(object, options \\ []) - def fix_type(%{"inReplyTo" => reply_id} = object, options) when is_binary(reply_id) do + def fix_type(%{"inReplyTo" => reply_id, "name" => _} = object, options) + when is_binary(reply_id) do reply = - if Federator.allowed_incoming_reply_depth?(options[:depth]) do - Object.normalize(reply_id, true) + with true <- Federator.allowed_incoming_reply_depth?(options[:depth]), + {:ok, object} <- get_obj_helper(reply_id, options) do + object end - if reply && (reply.data["type"] == "Question" and object["name"]) do + if reply && reply.data["type"] == "Question" do Map.put(object, "type", "Answer") else object @@ -480,8 +468,7 @@ def handle_incoming( {:ok, %User{} = follower} <- User.get_or_fetch_by_ap_id(follower), {:ok, activity} <- ActivityPub.follow(follower, followed, id, false) do with deny_follow_blocked <- Pleroma.Config.get([:user, :deny_follow_blocked]), - {_, false} <- - {:user_blocked, User.blocks?(followed, follower) && deny_follow_blocked}, + {_, false} <- {:user_blocked, User.blocks?(followed, follower) && deny_follow_blocked}, {_, false} <- {:user_locked, User.locked?(followed)}, {_, {:ok, follower}} <- {:follow, User.follow(follower, followed)}, {_, {:ok, _}} <- @@ -609,16 +596,22 @@ def handle_incoming( with %User{ap_id: ^actor_id} = actor <- User.get_cached_by_ap_id(object["id"]) do {:ok, new_user_data} = ActivityPub.user_data_from_user_object(object) - banner = new_user_data[:info]["banner"] - locked = new_user_data[:info]["locked"] || false + banner = new_user_data[:info][:banner] + locked = new_user_data[:info][:locked] || false + attachment = get_in(new_user_data, [:info, :source_data, "attachment"]) || [] + + fields = + attachment + |> Enum.filter(fn %{"type" => t} -> t == "PropertyValue" end) + |> Enum.map(fn fields -> Map.take(fields, ["name", "value"]) end) update_data = new_user_data |> Map.take([:name, :bio, :avatar]) - |> Map.put(:info, %{"banner" => banner, "locked" => locked}) + |> Map.put(:info, %{banner: banner, locked: locked, fields: fields}) actor - |> User.upgrade_changeset(update_data) + |> User.upgrade_changeset(update_data, true) |> User.update_and_set_cache() ActivityPub.update(%{ @@ -656,20 +649,7 @@ def handle_incoming( nil -> case User.get_cached_by_ap_id(object_id) do %User{ap_id: ^actor} = user -> - {:ok, followers} = User.get_followers(user) - - Enum.each(followers, fn follower -> - User.unfollow(follower, user) - end) - - {:ok, friends} = User.get_friends(user) - - Enum.each(friends, fn followed -> - User.unfollow(user, followed) - end) - - User.invalidate_cache(user) - Repo.delete(user) + User.delete(user) nil -> :error @@ -727,8 +707,7 @@ def handle_incoming( } = _data, _options ) do - with true <- Pleroma.Config.get([:activitypub, :accept_blocks]), - %User{local: true} = blocked <- User.get_cached_by_ap_id(blocked), + with %User{local: true} = blocked <- User.get_cached_by_ap_id(blocked), {:ok, %User{} = blocker} <- User.get_or_fetch_by_ap_id(blocker), {:ok, activity} <- ActivityPub.unblock(blocker, blocked, id, false) do User.unblock(blocker, blocked) @@ -742,8 +721,7 @@ def handle_incoming( %{"type" => "Block", "object" => blocked, "actor" => blocker, "id" => id} = _data, _options ) do - with true <- Pleroma.Config.get([:activitypub, :accept_blocks]), - %User{local: true} = blocked = User.get_cached_by_ap_id(blocked), + with %User{local: true} = blocked = User.get_cached_by_ap_id(blocked), {:ok, %User{} = blocker} = User.get_or_fetch_by_ap_id(blocker), {:ok, activity} <- ActivityPub.block(blocker, blocked, id, false) do User.unfollow(blocker, blocked) @@ -798,7 +776,6 @@ def prepare_object(object) do |> add_mention_tags |> add_emoji_tags |> add_attributed_to - |> add_likes |> prepare_attachments |> set_conversation |> set_reply_to_uri @@ -985,22 +962,6 @@ def add_attributed_to(object) do |> Map.put("attributedTo", attributed_to) end - def add_likes(%{"id" => id, "like_count" => likes} = object) do - likes = %{ - "id" => "#{id}/likes", - "first" => "#{id}/likes?page=1", - "type" => "OrderedCollection", - "totalItems" => likes - } - - object - |> Map.put("likes", likes) - end - - def add_likes(object) do - object - end - def prepare_attachments(object) do attachments = (object["attachment"] || []) @@ -1016,6 +977,7 @@ def prepare_attachments(object) do defp strip_internal_fields(object) do object |> Map.drop([ + "likes", "like_count", "announcements", "announcement_count", @@ -1090,10 +1052,6 @@ def upgrade_user_from_ap_id(ap_id) do PleromaJobQueue.enqueue(:transmogrifier, __MODULE__, [:user_upgrade, user]) end - if Pleroma.Config.get([:instance, :external_user_synchronization]) do - update_following_followers_counters(user) - end - {:ok, user} else %User{} = user -> {:ok, user} @@ -1126,27 +1084,4 @@ def maybe_fix_user_object(data) do data |> maybe_fix_user_url end - - def update_following_followers_counters(user) do - info = %{} - - following = fetch_counter(user.following_address) - info = if following, do: Map.put(info, :following_count, following), else: info - - followers = fetch_counter(user.follower_address) - info = if followers, do: Map.put(info, :follower_count, followers), else: info - - User.set_info_cache(user, info) - end - - defp fetch_counter(url) do - with {:ok, %{body: body, status: code}} when code in 200..299 <- - Pleroma.HTTP.get( - url, - [{:Accept, "application/activity+json"}] - ), - {:ok, data} <- Jason.decode(body) do - data["totalItems"] - end - end end diff --git a/lib/pleroma/web/activity_pub/utils.ex b/lib/pleroma/web/activity_pub/utils.ex index c146f59d4..1c3058658 100644 --- a/lib/pleroma/web/activity_pub/utils.ex +++ b/lib/pleroma/web/activity_pub/utils.ex @@ -18,6 +18,7 @@ defmodule Pleroma.Web.ActivityPub.Utils do import Ecto.Query require Logger + require Pleroma.Constants @supported_object_types ["Article", "Note", "Video", "Page", "Question", "Answer"] @supported_report_states ~w(open closed resolved) @@ -250,20 +251,6 @@ def insert_full_object(%{"object" => %{"type" => type} = object_data} = map) def insert_full_object(map), do: {:ok, map, nil} - def update_object_in_activities(%{data: %{"id" => id}} = object) do - # TODO - # Update activities that already had this. Could be done in a seperate process. - # Alternatively, just don't do this and fetch the current object each time. Most - # could probably be taken from cache. - relevant_activities = Activity.get_all_create_by_object_ap_id(id) - - Enum.map(relevant_activities, fn activity -> - new_activity_data = activity.data |> Map.put("object", object.data) - changeset = Changeset.change(activity, data: new_activity_data) - Repo.update(changeset) - end) - end - #### Like-related helpers @doc """ @@ -346,8 +333,7 @@ def update_element_in_object(property, element, object) do |> Map.put("#{property}_count", length(element)) |> Map.put("#{property}s", element), changeset <- Changeset.change(object, data: new_data), - {:ok, object} <- Object.update_and_set_cache(changeset), - _ <- update_object_in_activities(object) do + {:ok, object} <- Object.update_and_set_cache(changeset) do {:ok, object} end end @@ -388,6 +374,7 @@ def update_follow_state_for_all( [state, actor, object] ) + User.set_follow_state_cache(actor, object, state) activity = Activity.get_by_id(activity.id) {:ok, activity} rescue @@ -396,12 +383,16 @@ def update_follow_state_for_all( end end - def update_follow_state(%Activity{} = activity, state) do + def update_follow_state( + %Activity{data: %{"actor" => actor, "object" => object}} = activity, + state + ) do with new_data <- activity.data |> Map.put("state", state), changeset <- Changeset.change(activity, data: new_data), - {:ok, activity} <- Repo.update(changeset) do + {:ok, activity} <- Repo.update(changeset), + _ <- User.set_follow_state_cache(actor, object, state) do {:ok, activity} end end @@ -418,7 +409,7 @@ def make_follow_data( "type" => "Follow", "actor" => follower_id, "to" => [followed_id], - "cc" => ["https://www.w3.org/ns/activitystreams#Public"], + "cc" => [Pleroma.Constants.as_public()], "object" => followed_id, "state" => "pending" } @@ -510,7 +501,7 @@ def make_announce_data( "actor" => ap_id, "object" => id, "to" => [user.follower_address, object.data["actor"]], - "cc" => ["https://www.w3.org/ns/activitystreams#Public"], + "cc" => [Pleroma.Constants.as_public()], "context" => object.data["context"] } @@ -530,7 +521,7 @@ def make_unannounce_data( "actor" => ap_id, "object" => activity.data, "to" => [user.follower_address, activity.data["actor"]], - "cc" => ["https://www.w3.org/ns/activitystreams#Public"], + "cc" => [Pleroma.Constants.as_public()], "context" => context } @@ -547,7 +538,7 @@ def make_unlike_data( "actor" => ap_id, "object" => activity.data, "to" => [user.follower_address, activity.data["actor"]], - "cc" => ["https://www.w3.org/ns/activitystreams#Public"], + "cc" => [Pleroma.Constants.as_public()], "context" => context } @@ -556,7 +547,7 @@ def make_unlike_data( def add_announce_to_object( %Activity{ - data: %{"actor" => actor, "cc" => ["https://www.w3.org/ns/activitystreams#Public"]} + data: %{"actor" => actor, "cc" => [Pleroma.Constants.as_public()]} }, object ) do @@ -765,7 +756,7 @@ defp get_updated_targets( ) do cc = Map.get(data, "cc", []) follower_address = User.get_cached_by_ap_id(data["actor"]).follower_address - public = "https://www.w3.org/ns/activitystreams#Public" + public = Pleroma.Constants.as_public() case visibility do "public" -> diff --git a/lib/pleroma/web/activity_pub/views/object_view.ex b/lib/pleroma/web/activity_pub/views/object_view.ex index 6028b773c..94d05f49b 100644 --- a/lib/pleroma/web/activity_pub/views/object_view.ex +++ b/lib/pleroma/web/activity_pub/views/object_view.ex @@ -66,8 +66,10 @@ def collection(collection, iri, page) do "orderedItems" => items } - if offset < total do + if offset + length(items) < total do Map.put(map, "next", "#{iri}?page=#{page + 1}") + else + map end end end diff --git a/lib/pleroma/web/activity_pub/views/user_view.ex b/lib/pleroma/web/activity_pub/views/user_view.ex index 639519e0a..7be734b26 100644 --- a/lib/pleroma/web/activity_pub/views/user_view.ex +++ b/lib/pleroma/web/activity_pub/views/user_view.ex @@ -65,7 +65,7 @@ def render("user.json", %{user: %User{nickname: nil} = user}), do: render("service.json", %{user: user}) def render("user.json", %{user: %User{nickname: "internal." <> _} = user}), - do: render("service.json", %{user: user}) + do: render("service.json", %{user: user}) |> Map.put("preferredUsername", user.nickname) def render("user.json", %{user: user}) do {:ok, user} = User.ensure_keys_present(user) @@ -80,6 +80,17 @@ def render("user.json", %{user: user}) do |> Transmogrifier.add_emoji_tags() |> Map.get("tag", []) + fields = + user.info + |> User.Info.fields() + |> Enum.map(fn %{"name" => name, "value" => value} -> + %{ + "name" => Pleroma.HTML.strip_tags(name), + "value" => Pleroma.HTML.filter_tags(value, Pleroma.HTML.Scrubber.LinksOnly) + } + end) + |> Enum.map(&Map.put(&1, "type", "PropertyValue")) + %{ "id" => user.ap_id, "type" => "Person", @@ -98,6 +109,7 @@ def render("user.json", %{user: user}) do "publicKeyPem" => public_key }, "endpoints" => endpoints, + "attachment" => fields, "tag" => (user.info.source_data["tag"] || []) ++ user_tags } |> Map.merge(maybe_make_image(&User.avatar_url/2, "icon", user)) diff --git a/lib/pleroma/web/activity_pub/visibility.ex b/lib/pleroma/web/activity_pub/visibility.ex index 097fceb08..dfb166b65 100644 --- a/lib/pleroma/web/activity_pub/visibility.ex +++ b/lib/pleroma/web/activity_pub/visibility.ex @@ -8,14 +8,14 @@ defmodule Pleroma.Web.ActivityPub.Visibility do alias Pleroma.Repo alias Pleroma.User - @public "https://www.w3.org/ns/activitystreams#Public" + require Pleroma.Constants @spec is_public?(Object.t() | Activity.t() | map()) :: boolean() def is_public?(%Object{data: %{"type" => "Tombstone"}}), do: false def is_public?(%Object{data: data}), do: is_public?(data) def is_public?(%Activity{data: data}), do: is_public?(data) def is_public?(%{"directMessage" => true}), do: false - def is_public?(data), do: @public in (data["to"] ++ (data["cc"] || [])) + def is_public?(data), do: Pleroma.Constants.as_public() in (data["to"] ++ (data["cc"] || [])) def is_private?(activity) do with false <- is_public?(activity), @@ -73,10 +73,10 @@ def get_visibility(object) do cc = object.data["cc"] || [] cond do - @public in to -> + Pleroma.Constants.as_public() in to -> "public" - @public in cc -> + Pleroma.Constants.as_public() in cc -> "unlisted" # this should use the sql for the object's activity diff --git a/lib/pleroma/web/admin_api/admin_api_controller.ex b/lib/pleroma/web/admin_api/admin_api_controller.ex index 1ae5acd91..2d3d0adc4 100644 --- a/lib/pleroma/web/admin_api/admin_api_controller.ex +++ b/lib/pleroma/web/admin_api/admin_api_controller.ex @@ -379,6 +379,16 @@ def status_delete(%{assigns: %{user: user}} = conn, %{"id" => id}) do end end + def migrate_to_db(conn, _params) do + Mix.Tasks.Pleroma.Config.run(["migrate_to_db"]) + json(conn, %{}) + end + + def migrate_from_db(conn, _params) do + Mix.Tasks.Pleroma.Config.run(["migrate_from_db", Pleroma.Config.get(:env), "true"]) + json(conn, %{}) + end + def config_show(conn, _params) do configs = Pleroma.Repo.all(Config) @@ -392,9 +402,9 @@ def config_update(conn, %{"configs" => configs}) do if Pleroma.Config.get([:instance, :dynamic_configuration]) do updated = Enum.map(configs, fn - %{"group" => group, "key" => key, "delete" => "true"} -> - {:ok, _} = Config.delete(%{group: group, key: key}) - nil + %{"group" => group, "key" => key, "delete" => "true"} = params -> + {:ok, config} = Config.delete(%{group: group, key: key, subkeys: params["subkeys"]}) + config %{"group" => group, "key" => key, "value" => value} -> {:ok, config} = Config.update_or_create(%{group: group, key: key, value: value}) diff --git a/lib/pleroma/web/admin_api/config.ex b/lib/pleroma/web/admin_api/config.ex index dde05ea7b..a10cc779b 100644 --- a/lib/pleroma/web/admin_api/config.ex +++ b/lib/pleroma/web/admin_api/config.ex @@ -55,8 +55,19 @@ def update_or_create(params) do @spec delete(map()) :: {:ok, Config.t()} | {:error, Changeset.t()} def delete(params) do - with %Config{} = config <- Config.get_by_params(params) do - Repo.delete(config) + with %Config{} = config <- Config.get_by_params(Map.delete(params, :subkeys)) do + if params[:subkeys] do + updated_value = + Keyword.drop( + :erlang.binary_to_term(config.value), + Enum.map(params[:subkeys], &do_transform_string(&1)) + ) + + Config.update(config, %{value: updated_value}) + else + Repo.delete(config) + {:ok, nil} + end else nil -> err = diff --git a/lib/pleroma/web/auth/authenticator.ex b/lib/pleroma/web/auth/authenticator.ex index d4e0ffa80..dd49987f7 100644 --- a/lib/pleroma/web/auth/authenticator.ex +++ b/lib/pleroma/web/auth/authenticator.ex @@ -21,8 +21,7 @@ def get_user(plug), do: implementation().get_user(plug) def create_from_registration(plug, registration), do: implementation().create_from_registration(plug, registration) - @callback get_registration(Plug.Conn.t()) :: - {:ok, Registration.t()} | {:error, any()} + @callback get_registration(Plug.Conn.t()) :: {:ok, Registration.t()} | {:error, any()} def get_registration(plug), do: implementation().get_registration(plug) @callback handle_error(Plug.Conn.t(), any()) :: any() diff --git a/lib/pleroma/web/chat_channel.ex b/lib/pleroma/web/chat_channel.ex index f63f4bda1..b543909f1 100644 --- a/lib/pleroma/web/chat_channel.ex +++ b/lib/pleroma/web/chat_channel.ex @@ -33,9 +33,11 @@ def handle_in("new_msg", %{"text" => text}, %{assigns: %{user_name: user_name}} end defmodule Pleroma.Web.ChatChannel.ChatChannelState do + use Agent + @max_messages 20 - def start_link do + def start_link(_) do Agent.start_link(fn -> %{max_id: 1, messages: []} end, name: __MODULE__) end diff --git a/lib/pleroma/web/common_api/common_api.ex b/lib/pleroma/web/common_api/common_api.ex index 261d60392..69120cc19 100644 --- a/lib/pleroma/web/common_api/common_api.ex +++ b/lib/pleroma/web/common_api/common_api.ex @@ -5,6 +5,7 @@ defmodule Pleroma.Web.CommonAPI do alias Pleroma.Activity alias Pleroma.ActivityExpiration + alias Pleroma.Conversation.Participation alias Pleroma.Formatter alias Pleroma.Object alias Pleroma.ThreadMute @@ -172,21 +173,25 @@ defp normalize_and_validate_choice_indices(choices, count) do end) end - def get_visibility(%{"visibility" => visibility}, in_reply_to) + def get_visibility(_, _, %Participation{}) do + {"direct", "direct"} + end + + def get_visibility(%{"visibility" => visibility}, in_reply_to, _) when visibility in ~w{public unlisted private direct}, do: {visibility, get_replied_to_visibility(in_reply_to)} - def get_visibility(%{"visibility" => "list:" <> list_id}, in_reply_to) do + def get_visibility(%{"visibility" => "list:" <> list_id}, in_reply_to, _) do visibility = {:list, String.to_integer(list_id)} {visibility, get_replied_to_visibility(in_reply_to)} end - def get_visibility(_, in_reply_to) when not is_nil(in_reply_to) do + def get_visibility(_, in_reply_to, _) when not is_nil(in_reply_to) do visibility = get_replied_to_visibility(in_reply_to) {visibility, visibility} end - def get_visibility(_, in_reply_to), do: {"public", get_replied_to_visibility(in_reply_to)} + def get_visibility(_, in_reply_to, _), do: {"public", get_replied_to_visibility(in_reply_to)} def get_replied_to_visibility(nil), do: nil @@ -212,7 +217,9 @@ def post(user, %{"status" => status} = data) do with status <- String.trim(status), attachments <- attachments_from_ids(data), in_reply_to <- get_replied_to_activity(data["in_reply_to_status_id"]), - {visibility, in_reply_to_visibility} <- get_visibility(data, in_reply_to), + in_reply_to_conversation <- Participation.get(data["in_reply_to_conversation_id"]), + {visibility, in_reply_to_visibility} <- + get_visibility(data, in_reply_to, in_reply_to_conversation), {_, false} <- {:private_to_public, in_reply_to_visibility == "direct" && visibility != "direct"}, {content_html, mentions, tags} <- @@ -225,8 +232,9 @@ def post(user, %{"status" => status} = data) do mentioned_users <- for({_, mentioned_user} <- mentions, do: mentioned_user.ap_id), addressed_users <- get_addressed_users(mentioned_users, data["to"]), {poll, poll_emoji} <- make_poll_data(data), - {to, cc} <- get_to_and_cc(user, addressed_users, in_reply_to, visibility), - context <- make_context(in_reply_to), + {to, cc} <- + get_to_and_cc(user, addressed_users, in_reply_to, visibility, in_reply_to_conversation), + context <- make_context(in_reply_to, in_reply_to_conversation), cw <- data["spoiler_text"] || "", sensitive <- data["sensitive"] || Enum.member?(tags, {"#nsfw", "nsfw"}), {:ok, expires_at} <- check_expiry_date(data["expires_at"]), @@ -321,8 +329,7 @@ def pin(id_or_ap_id, %{ap_id: user_ap_id} = user) do } } = activity <- get_by_id_or_ap_id(id_or_ap_id), true <- Visibility.is_public?(activity), - %{valid?: true} = info_changeset <- - User.Info.add_pinnned_activity(user.info, activity), + %{valid?: true} = info_changeset <- User.Info.add_pinnned_activity(user.info, activity), changeset <- Ecto.Changeset.change(user) |> Ecto.Changeset.put_embed(:info, info_changeset), {:ok, _user} <- User.update_and_set_cache(changeset) do diff --git a/lib/pleroma/web/common_api/utils.ex b/lib/pleroma/web/common_api/utils.ex index 94462c3dd..61b96aba9 100644 --- a/lib/pleroma/web/common_api/utils.ex +++ b/lib/pleroma/web/common_api/utils.ex @@ -8,6 +8,7 @@ defmodule Pleroma.Web.CommonAPI.Utils do alias Calendar.Strftime alias Pleroma.Activity alias Pleroma.Config + alias Pleroma.Conversation.Participation alias Pleroma.Formatter alias Pleroma.Object alias Pleroma.Plugs.AuthenticationPlug @@ -19,11 +20,17 @@ defmodule Pleroma.Web.CommonAPI.Utils do alias Pleroma.Web.MediaProxy require Logger + require Pleroma.Constants # This is a hack for twidere. def get_by_id_or_ap_id(id) do activity = - Activity.get_by_id_with_object(id) || Activity.get_create_by_object_ap_id_with_object(id) + with true <- Pleroma.FlakeId.is_flake_id?(id), + %Activity{} = activity <- Activity.get_by_id_with_object(id) do + activity + else + _ -> Activity.get_create_by_object_ap_id_with_object(id) + end activity && if activity.data["type"] == "Create" do @@ -41,32 +48,61 @@ def get_replied_to_activity(id) when not is_nil(id) do def get_replied_to_activity(_), do: nil - def attachments_from_ids(data) do - if Map.has_key?(data, "descriptions") do - attachments_from_ids_descs(data["media_ids"], data["descriptions"]) - else - attachments_from_ids_no_descs(data["media_ids"]) - end + def attachments_from_ids(%{"media_ids" => ids, "descriptions" => desc} = _) do + attachments_from_ids_descs(ids, desc) end - def attachments_from_ids_no_descs(ids) do - Enum.map(ids || [], fn media_id -> - Repo.get(Object, media_id).data - end) + def attachments_from_ids(%{"media_ids" => ids} = _) do + attachments_from_ids_no_descs(ids) end + def attachments_from_ids(_), do: [] + + def attachments_from_ids_no_descs([]), do: [] + + def attachments_from_ids_no_descs(ids) do + Enum.map(ids, fn media_id -> + case Repo.get(Object, media_id) do + %Object{data: data} = _ -> data + _ -> nil + end + end) + |> Enum.filter(& &1) + end + + def attachments_from_ids_descs([], _), do: [] + def attachments_from_ids_descs(ids, descs_str) do {_, descs} = Jason.decode(descs_str) - Enum.map(ids || [], fn media_id -> - Map.put(Repo.get(Object, media_id).data, "name", descs[media_id]) + Enum.map(ids, fn media_id -> + case Repo.get(Object, media_id) do + %Object{data: data} = _ -> + Map.put(data, "name", descs[media_id]) + + _ -> + nil + end end) + |> Enum.filter(& &1) end - @spec get_to_and_cc(User.t(), list(String.t()), Activity.t() | nil, String.t()) :: + @spec get_to_and_cc( + User.t(), + list(String.t()), + Activity.t() | nil, + String.t(), + Participation.t() | nil + ) :: {list(String.t()), list(String.t())} - def get_to_and_cc(user, mentioned_users, inReplyTo, "public") do - to = ["https://www.w3.org/ns/activitystreams#Public" | mentioned_users] + + def get_to_and_cc(_, _, _, _, %Participation{} = participation) do + participation = Repo.preload(participation, :recipients) + {Enum.map(participation.recipients, & &1.ap_id), []} + end + + def get_to_and_cc(user, mentioned_users, inReplyTo, "public", _) do + to = [Pleroma.Constants.as_public() | mentioned_users] cc = [user.follower_address] if inReplyTo do @@ -76,9 +112,9 @@ def get_to_and_cc(user, mentioned_users, inReplyTo, "public") do end end - def get_to_and_cc(user, mentioned_users, inReplyTo, "unlisted") do + def get_to_and_cc(user, mentioned_users, inReplyTo, "unlisted", _) do to = [user.follower_address | mentioned_users] - cc = ["https://www.w3.org/ns/activitystreams#Public"] + cc = [Pleroma.Constants.as_public()] if inReplyTo do {Enum.uniq([inReplyTo.data["actor"] | to]), cc} @@ -87,12 +123,12 @@ def get_to_and_cc(user, mentioned_users, inReplyTo, "unlisted") do end end - def get_to_and_cc(user, mentioned_users, inReplyTo, "private") do - {to, cc} = get_to_and_cc(user, mentioned_users, inReplyTo, "direct") + def get_to_and_cc(user, mentioned_users, inReplyTo, "private", _) do + {to, cc} = get_to_and_cc(user, mentioned_users, inReplyTo, "direct", nil) {[user.follower_address | to], cc} end - def get_to_and_cc(_user, mentioned_users, inReplyTo, "direct") do + def get_to_and_cc(_user, mentioned_users, inReplyTo, "direct", _) do if inReplyTo do {Enum.uniq([inReplyTo.data["actor"] | mentioned_users]), []} else @@ -100,7 +136,7 @@ def get_to_and_cc(_user, mentioned_users, inReplyTo, "direct") do end end - def get_to_and_cc(_user, mentions, _inReplyTo, {:list, _}), do: {mentions, []} + def get_to_and_cc(_user, mentions, _inReplyTo, {:list, _}, _), do: {mentions, []} def get_addressed_users(_, to) when is_list(to) do User.get_ap_ids_by_nicknames(to) @@ -230,8 +266,12 @@ defp maybe_add_nsfw_tag({text, mentions, tags}, %{"sensitive" => sensitive}) defp maybe_add_nsfw_tag(data, _), do: data - def make_context(%Activity{data: %{"context" => context}}), do: context - def make_context(_), do: Utils.generate_context_id() + def make_context(_, %Participation{} = participation) do + Repo.preload(participation, :conversation).conversation.ap_id + end + + def make_context(%Activity{data: %{"context" => context}}, _), do: context + def make_context(_, _), do: Utils.generate_context_id() def maybe_add_attachments(parsed, _attachments, true = _no_links), do: parsed @@ -241,20 +281,18 @@ def maybe_add_attachments({text, mentions, tags}, attachments, _no_links) do end def add_attachments(text, attachments) do - attachment_text = - Enum.map(attachments, fn - %{"url" => [%{"href" => href} | _]} = attachment -> - name = attachment["name"] || URI.decode(Path.basename(href)) - href = MediaProxy.url(href) - "#{shortname(name)}" - - _ -> - "" - end) - + attachment_text = Enum.map(attachments, &build_attachment_link/1) Enum.join([text | attachment_text], "
") end + defp build_attachment_link(%{"url" => [%{"href" => href} | _]} = attachment) do + name = attachment["name"] || URI.decode(Path.basename(href)) + href = MediaProxy.url(href) + "#{shortname(name)}" + end + + defp build_attachment_link(_), do: "" + def format_input(text, format, options \\ []) @doc """ @@ -314,7 +352,7 @@ def make_note_data( sensitive \\ false, merge \\ %{} ) do - object = %{ + %{ "type" => "Note", "to" => to, "cc" => cc, @@ -324,18 +362,20 @@ def make_note_data( "context" => context, "attachment" => attachments, "actor" => actor, - "tag" => tags |> Enum.map(fn {_, tag} -> tag end) |> Enum.uniq() + "tag" => Keyword.values(tags) |> Enum.uniq() } + |> add_in_reply_to(in_reply_to) + |> Map.merge(merge) + end - object = - with false <- is_nil(in_reply_to), - %Object{} = in_reply_to_object <- Object.normalize(in_reply_to) do - Map.put(object, "inReplyTo", in_reply_to_object.data["id"]) - else - _ -> object - end + defp add_in_reply_to(object, nil), do: object - Map.merge(object, merge) + defp add_in_reply_to(object, in_reply_to) do + with %Object{} = in_reply_to_object <- Object.normalize(in_reply_to) do + Map.put(object, "inReplyTo", in_reply_to_object.data["id"]) + else + _ -> object + end end def format_naive_asctime(date) do @@ -367,17 +407,16 @@ def to_masto_date(%NaiveDateTime{} = date) do |> String.replace(~r/(\.\d+)?$/, ".000Z", global: false) end - def to_masto_date(date) do - try do - date - |> NaiveDateTime.from_iso8601!() - |> NaiveDateTime.to_iso8601() - |> String.replace(~r/(\.\d+)?$/, ".000Z", global: false) - rescue - _e -> "" + def to_masto_date(date) when is_binary(date) do + with {:ok, date} <- NaiveDateTime.from_iso8601(date) do + to_masto_date(date) + else + _ -> "" end end + def to_masto_date(_), do: "" + defp shortname(name) do if String.length(name) < 30 do name @@ -422,7 +461,7 @@ def maybe_notify_mentioned_recipients( object_data = cond do - !is_nil(object) -> + not is_nil(object) -> object.data is_map(data["object"]) -> @@ -466,9 +505,9 @@ def maybe_notify_subscribers(recipients, _), do: recipients def maybe_extract_mentions(%{"tag" => tag}) do tag - |> Enum.filter(fn x -> is_map(x) end) - |> Enum.filter(fn x -> x["type"] == "Mention" end) + |> Enum.filter(fn x -> is_map(x) && x["type"] == "Mention" end) |> Enum.map(fn x -> x["href"] end) + |> Enum.uniq() end def maybe_extract_mentions(_), do: [] diff --git a/lib/pleroma/web/controller_helper.ex b/lib/pleroma/web/controller_helper.ex index 8a753bb4f..eeac9f503 100644 --- a/lib/pleroma/web/controller_helper.ex +++ b/lib/pleroma/web/controller_helper.ex @@ -33,4 +33,80 @@ defp param_to_integer(val, default) when is_binary(val) do end defp param_to_integer(_, default), do: default + + def add_link_headers( + conn, + method, + activities, + param \\ nil, + params \\ %{}, + func3 \\ nil, + func4 \\ nil + ) do + params = + conn.params + |> Map.drop(["since_id", "max_id", "min_id"]) + |> Map.merge(params) + + last = List.last(activities) + + func3 = func3 || (&mastodon_api_url/3) + func4 = func4 || (&mastodon_api_url/4) + + if last do + max_id = last.id + + limit = + params + |> Map.get("limit", "20") + |> String.to_integer() + + min_id = + if length(activities) <= limit do + activities + |> List.first() + |> Map.get(:id) + else + activities + |> Enum.at(limit * -1) + |> Map.get(:id) + end + + {next_url, prev_url} = + if param do + { + func4.( + Pleroma.Web.Endpoint, + method, + param, + Map.merge(params, %{max_id: max_id}) + ), + func4.( + Pleroma.Web.Endpoint, + method, + param, + Map.merge(params, %{min_id: min_id}) + ) + } + else + { + func3.( + Pleroma.Web.Endpoint, + method, + Map.merge(params, %{max_id: max_id}) + ), + func3.( + Pleroma.Web.Endpoint, + method, + Map.merge(params, %{min_id: min_id}) + ) + } + end + + conn + |> put_resp_header("link", "<#{next_url}>; rel=\"next\", <#{prev_url}>; rel=\"prev\"") + else + conn + end + end end diff --git a/lib/pleroma/web/fallback_redirect_controller.ex b/lib/pleroma/web/fallback_redirect_controller.ex new file mode 100644 index 000000000..5fbf3695f --- /dev/null +++ b/lib/pleroma/web/fallback_redirect_controller.ex @@ -0,0 +1,77 @@ +# Pleroma: A lightweight social networking server +# Copyright © 2017-2019 Pleroma Authors +# SPDX-License-Identifier: AGPL-3.0-only + +defmodule Fallback.RedirectController do + use Pleroma.Web, :controller + require Logger + alias Pleroma.User + alias Pleroma.Web.Metadata + + def api_not_implemented(conn, _params) do + conn + |> put_status(404) + |> json(%{error: "Not implemented"}) + end + + def redirector(conn, _params, code \\ 200) + + # redirect to admin section + # /pleroma/admin -> /pleroma/admin/ + # + def redirector(conn, %{"path" => ["pleroma", "admin"]} = _, _code) do + redirect(conn, to: "/pleroma/admin/") + end + + def redirector(conn, _params, code) do + conn + |> put_resp_content_type("text/html") + |> send_file(code, index_file_path()) + end + + def redirector_with_meta(conn, %{"maybe_nickname_or_id" => maybe_nickname_or_id} = params) do + with %User{} = user <- User.get_cached_by_nickname_or_id(maybe_nickname_or_id) do + redirector_with_meta(conn, %{user: user}) + else + nil -> + redirector(conn, params) + end + end + + def redirector_with_meta(conn, params) do + {:ok, index_content} = File.read(index_file_path()) + + tags = + try do + Metadata.build_tags(params) + rescue + e -> + Logger.error( + "Metadata rendering for #{conn.request_path} failed.\n" <> + Exception.format(:error, e, __STACKTRACE__) + ) + + "" + end + + response = String.replace(index_content, "", tags) + + conn + |> put_resp_content_type("text/html") + |> send_resp(200, response) + end + + def index_file_path do + Pleroma.Plugs.InstanceStatic.file_path("index.html") + end + + def registration_page(conn, params) do + redirector(conn, params) + end + + def empty(conn, _params) do + conn + |> put_status(204) + |> text("") + end +end diff --git a/lib/pleroma/web/federator/retry_queue.ex b/lib/pleroma/web/federator/retry_queue.ex index 3db948c2e..9eab8c218 100644 --- a/lib/pleroma/web/federator/retry_queue.ex +++ b/lib/pleroma/web/federator/retry_queue.ex @@ -13,7 +13,7 @@ def init(args) do {:ok, %{args | queue_table: queue_table, running_jobs: :sets.new()}} end - def start_link do + def start_link(_) do enabled = if Pleroma.Config.get(:env) == :test, do: true, diff --git a/lib/pleroma/web/mailer/subscription_controller.ex b/lib/pleroma/web/mailer/subscription_controller.ex new file mode 100644 index 000000000..478a83518 --- /dev/null +++ b/lib/pleroma/web/mailer/subscription_controller.ex @@ -0,0 +1,20 @@ +defmodule Pleroma.Web.Mailer.SubscriptionController do + use Pleroma.Web, :controller + + alias Pleroma.JWT + alias Pleroma.Repo + alias Pleroma.User + + def unsubscribe(conn, %{"token" => encoded_token}) do + with {:ok, token} <- Base.decode64(encoded_token), + {:ok, claims} <- JWT.verify_and_validate(token), + %{"act" => %{"unsubscribe" => type}, "sub" => uid} <- claims, + %User{} = user <- Repo.get(User, uid), + {:ok, _user} <- User.switch_email_notifications(user, type, false) do + render(conn, "unsubscribe_success.html", email: user.email) + else + _err -> + render(conn, "unsubscribe_failure.html") + end + end +end diff --git a/lib/pleroma/web/mastodon_api/mastodon_api.ex b/lib/pleroma/web/mastodon_api/mastodon_api.ex index 46944dcbc..ac01d1ff3 100644 --- a/lib/pleroma/web/mastodon_api/mastodon_api.ex +++ b/lib/pleroma/web/mastodon_api/mastodon_api.ex @@ -13,10 +13,8 @@ defmodule Pleroma.Web.MastodonAPI.MastodonAPI do alias Pleroma.User alias Pleroma.Web.CommonAPI + @spec follow(User.t(), User.t(), map) :: {:ok, User.t()} | {:error, String.t()} def follow(follower, followed, params \\ %{}) do - options = cast_params(params) - reblogs = options[:reblogs] - result = if not User.following?(follower, followed) do CommonAPI.follow(follower, followed) @@ -24,19 +22,25 @@ def follow(follower, followed, params \\ %{}) do {:ok, follower, followed, nil} end - with {:ok, follower, followed, _} <- result do - reblogs - |> case do - false -> CommonAPI.hide_reblogs(follower, followed) - _ -> CommonAPI.show_reblogs(follower, followed) - end - |> case do + with {:ok, follower, _followed, _} <- result do + options = cast_params(params) + + case reblogs_visibility(options[:reblogs], result) do {:ok, follower} -> {:ok, follower} _ -> {:ok, follower} end end end + defp reblogs_visibility(false, {:ok, follower, followed, _}) do + CommonAPI.hide_reblogs(follower, followed) + end + + defp reblogs_visibility(_, {:ok, follower, followed, _}) do + CommonAPI.show_reblogs(follower, followed) + end + + @spec get_followers(User.t(), map()) :: list(User.t()) def get_followers(user, params \\ %{}) do user |> User.get_followers_query() diff --git a/lib/pleroma/web/mastodon_api/mastodon_api_controller.ex b/lib/pleroma/web/mastodon_api/mastodon_api_controller.ex index d660f3f05..53cf95fbb 100644 --- a/lib/pleroma/web/mastodon_api/mastodon_api_controller.ex +++ b/lib/pleroma/web/mastodon_api/mastodon_api_controller.ex @@ -4,6 +4,10 @@ defmodule Pleroma.Web.MastodonAPI.MastodonAPIController do use Pleroma.Web, :controller + + import Pleroma.Web.ControllerHelper, + only: [json_response: 3, add_link_headers: 5, add_link_headers: 4, add_link_headers: 3] + alias Ecto.Changeset alias Pleroma.Activity alias Pleroma.Bookmark @@ -46,6 +50,7 @@ defmodule Pleroma.Web.MastodonAPI.MastodonAPIController do import Ecto.Query require Logger + require Pleroma.Constants @rate_limited_relations_actions ~w(follow unfollow)a @@ -74,6 +79,7 @@ defmodule Pleroma.Web.MastodonAPI.MastodonAPIController do plug(RateLimiter, :app_account_creation when action == :account_register) plug(RateLimiter, :search when action in [:search, :search2, :account_search]) plug(RateLimiter, :password_reset when action == :password_reset) + plug(RateLimiter, :account_confirmation_resend when action == :account_confirmation_resend) @local_mastodon_name "Mastodon-Local" @@ -132,7 +138,9 @@ def update_credentials(%{assigns: %{user: user}} = conn, params) do emojis_text = (user_params["display_name"] || "") <> (user_params["note"] || "") user_info_emojis = - ((user.info.emoji || []) ++ Formatter.get_emoji_map(emojis_text)) + user.info + |> Map.get(:emoji, []) + |> Enum.concat(Formatter.get_emoji_map(emojis_text)) |> Enum.dedup() info_params = @@ -151,6 +159,12 @@ def update_credentials(%{assigns: %{user: user}} = conn, params) do end) end) |> add_if_present(params, "default_scope", :default_scope) + |> add_if_present(params, "fields", :fields, fn fields -> + fields = Enum.map(fields, fn f -> Map.update!(f, "value", &AutoLinker.link(&1)) end) + + {:ok, fields} + end) + |> add_if_present(params, "fields", :raw_fields) |> add_if_present(params, "pleroma_settings_store", :pleroma_settings_store, fn value -> {:ok, Map.merge(user.info.pleroma_settings_store, value)} end) @@ -337,71 +351,6 @@ def custom_emojis(conn, _params) do json(conn, mastodon_emoji) end - defp add_link_headers(conn, method, activities, param \\ nil, params \\ %{}) do - params = - conn.params - |> Map.drop(["since_id", "max_id", "min_id"]) - |> Map.merge(params) - - last = List.last(activities) - - if last do - max_id = last.id - - limit = - params - |> Map.get("limit", "20") - |> String.to_integer() - - min_id = - if length(activities) <= limit do - activities - |> List.first() - |> Map.get(:id) - else - activities - |> Enum.at(limit * -1) - |> Map.get(:id) - end - - {next_url, prev_url} = - if param do - { - mastodon_api_url( - Pleroma.Web.Endpoint, - method, - param, - Map.merge(params, %{max_id: max_id}) - ), - mastodon_api_url( - Pleroma.Web.Endpoint, - method, - param, - Map.merge(params, %{min_id: min_id}) - ) - } - else - { - mastodon_api_url( - Pleroma.Web.Endpoint, - method, - Map.merge(params, %{max_id: max_id}) - ), - mastodon_api_url( - Pleroma.Web.Endpoint, - method, - Map.merge(params, %{min_id: min_id}) - ) - } - end - - conn - |> put_resp_header("link", "<#{next_url}>; rel=\"next\", <#{prev_url}>; rel=\"prev\"") - else - conn - end - end - def home_timeline(%{assigns: %{user: user}} = conn, params) do params = params @@ -430,6 +379,7 @@ def public_timeline(%{assigns: %{user: user}} = conn, params) do |> Map.put("local_only", local_only) |> Map.put("blocking_user", user) |> Map.put("muting_user", user) + |> Map.put("user", user) |> ActivityPub.fetch_public_activities() |> Enum.reverse() @@ -491,12 +441,9 @@ def get_context(%{assigns: %{user: user}} = conn, %{"id" => id}) do activities <- ActivityPub.fetch_activities_for_context(activity.data["context"], %{ "blocking_user" => user, - "user" => user + "user" => user, + "exclude_id" => activity.id }), - activities <- - activities |> Enum.filter(fn %{id: aid} -> to_string(aid) != to_string(id) end), - activities <- - activities |> Enum.filter(fn %{data: %{"type" => type}} -> type == "Create" end), grouped_activities <- Enum.group_by(activities, fn %{id: id} -> id < activity.id end) do result = %{ ancestors: @@ -531,8 +478,8 @@ def get_poll(%{assigns: %{user: user}} = conn, %{"id" => id}) do |> put_view(StatusView) |> try_render("poll.json", %{object: object, for: user}) else - nil -> render_error(conn, :not_found, "Record not found") - false -> render_error(conn, :not_found, "Record not found") + error when is_nil(error) or error == false -> + render_error(conn, :not_found, "Record not found") end end @@ -880,8 +827,8 @@ def get_mascot(%{assigns: %{user: user}} = conn, _params) do end def favourited_by(%{assigns: %{user: user}} = conn, %{"id" => id}) do - with %Activity{data: %{"object" => object}} <- Activity.get_by_id(id), - %Object{data: %{"likes" => likes}} <- Object.normalize(object) do + with %Activity{} = activity <- Activity.get_by_id_with_object(id), + %Object{data: %{"likes" => likes}} <- Object.normalize(activity) do q = from(u in User, where: u.ap_id in ^likes) users = @@ -897,8 +844,8 @@ def favourited_by(%{assigns: %{user: user}} = conn, %{"id" => id}) do end def reblogged_by(%{assigns: %{user: user}} = conn, %{"id" => id}) do - with %Activity{data: %{"object" => object}} <- Activity.get_by_id(id), - %Object{data: %{"announcements" => announces}} <- Object.normalize(object) do + with %Activity{} = activity <- Activity.get_by_id_with_object(id), + %Object{data: %{"announcements" => announces}} <- Object.normalize(activity) do q = from(u in User, where: u.ap_id in ^announces) users = @@ -939,6 +886,7 @@ def hashtag_timeline(%{assigns: %{user: user}} = conn, params) do |> Map.put("local_only", local_only) |> Map.put("blocking_user", user) |> Map.put("muting_user", user) + |> Map.put("user", user) |> Map.put("tag", tags) |> Map.put("tag_all", tag_all) |> Map.put("tag_reject", tag_reject) @@ -1220,10 +1168,9 @@ def user_favourites(%{assigns: %{user: for_user}} = conn, %{"id" => id} = params recipients = if for_user do - ["https://www.w3.org/ns/activitystreams#Public"] ++ - [for_user.ap_id | for_user.following] + [Pleroma.Constants.as_public()] ++ [for_user.ap_id | for_user.following] else - ["https://www.w3.org/ns/activitystreams#Public"] + [Pleroma.Constants.as_public()] end activities = @@ -1346,6 +1293,7 @@ def list_timeline(%{assigns: %{user: user}} = conn, %{"list_id" => id} = params) params |> Map.put("type", "Create") |> Map.put("blocking_user", user) + |> Map.put("user", user) |> Map.put("muting_user", user) # we must filter the following list for the user to avoid leaking statuses the user @@ -1686,45 +1634,35 @@ def suggestions(%{assigns: %{user: user}} = conn, _) do |> String.replace("{{user}}", user) with {:ok, %{status: 200, body: body}} <- - HTTP.get( - url, - [], - adapter: [ - recv_timeout: timeout, - pool: :default - ] - ), + HTTP.get(url, [], adapter: [recv_timeout: timeout, pool: :default]), {:ok, data} <- Jason.decode(body) do data = data |> Enum.slice(0, limit) |> Enum.map(fn x -> - Map.put( - x, - "id", - case User.get_or_fetch(x["acct"]) do - {:ok, %User{id: id}} -> id - _ -> 0 - end - ) - end) - |> Enum.map(fn x -> - Map.put(x, "avatar", MediaProxy.url(x["avatar"])) - end) - |> Enum.map(fn x -> - Map.put(x, "avatar_static", MediaProxy.url(x["avatar_static"])) + x + |> Map.put("id", fetch_suggestion_id(x)) + |> Map.put("avatar", MediaProxy.url(x["avatar"])) + |> Map.put("avatar_static", MediaProxy.url(x["avatar_static"])) end) - conn - |> json(data) + json(conn, data) else - e -> Logger.error("Could not retrieve suggestions at fetch #{url}, #{inspect(e)}") + e -> + Logger.error("Could not retrieve suggestions at fetch #{url}, #{inspect(e)}") end else json(conn, []) end end + defp fetch_suggestion_id(attrs) do + case User.get_or_fetch(attrs["acct"]) do + {:ok, %User{id: id}} -> id + _ -> 0 + end + end + def status_card(%{assigns: %{user: user}} = conn, %{"id" => status_id}) do with %Activity{} = activity <- Activity.get_by_id(status_id), true <- Visibility.visible_for_user?(activity, user) do @@ -1803,7 +1741,7 @@ def conversations(%{assigns: %{user: user}} = conn, params) do conversations = Enum.map(participations, fn participation -> - ConversationView.render("participation.json", %{participation: participation, user: user}) + ConversationView.render("participation.json", %{participation: participation, for: user}) end) conn @@ -1816,7 +1754,7 @@ def conversation_read(%{assigns: %{user: user}} = conn, %{"id" => participation_ Repo.get_by(Participation, id: participation_id, user_id: user.id), {:ok, participation} <- Participation.mark_as_read(participation) do participation_view = - ConversationView.render("participation.json", %{participation: participation, user: user}) + ConversationView.render("participation.json", %{participation: participation, for: user}) conn |> json(participation_view) @@ -1839,6 +1777,16 @@ def password_reset(conn, params) do end end + def account_confirmation_resend(conn, params) do + nickname_or_email = params["email"] || params["nickname"] + + with %User{} = user <- User.get_by_nickname_or_email(nickname_or_email), + {:ok, _} <- User.try_send_confirmation_email(user) do + conn + |> json_response(:no_content, "") + end + end + def try_render(conn, target, params) when is_binary(target) do case render(conn, target, params) do diff --git a/lib/pleroma/web/mastodon_api/views/account_view.ex b/lib/pleroma/web/mastodon_api/views/account_view.ex index befb35c26..169116d0d 100644 --- a/lib/pleroma/web/mastodon_api/views/account_view.ex +++ b/lib/pleroma/web/mastodon_api/views/account_view.ex @@ -28,7 +28,7 @@ def render("mention.json", %{user: user}) do id: to_string(user.id), acct: user.nickname, username: username_from_nickname(user.nickname), - url: user.ap_id + url: User.profile_url(user) } end @@ -37,11 +37,11 @@ def render("relationship.json", %{user: nil, target: _target}) do end def render("relationship.json", %{user: %User{} = user, target: %User{} = target}) do - follow_activity = Pleroma.Web.ActivityPub.Utils.fetch_latest_follow(user, target) + follow_state = User.get_cached_follow_state(user, target) requested = - if follow_activity && !User.following?(target, user) do - follow_activity.data["state"] == "pending" + if follow_state && !User.following?(user, target) do + follow_state == "pending" else false end @@ -50,13 +50,13 @@ def render("relationship.json", %{user: %User{} = user, target: %User{} = target id: to_string(target.id), following: User.following?(user, target), followed_by: User.following?(target, user), - blocking: User.blocks?(user, target), - blocked_by: User.blocks?(target, user), + blocking: User.blocks_ap_id?(user, target), + blocked_by: User.blocks_ap_id?(target, user), muting: User.mutes?(user, target), muting_notifications: User.muted_notifications?(user, target), subscribing: User.subscribed_to?(user, target), requested: requested, - domain_blocking: false, + domain_blocking: User.blocks_domain?(user, target), showing_reblogs: User.showing_reblogs?(user, target), endorsed: false } @@ -72,6 +72,13 @@ defp do_render("account.json", %{user: user} = opts) do image = User.avatar_url(user) |> MediaProxy.url() header = User.banner_url(user) |> MediaProxy.url() user_info = User.get_cached_user_info(user) + + following_count = + ((!user.info.hide_follows or opts[:for] == user) && user_info.following_count) || 0 + + followers_count = + ((!user.info.hide_followers or opts[:for] == user) && user_info.follower_count) || 0 + bot = (user.info.source_data["type"] || "Person") in ["Application", "Service"] emojis = @@ -87,12 +94,18 @@ defp do_render("account.json", %{user: user} = opts) do end) fields = - (user.info.source_data["attachment"] || []) - |> Enum.filter(fn %{"type" => t} -> t == "PropertyValue" end) - |> Enum.map(fn fields -> Map.take(fields, ["name", "value"]) end) + user.info + |> User.Info.fields() + |> Enum.map(fn %{"name" => name, "value" => value} -> + %{ + "name" => Pleroma.HTML.strip_tags(name), + "value" => Pleroma.HTML.filter_tags(value, Pleroma.HTML.Scrubber.LinksOnly) + } + end) + + raw_fields = Map.get(user.info, :raw_fields, []) bio = HTML.filter_tags(user.bio, User.html_filter_policy(opts[:for])) - relationship = render("relationship.json", %{user: opts[:for], target: user}) %{ @@ -102,11 +115,11 @@ defp do_render("account.json", %{user: user} = opts) do display_name: display_name, locked: user_info.locked, created_at: Utils.to_masto_date(user.inserted_at), - followers_count: user_info.follower_count, - following_count: user_info.following_count, + followers_count: followers_count, + following_count: following_count, statuses_count: user_info.note_count, note: bio || "", - url: user.ap_id, + url: User.profile_url(user), avatar: image, avatar_static: image, header: header, @@ -117,6 +130,7 @@ defp do_render("account.json", %{user: user} = opts) do source: %{ note: HTML.strip_tags((user.bio || "") |> String.replace("
", "\n")), sensitive: false, + fields: raw_fields, pleroma: %{} }, diff --git a/lib/pleroma/web/mastodon_api/views/conversation_view.ex b/lib/pleroma/web/mastodon_api/views/conversation_view.ex index 38bdec737..40acc07b3 100644 --- a/lib/pleroma/web/mastodon_api/views/conversation_view.ex +++ b/lib/pleroma/web/mastodon_api/views/conversation_view.ex @@ -11,8 +11,8 @@ defmodule Pleroma.Web.MastodonAPI.ConversationView do alias Pleroma.Web.MastodonAPI.AccountView alias Pleroma.Web.MastodonAPI.StatusView - def render("participation.json", %{participation: participation, user: user}) do - participation = Repo.preload(participation, conversation: :users) + def render("participation.json", %{participation: participation, for: user}) do + participation = Repo.preload(participation, conversation: [], recipients: []) last_activity_id = with nil <- participation.last_activity_id do @@ -28,7 +28,7 @@ def render("participation.json", %{participation: participation, user: user}) do # Conversations return all users except the current user. users = - participation.conversation.users + participation.recipients |> Enum.reject(&(&1.id == user.id)) accounts = diff --git a/lib/pleroma/web/mastodon_api/views/status_view.ex b/lib/pleroma/web/mastodon_api/views/status_view.ex index 7264dcafb..a4ee0b5dd 100644 --- a/lib/pleroma/web/mastodon_api/views/status_view.ex +++ b/lib/pleroma/web/mastodon_api/views/status_view.ex @@ -5,8 +5,12 @@ defmodule Pleroma.Web.MastodonAPI.StatusView do use Pleroma.Web, :view + require Pleroma.Constants + alias Pleroma.Activity alias Pleroma.ActivityExpiration + alias Pleroma.Conversation + alias Pleroma.Conversation.Participation alias Pleroma.HTML alias Pleroma.Object alias Pleroma.Repo @@ -25,19 +29,19 @@ defp get_replied_to_activities([]), do: %{} defp get_replied_to_activities(activities) do activities |> Enum.map(fn - %{data: %{"type" => "Create", "object" => object}} -> - object = Object.normalize(object) - object.data["inReplyTo"] != "" && object.data["inReplyTo"] + %{data: %{"type" => "Create"}} = activity -> + object = Object.normalize(activity) + object && object.data["inReplyTo"] != "" && object.data["inReplyTo"] _ -> nil end) |> Enum.filter(& &1) - |> Activity.create_by_object_ap_id() + |> Activity.create_by_object_ap_id_with_object() |> Repo.all() |> Enum.reduce(%{}, fn activity, acc -> object = Object.normalize(activity) - Map.put(acc, object.data["id"], activity) + if object, do: Map.put(acc, object.data["id"], activity), else: acc end) end @@ -69,12 +73,14 @@ defp reblogged?(activity, user) do def render("index.json", opts) do replied_to_activities = get_replied_to_activities(opts.activities) + parallel = unless is_nil(opts[:parallel]), do: opts[:parallel], else: true opts.activities |> safe_render_many( StatusView, "status.json", - Map.put(opts, :replied_to_activities, replied_to_activities) + Map.put(opts, :replied_to_activities, replied_to_activities), + parallel ) end @@ -89,6 +95,7 @@ def render( reblogged_activity = Activity.create_by_object_ap_id(activity_object.data["id"]) |> Activity.with_preloaded_bookmark(opts[:for]) + |> Activity.with_set_thread_muted_field(opts[:for]) |> Repo.one() reblogged = render("status.json", Map.put(opts, :activity, reblogged_activity)) @@ -143,6 +150,7 @@ def render("status.json", %{activity: %{data: %{"object" => _object}} = activity object = Object.normalize(activity) user = get_user(activity.data["actor"]) + user_follower_address = user.follower_address like_count = object.data["like_count"] || 0 announcement_count = object.data["announcement_count"] || 0 @@ -158,7 +166,11 @@ def render("status.json", %{activity: %{data: %{"object" => _object}} = activity mentions = (object.data["to"] ++ tag_mentions) |> Enum.uniq() - |> Enum.map(fn ap_id -> User.get_cached_by_ap_id(ap_id) end) + |> Enum.map(fn + Pleroma.Constants.as_public() -> nil + ^user_follower_address -> nil + ap_id -> User.get_cached_by_ap_id(ap_id) + end) |> Enum.filter(& &1) |> Enum.map(fn user -> AccountView.render("mention.json", %{user: user}) end) @@ -178,7 +190,7 @@ def render("status.json", %{activity: %{data: %{"object" => _object}} = activity thread_muted? = case activity.thread_muted? do thread_muted? when is_boolean(thread_muted?) -> thread_muted? - nil -> CommonAPI.thread_muted?(user, activity) + nil -> (opts[:for] && CommonAPI.thread_muted?(opts[:for], activity)) || false end attachment_data = object.data["attachment"] || [] @@ -232,7 +244,20 @@ def render("status.json", %{activity: %{data: %{"object" => _object}} = activity if user.local do Pleroma.Web.Router.Helpers.o_status_url(Pleroma.Web.Endpoint, :notice, activity) else - object.data["external_url"] || object.data["id"] + object.data["url"] || object.data["external_url"] || object.data["id"] + end + + direct_conversation_id = + with {_, true} <- {:include_id, opts[:with_direct_conversation_id]}, + {_, %User{} = for_user} <- {:for_user, opts[:for]}, + %{data: %{"context" => context}} when is_binary(context) <- activity, + %Conversation{} = conversation <- Conversation.get_for_ap_id(context), + %Participation{id: participation_id} <- + Participation.for_user_and_conversation(for_user, conversation) do + participation_id + else + _e -> + nil end %{ @@ -273,7 +298,8 @@ def render("status.json", %{activity: %{data: %{"object" => _object}} = activity in_reply_to_account_acct: reply_to_user && reply_to_user.nickname, content: %{"text/plain" => content_plaintext}, spoiler_text: %{"text/plain" => summary_plaintext}, - expires_at: expires_at + expires_at: expires_at, + direct_conversation_id: direct_conversation_id } } end diff --git a/lib/pleroma/web/media_proxy/media_proxy.ex b/lib/pleroma/web/media_proxy/media_proxy.ex index a661e9bb7..1725ab071 100644 --- a/lib/pleroma/web/media_proxy/media_proxy.ex +++ b/lib/pleroma/web/media_proxy/media_proxy.ex @@ -4,6 +4,7 @@ defmodule Pleroma.Web.MediaProxy do alias Pleroma.Config + alias Pleroma.Upload alias Pleroma.Web @base64_opts [padding: false] @@ -26,7 +27,18 @@ defp local?(url), do: String.starts_with?(url, Pleroma.Web.base_url()) defp whitelisted?(url) do %{host: domain} = URI.parse(url) - Enum.any?(Config.get([:media_proxy, :whitelist]), fn pattern -> + mediaproxy_whitelist = Config.get([:media_proxy, :whitelist]) + + upload_base_url_domain = + if !is_nil(Config.get([Upload, :base_url])) do + [URI.parse(Config.get([Upload, :base_url])).host] + else + [] + end + + whitelist = mediaproxy_whitelist ++ upload_base_url_domain + + Enum.any?(whitelist, fn pattern -> String.equivalent?(domain, pattern) end) end diff --git a/lib/pleroma/web/nodeinfo/nodeinfo_controller.ex b/lib/pleroma/web/nodeinfo/nodeinfo_controller.ex index a1d7fcc7d..ee14cfd6b 100644 --- a/lib/pleroma/web/nodeinfo/nodeinfo_controller.ex +++ b/lib/pleroma/web/nodeinfo/nodeinfo_controller.ex @@ -34,64 +34,18 @@ def schemas(conn, _params) do def raw_nodeinfo do stats = Stats.get_stats() - exclusions = Config.get([:instance, :mrf_transparency_exclusions]) - - mrf_simple = - Config.get(:mrf_simple) - |> Enum.map(fn {k, v} -> {k, Enum.reject(v, fn v -> v in exclusions end)} end) - |> Enum.into(%{}) - - # This horror is needed to convert regex sigils to strings - mrf_keyword = - Config.get(:mrf_keyword, []) - |> Enum.map(fn {key, value} -> - {key, - Enum.map(value, fn - {pattern, replacement} -> - %{ - "pattern" => - if not is_binary(pattern) do - inspect(pattern) - else - pattern - end, - "replacement" => replacement - } - - pattern -> - if not is_binary(pattern) do - inspect(pattern) - else - pattern - end - end)} - end) - |> Enum.into(%{}) - - mrf_policies = - MRF.get_policies() - |> Enum.map(fn policy -> to_string(policy) |> String.split(".") |> List.last() end) - quarantined = Config.get([:instance, :quarantined_instances], []) staff_accounts = User.all_superusers() |> Enum.map(fn u -> u.ap_id end) - mrf_user_allowlist = - Config.get([:mrf_user_allowlist], []) - |> Enum.into(%{}, fn {k, v} -> {k, length(v)} end) - federation_response = if Config.get([:instance, :mrf_transparency]) do - %{ - mrf_policies: mrf_policies, - mrf_simple: mrf_simple, - mrf_keyword: mrf_keyword, - mrf_user_allowlist: mrf_user_allowlist, - quarantined_instances: quarantined, - exclusions: length(exclusions) > 0 - } + {:ok, data} = MRF.describe() + + data + |> Map.merge(%{quarantined_instances: quarantined}) else %{} end @@ -165,6 +119,7 @@ def raw_nodeinfo do }, accountActivationRequired: Config.get([:instance, :account_activation_required], false), invitesEnabled: Config.get([:instance, :invites_enabled], false), + mailerEnabled: Config.get([Pleroma.Emails.Mailer, :enabled], false), features: features, restrictedNicknames: Config.get([Pleroma.User, :restricted_nicknames]), skipThreadContainment: Config.get([:instance, :skip_thread_containment], false) diff --git a/lib/pleroma/web/oauth/oauth_controller.ex b/lib/pleroma/web/oauth/oauth_controller.ex index ef53b7ae3..81eae2c8b 100644 --- a/lib/pleroma/web/oauth/oauth_controller.ex +++ b/lib/pleroma/web/oauth/oauth_controller.ex @@ -365,8 +365,7 @@ def registration_details(%Plug.Conn{} = conn, %{"authorization" => auth_attrs}) def register(%Plug.Conn{} = conn, %{"authorization" => _, "op" => "connect"} = params) do with registration_id when not is_nil(registration_id) <- get_session_registration_id(conn), %Registration{} = registration <- Repo.get(Registration, registration_id), - {_, {:ok, auth}} <- - {:create_authorization, do_create_authorization(conn, params)}, + {_, {:ok, auth}} <- {:create_authorization, do_create_authorization(conn, params)}, %User{} = user <- Repo.preload(auth, :user).user, {:ok, _updated_registration} <- Registration.bind_to_user(registration, user) do conn diff --git a/lib/pleroma/web/oauth/token.ex b/lib/pleroma/web/oauth/token.ex index 90c304487..40f131b57 100644 --- a/lib/pleroma/web/oauth/token.ex +++ b/lib/pleroma/web/oauth/token.ex @@ -44,8 +44,7 @@ def get_by_refresh_token(%App{id: app_id} = _app, token) do |> Repo.find_resource() end - @spec exchange_token(App.t(), Authorization.t()) :: - {:ok, Token.t()} | {:error, Changeset.t()} + @spec exchange_token(App.t(), Authorization.t()) :: {:ok, Token.t()} | {:error, Changeset.t()} def exchange_token(app, auth) do with {:ok, auth} <- Authorization.use_token(auth), true <- auth.app_id == app.id do diff --git a/lib/pleroma/web/oauth/token/clean_worker.ex b/lib/pleroma/web/oauth/token/clean_worker.ex index dca852449..f50098302 100644 --- a/lib/pleroma/web/oauth/token/clean_worker.ex +++ b/lib/pleroma/web/oauth/token/clean_worker.ex @@ -6,36 +6,30 @@ defmodule Pleroma.Web.OAuth.Token.CleanWorker do @moduledoc """ The module represents functions to clean an expired oauth tokens. """ + use GenServer + + @ten_seconds 10_000 + @one_day 86_400_000 - # 10 seconds - @start_interval 10_000 @interval Pleroma.Config.get( - # 24 hours [:oauth2, :clean_expired_tokens_interval], - 86_400_000 + @one_day ) - @queue :background alias Pleroma.Web.OAuth.Token - def start_link, do: GenServer.start_link(__MODULE__, nil) + def start_link(_), do: GenServer.start_link(__MODULE__, %{}) def init(_) do - if Pleroma.Config.get([:oauth2, :clean_expired_tokens], false) do - Process.send_after(self(), :perform, @start_interval) - {:ok, nil} - else - :ignore - end + Process.send_after(self(), :perform, @ten_seconds) + {:ok, nil} end @doc false def handle_info(:perform, state) do + Token.delete_expired_tokens() + Process.send_after(self(), :perform, @interval) - PleromaJobQueue.enqueue(@queue, __MODULE__, [:clean]) {:noreply, state} end - - # Job Worker Callbacks - def perform(:clean), do: Token.delete_expired_tokens() end diff --git a/lib/pleroma/web/ostatus/activity_representer.ex b/lib/pleroma/web/ostatus/activity_representer.ex index 95037125d..8e55b9f0b 100644 --- a/lib/pleroma/web/ostatus/activity_representer.ex +++ b/lib/pleroma/web/ostatus/activity_representer.ex @@ -9,6 +9,7 @@ defmodule Pleroma.Web.OStatus.ActivityRepresenter do alias Pleroma.Web.OStatus.UserRepresenter require Logger + require Pleroma.Constants defp get_href(id) do with %Object{data: %{"external_url" => external_url}} <- Object.get_cached_by_ap_id(id) do @@ -34,7 +35,7 @@ defp get_mentions(to) do Enum.map(to, fn id -> cond do # Special handling for the AP/Ostatus public collections - "https://www.w3.org/ns/activitystreams#Public" == id -> + Pleroma.Constants.as_public() == id -> {:link, [ rel: "mentioned", @@ -182,6 +183,7 @@ def to_simple_form(%{data: %{"type" => "Announce"}} = activity, user, with_autho author = if with_author, do: [{:author, UserRepresenter.to_simple_form(user)}], else: [] retweeted_activity = Activity.get_create_by_object_ap_id(activity.data["object"]) + retweeted_object = Object.normalize(retweeted_activity) retweeted_user = User.get_cached_by_ap_id(retweeted_activity.data["actor"]) retweeted_xml = to_simple_form(retweeted_activity, retweeted_user, true) @@ -196,7 +198,7 @@ def to_simple_form(%{data: %{"type" => "Announce"}} = activity, user, with_autho {:"activity:verb", ['http://activitystrea.ms/schema/1.0/share']}, {:id, h.(activity.data["id"])}, {:title, ['#{user.nickname} repeated a notice']}, - {:content, [type: 'html'], ['RT #{retweeted_activity.data["object"]["content"]}']}, + {:content, [type: 'html'], ['RT #{retweeted_object.data["content"]}']}, {:published, h.(inserted_at)}, {:updated, h.(updated_at)}, {:"ostatus:conversation", [ref: h.(activity.data["context"])], diff --git a/lib/pleroma/web/ostatus/handlers/follow_handler.ex b/lib/pleroma/web/ostatus/handlers/follow_handler.ex index 263d3b2dc..24513972e 100644 --- a/lib/pleroma/web/ostatus/handlers/follow_handler.ex +++ b/lib/pleroma/web/ostatus/handlers/follow_handler.ex @@ -9,14 +9,18 @@ defmodule Pleroma.Web.OStatus.FollowHandler do alias Pleroma.Web.XML def handle(entry, doc) do - with {:ok, actor} <- OStatus.find_make_or_update_user(doc), + with {:ok, actor} <- OStatus.find_make_or_update_actor(doc), id when not is_nil(id) <- XML.string_from_xpath("/entry/id", entry), followed_uri when not is_nil(followed_uri) <- XML.string_from_xpath("/entry/activity:object/id", entry), {:ok, followed} <- OStatus.find_or_make_user(followed_uri), + {:locked, false} <- {:locked, followed.info.locked}, {:ok, activity} <- ActivityPub.follow(actor, followed, id, false) do User.follow(actor, followed) {:ok, activity} + else + {:locked, true} -> + {:error, "It's not possible to follow locked accounts over OStatus"} end end end diff --git a/lib/pleroma/web/ostatus/handlers/note_handler.ex b/lib/pleroma/web/ostatus/handlers/note_handler.ex index 8e0adad91..7fae14f7b 100644 --- a/lib/pleroma/web/ostatus/handlers/note_handler.ex +++ b/lib/pleroma/web/ostatus/handlers/note_handler.ex @@ -4,6 +4,7 @@ defmodule Pleroma.Web.OStatus.NoteHandler do require Logger + require Pleroma.Constants alias Pleroma.Activity alias Pleroma.Object @@ -49,7 +50,7 @@ def get_people_mentions(entry) do def get_collection_mentions(entry) do transmogrify = fn "http://activityschema.org/collection/public" -> - "https://www.w3.org/ns/activitystreams#Public" + Pleroma.Constants.as_public() group -> group @@ -110,7 +111,7 @@ def handle_note(entry, doc \\ nil, options \\ []) do with id <- XML.string_from_xpath("//id", entry), activity when is_nil(activity) <- Activity.get_create_by_object_ap_id_with_object(id), [author] <- :xmerl_xpath.string('//author[1]', doc), - {:ok, actor} <- OStatus.find_make_or_update_user(author), + {:ok, actor} <- OStatus.find_make_or_update_actor(author), content_html <- OStatus.get_content(entry), cw <- OStatus.get_cw(entry), in_reply_to <- XML.string_from_xpath("//thr:in-reply-to[1]/@ref", entry), @@ -126,7 +127,7 @@ def handle_note(entry, doc \\ nil, options \\ []) do to <- make_to_list(actor, mentions), date <- XML.string_from_xpath("//published", entry), unlisted <- XML.string_from_xpath("//mastodon:scope", entry) == "unlisted", - cc <- if(unlisted, do: ["https://www.w3.org/ns/activitystreams#Public"], else: []), + cc <- if(unlisted, do: [Pleroma.Constants.as_public()], else: []), note <- CommonAPI.Utils.make_note_data( actor.ap_id, diff --git a/lib/pleroma/web/ostatus/handlers/unfollow_handler.ex b/lib/pleroma/web/ostatus/handlers/unfollow_handler.ex index 6596ada3b..2062432e3 100644 --- a/lib/pleroma/web/ostatus/handlers/unfollow_handler.ex +++ b/lib/pleroma/web/ostatus/handlers/unfollow_handler.ex @@ -9,7 +9,7 @@ defmodule Pleroma.Web.OStatus.UnfollowHandler do alias Pleroma.Web.XML def handle(entry, doc) do - with {:ok, actor} <- OStatus.find_make_or_update_user(doc), + with {:ok, actor} <- OStatus.find_make_or_update_actor(doc), id when not is_nil(id) <- XML.string_from_xpath("/entry/id", entry), followed_uri when not is_nil(followed_uri) <- XML.string_from_xpath("/entry/activity:object/id", entry), diff --git a/lib/pleroma/web/ostatus/ostatus.ex b/lib/pleroma/web/ostatus/ostatus.ex index 502410c83..331cbc0b7 100644 --- a/lib/pleroma/web/ostatus/ostatus.ex +++ b/lib/pleroma/web/ostatus/ostatus.ex @@ -56,7 +56,7 @@ def remote_follow_path do def handle_incoming(xml_string, options \\ []) do with doc when doc != :error <- parse_document(xml_string) do - with {:ok, actor_user} <- find_make_or_update_user(doc), + with {:ok, actor_user} <- find_make_or_update_actor(doc), do: Pleroma.Instances.set_reachable(actor_user.ap_id) entries = :xmerl_xpath.string('//entry', doc) @@ -120,7 +120,7 @@ def handle_incoming(xml_string, options \\ []) do end def make_share(entry, doc, retweeted_activity) do - with {:ok, actor} <- find_make_or_update_user(doc), + with {:ok, actor} <- find_make_or_update_actor(doc), %Object{} = object <- Object.normalize(retweeted_activity), id when not is_nil(id) <- string_from_xpath("/entry/id", entry), {:ok, activity, _object} = ActivityPub.announce(actor, object, id, false) do @@ -138,7 +138,7 @@ def handle_share(entry, doc) do end def make_favorite(entry, doc, favorited_activity) do - with {:ok, actor} <- find_make_or_update_user(doc), + with {:ok, actor} <- find_make_or_update_actor(doc), %Object{} = object <- Object.normalize(favorited_activity), id when not is_nil(id) <- string_from_xpath("/entry/id", entry), {:ok, activity, _object} = ActivityPub.like(actor, object, id, false) do @@ -264,11 +264,18 @@ def maybe_update_ostatus(doc, user) do end end - def find_make_or_update_user(doc) do + def find_make_or_update_actor(doc) do uri = string_from_xpath("//author/uri[1]", doc) - with {:ok, user} <- find_or_make_user(uri) do + with {:ok, %User{} = user} <- find_or_make_user(uri), + {:ap_enabled, false} <- {:ap_enabled, User.ap_enabled?(user)} do maybe_update(doc, user) + else + {:ap_enabled, true} -> + {:error, :invalid_protocol} + + _ -> + {:error, :unknown_user} end end diff --git a/lib/pleroma/web/ostatus/ostatus_controller.ex b/lib/pleroma/web/ostatus/ostatus_controller.ex index 372d52899..fdba0f77f 100644 --- a/lib/pleroma/web/ostatus/ostatus_controller.ex +++ b/lib/pleroma/web/ostatus/ostatus_controller.ex @@ -5,6 +5,7 @@ defmodule Pleroma.Web.OStatus.OStatusController do use Pleroma.Web, :controller + alias Fallback.RedirectController alias Pleroma.Activity alias Pleroma.Object alias Pleroma.User @@ -12,42 +13,49 @@ defmodule Pleroma.Web.OStatus.OStatusController do alias Pleroma.Web.ActivityPub.ActivityPubController alias Pleroma.Web.ActivityPub.ObjectView alias Pleroma.Web.ActivityPub.Visibility + alias Pleroma.Web.Endpoint alias Pleroma.Web.Federator + alias Pleroma.Web.Metadata.PlayerView alias Pleroma.Web.OStatus alias Pleroma.Web.OStatus.ActivityRepresenter alias Pleroma.Web.OStatus.FeedRepresenter + alias Pleroma.Web.Router alias Pleroma.Web.XML + plug( + Pleroma.Plugs.RateLimiter, + {:ap_routes, params: ["uuid"]} when action in [:object, :activity] + ) + plug(Pleroma.Web.FederatingPlug when action in [:salmon_incoming]) + plug( + Pleroma.Plugs.SetFormatPlug + when action in [:feed_redirect, :object, :activity, :notice] + ) + action_fallback(:errors) + def feed_redirect(%{assigns: %{format: "html"}} = conn, %{"nickname" => nickname}) do + with {_, %User{} = user} <- + {:fetch_user, User.get_cached_by_nickname_or_id(nickname)} do + RedirectController.redirector_with_meta(conn, %{user: user}) + end + end + + def feed_redirect(%{assigns: %{format: format}} = conn, _params) + when format in ["json", "activity+json"] do + ActivityPubController.call(conn, :user) + end + def feed_redirect(conn, %{"nickname" => nickname}) do - case get_format(conn) do - "html" -> - with %User{} = user <- User.get_cached_by_nickname_or_id(nickname) do - Fallback.RedirectController.redirector_with_meta(conn, %{user: user}) - else - nil -> {:error, :not_found} - end - - "activity+json" -> - ActivityPubController.call(conn, :user) - - "json" -> - ActivityPubController.call(conn, :user) - - _ -> - with %User{} = user <- User.get_cached_by_nickname(nickname) do - redirect(conn, external: OStatus.feed_path(user)) - else - nil -> {:error, :not_found} - end + with {_, %User{} = user} <- {:fetch_user, User.get_cached_by_nickname(nickname)} do + redirect(conn, external: OStatus.feed_path(user)) end end def feed(conn, %{"nickname" => nickname} = params) do - with %User{} = user <- User.get_cached_by_nickname(nickname) do + with {_, %User{} = user} <- {:fetch_user, User.get_cached_by_nickname(nickname)} do query_params = Map.take(params, ["max_id"]) |> Map.merge(%{"whole_db" => true, "actor_id" => user.ap_id}) @@ -65,8 +73,6 @@ def feed(conn, %{"nickname" => nickname} = params) do conn |> put_resp_content_type("application/atom+xml") |> send_resp(200, response) - else - nil -> {:error, :not_found} end end @@ -97,93 +103,82 @@ def salmon_incoming(conn, _) do |> send_resp(200, "") end - def object(conn, %{"uuid" => uuid}) do - if get_format(conn) in ["activity+json", "json"] do - ActivityPubController.call(conn, :object) - else - with id <- o_status_url(conn, :object, uuid), - {_, %Activity{} = activity} <- - {:activity, Activity.get_create_by_object_ap_id_with_object(id)}, - {_, true} <- {:public?, Visibility.is_public?(activity)}, - %User{} = user <- User.get_cached_by_ap_id(activity.data["actor"]) do - case get_format(conn) do - "html" -> redirect(conn, to: "/notice/#{activity.id}") - _ -> represent_activity(conn, nil, activity, user) - end - else - {:public?, false} -> - {:error, :not_found} + def object(%{assigns: %{format: format}} = conn, %{"uuid" => _uuid}) + when format in ["json", "activity+json"] do + ActivityPubController.call(conn, :object) + end - {:activity, nil} -> - {:error, :not_found} - - e -> - e + def object(%{assigns: %{format: format}} = conn, %{"uuid" => uuid}) do + with id <- o_status_url(conn, :object, uuid), + {_, %Activity{} = activity} <- + {:activity, Activity.get_create_by_object_ap_id_with_object(id)}, + {_, true} <- {:public?, Visibility.is_public?(activity)}, + %User{} = user <- User.get_cached_by_ap_id(activity.data["actor"]) do + case format do + "html" -> redirect(conn, to: "/notice/#{activity.id}") + _ -> represent_activity(conn, nil, activity, user) end + else + reason when reason in [{:public?, false}, {:activity, nil}] -> + {:error, :not_found} + + e -> + e end end - def activity(conn, %{"uuid" => uuid}) do - if get_format(conn) in ["activity+json", "json"] do - ActivityPubController.call(conn, :activity) - else - with id <- o_status_url(conn, :activity, uuid), - {_, %Activity{} = activity} <- {:activity, Activity.normalize(id)}, - {_, true} <- {:public?, Visibility.is_public?(activity)}, - %User{} = user <- User.get_cached_by_ap_id(activity.data["actor"]) do - case format = get_format(conn) do - "html" -> redirect(conn, to: "/notice/#{activity.id}") - _ -> represent_activity(conn, format, activity, user) - end - else - {:public?, false} -> - {:error, :not_found} + def activity(%{assigns: %{format: format}} = conn, %{"uuid" => _uuid}) + when format in ["json", "activity+json"] do + ActivityPubController.call(conn, :activity) + end - {:activity, nil} -> - {:error, :not_found} - - e -> - e + def activity(%{assigns: %{format: format}} = conn, %{"uuid" => uuid}) do + with id <- o_status_url(conn, :activity, uuid), + {_, %Activity{} = activity} <- {:activity, Activity.normalize(id)}, + {_, true} <- {:public?, Visibility.is_public?(activity)}, + %User{} = user <- User.get_cached_by_ap_id(activity.data["actor"]) do + case format do + "html" -> redirect(conn, to: "/notice/#{activity.id}") + _ -> represent_activity(conn, format, activity, user) end + else + reason when reason in [{:public?, false}, {:activity, nil}] -> + {:error, :not_found} + + e -> + e end end - def notice(conn, %{"id" => id}) do + def notice(%{assigns: %{format: format}} = conn, %{"id" => id}) do with {_, %Activity{} = activity} <- {:activity, Activity.get_by_id_with_object(id)}, {_, true} <- {:public?, Visibility.is_public?(activity)}, %User{} = user <- User.get_cached_by_ap_id(activity.data["actor"]) do - case format = get_format(conn) do - "html" -> - if activity.data["type"] == "Create" do - %Object{} = object = Object.normalize(activity) + cond do + format == "html" && activity.data["type"] == "Create" -> + %Object{} = object = Object.normalize(activity) - Fallback.RedirectController.redirector_with_meta(conn, %{ + RedirectController.redirector_with_meta( + conn, + %{ activity_id: activity.id, object: object, - url: - Pleroma.Web.Router.Helpers.o_status_url( - Pleroma.Web.Endpoint, - :notice, - activity.id - ), + url: Router.Helpers.o_status_url(Endpoint, :notice, activity.id), user: user - }) - else - Fallback.RedirectController.redirector(conn, nil) - end + } + ) - _ -> + format == "html" -> + RedirectController.redirector(conn, nil) + + true -> represent_activity(conn, format, activity, user) end else - {:public?, false} -> + reason when reason in [{:public?, false}, {:activity, nil}] -> conn |> put_status(404) - |> Fallback.RedirectController.redirector(nil, 404) - - {:activity, nil} -> - conn - |> Fallback.RedirectController.redirector(nil, 404) + |> RedirectController.redirector(nil, 404) e -> e @@ -204,13 +199,13 @@ def notice_player(conn, %{"id" => id}) do "content-security-policy", "default-src 'none';style-src 'self' 'unsafe-inline';img-src 'self' data: https:; media-src 'self' https:;" ) - |> put_view(Pleroma.Web.Metadata.PlayerView) + |> put_view(PlayerView) |> render("player.html", url) else _error -> conn |> put_status(404) - |> Fallback.RedirectController.redirector(nil, 404) + |> RedirectController.redirector(nil, 404) end end @@ -248,6 +243,8 @@ def errors(conn, {:error, :not_found}) do render_error(conn, :not_found, "Not found") end + def errors(conn, {:fetch_user, nil}), do: errors(conn, {:error, :not_found}) + def errors(conn, _) do render_error(conn, :internal_server_error, "Something went wrong") end diff --git a/lib/pleroma/web/pleroma_api/pleroma_api_controller.ex b/lib/pleroma/web/pleroma_api/pleroma_api_controller.ex new file mode 100644 index 000000000..b6d2bf86b --- /dev/null +++ b/lib/pleroma/web/pleroma_api/pleroma_api_controller.ex @@ -0,0 +1,73 @@ +# Pleroma: A lightweight social networking server +# Copyright © 2017-2019 Pleroma Authors +# SPDX-License-Identifier: AGPL-3.0-only + +defmodule Pleroma.Web.PleromaAPI.PleromaAPIController do + use Pleroma.Web, :controller + + import Pleroma.Web.ControllerHelper, only: [add_link_headers: 7] + + alias Pleroma.Conversation.Participation + alias Pleroma.Web.ActivityPub.ActivityPub + alias Pleroma.Web.MastodonAPI.ConversationView + alias Pleroma.Web.MastodonAPI.StatusView + + def conversation(%{assigns: %{user: user}} = conn, %{"id" => participation_id}) do + with %Participation{} = participation <- Participation.get(participation_id), + true <- user.id == participation.user_id do + conn + |> put_view(ConversationView) + |> render("participation.json", %{participation: participation, for: user}) + end + end + + def conversation_statuses( + %{assigns: %{user: user}} = conn, + %{"id" => participation_id} = params + ) do + params = + params + |> Map.put("blocking_user", user) + |> Map.put("muting_user", user) + |> Map.put("user", user) + + participation = + participation_id + |> Participation.get(preload: [:conversation]) + + if user.id == participation.user_id do + activities = + participation.conversation.ap_id + |> ActivityPub.fetch_activities_for_context(params) + |> Enum.reverse() + + conn + |> add_link_headers( + :conversation_statuses, + activities, + participation_id, + params, + nil, + &pleroma_api_url/4 + ) + |> put_view(StatusView) + |> render("index.json", %{activities: activities, for: user, as: :activity}) + end + end + + def update_conversation( + %{assigns: %{user: user}} = conn, + %{"id" => participation_id, "recipients" => recipients} + ) do + participation = + participation_id + |> Participation.get() + + with true <- user.id == participation.user_id, + {:ok, participation} <- Participation.set_recipients(participation, recipients) do + conn + |> put_view(ConversationView) + |> render("participation.json", %{participation: participation, for: user}) + end + end +end diff --git a/lib/pleroma/web/rich_media/parsers/ttl/aws_signed_url.ex b/lib/pleroma/web/rich_media/parsers/ttl/aws_signed_url.ex index 014c0935f..0dc1efdaf 100644 --- a/lib/pleroma/web/rich_media/parsers/ttl/aws_signed_url.ex +++ b/lib/pleroma/web/rich_media/parsers/ttl/aws_signed_url.ex @@ -19,8 +19,7 @@ defp is_aws_signed_url(nil), do: nil defp is_aws_signed_url(image) when is_binary(image) do %URI{host: host, query: query} = URI.parse(image) - if String.contains?(host, "amazonaws.com") and - String.contains?(query, "X-Amz-Expires") do + if String.contains?(host, "amazonaws.com") and String.contains?(query, "X-Amz-Expires") do image else nil diff --git a/lib/pleroma/web/rich_media/parsers/twitter_card.ex b/lib/pleroma/web/rich_media/parsers/twitter_card.ex index e4efe2dd0..afaa98f3d 100644 --- a/lib/pleroma/web/rich_media/parsers/twitter_card.ex +++ b/lib/pleroma/web/rich_media/parsers/twitter_card.ex @@ -3,13 +3,20 @@ # SPDX-License-Identifier: AGPL-3.0-only defmodule Pleroma.Web.RichMedia.Parsers.TwitterCard do + alias Pleroma.Web.RichMedia.Parsers.MetaTagsParser + + @spec parse(String.t(), map()) :: {:ok, map()} | {:error, String.t()} def parse(html, data) do - Pleroma.Web.RichMedia.Parsers.MetaTagsParser.parse( - html, - data, - "twitter", - "No twitter card metadata found", - "name" - ) + data + |> parse_name_attrs(html) + |> parse_property_attrs(html) + end + + defp parse_name_attrs(data, html) do + MetaTagsParser.parse(html, data, "twitter", %{}, "name") + end + + defp parse_property_attrs({_, data}, html) do + MetaTagsParser.parse(html, data, "twitter", "No twitter card metadata found", "property") end end diff --git a/lib/pleroma/web/router.ex b/lib/pleroma/web/router.ex index a9f3826fc..1eb6f7b9d 100644 --- a/lib/pleroma/web/router.ex +++ b/lib/pleroma/web/router.ex @@ -196,6 +196,8 @@ defmodule Pleroma.Web.Router do get("/config", AdminAPIController, :config_show) post("/config", AdminAPIController, :config_update) + get("/config/migrate_to_db", AdminAPIController, :migrate_to_db) + get("/config/migrate_from_db", AdminAPIController, :migrate_from_db) end scope "/", Pleroma.Web.TwitterAPI do @@ -257,6 +259,21 @@ defmodule Pleroma.Web.Router do end end + scope "/api/v1/pleroma", Pleroma.Web.PleromaAPI do + pipe_through(:authenticated_api) + + scope [] do + pipe_through(:oauth_read) + get("/conversations/:id/statuses", PleromaAPIController, :conversation_statuses) + get("/conversations/:id", PleromaAPIController, :conversation) + end + + scope [] do + pipe_through(:oauth_write) + patch("/conversations/:id", PleromaAPIController, :update_conversation) + end + end + scope "/api/v1", Pleroma.Web.MastodonAPI do pipe_through(:authenticated_api) @@ -412,6 +429,12 @@ defmodule Pleroma.Web.Router do get("/accounts/search", SearchController, :account_search) + post( + "/pleroma/accounts/confirmation_resend", + MastodonAPIController, + :account_confirmation_resend + ) + scope [] do pipe_through(:oauth_read_or_public) @@ -600,6 +623,8 @@ defmodule Pleroma.Web.Router do post("/push/hub/:nickname", Websub.WebsubController, :websub_subscription_request) get("/push/subscriptions/:id", Websub.WebsubController, :websub_subscription_confirmation) post("/push/subscriptions/:id", Websub.WebsubController, :websub_incoming) + + get("/mailer/unsubscribe/:token", Mailer.SubscriptionController, :unsubscribe) end pipeline :activitypub do @@ -692,7 +717,7 @@ defmodule Pleroma.Web.Router do post("/auth/password", MastodonAPIController, :password_reset) scope [] do - pipe_through(:oauth_read_or_public) + pipe_through(:oauth_read) get("/web/*path", MastodonAPIController, :index) end end @@ -729,68 +754,3 @@ defmodule Pleroma.Web.Router do options("/*path", RedirectController, :empty) end end - -defmodule Fallback.RedirectController do - use Pleroma.Web, :controller - require Logger - alias Pleroma.User - alias Pleroma.Web.Metadata - - def api_not_implemented(conn, _params) do - conn - |> put_status(404) - |> json(%{error: "Not implemented"}) - end - - def redirector(conn, _params, code \\ 200) do - conn - |> put_resp_content_type("text/html") - |> send_file(code, index_file_path()) - end - - def redirector_with_meta(conn, %{"maybe_nickname_or_id" => maybe_nickname_or_id} = params) do - with %User{} = user <- User.get_cached_by_nickname_or_id(maybe_nickname_or_id) do - redirector_with_meta(conn, %{user: user}) - else - nil -> - redirector(conn, params) - end - end - - def redirector_with_meta(conn, params) do - {:ok, index_content} = File.read(index_file_path()) - - tags = - try do - Metadata.build_tags(params) - rescue - e -> - Logger.error( - "Metadata rendering for #{conn.request_path} failed.\n" <> - Exception.format(:error, e, __STACKTRACE__) - ) - - "" - end - - response = String.replace(index_content, "", tags) - - conn - |> put_resp_content_type("text/html") - |> send_resp(200, response) - end - - def index_file_path do - Pleroma.Plugs.InstanceStatic.file_path("index.html") - end - - def registration_page(conn, params) do - redirector(conn, params) - end - - def empty(conn, _params) do - conn - |> put_status(204) - |> text("") - end -end diff --git a/lib/pleroma/web/streamer.ex b/lib/pleroma/web/streamer.ex index 86e2dc4dd..587c43f40 100644 --- a/lib/pleroma/web/streamer.ex +++ b/lib/pleroma/web/streamer.ex @@ -18,7 +18,7 @@ defmodule Pleroma.Web.Streamer do @keepalive_interval :timer.seconds(30) - def start_link do + def start_link(_) do GenServer.start_link(__MODULE__, %{}, name: __MODULE__) end @@ -35,28 +35,21 @@ def stream(topic, item) do end def init(args) do - spawn(fn -> - # 30 seconds - Process.sleep(@keepalive_interval) - GenServer.cast(__MODULE__, %{action: :ping}) - end) + Process.send_after(self(), %{action: :ping}, @keepalive_interval) {:ok, args} end - def handle_cast(%{action: :ping}, topics) do - Map.values(topics) + def handle_info(%{action: :ping}, topics) do + topics + |> Map.values() |> List.flatten() |> Enum.each(fn socket -> Logger.debug("Sending keepalive ping") send(socket.transport_pid, {:text, ""}) end) - spawn(fn -> - # 30 seconds - Process.sleep(@keepalive_interval) - GenServer.cast(__MODULE__, %{action: :ping}) - end) + Process.send_after(self(), %{action: :ping}, @keepalive_interval) {:noreply, topics} end @@ -120,8 +113,7 @@ def handle_cast( |> Map.get("#{topic}:#{item.user_id}", []) |> Enum.each(fn socket -> with %User{} = user <- User.get_cached_by_ap_id(socket.assigns[:user].ap_id), - true <- should_send?(user, item), - false <- CommonAPI.thread_muted?(user, item.activity) do + true <- should_send?(user, item) do send( socket.transport_pid, {:text, represent_notification(socket.assigns[:user], item)} @@ -209,7 +201,7 @@ def represent_conversation(%Participation{} = participation) do payload: Pleroma.Web.MastodonAPI.ConversationView.render("participation.json", %{ participation: participation, - user: participation.user + for: participation.user }) |> Jason.encode!() } @@ -234,11 +226,17 @@ defp should_send?(%User{} = user, %Activity{} = item) do blocks = user.info.blocks || [] mutes = user.info.mutes || [] reblog_mutes = user.info.muted_reblogs || [] + domain_blocks = Pleroma.Web.ActivityPub.MRF.subdomains_regex(user.info.domain_blocks) with parent when not is_nil(parent) <- Object.normalize(item), true <- Enum.all?([blocks, mutes, reblog_mutes], &(item.actor not in &1)), true <- Enum.all?([blocks, mutes], &(parent.data["actor"] not in &1)), - true <- thread_containment(item, user) do + %{host: item_host} <- URI.parse(item.actor), + %{host: parent_host} <- URI.parse(parent.data["actor"]), + false <- Pleroma.Web.ActivityPub.MRF.subdomain_match?(domain_blocks, item_host), + false <- Pleroma.Web.ActivityPub.MRF.subdomain_match?(domain_blocks, parent_host), + true <- thread_containment(item, user), + false <- CommonAPI.thread_muted?(user, item) do true else _ -> false diff --git a/lib/pleroma/web/templates/email/digest.html.eex b/lib/pleroma/web/templates/email/digest.html.eex new file mode 100644 index 000000000..860df5f9c --- /dev/null +++ b/lib/pleroma/web/templates/email/digest.html.eex @@ -0,0 +1,568 @@ + + + + + + + + + + + + <%= @email.subject %>< + + + + + + + + + + + + + + + + + + + diff --git a/lib/pleroma/web/templates/layout/app.html.eex b/lib/pleroma/web/templates/layout/app.html.eex index b3cf9ed11..5836ec1e0 100644 --- a/lib/pleroma/web/templates/layout/app.html.eex +++ b/lib/pleroma/web/templates/layout/app.html.eex @@ -36,6 +36,11 @@ margin-bottom: 20px; } + a { + color: color: #d8a070; + text-decoration: none; + } + form { width: 100%; } diff --git a/lib/pleroma/web/templates/layout/email.html.eex b/lib/pleroma/web/templates/layout/email.html.eex new file mode 100644 index 000000000..f6dcd7f0f --- /dev/null +++ b/lib/pleroma/web/templates/layout/email.html.eex @@ -0,0 +1,10 @@ + + + + + <%= @email.subject %> + + + <%= render @view_module, @view_template, assigns %> + + \ No newline at end of file diff --git a/lib/pleroma/web/templates/mailer/subscription/unsubscribe_failure.html.eex b/lib/pleroma/web/templates/mailer/subscription/unsubscribe_failure.html.eex new file mode 100644 index 000000000..7b476f02d --- /dev/null +++ b/lib/pleroma/web/templates/mailer/subscription/unsubscribe_failure.html.eex @@ -0,0 +1 @@ +

UNSUBSCRIBE FAILURE

diff --git a/lib/pleroma/web/templates/mailer/subscription/unsubscribe_success.html.eex b/lib/pleroma/web/templates/mailer/subscription/unsubscribe_success.html.eex new file mode 100644 index 000000000..6dfa2c185 --- /dev/null +++ b/lib/pleroma/web/templates/mailer/subscription/unsubscribe_success.html.eex @@ -0,0 +1 @@ +

UNSUBSCRIBE SUCCESSFUL

diff --git a/lib/pleroma/web/twitter_api/controllers/util_controller.ex b/lib/pleroma/web/twitter_api/controllers/util_controller.ex index 9e4da7dca..3405bd3b7 100644 --- a/lib/pleroma/web/twitter_api/controllers/util_controller.ex +++ b/lib/pleroma/web/twitter_api/controllers/util_controller.ex @@ -15,11 +15,11 @@ defmodule Pleroma.Web.TwitterAPI.UtilController do alias Pleroma.Plugs.AuthenticationPlug alias Pleroma.User alias Pleroma.Web - alias Pleroma.Web.ActivityPub.ActivityPub alias Pleroma.Web.CommonAPI - alias Pleroma.Web.OStatus alias Pleroma.Web.WebFinger + plug(Pleroma.Plugs.SetFormatPlug when action in [:config, :version]) + def help_test(conn, _params) do json(conn, "ok") end @@ -60,27 +60,25 @@ def remote_follow(%{assigns: %{user: user}} = conn, %{"acct" => acct}) do %Activity{id: activity_id} = Activity.get_create_by_object_ap_id(object.data["id"]) redirect(conn, to: "/notice/#{activity_id}") else - {err, followee} = OStatus.find_or_make_user(acct) - avatar = User.avatar_url(followee) - name = followee.nickname - id = followee.id - - if !!user do + with {:ok, followee} <- User.get_or_fetch(acct) do conn - |> render("follow.html", %{error: err, acct: acct, avatar: avatar, name: name, id: id}) - else - conn - |> render("follow_login.html", %{ + |> render(follow_template(user), %{ error: false, acct: acct, - avatar: avatar, - name: name, - id: id + avatar: User.avatar_url(followee), + name: followee.nickname, + id: followee.id }) + else + {:error, _reason} -> + render(conn, follow_template(user), %{error: :error}) end end end + defp follow_template(%User{} = _user), do: "follow.html" + defp follow_template(_), do: "follow_login.html" + defp is_status?(acct) do case Pleroma.Object.Fetcher.fetch_and_contain_remote_object_from_id(acct) do {:ok, %{"type" => type}} when type in ["Article", "Note", "Video", "Page", "Question"] -> @@ -94,50 +92,53 @@ defp is_status?(acct) do def do_remote_follow(conn, %{ "authorization" => %{"name" => username, "password" => password, "id" => id} }) do - followee = User.get_cached_by_id(id) - avatar = User.avatar_url(followee) - name = followee.nickname - - with %User{} = user <- User.get_cached_by_nickname(username), - true <- AuthenticationPlug.checkpw(password, user.password_hash), - %User{} = _followed <- User.get_cached_by_id(id), - {:ok, follower} <- User.follow(user, followee), - {:ok, _activity} <- ActivityPub.follow(follower, followee) do + with %User{} = followee <- User.get_cached_by_id(id), + {_, %User{} = user, _} <- {:auth, User.get_cached_by_nickname(username), followee}, + {_, true, _} <- { + :auth, + AuthenticationPlug.checkpw(password, user.password_hash), + followee + }, + {:ok, _follower, _followee, _activity} <- CommonAPI.follow(user, followee) do conn |> render("followed.html", %{error: false}) else # Was already following user {:error, "Could not follow user:" <> _rest} -> - render(conn, "followed.html", %{error: false}) + render(conn, "followed.html", %{error: "Error following account"}) - _e -> + {:auth, _, followee} -> conn |> render("follow_login.html", %{ error: "Wrong username or password", id: id, - name: name, - avatar: avatar + name: followee.nickname, + avatar: User.avatar_url(followee) }) + + e -> + Logger.debug("Remote follow failed with error #{inspect(e)}") + render(conn, "followed.html", %{error: "Something went wrong."}) end end def do_remote_follow(%{assigns: %{user: user}} = conn, %{"user" => %{"id" => id}}) do - with %User{} = followee <- User.get_cached_by_id(id), - {:ok, follower} <- User.follow(user, followee), - {:ok, _activity} <- ActivityPub.follow(follower, followee) do + with {:fetch_user, %User{} = followee} <- {:fetch_user, User.get_cached_by_id(id)}, + {:ok, _follower, _followee, _activity} <- CommonAPI.follow(user, followee) do conn |> render("followed.html", %{error: false}) else # Was already following user {:error, "Could not follow user:" <> _rest} -> - conn - |> render("followed.html", %{error: false}) + render(conn, "followed.html", %{error: "Error following account"}) + + {:fetch_user, error} -> + Logger.debug("Remote follow failed with error #{inspect(error)}") + render(conn, "followed.html", %{error: "Could not find user"}) e -> Logger.debug("Remote follow failed with error #{inspect(e)}") - - conn - |> render("followed.html", %{error: inspect(e)}) + render(conn, "followed.html", %{error: "Something went wrong."}) end end @@ -152,67 +153,70 @@ def notifications_read(%{assigns: %{user: user}} = conn, %{"id" => notification_ end end + def config(%{assigns: %{format: "xml"}} = conn, _params) do + instance = Pleroma.Config.get(:instance) + + response = """ + + + #{Keyword.get(instance, :name)} + #{Web.base_url()} + #{Keyword.get(instance, :limit)} + #{!Keyword.get(instance, :registrations_open)} + + + """ + + conn + |> put_resp_content_type("application/xml") + |> send_resp(200, response) + end + def config(conn, _params) do instance = Pleroma.Config.get(:instance) - case get_format(conn) do - "xml" -> - response = """ - - - #{Keyword.get(instance, :name)} - #{Web.base_url()} - #{Keyword.get(instance, :limit)} - #{!Keyword.get(instance, :registrations_open)} - - - """ + vapid_public_key = Keyword.get(Pleroma.Web.Push.vapid_config(), :public_key) - conn - |> put_resp_content_type("application/xml") - |> send_resp(200, response) + uploadlimit = %{ + uploadlimit: to_string(Keyword.get(instance, :upload_limit)), + avatarlimit: to_string(Keyword.get(instance, :avatar_upload_limit)), + backgroundlimit: to_string(Keyword.get(instance, :background_upload_limit)), + bannerlimit: to_string(Keyword.get(instance, :banner_upload_limit)) + } - _ -> - vapid_public_key = Keyword.get(Pleroma.Web.Push.vapid_config(), :public_key) + data = %{ + name: Keyword.get(instance, :name), + description: Keyword.get(instance, :description), + server: Web.base_url(), + textlimit: to_string(Keyword.get(instance, :limit)), + uploadlimit: uploadlimit, + closed: bool_to_val(Keyword.get(instance, :registrations_open), "0", "1"), + private: bool_to_val(Keyword.get(instance, :public, true), "0", "1"), + vapidPublicKey: vapid_public_key, + accountActivationRequired: + bool_to_val(Keyword.get(instance, :account_activation_required, false)), + invitesEnabled: bool_to_val(Keyword.get(instance, :invites_enabled, false)), + safeDMMentionsEnabled: bool_to_val(Pleroma.Config.get([:instance, :safe_dm_mentions])) + } - uploadlimit = %{ - uploadlimit: to_string(Keyword.get(instance, :upload_limit)), - avatarlimit: to_string(Keyword.get(instance, :avatar_upload_limit)), - backgroundlimit: to_string(Keyword.get(instance, :background_upload_limit)), - bannerlimit: to_string(Keyword.get(instance, :banner_upload_limit)) - } - - data = %{ - name: Keyword.get(instance, :name), - description: Keyword.get(instance, :description), - server: Web.base_url(), - textlimit: to_string(Keyword.get(instance, :limit)), - uploadlimit: uploadlimit, - closed: if(Keyword.get(instance, :registrations_open), do: "0", else: "1"), - private: if(Keyword.get(instance, :public, true), do: "0", else: "1"), - vapidPublicKey: vapid_public_key, - accountActivationRequired: - if(Keyword.get(instance, :account_activation_required, false), do: "1", else: "0"), - invitesEnabled: if(Keyword.get(instance, :invites_enabled, false), do: "1", else: "0"), - safeDMMentionsEnabled: - if(Pleroma.Config.get([:instance, :safe_dm_mentions]), do: "1", else: "0") - } + managed_config = Keyword.get(instance, :managed_config) + data = + if managed_config do pleroma_fe = Pleroma.Config.get([:frontend_configurations, :pleroma_fe]) + Map.put(data, "pleromafe", pleroma_fe) + else + data + end - managed_config = Keyword.get(instance, :managed_config) - - data = - if managed_config do - data |> Map.put("pleromafe", pleroma_fe) - else - data - end - - json(conn, %{site: data}) - end + json(conn, %{site: data}) end + defp bool_to_val(true), do: "1" + defp bool_to_val(_), do: "0" + defp bool_to_val(true, val, _), do: val + defp bool_to_val(_, _, val), do: val + def frontend_configurations(conn, _params) do config = Pleroma.Config.get(:frontend_configurations, %{}) @@ -221,20 +225,16 @@ def frontend_configurations(conn, _params) do json(conn, config) end - def version(conn, _params) do + def version(%{assigns: %{format: "xml"}} = conn, _params) do version = Pleroma.Application.named_version() - case get_format(conn) do - "xml" -> - response = "#{version}" + conn + |> put_resp_content_type("application/xml") + |> send_resp(200, "#{version}") + end - conn - |> put_resp_content_type("application/xml") - |> send_resp(200, response) - - _ -> - json(conn, version) - end + def version(conn, _params) do + json(conn, Pleroma.Application.named_version()) end def emoji(conn, _params) do diff --git a/lib/pleroma/web/twitter_api/twitter_api.ex b/lib/pleroma/web/twitter_api/twitter_api.ex index bb5dda204..80082ea84 100644 --- a/lib/pleroma/web/twitter_api/twitter_api.ex +++ b/lib/pleroma/web/twitter_api/twitter_api.ex @@ -15,6 +15,8 @@ defmodule Pleroma.Web.TwitterAPI.TwitterAPI do import Ecto.Query + require Pleroma.Constants + def create_status(%User{} = user, %{"status" => _} = data) do CommonAPI.post(user, data) end @@ -286,7 +288,7 @@ def search(_user, %{"q" => query} = params) do from( [a, o] in Activity.with_preloaded_object(Activity), where: fragment("?->>'type' = 'Create'", a.data), - where: "https://www.w3.org/ns/activitystreams#Public" in a.recipients, + where: ^Pleroma.Constants.as_public() in a.recipients, where: fragment( "to_tsvector('english', ?->>'content') @@ plainto_tsquery('english', ?)", diff --git a/lib/pleroma/web/twitter_api/views/activity_view.ex b/lib/pleroma/web/twitter_api/views/activity_view.ex index e84af84dc..abae63877 100644 --- a/lib/pleroma/web/twitter_api/views/activity_view.ex +++ b/lib/pleroma/web/twitter_api/views/activity_view.ex @@ -19,6 +19,7 @@ defmodule Pleroma.Web.TwitterAPI.ActivityView do import Ecto.Query require Logger + require Pleroma.Constants defp query_context_ids([]), do: [] @@ -91,7 +92,7 @@ defp get_user(ap_id, opts) do String.ends_with?(ap_id, "/followers") -> nil - ap_id == "https://www.w3.org/ns/activitystreams#Public" -> + ap_id == Pleroma.Constants.as_public() -> nil user = User.get_cached_by_ap_id(ap_id) -> diff --git a/lib/pleroma/web/twitter_api/views/notification_view.ex b/lib/pleroma/web/twitter_api/views/notification_view.ex index e7c7a7496..085cd5aa3 100644 --- a/lib/pleroma/web/twitter_api/views/notification_view.ex +++ b/lib/pleroma/web/twitter_api/views/notification_view.ex @@ -10,6 +10,8 @@ defmodule Pleroma.Web.TwitterAPI.NotificationView do alias Pleroma.Web.TwitterAPI.ActivityView alias Pleroma.Web.TwitterAPI.UserView + require Pleroma.Constants + defp get_user(ap_id, opts) do cond do user = opts[:users][ap_id] -> @@ -18,7 +20,7 @@ defp get_user(ap_id, opts) do String.ends_with?(ap_id, "/followers") -> nil - ap_id == "https://www.w3.org/ns/activitystreams#Public" -> + ap_id == Pleroma.Constants.as_public() -> nil true -> diff --git a/lib/pleroma/web/twitter_api/views/user_view.ex b/lib/pleroma/web/twitter_api/views/user_view.ex index 8d8892068..8a7d2fc72 100644 --- a/lib/pleroma/web/twitter_api/views/user_view.ex +++ b/lib/pleroma/web/twitter_api/views/user_view.ex @@ -74,12 +74,15 @@ defp do_render("user.json", %{user: user = %User{}} = assigns) do |> HTML.filter_tags(User.html_filter_policy(for_user)) |> Formatter.emojify(emoji) - # ``fields`` is an array of mastodon profile field, containing ``{"name": "…", "value": "…"}``. - # For example: [{"name": "Pronoun", "value": "she/her"}, …] fields = - (user.info.source_data["attachment"] || []) - |> Enum.filter(fn %{"type" => t} -> t == "PropertyValue" end) - |> Enum.map(fn fields -> Map.take(fields, ["name", "value"]) end) + user.info + |> User.Info.fields() + |> Enum.map(fn %{"name" => name, "value" => value} -> + %{ + "name" => Pleroma.HTML.strip_tags(name), + "value" => Pleroma.HTML.filter_tags(value, Pleroma.HTML.Scrubber.LinksOnly) + } + end) data = %{ diff --git a/lib/pleroma/web/views/email_view.ex b/lib/pleroma/web/views/email_view.ex new file mode 100644 index 000000000..b506a234b --- /dev/null +++ b/lib/pleroma/web/views/email_view.ex @@ -0,0 +1,15 @@ +defmodule Pleroma.Web.EmailView do + use Pleroma.Web, :view + import Phoenix.HTML + import Phoenix.HTML.Link + + def avatar_url(user) do + Pleroma.User.avatar_url(user) + end + + def format_date(date) when is_binary(date) do + date + |> Timex.parse!("{ISO:Extended:Z}") + |> Timex.format!("{Mshort} {D}, {YYYY} {h24}:{m}") + end +end diff --git a/lib/pleroma/web/views/mailer/subscription_view.ex b/lib/pleroma/web/views/mailer/subscription_view.ex new file mode 100644 index 000000000..fc3d20816 --- /dev/null +++ b/lib/pleroma/web/views/mailer/subscription_view.ex @@ -0,0 +1,3 @@ +defmodule Pleroma.Web.Mailer.SubscriptionView do + use Pleroma.Web, :view +end diff --git a/lib/pleroma/web/web.ex b/lib/pleroma/web/web.ex index b42f6887e..bfb6c7287 100644 --- a/lib/pleroma/web/web.ex +++ b/lib/pleroma/web/web.ex @@ -58,17 +58,31 @@ def safe_render(view, template, assigns \\ %{}) do rescue error -> Logger.error( - "#{__MODULE__} failed to render #{inspect({view, template})}: #{inspect(error)}" + "#{__MODULE__} failed to render #{inspect({view, template})}\n" <> + Exception.format(:error, error, __STACKTRACE__) ) - Logger.error(inspect(__STACKTRACE__)) nil end @doc """ - Same as `render_many/4` but wrapped in rescue block. + Same as `render_many/4` but wrapped in rescue block and parallelized (unless disabled by passing false as a fifth argument). """ - def safe_render_many(collection, view, template, assigns \\ %{}) do + def safe_render_many(collection, view, template, assigns \\ %{}, parallel \\ true) + + def safe_render_many(collection, view, template, assigns, true) do + Enum.map(collection, fn resource -> + Task.async(fn -> + as = Map.get(assigns, :as) || view.__resource__ + assigns = Map.put(assigns, as, resource) + safe_render(view, template, assigns) + end) + end) + |> Enum.map(&Task.await(&1, :infinity)) + |> Enum.filter(& &1) + end + + def safe_render_many(collection, view, template, assigns, false) do Enum.map(collection, fn resource -> as = Map.get(assigns, :as) || view.__resource__ assigns = Map.put(assigns, as, resource) diff --git a/lib/pleroma/web/web_finger/web_finger.ex b/lib/pleroma/web/web_finger/web_finger.ex index fa34c7ced..ecb39ee50 100644 --- a/lib/pleroma/web/web_finger/web_finger.ex +++ b/lib/pleroma/web/web_finger/web_finger.ex @@ -86,11 +86,17 @@ def represent_user(user, "XML") do |> XmlBuilder.to_doc() end - defp get_magic_key(magic_key) do - "data:application/magic-public-key," <> magic_key = magic_key + defp get_magic_key("data:application/magic-public-key," <> magic_key) do {:ok, magic_key} - rescue - MatchError -> {:error, "Missing magic key data."} + end + + defp get_magic_key(nil) do + Logger.debug("Undefined magic key.") + {:ok, nil} + end + + defp get_magic_key(_) do + {:error, "Missing magic key data."} end defp webfinger_from_xml(doc) do @@ -187,6 +193,7 @@ def find_lrdd_template(domain) do end end + @spec finger(String.t()) :: {:ok, map()} | {:error, any()} def finger(account) do account = String.trim_leading(account, "@") @@ -220,8 +227,6 @@ def finger(account) do else with {:ok, doc} <- Jason.decode(body) do webfinger_from_json(doc) - else - {:error, e} -> e end end else diff --git a/lib/pleroma/web/web_finger/web_finger_controller.ex b/lib/pleroma/web/web_finger/web_finger_controller.ex index b77c75ec5..896eb15f9 100644 --- a/lib/pleroma/web/web_finger/web_finger_controller.ex +++ b/lib/pleroma/web/web_finger/web_finger_controller.ex @@ -7,6 +7,7 @@ defmodule Pleroma.Web.WebFinger.WebFingerController do alias Pleroma.Web.WebFinger + plug(Pleroma.Plugs.SetFormatPlug) plug(Pleroma.Web.FederatingPlug) def host_meta(conn, _params) do @@ -17,30 +18,28 @@ def host_meta(conn, _params) do |> send_resp(200, xml) end - def webfinger(conn, %{"resource" => resource}) do - case get_format(conn) do - n when n in ["xml", "xrd+xml"] -> - with {:ok, response} <- WebFinger.webfinger(resource, "XML") do - conn - |> put_resp_content_type("application/xrd+xml") - |> send_resp(200, response) - else - _e -> send_resp(conn, 404, "Couldn't find user") - end - - n when n in ["json", "jrd+json"] -> - with {:ok, response} <- WebFinger.webfinger(resource, "JSON") do - json(conn, response) - else - _e -> send_resp(conn, 404, "Couldn't find user") - end - - _ -> - send_resp(conn, 404, "Unsupported format") + def webfinger(%{assigns: %{format: format}} = conn, %{"resource" => resource}) + when format in ["xml", "xrd+xml"] do + with {:ok, response} <- WebFinger.webfinger(resource, "XML") do + conn + |> put_resp_content_type("application/xrd+xml") + |> send_resp(200, response) + else + _e -> send_resp(conn, 404, "Couldn't find user") end end - def webfinger(conn, _params) do - send_resp(conn, 400, "Bad Request") + def webfinger(%{assigns: %{format: format}} = conn, %{"resource" => resource}) + when format in ["json", "jrd+json"] do + with {:ok, response} <- WebFinger.webfinger(resource, "JSON") do + json(conn, response) + else + _e -> + conn + |> put_status(404) + |> json("Couldn't find user") + end end + + def webfinger(conn, _params), do: send_resp(conn, 400, "Bad Request") end diff --git a/mix.exs b/mix.exs index e69940c5d..3170d6f2d 100644 --- a/mix.exs +++ b/mix.exs @@ -95,7 +95,7 @@ defp oauth_deps do defp deps do [ {:phoenix, "~> 1.4.8"}, - {:tzdata, "~> 1.0"}, + {:tzdata, "~> 0.5.21"}, {:plug_cowboy, "~> 2.0"}, {:phoenix_pubsub, "~> 1.1"}, {:phoenix_ecto, "~> 4.0"}, @@ -114,8 +114,9 @@ defp deps do {:tesla, "~> 1.2"}, {:jason, "~> 1.0"}, {:mogrify, "~> 0.6.1"}, - {:ex_aws, "~> 2.0"}, + {:ex_aws, "~> 2.1"}, {:ex_aws_s3, "~> 2.0"}, + {:sweet_xml, "~> 0.6.6"}, {:earmark, "~> 1.3"}, {:bbcode, "~> 0.1.1"}, {:ex_machina, "~> 2.3", only: :test}, @@ -127,6 +128,7 @@ defp deps do {:ex_doc, "~> 0.20.2", only: :dev, runtime: false}, {:web_push_encryption, "~> 0.2.1"}, {:swoosh, "~> 0.23.2"}, + {:phoenix_swoosh, "~> 0.2"}, {:gen_smtp, "~> 0.13"}, {:websocket_client, git: "https://github.com/jeremyong/websocket_client.git", only: :test}, {:floki, "~> 0.20.0"}, @@ -139,7 +141,7 @@ defp deps do {:http_signatures, git: "https://git.pleroma.social/pleroma/http_signatures.git", ref: "293d77bb6f4a67ac8bde1428735c3b42f22cbb30"}, - {:pleroma_job_queue, "~> 0.2.0"}, + {:pleroma_job_queue, "~> 0.3"}, {:telemetry, "~> 0.3"}, {:prometheus_ex, "~> 3.0"}, {:prometheus_plugs, "~> 1.1"}, @@ -147,9 +149,11 @@ defp deps do {:prometheus_ecto, "~> 1.4"}, {:recon, github: "ferd/recon", tag: "2.4.0"}, {:quack, "~> 0.1.1"}, + {:joken, "~> 2.0"}, {:benchee, "~> 1.0"}, {:esshd, "~> 0.1.0", runtime: Application.get_env(:esshd, :enabled, false)}, {:ex_rated, "~> 1.3"}, + {:ex_const, "~> 0.2"}, {:plug_static_index_html, "~> 1.0.0"}, {:excoveralls, "~> 0.11.1", only: :test}, {:mox, "~> 0.5", only: :test} @@ -189,12 +193,13 @@ defp version(version) do tag = String.trim(tag), {describe, 0} <- System.cmd("git", ["describe", "--tags", "--abbrev=8"]), describe = String.trim(describe), - ahead <- String.replace(describe, tag, "") do + ahead <- String.replace(describe, tag, ""), + ahead <- String.trim_leading(ahead, "-") do {String.replace_prefix(tag, "v", ""), if(ahead != "", do: String.trim(ahead))} else _ -> {commit_hash, 0} = System.cmd("git", ["rev-parse", "--short", "HEAD"]) - {nil, "-0-g" <> String.trim(commit_hash)} + {nil, "0-g" <> String.trim(commit_hash)} end if git_tag && version != git_tag do @@ -206,14 +211,15 @@ defp version(version) do # Branch name as pre-release version component, denoted with a dot branch_name = with {branch_name, 0} <- System.cmd("git", ["rev-parse", "--abbrev-ref", "HEAD"]), + branch_name <- String.trim(branch_name), branch_name <- System.get_env("PLEROMA_BUILD_BRANCH") || branch_name, - true <- branch_name != "master" do + true <- branch_name not in ["master", "HEAD"] do branch_name = branch_name |> String.trim() |> String.replace(identifier_filter, "-") - "." <> branch_name + branch_name end build_name = @@ -233,6 +239,17 @@ defp version(version) do env_override -> env_override end + # Pre-release version, denoted by appending a hyphen + # and a series of dot separated identifiers + pre_release = + [git_pre_release, branch_name] + |> Enum.filter(fn string -> string && string != "" end) + |> Enum.join(".") + |> (fn + "" -> nil + string -> "-" <> String.replace(string, identifier_filter, "-") + end).() + # Build metadata, denoted with a plus sign build_metadata = [build_name, env_name] @@ -243,7 +260,7 @@ defp version(version) do string -> "+" <> String.replace(string, identifier_filter, "-") end).() - [version, git_pre_release, branch_name, build_metadata] + [version, pre_release, build_metadata] |> Enum.filter(fn string -> string && string != "" end) |> Enum.join() end diff --git a/mix.lock b/mix.lock index 5f20878d3..2639e96e9 100644 --- a/mix.lock +++ b/mix.lock @@ -5,16 +5,17 @@ "bbcode": {:hex, :bbcode, "0.1.1", "0023e2c7814119b2e620b7add67182e3f6019f92bfec9a22da7e99821aceba70", [:mix], [{:nimble_parsec, "~> 0.5", [hex: :nimble_parsec, repo: "hexpm", optional: false]}], "hexpm"}, "benchee": {:hex, :benchee, "1.0.1", "66b211f9bfd84bd97e6d1beaddf8fc2312aaabe192f776e8931cb0c16f53a521", [:mix], [{:deep_merge, "~> 1.0", [hex: :deep_merge, repo: "hexpm", optional: false]}], "hexpm"}, "bunt": {:hex, :bunt, "0.2.0", "951c6e801e8b1d2cbe58ebbd3e616a869061ddadcc4863d0a2182541acae9a38", [:mix], [], "hexpm"}, - "cachex": {:hex, :cachex, "3.0.2", "1351caa4e26e29f7d7ec1d29b53d6013f0447630bbf382b4fb5d5bad0209f203", [:mix], [{:eternal, "~> 1.2", [hex: :eternal, repo: "hexpm", optional: false]}, {:unsafe, "~> 1.0", [hex: :unsafe, repo: "hexpm", optional: false]}], "hexpm"}, + "cachex": {:hex, :cachex, "3.0.3", "4e2d3e05814a5738f5ff3903151d5c25636d72a3527251b753f501ad9c657967", [:mix], [{:eternal, "~> 1.2", [hex: :eternal, repo: "hexpm", optional: false]}, {:unsafe, "~> 1.0", [hex: :unsafe, repo: "hexpm", optional: false]}], "hexpm"}, "calendar": {:hex, :calendar, "0.17.6", "ec291cb2e4ba499c2e8c0ef5f4ace974e2f9d02ae9e807e711a9b0c7850b9aee", [:mix], [{:tzdata, "~> 0.5.20 or ~> 0.1.201603 or ~> 1.0", [hex: :tzdata, repo: "hexpm", optional: false]}], "hexpm"}, "certifi": {:hex, :certifi, "2.5.1", "867ce347f7c7d78563450a18a6a28a8090331e77fa02380b4a21962a65d36ee5", [:rebar3], [{:parse_trans, "~>3.3", [hex: :parse_trans, repo: "hexpm", optional: false]}], "hexpm"}, "combine": {:hex, :combine, "0.10.0", "eff8224eeb56498a2af13011d142c5e7997a80c8f5b97c499f84c841032e429f", [:mix], [], "hexpm"}, - "comeonin": {:hex, :comeonin, "4.1.1", "c7304fc29b45b897b34142a91122bc72757bc0c295e9e824999d5179ffc08416", [:mix], [{:argon2_elixir, "~> 1.2", [hex: :argon2_elixir, repo: "hexpm", optional: true]}, {:bcrypt_elixir, "~> 0.12.1 or ~> 1.0", [hex: :bcrypt_elixir, repo: "hexpm", optional: true]}, {:pbkdf2_elixir, "~> 0.12", [hex: :pbkdf2_elixir, repo: "hexpm", optional: true]}], "hexpm"}, + "comeonin": {:hex, :comeonin, "4.1.2", "3eb5620fd8e35508991664b4c2b04dd41e52f1620b36957be837c1d7784b7592", [:mix], [{:argon2_elixir, "~> 1.2", [hex: :argon2_elixir, repo: "hexpm", optional: true]}, {:bcrypt_elixir, "~> 0.12.1 or ~> 1.0", [hex: :bcrypt_elixir, repo: "hexpm", optional: true]}, {:pbkdf2_elixir, "~> 0.12", [hex: :pbkdf2_elixir, repo: "hexpm", optional: true]}], "hexpm"}, "connection": {:hex, :connection, "1.0.4", "a1cae72211f0eef17705aaededacac3eb30e6625b04a6117c1b2db6ace7d5976", [:mix], [], "hexpm"}, "cors_plug": {:hex, :cors_plug, "1.5.2", "72df63c87e4f94112f458ce9d25800900cc88608c1078f0e4faddf20933eda6e", [:mix], [{:plug, "~> 1.3 or ~> 1.4 or ~> 1.5", [hex: :plug, repo: "hexpm", optional: false]}], "hexpm"}, "cowboy": {:hex, :cowboy, "2.6.3", "99aa50e94e685557cad82e704457336a453d4abcb77839ad22dbe71f311fcc06", [:rebar3], [{:cowlib, "~> 2.7.3", [hex: :cowlib, repo: "hexpm", optional: false]}, {:ranch, "~> 1.7.1", [hex: :ranch, repo: "hexpm", optional: false]}], "hexpm"}, "cowlib": {:hex, :cowlib, "2.7.3", "a7ffcd0917e6d50b4d5fb28e9e2085a0ceb3c97dea310505f7460ff5ed764ce9", [:rebar3], [], "hexpm"}, "credo": {:hex, :credo, "0.9.3", "76fa3e9e497ab282e0cf64b98a624aa11da702854c52c82db1bf24e54ab7c97a", [:mix], [{:bunt, "~> 0.2.0", [hex: :bunt, repo: "hexpm", optional: false]}, {:poison, ">= 0.0.0", [hex: :poison, repo: "hexpm", optional: false]}], "hexpm"}, + "crontab": {:hex, :crontab, "1.1.7", "b9219f0bdc8678b94143655a8f229716c5810c0636a4489f98c0956137e53985", [:mix], [{:ecto, "~> 1.0 or ~> 2.0 or ~> 3.0", [hex: :ecto, repo: "hexpm", optional: true]}], "hexpm"}, "crypt": {:git, "https://github.com/msantos/crypt", "1f2b58927ab57e72910191a7ebaeff984382a1d3", [ref: "1f2b58927ab57e72910191a7ebaeff984382a1d3"]}, "db_connection": {:hex, :db_connection, "2.0.6", "bde2f85d047969c5b5800cb8f4b3ed6316c8cb11487afedac4aa5f93fd39abfa", [:mix], [{:connection, "~> 1.0.2", [hex: :connection, repo: "hexpm", optional: false]}], "hexpm"}, "decimal": {:hex, :decimal, "1.8.0", "ca462e0d885f09a1c5a342dbd7c1dcf27ea63548c65a65e67334f4b61803822e", [:mix], [], "hexpm"}, @@ -27,6 +28,7 @@ "ex2ms": {:hex, :ex2ms, "1.5.0", "19e27f9212be9a96093fed8cdfbef0a2b56c21237196d26760f11dfcfae58e97", [:mix], [], "hexpm"}, "ex_aws": {:hex, :ex_aws, "2.1.0", "b92651527d6c09c479f9013caa9c7331f19cba38a650590d82ebf2c6c16a1d8a", [:mix], [{:configparser_ex, "~> 2.0", [hex: :configparser_ex, repo: "hexpm", optional: true]}, {:hackney, "1.6.3 or 1.6.5 or 1.7.1 or 1.8.6 or ~> 1.9", [hex: :hackney, repo: "hexpm", optional: true]}, {:jsx, "~> 2.8", [hex: :jsx, repo: "hexpm", optional: true]}, {:poison, ">= 1.2.0", [hex: :poison, repo: "hexpm", optional: true]}, {:sweet_xml, "~> 0.6", [hex: :sweet_xml, repo: "hexpm", optional: true]}, {:xml_builder, "~> 0.1.0", [hex: :xml_builder, repo: "hexpm", optional: true]}], "hexpm"}, "ex_aws_s3": {:hex, :ex_aws_s3, "2.0.1", "9e09366e77f25d3d88c5393824e613344631be8db0d1839faca49686e99b6704", [:mix], [{:ex_aws, "~> 2.0", [hex: :ex_aws, repo: "hexpm", optional: false]}, {:sweet_xml, ">= 0.0.0", [hex: :sweet_xml, repo: "hexpm", optional: true]}], "hexpm"}, + "ex_const": {:hex, :ex_const, "0.2.4", "d06e540c9d834865b012a17407761455efa71d0ce91e5831e86881b9c9d82448", [:mix], [], "hexpm"}, "ex_doc": {:hex, :ex_doc, "0.20.2", "1bd0dfb0304bade58beb77f20f21ee3558cc3c753743ae0ddbb0fd7ba2912331", [:mix], [{:earmark, "~> 1.3", [hex: :earmark, repo: "hexpm", optional: false]}, {:makeup_elixir, "~> 0.10", [hex: :makeup_elixir, repo: "hexpm", optional: false]}], "hexpm"}, "ex_machina": {:hex, :ex_machina, "2.3.0", "92a5ad0a8b10ea6314b876a99c8c9e3f25f4dde71a2a835845b136b9adaf199a", [:mix], [{:ecto, "~> 2.2 or ~> 3.0", [hex: :ecto, repo: "hexpm", optional: true]}, {:ecto_sql, "~> 3.0", [hex: :ecto_sql, repo: "hexpm", optional: true]}], "hexpm"}, "ex_rated": {:hex, :ex_rated, "1.3.3", "30ecbdabe91f7eaa9d37fa4e81c85ba420f371babeb9d1910adbcd79ec798d27", [:mix], [{:ex2ms, "~> 1.5", [hex: :ex2ms, repo: "hexpm", optional: false]}], "hexpm"}, @@ -42,14 +44,15 @@ "httpoison": {:hex, :httpoison, "1.2.0", "2702ed3da5fd7a8130fc34b11965c8cfa21ade2f232c00b42d96d4967c39a3a3", [:mix], [{:hackney, "~> 1.8", [hex: :hackney, repo: "hexpm", optional: false]}], "hexpm"}, "idna": {:hex, :idna, "6.0.0", "689c46cbcdf3524c44d5f3dde8001f364cd7608a99556d8fbd8239a5798d4c10", [:rebar3], [{:unicode_util_compat, "0.4.1", [hex: :unicode_util_compat, repo: "hexpm", optional: false]}], "hexpm"}, "jason": {:hex, :jason, "1.1.2", "b03dedea67a99223a2eaf9f1264ce37154564de899fd3d8b9a21b1a6fd64afe7", [:mix], [{:decimal, "~> 1.0", [hex: :decimal, repo: "hexpm", optional: true]}], "hexpm"}, - "jose": {:hex, :jose, "1.8.4", "7946d1e5c03a76ac9ef42a6e6a20001d35987afd68c2107bcd8f01a84e75aa73", [:mix, :rebar3], [{:base64url, "~> 0.0.1", [hex: :base64url, repo: "hexpm", optional: false]}], "hexpm"}, + "joken": {:hex, :joken, "2.0.1", "ec9ab31bf660f343380da033b3316855197c8d4c6ef597fa3fcb451b326beb14", [:mix], [{:jose, "~> 1.9", [hex: :jose, repo: "hexpm", optional: false]}], "hexpm"}, + "jose": {:hex, :jose, "1.9.0", "4167c5f6d06ffaebffd15cdb8da61a108445ef5e85ab8f5a7ad926fdf3ada154", [:mix, :rebar3], [{:base64url, "~> 0.0.1", [hex: :base64url, repo: "hexpm", optional: false]}], "hexpm"}, "makeup": {:hex, :makeup, "0.8.0", "9cf32aea71c7fe0a4b2e9246c2c4978f9070257e5c9ce6d4a28ec450a839b55f", [:mix], [{:nimble_parsec, "~> 0.5.0", [hex: :nimble_parsec, repo: "hexpm", optional: false]}], "hexpm"}, "makeup_elixir": {:hex, :makeup_elixir, "0.13.0", "be7a477997dcac2e48a9d695ec730b2d22418292675c75aa2d34ba0909dcdeda", [:mix], [{:makeup, "~> 0.8", [hex: :makeup, repo: "hexpm", optional: false]}], "hexpm"}, "meck": {:hex, :meck, "0.8.13", "ffedb39f99b0b99703b8601c6f17c7f76313ee12de6b646e671e3188401f7866", [:rebar3], [], "hexpm"}, "metrics": {:hex, :metrics, "1.0.1", "25f094dea2cda98213cecc3aeff09e940299d950904393b2a29d191c346a8486", [:rebar3], [], "hexpm"}, "mime": {:hex, :mime, "1.3.1", "30ce04ab3175b6ad0bdce0035cba77bba68b813d523d1aac73d9781b4d193cf8", [:mix], [], "hexpm"}, "mimerl": {:hex, :mimerl, "1.2.0", "67e2d3f571088d5cfd3e550c383094b47159f3eee8ffa08e64106cdf5e981be3", [:rebar3], [], "hexpm"}, - "mochiweb": {:hex, :mochiweb, "2.15.0", "e1daac474df07651e5d17cc1e642c4069c7850dc4508d3db7263a0651330aacc", [:rebar3], [], "hexpm"}, + "mochiweb": {:hex, :mochiweb, "2.18.0", "eb55f1db3e6e960fac4e6db4e2db9ec3602cc9f30b86cd1481d56545c3145d2e", [:rebar3], [], "hexpm"}, "mock": {:hex, :mock, "0.3.3", "42a433794b1291a9cf1525c6d26b38e039e0d3a360732b5e467bfc77ef26c914", [:mix], [{:meck, "~> 0.8.13", [hex: :meck, repo: "hexpm", optional: false]}], "hexpm"}, "mogrify": {:hex, :mogrify, "0.6.1", "de1b527514f2d95a7bbe9642eb556061afb337e220cf97adbf3a4e6438ed70af", [:mix], [], "hexpm"}, "mox": {:hex, :mox, "0.5.1", "f86bb36026aac1e6f924a4b6d024b05e9adbed5c63e8daa069bd66fb3292165b", [:mix], [], "hexpm"}, @@ -60,34 +63,34 @@ "phoenix_ecto": {:hex, :phoenix_ecto, "4.0.0", "c43117a136e7399ea04ecaac73f8f23ee0ffe3e07acfcb8062fe5f4c9f0f6531", [:mix], [{:ecto, "~> 3.0", [hex: :ecto, repo: "hexpm", optional: false]}, {:phoenix_html, "~> 2.9", [hex: :phoenix_html, repo: "hexpm", optional: true]}, {:plug, "~> 1.0", [hex: :plug, repo: "hexpm", optional: false]}], "hexpm"}, "phoenix_html": {:hex, :phoenix_html, "2.13.1", "fa8f034b5328e2dfa0e4131b5569379003f34bc1fafdaa84985b0b9d2f12e68b", [:mix], [{:plug, "~> 1.5", [hex: :plug, repo: "hexpm", optional: false]}], "hexpm"}, "phoenix_pubsub": {:hex, :phoenix_pubsub, "1.1.2", "496c303bdf1b2e98a9d26e89af5bba3ab487ba3a3735f74bf1f4064d2a845a3e", [:mix], [], "hexpm"}, - "pleroma_job_queue": {:hex, :pleroma_job_queue, "0.2.0", "879e660aa1cebe8dc6f0aaaa6aa48b4875e89cd961d4a585fd128e0773b31a18", [:mix], [], "hexpm"}, + "phoenix_swoosh": {:hex, :phoenix_swoosh, "0.2.0", "a7e0b32077cd6d2323ae15198839b05d9caddfa20663fd85787479e81f89520e", [:mix], [{:phoenix, "~> 1.0", [hex: :phoenix, repo: "hexpm", optional: false]}, {:phoenix_html, "~> 2.2", [hex: :phoenix_html, repo: "hexpm", optional: false]}, {:swoosh, "~> 0.1", [hex: :swoosh, repo: "hexpm", optional: false]}], "hexpm"}, + "pleroma_job_queue": {:hex, :pleroma_job_queue, "0.3.0", "b84538d621f0c3d6fcc1cff9d5648d3faaf873b8b21b94e6503428a07a48ec47", [:mix], [{:crontab, "~> 1.1", [hex: :crontab, repo: "hexpm", optional: false]}], "hexpm"}, "plug": {:hex, :plug, "1.8.2", "0bcce1daa420f189a6491f3940cc77ea7fb1919761175c9c3b59800d897440fc", [:mix], [{:mime, "~> 1.0", [hex: :mime, repo: "hexpm", optional: false]}, {:plug_crypto, "~> 1.0", [hex: :plug_crypto, repo: "hexpm", optional: false]}, {:telemetry, "~> 0.4", [hex: :telemetry, repo: "hexpm", optional: true]}], "hexpm"}, "plug_cowboy": {:hex, :plug_cowboy, "2.1.0", "b75768153c3a8a9e8039d4b25bb9b14efbc58e9c4a6e6a270abff1cd30cbe320", [:mix], [{:cowboy, "~> 2.5", [hex: :cowboy, repo: "hexpm", optional: false]}, {:plug, "~> 1.7", [hex: :plug, repo: "hexpm", optional: false]}], "hexpm"}, "plug_crypto": {:hex, :plug_crypto, "1.0.0", "18e49317d3fa343f24620ed22795ec29d4a5e602d52d1513ccea0b07d8ea7d4d", [:mix], [], "hexpm"}, "plug_static_index_html": {:hex, :plug_static_index_html, "1.0.0", "840123d4d3975585133485ea86af73cb2600afd7f2a976f9f5fd8b3808e636a0", [:mix], [{:plug, "~> 1.0", [hex: :plug, repo: "hexpm", optional: false]}], "hexpm"}, "poison": {:hex, :poison, "3.1.0", "d9eb636610e096f86f25d9a46f35a9facac35609a7591b3be3326e99a0484665", [:mix], [], "hexpm"}, - "poolboy": {:hex, :poolboy, "1.5.2", "392b007a1693a64540cead79830443abf5762f5d30cf50bc95cb2c1aaafa006b", [:rebar3], [], "hexpm"}, "postgrex": {:hex, :postgrex, "0.14.3", "5754dee2fdf6e9e508cbf49ab138df964278700b764177e8f3871e658b345a1e", [:mix], [{:connection, "~> 1.0", [hex: :connection, repo: "hexpm", optional: false]}, {:db_connection, "~> 2.0", [hex: :db_connection, repo: "hexpm", optional: false]}, {:decimal, "~> 1.5", [hex: :decimal, repo: "hexpm", optional: false]}, {:jason, "~> 1.0", [hex: :jason, repo: "hexpm", optional: true]}], "hexpm"}, "prometheus": {:hex, :prometheus, "4.4.1", "1e96073b3ed7788053768fea779cbc896ddc3bdd9ba60687f2ad50b252ac87d6", [:mix, :rebar3], [], "hexpm"}, "prometheus_ecto": {:hex, :prometheus_ecto, "1.4.1", "6c768ea9654de871e5b32fab2eac348467b3021604ebebbcbd8bcbe806a65ed5", [:mix], [{:ecto, "~> 2.0 or ~> 3.0", [hex: :ecto, repo: "hexpm", optional: false]}, {:prometheus_ex, "~> 1.1 or ~> 2.0 or ~> 3.0", [hex: :prometheus_ex, repo: "hexpm", optional: false]}], "hexpm"}, "prometheus_ex": {:hex, :prometheus_ex, "3.0.5", "fa58cfd983487fc5ead331e9a3e0aa622c67232b3ec71710ced122c4c453a02f", [:mix], [{:prometheus, "~> 4.0", [hex: :prometheus, repo: "hexpm", optional: false]}], "hexpm"}, "prometheus_phoenix": {:hex, :prometheus_phoenix, "1.3.0", "c4b527e0b3a9ef1af26bdcfbfad3998f37795b9185d475ca610fe4388fdd3bb5", [:mix], [{:phoenix, "~> 1.4", [hex: :phoenix, repo: "hexpm", optional: false]}, {:prometheus_ex, "~> 1.3 or ~> 2.0 or ~> 3.0", [hex: :prometheus_ex, repo: "hexpm", optional: false]}], "hexpm"}, "prometheus_plugs": {:hex, :prometheus_plugs, "1.1.5", "25933d48f8af3a5941dd7b621c889749894d8a1082a6ff7c67cc99dec26377c5", [:mix], [{:accept, "~> 0.1", [hex: :accept, repo: "hexpm", optional: false]}, {:plug, "~> 1.0", [hex: :plug, repo: "hexpm", optional: false]}, {:prometheus_ex, "~> 1.1 or ~> 2.0 or ~> 3.0", [hex: :prometheus_ex, repo: "hexpm", optional: false]}, {:prometheus_process_collector, "~> 1.1", [hex: :prometheus_process_collector, repo: "hexpm", optional: true]}], "hexpm"}, - "prometheus_process_collector": {:hex, :prometheus_process_collector, "1.4.0", "6dbd39e3165b9ef1c94a7a820e9ffe08479f949dcdd431ed4aaea7b250eebfde", [:rebar3], [{:prometheus, "~> 4.0", [hex: :prometheus, repo: "hexpm", optional: false]}], "hexpm"}, "quack": {:hex, :quack, "0.1.1", "cca7b4da1a233757fdb44b3334fce80c94785b3ad5a602053b7a002b5a8967bf", [:mix], [{:poison, ">= 1.0.0", [hex: :poison, repo: "hexpm", optional: false]}, {:tesla, "~> 1.2.0", [hex: :tesla, repo: "hexpm", optional: false]}], "hexpm"}, "ranch": {:hex, :ranch, "1.7.1", "6b1fab51b49196860b733a49c07604465a47bdb78aa10c1c16a3d199f7f8c881", [:rebar3], [], "hexpm"}, "recon": {:git, "https://github.com/ferd/recon.git", "75d70c7c08926d2f24f1ee6de14ee50fe8a52763", [tag: "2.4.0"]}, "ssl_verify_fun": {:hex, :ssl_verify_fun, "1.1.4", "f0eafff810d2041e93f915ef59899c923f4568f4585904d010387ed74988e77b", [:make, :mix, :rebar3], [], "hexpm"}, + "sweet_xml": {:hex, :sweet_xml, "0.6.6", "fc3e91ec5dd7c787b6195757fbcf0abc670cee1e4172687b45183032221b66b8", [:mix], [], "hexpm"}, "swoosh": {:hex, :swoosh, "0.23.2", "7dda95ff0bf54a2298328d6899c74dae1223777b43563ccebebb4b5d2b61df38", [:mix], [{:cowboy, "~> 1.0.1 or ~> 1.1 or ~> 2.4", [hex: :cowboy, repo: "hexpm", optional: true]}, {:gen_smtp, "~> 0.13", [hex: :gen_smtp, repo: "hexpm", optional: true]}, {:hackney, "~> 1.9", [hex: :hackney, repo: "hexpm", optional: false]}, {:jason, "~> 1.0", [hex: :jason, repo: "hexpm", optional: false]}, {:mail, "~> 0.2", [hex: :mail, repo: "hexpm", optional: true]}, {:mime, "~> 1.1", [hex: :mime, repo: "hexpm", optional: false]}, {:plug_cowboy, ">= 1.0.0", [hex: :plug_cowboy, repo: "hexpm", optional: true]}], "hexpm"}, "syslog": {:git, "https://github.com/Vagabond/erlang-syslog.git", "4a6c6f2c996483e86c1320e9553f91d337bcb6aa", [tag: "1.0.5"]}, "telemetry": {:hex, :telemetry, "0.4.0", "8339bee3fa8b91cb84d14c2935f8ecf399ccd87301ad6da6b71c09553834b2ab", [:rebar3], [], "hexpm"}, "tesla": {:hex, :tesla, "1.2.1", "864783cc27f71dd8c8969163704752476cec0f3a51eb3b06393b3971dc9733ff", [:mix], [{:exjsx, ">= 3.0.0", [hex: :exjsx, repo: "hexpm", optional: true]}, {:fuse, "~> 2.4", [hex: :fuse, repo: "hexpm", optional: true]}, {:hackney, "~> 1.6", [hex: :hackney, repo: "hexpm", optional: true]}, {:ibrowse, "~> 4.4.0", [hex: :ibrowse, repo: "hexpm", optional: true]}, {:jason, ">= 1.0.0", [hex: :jason, repo: "hexpm", optional: true]}, {:mime, "~> 1.0", [hex: :mime, repo: "hexpm", optional: false]}, {:poison, ">= 1.0.0", [hex: :poison, repo: "hexpm", optional: true]}], "hexpm"}, "timex": {:hex, :timex, "3.6.1", "efdf56d0e67a6b956cc57774353b0329c8ab7726766a11547e529357ffdc1d56", [:mix], [{:combine, "~> 0.10", [hex: :combine, repo: "hexpm", optional: false]}, {:gettext, "~> 0.10", [hex: :gettext, repo: "hexpm", optional: false]}, {:tzdata, "~> 0.1.8 or ~> 0.5 or ~> 1.0.0", [hex: :tzdata, repo: "hexpm", optional: false]}], "hexpm"}, "trailing_format_plug": {:hex, :trailing_format_plug, "0.0.7", "64b877f912cf7273bed03379936df39894149e35137ac9509117e59866e10e45", [:mix], [{:plug, "> 0.12.0", [hex: :plug, repo: "hexpm", optional: false]}], "hexpm"}, - "tzdata": {:hex, :tzdata, "1.0.1", "f6027a331af7d837471248e62733c6ebee86a72e57c613aa071ebb1f750fc71a", [:mix], [{:hackney, "~> 1.0", [hex: :hackney, repo: "hexpm", optional: false]}], "hexpm"}, + "tzdata": {:hex, :tzdata, "0.5.21", "8cbf3607fcce69636c672d5be2bbb08687fe26639a62bdcc283d267277db7cf0", [:mix], [{:hackney, "~> 1.0", [hex: :hackney, repo: "hexpm", optional: false]}], "hexpm"}, "ueberauth": {:hex, :ueberauth, "0.6.1", "9e90d3337dddf38b1ca2753aca9b1e53d8a52b890191cdc55240247c89230412", [:mix], [{:plug, "~> 1.5", [hex: :plug, repo: "hexpm", optional: false]}], "hexpm"}, "unicode_util_compat": {:hex, :unicode_util_compat, "0.4.1", "d869e4c68901dd9531385bb0c8c40444ebf624e60b6962d95952775cac5e90cd", [:rebar3], [], "hexpm"}, - "unsafe": {:hex, :unsafe, "1.0.0", "7c21742cd05380c7875546b023481d3a26f52df8e5dfedcb9f958f322baae305", [:mix], [], "hexpm"}, + "unsafe": {:hex, :unsafe, "1.0.1", "a27e1874f72ee49312e0a9ec2e0b27924214a05e3ddac90e91727bc76f8613d8", [:mix], [], "hexpm"}, "web_push_encryption": {:hex, :web_push_encryption, "0.2.1", "d42cecf73420d9dc0053ba3299cc8c8d6ff2be2487d67ca2a57265868e4d9a98", [:mix], [{:httpoison, "~> 1.0", [hex: :httpoison, repo: "hexpm", optional: false]}, {:jose, "~> 1.8", [hex: :jose, repo: "hexpm", optional: false]}, {:poison, "~> 3.0", [hex: :poison, repo: "hexpm", optional: false]}], "hexpm"}, "websocket_client": {:git, "https://github.com/jeremyong/websocket_client.git", "9a6f65d05ebf2725d62fb19262b21f1805a59fbf", []}, } diff --git a/priv/repo/migrations/20190205114625_create_thread_mutes.exs b/priv/repo/migrations/20190205114625_create_thread_mutes.exs index 7e44db121..baaf07253 100644 --- a/priv/repo/migrations/20190205114625_create_thread_mutes.exs +++ b/priv/repo/migrations/20190205114625_create_thread_mutes.exs @@ -6,7 +6,7 @@ def change do add :user_id, references(:users, type: :uuid, on_delete: :delete_all) add :context, :string end - + create_if_not_exists unique_index(:thread_mutes, [:user_id, :context], name: :unique_index) end end diff --git a/priv/repo/migrations/20190412052952_add_user_info_fields.exs b/priv/repo/migrations/20190412052952_add_user_info_fields.exs new file mode 100644 index 000000000..646c91f32 --- /dev/null +++ b/priv/repo/migrations/20190412052952_add_user_info_fields.exs @@ -0,0 +1,20 @@ +defmodule Pleroma.Repo.Migrations.AddEmailNotificationsToUserInfo do + use Ecto.Migration + + def up do + execute(" + UPDATE users + SET info = info || '{ + \"email_notifications\": { + \"digest\": false + } + }'") + end + + def down do + execute(" + UPDATE users + SET info = info - 'email_notifications' + ") + end +end diff --git a/priv/repo/migrations/20190413085040_add_signin_and_last_digest_dates_to_user.exs b/priv/repo/migrations/20190413085040_add_signin_and_last_digest_dates_to_user.exs new file mode 100644 index 000000000..4312b171f --- /dev/null +++ b/priv/repo/migrations/20190413085040_add_signin_and_last_digest_dates_to_user.exs @@ -0,0 +1,9 @@ +defmodule Pleroma.Repo.Migrations.AddSigninAndLastDigestDatesToUser do + use Ecto.Migration + + def change do + alter table(:users) do + add(:last_digest_emailed_at, :naive_datetime, default: fragment("now()")) + end + end +end diff --git a/priv/repo/migrations/20190801154554_create_conversation_participation_recipient_ships.exs b/priv/repo/migrations/20190801154554_create_conversation_participation_recipient_ships.exs new file mode 100644 index 000000000..c6e3469d5 --- /dev/null +++ b/priv/repo/migrations/20190801154554_create_conversation_participation_recipient_ships.exs @@ -0,0 +1,13 @@ +defmodule Pleroma.Repo.Migrations.CreateConversationParticipationRecipientShips do + use Ecto.Migration + + def change do + create_if_not_exists table(:conversation_participation_recipient_ships) do + add(:user_id, references(:users, type: :uuid, on_delete: :delete_all)) + add(:participation_id, references(:conversation_participations, on_delete: :delete_all)) + end + + create_if_not_exists index(:conversation_participation_recipient_ships, [:user_id]) + create_if_not_exists index(:conversation_participation_recipient_ships, [:participation_id]) + end +end diff --git a/priv/repo/migrations/20190823000549_add_likes_index_to_objects.exs b/priv/repo/migrations/20190823000549_add_likes_index_to_objects.exs new file mode 100644 index 000000000..13f3d6e83 --- /dev/null +++ b/priv/repo/migrations/20190823000549_add_likes_index_to_objects.exs @@ -0,0 +1,7 @@ +defmodule Pleroma.Repo.Migrations.AddLikesIndexToObjects do + use Ecto.Migration + + def change do + create_if_not_exists index(:objects, ["(data->'likes')"], using: :gin, name: :objects_likes) + end +end diff --git a/priv/static/adminfe/chunk-0e18.e12401fb.css b/priv/static/adminfe/chunk-0e18.e12401fb.css new file mode 100644 index 000000000..ba85e77d5 Binary files /dev/null and b/priv/static/adminfe/chunk-0e18.e12401fb.css differ diff --git a/priv/static/adminfe/chunk-56c9.c27dac5e.css b/priv/static/adminfe/chunk-1fbf.d7a1893c.css similarity index 95% rename from priv/static/adminfe/chunk-56c9.c27dac5e.css rename to priv/static/adminfe/chunk-1fbf.d7a1893c.css index 2b4283ec5..4672a9f75 100644 Binary files a/priv/static/adminfe/chunk-56c9.c27dac5e.css and b/priv/static/adminfe/chunk-1fbf.d7a1893c.css differ diff --git a/priv/static/adminfe/chunk-f018.0d22684d.css b/priv/static/adminfe/chunk-2325.0d22684d.css similarity index 100% rename from priv/static/adminfe/chunk-f018.0d22684d.css rename to priv/static/adminfe/chunk-2325.0d22684d.css diff --git a/priv/static/adminfe/chunk-5e57.ac97b15a.css b/priv/static/adminfe/chunk-5e57.ac97b15a.css new file mode 100644 index 000000000..0c9284744 Binary files /dev/null and b/priv/static/adminfe/chunk-5e57.ac97b15a.css differ diff --git a/priv/static/adminfe/chunk-5eaf.1a04e979.css b/priv/static/adminfe/chunk-e547.e4b6230b.css similarity index 60% rename from priv/static/adminfe/chunk-5eaf.1a04e979.css rename to priv/static/adminfe/chunk-e547.e4b6230b.css index a09287f58..f740543a0 100644 Binary files a/priv/static/adminfe/chunk-5eaf.1a04e979.css and b/priv/static/adminfe/chunk-e547.e4b6230b.css differ diff --git a/priv/static/adminfe/chunk-elementUI.e5cd8da6.css b/priv/static/adminfe/chunk-elementUI.e5cd8da6.css new file mode 100644 index 000000000..3fef5e5fd Binary files /dev/null and b/priv/static/adminfe/chunk-elementUI.e5cd8da6.css differ diff --git a/priv/static/adminfe/chunk-elementUI.f74c256b.css b/priv/static/adminfe/chunk-elementUI.f74c256b.css deleted file mode 100644 index c8d56344e..000000000 Binary files a/priv/static/adminfe/chunk-elementUI.f74c256b.css and /dev/null differ diff --git a/priv/static/adminfe/index.html b/priv/static/adminfe/index.html index 2ef6362ba..c31247c03 100644 --- a/priv/static/adminfe/index.html +++ b/priv/static/adminfe/index.html @@ -1 +1 @@ -Admin FE
\ No newline at end of file +Admin FE
\ No newline at end of file diff --git a/priv/static/adminfe/static/fonts/element-icons.2fad952.woff b/priv/static/adminfe/static/fonts/element-icons.2fad952.woff deleted file mode 100644 index 28da65d49..000000000 Binary files a/priv/static/adminfe/static/fonts/element-icons.2fad952.woff and /dev/null differ diff --git a/priv/static/adminfe/static/fonts/element-icons.535877f.woff b/priv/static/adminfe/static/fonts/element-icons.535877f.woff new file mode 100644 index 000000000..02b9a2539 Binary files /dev/null and b/priv/static/adminfe/static/fonts/element-icons.535877f.woff differ diff --git a/priv/static/adminfe/static/fonts/element-icons.6f0a763.ttf b/priv/static/adminfe/static/fonts/element-icons.6f0a763.ttf deleted file mode 100644 index 73bc90f4a..000000000 Binary files a/priv/static/adminfe/static/fonts/element-icons.6f0a763.ttf and /dev/null differ diff --git a/priv/static/adminfe/static/fonts/element-icons.732389d.ttf b/priv/static/adminfe/static/fonts/element-icons.732389d.ttf new file mode 100644 index 000000000..91b74de36 Binary files /dev/null and b/priv/static/adminfe/static/fonts/element-icons.732389d.ttf differ diff --git a/priv/static/adminfe/static/js/app.4137ad8f.js b/priv/static/adminfe/static/js/app.4137ad8f.js deleted file mode 100644 index bb4b6ec49..000000000 Binary files a/priv/static/adminfe/static/js/app.4137ad8f.js and /dev/null differ diff --git a/priv/static/adminfe/static/js/app.8e186193.js b/priv/static/adminfe/static/js/app.8e186193.js new file mode 100644 index 000000000..207bbeaa6 Binary files /dev/null and b/priv/static/adminfe/static/js/app.8e186193.js differ diff --git a/priv/static/adminfe/static/js/chunk-0e18.208cd826.js b/priv/static/adminfe/static/js/chunk-0e18.208cd826.js new file mode 100644 index 000000000..eb7100ecd Binary files /dev/null and b/priv/static/adminfe/static/js/chunk-0e18.208cd826.js differ diff --git a/priv/static/adminfe/static/js/chunk-1fbf.616fb309.js b/priv/static/adminfe/static/js/chunk-1fbf.616fb309.js new file mode 100644 index 000000000..5ad34d801 Binary files /dev/null and b/priv/static/adminfe/static/js/chunk-1fbf.616fb309.js differ diff --git a/priv/static/adminfe/static/js/chunk-f018.e1a7a454.js b/priv/static/adminfe/static/js/chunk-2325.154a537b.js similarity index 98% rename from priv/static/adminfe/static/js/chunk-f018.e1a7a454.js rename to priv/static/adminfe/static/js/chunk-2325.154a537b.js index 9c06e442c..3fe9add82 100644 Binary files a/priv/static/adminfe/static/js/chunk-f018.e1a7a454.js and b/priv/static/adminfe/static/js/chunk-2325.154a537b.js differ diff --git a/priv/static/adminfe/static/js/chunk-56c9.28e35fc3.js b/priv/static/adminfe/static/js/chunk-56c9.28e35fc3.js deleted file mode 100644 index 6f92e0e72..000000000 Binary files a/priv/static/adminfe/static/js/chunk-56c9.28e35fc3.js and /dev/null differ diff --git a/priv/static/adminfe/static/js/chunk-5e57.7313703a.js b/priv/static/adminfe/static/js/chunk-5e57.7313703a.js new file mode 100644 index 000000000..e16366179 Binary files /dev/null and b/priv/static/adminfe/static/js/chunk-5e57.7313703a.js differ diff --git a/priv/static/adminfe/static/js/chunk-5eaf.5b76e416.js b/priv/static/adminfe/static/js/chunk-5eaf.5b76e416.js deleted file mode 100644 index 56f1b6891..000000000 Binary files a/priv/static/adminfe/static/js/chunk-5eaf.5b76e416.js and /dev/null differ diff --git a/priv/static/adminfe/static/js/chunk-7fe2.458f9da5.js b/priv/static/adminfe/static/js/chunk-7fe2.458f9da5.js new file mode 100644 index 000000000..4442e3e24 Binary files /dev/null and b/priv/static/adminfe/static/js/chunk-7fe2.458f9da5.js differ diff --git a/priv/static/adminfe/static/js/chunk-e547.d57d1b91.js b/priv/static/adminfe/static/js/chunk-e547.d57d1b91.js new file mode 100644 index 000000000..788164466 Binary files /dev/null and b/priv/static/adminfe/static/js/chunk-e547.d57d1b91.js differ diff --git a/priv/static/adminfe/static/js/chunk-elementUI.1911151b.js b/priv/static/adminfe/static/js/chunk-elementUI.1911151b.js new file mode 100644 index 000000000..d11c13e49 Binary files /dev/null and b/priv/static/adminfe/static/js/chunk-elementUI.1911151b.js differ diff --git a/priv/static/adminfe/static/js/chunk-elementUI.1fa5434b.js b/priv/static/adminfe/static/js/chunk-elementUI.1fa5434b.js deleted file mode 100644 index 8f6f193b7..000000000 Binary files a/priv/static/adminfe/static/js/chunk-elementUI.1fa5434b.js and /dev/null differ diff --git a/priv/static/adminfe/static/js/chunk-libs.d5609760.js b/priv/static/adminfe/static/js/chunk-libs.fb0b7f4a.js similarity index 71% rename from priv/static/adminfe/static/js/chunk-libs.d5609760.js rename to priv/static/adminfe/static/js/chunk-libs.fb0b7f4a.js index 5fa09cf69..e7f33e6c3 100644 Binary files a/priv/static/adminfe/static/js/chunk-libs.d5609760.js and b/priv/static/adminfe/static/js/chunk-libs.fb0b7f4a.js differ diff --git a/priv/static/adminfe/static/js/runtime.d8d12c12.js b/priv/static/adminfe/static/js/runtime.d8d12c12.js deleted file mode 100644 index 6d8ac5af6..000000000 Binary files a/priv/static/adminfe/static/js/runtime.d8d12c12.js and /dev/null differ diff --git a/priv/static/adminfe/static/js/runtime.f40c8ec4.js b/priv/static/adminfe/static/js/runtime.f40c8ec4.js new file mode 100644 index 000000000..12796dafa Binary files /dev/null and b/priv/static/adminfe/static/js/runtime.f40c8ec4.js differ diff --git a/priv/static/emoji/f_00b.png b/priv/static/emoji/f_00b.png deleted file mode 100644 index 3d00b89b0..000000000 Binary files a/priv/static/emoji/f_00b.png and /dev/null differ diff --git a/priv/static/emoji/f_00b11b.png b/priv/static/emoji/f_00b11b.png deleted file mode 100644 index 3e99ce464..000000000 Binary files a/priv/static/emoji/f_00b11b.png and /dev/null differ diff --git a/priv/static/emoji/f_00b33b.png b/priv/static/emoji/f_00b33b.png deleted file mode 100644 index 8f4929297..000000000 Binary files a/priv/static/emoji/f_00b33b.png and /dev/null differ diff --git a/priv/static/emoji/f_00h.png b/priv/static/emoji/f_00h.png deleted file mode 100644 index ba3da57c6..000000000 Binary files a/priv/static/emoji/f_00h.png and /dev/null differ diff --git a/priv/static/emoji/f_00t.png b/priv/static/emoji/f_00t.png deleted file mode 100644 index 31d98b433..000000000 Binary files a/priv/static/emoji/f_00t.png and /dev/null differ diff --git a/priv/static/emoji/f_01b.png b/priv/static/emoji/f_01b.png deleted file mode 100644 index 7bd2582c5..000000000 Binary files a/priv/static/emoji/f_01b.png and /dev/null differ diff --git a/priv/static/emoji/f_03b.png b/priv/static/emoji/f_03b.png deleted file mode 100644 index 9e4ff1bf7..000000000 Binary files a/priv/static/emoji/f_03b.png and /dev/null differ diff --git a/priv/static/emoji/f_10b.png b/priv/static/emoji/f_10b.png deleted file mode 100644 index 67c6493fc..000000000 Binary files a/priv/static/emoji/f_10b.png and /dev/null differ diff --git a/priv/static/emoji/f_11b.png b/priv/static/emoji/f_11b.png deleted file mode 100644 index b53328ba9..000000000 Binary files a/priv/static/emoji/f_11b.png and /dev/null differ diff --git a/priv/static/emoji/f_11b00b.png b/priv/static/emoji/f_11b00b.png deleted file mode 100644 index c4c30e11f..000000000 Binary files a/priv/static/emoji/f_11b00b.png and /dev/null differ diff --git a/priv/static/emoji/f_11b22b.png b/priv/static/emoji/f_11b22b.png deleted file mode 100644 index 47425e06e..000000000 Binary files a/priv/static/emoji/f_11b22b.png and /dev/null differ diff --git a/priv/static/emoji/f_11h.png b/priv/static/emoji/f_11h.png deleted file mode 100644 index 28342363a..000000000 Binary files a/priv/static/emoji/f_11h.png and /dev/null differ diff --git a/priv/static/emoji/f_11t.png b/priv/static/emoji/f_11t.png deleted file mode 100644 index dca67dc70..000000000 Binary files a/priv/static/emoji/f_11t.png and /dev/null differ diff --git a/priv/static/emoji/f_12b.png b/priv/static/emoji/f_12b.png deleted file mode 100644 index 9925adb7c..000000000 Binary files a/priv/static/emoji/f_12b.png and /dev/null differ diff --git a/priv/static/emoji/f_21b.png b/priv/static/emoji/f_21b.png deleted file mode 100644 index aa56d2cb2..000000000 Binary files a/priv/static/emoji/f_21b.png and /dev/null differ diff --git a/priv/static/emoji/f_22b.png b/priv/static/emoji/f_22b.png deleted file mode 100644 index 426878986..000000000 Binary files a/priv/static/emoji/f_22b.png and /dev/null differ diff --git a/priv/static/emoji/f_22b11b.png b/priv/static/emoji/f_22b11b.png deleted file mode 100644 index 4bdfb3107..000000000 Binary files a/priv/static/emoji/f_22b11b.png and /dev/null differ diff --git a/priv/static/emoji/f_22b33b.png b/priv/static/emoji/f_22b33b.png deleted file mode 100644 index adf94f811..000000000 Binary files a/priv/static/emoji/f_22b33b.png and /dev/null differ diff --git a/priv/static/emoji/f_22h.png b/priv/static/emoji/f_22h.png deleted file mode 100644 index 3b27e2de8..000000000 Binary files a/priv/static/emoji/f_22h.png and /dev/null differ diff --git a/priv/static/emoji/f_22t.png b/priv/static/emoji/f_22t.png deleted file mode 100644 index addd9fec7..000000000 Binary files a/priv/static/emoji/f_22t.png and /dev/null differ diff --git a/priv/static/emoji/f_23b.png b/priv/static/emoji/f_23b.png deleted file mode 100644 index beb69ab36..000000000 Binary files a/priv/static/emoji/f_23b.png and /dev/null differ diff --git a/priv/static/emoji/f_30b.png b/priv/static/emoji/f_30b.png deleted file mode 100644 index 41dbb2a5d..000000000 Binary files a/priv/static/emoji/f_30b.png and /dev/null differ diff --git a/priv/static/emoji/f_32b.png b/priv/static/emoji/f_32b.png deleted file mode 100644 index d8261e8a8..000000000 Binary files a/priv/static/emoji/f_32b.png and /dev/null differ diff --git a/priv/static/emoji/f_33b.png b/priv/static/emoji/f_33b.png deleted file mode 100644 index 71b8b914a..000000000 Binary files a/priv/static/emoji/f_33b.png and /dev/null differ diff --git a/priv/static/emoji/f_33b00b.png b/priv/static/emoji/f_33b00b.png deleted file mode 100644 index 65b6e24b8..000000000 Binary files a/priv/static/emoji/f_33b00b.png and /dev/null differ diff --git a/priv/static/emoji/f_33b22b.png b/priv/static/emoji/f_33b22b.png deleted file mode 100644 index d71a8ddd4..000000000 Binary files a/priv/static/emoji/f_33b22b.png and /dev/null differ diff --git a/priv/static/emoji/f_33h.png b/priv/static/emoji/f_33h.png deleted file mode 100644 index e141c5184..000000000 Binary files a/priv/static/emoji/f_33h.png and /dev/null differ diff --git a/priv/static/emoji/f_33t.png b/priv/static/emoji/f_33t.png deleted file mode 100644 index d5a23073d..000000000 Binary files a/priv/static/emoji/f_33t.png and /dev/null differ diff --git a/priv/templates/sample_config.eex b/priv/templates/sample_config.eex index ca9c7a2c2..dc75d4008 100644 --- a/priv/templates/sample_config.eex +++ b/priv/templates/sample_config.eex @@ -68,3 +68,5 @@ config :pleroma, Pleroma.Uploaders.Local, uploads: "<%= uploads_dir %>" # For using third-party S3 clones like wasabi, also do: # config :ex_aws, :s3, # host: "s3.wasabisys.com" + +config :joken, default_signer: "<%= jwt_secret %>" diff --git a/test/config/transfer_task_test.exs b/test/config/transfer_task_test.exs index dbeadbe87..9074f3b97 100644 --- a/test/config/transfer_task_test.exs +++ b/test/config/transfer_task_test.exs @@ -5,14 +5,8 @@ defmodule Pleroma.Config.TransferTaskTest do use Pleroma.DataCase - setup do - dynamic = Pleroma.Config.get([:instance, :dynamic_configuration]) - + clear_config([:instance, :dynamic_configuration]) do Pleroma.Config.put([:instance, :dynamic_configuration], true) - - on_exit(fn -> - Pleroma.Config.put([:instance, :dynamic_configuration], dynamic) - end) end test "transfer config values from db to env" do @@ -31,7 +25,7 @@ test "transfer config values from db to env" do value: [live: 15, com: 35] }) - Pleroma.Config.TransferTask.start_link() + Pleroma.Config.TransferTask.start_link([]) assert Application.get_env(:pleroma, :test_key) == [live: 2, com: 3] assert Application.get_env(:idna, :test_key) == [live: 15, com: 35] @@ -50,7 +44,7 @@ test "non existing atom" do }) assert ExUnit.CaptureLog.capture_log(fn -> - Pleroma.Config.TransferTask.start_link() + Pleroma.Config.TransferTask.start_link([]) end) =~ "updating env causes error, key: \"undefined_atom_key\", error: %ArgumentError{message: \"argument error\"}" end diff --git a/test/conversation/participation_test.exs b/test/conversation/participation_test.exs index 2a03e5d67..a27167d42 100644 --- a/test/conversation/participation_test.exs +++ b/test/conversation/participation_test.exs @@ -8,6 +8,50 @@ defmodule Pleroma.Conversation.ParticipationTest do alias Pleroma.Conversation.Participation alias Pleroma.Web.CommonAPI + test "getting a participation will also preload things" do + user = insert(:user) + other_user = insert(:user) + + {:ok, _activity} = + CommonAPI.post(user, %{"status" => "Hey @#{other_user.nickname}.", "visibility" => "direct"}) + + [participation] = Participation.for_user(user) + + participation = Participation.get(participation.id, preload: [:conversation]) + + assert %Pleroma.Conversation{} = participation.conversation + end + + test "for a new conversation, it sets the recipents of the participation" do + user = insert(:user) + other_user = insert(:user) + third_user = insert(:user) + + {:ok, activity} = + CommonAPI.post(user, %{"status" => "Hey @#{other_user.nickname}.", "visibility" => "direct"}) + + [participation] = Participation.for_user(user) + participation = Pleroma.Repo.preload(participation, :recipients) + + assert length(participation.recipients) == 2 + assert user in participation.recipients + assert other_user in participation.recipients + + # Mentioning another user in the same conversation will not add a new recipients. + + {:ok, _activity} = + CommonAPI.post(user, %{ + "in_reply_to_status_id" => activity.id, + "status" => "Hey @#{third_user.nickname}.", + "visibility" => "direct" + }) + + [participation] = Participation.for_user(user) + participation = Pleroma.Repo.preload(participation, :recipients) + + assert length(participation.recipients) == 2 + end + test "it creates a participation for a conversation and a user" do user = insert(:user) conversation = insert(:conversation) @@ -102,4 +146,23 @@ test "Doesn't die when the conversation gets empty" do [] = Participation.for_user_with_last_activity_id(user) end + + test "it sets recipients, always keeping the owner of the participation even when not explicitly set" do + user = insert(:user) + other_user = insert(:user) + + {:ok, _activity} = CommonAPI.post(user, %{"status" => ".", "visibility" => "direct"}) + [participation] = Participation.for_user_with_last_activity_id(user) + + participation = Repo.preload(participation, :recipients) + + assert participation.recipients |> length() == 1 + assert user in participation.recipients + + {:ok, participation} = Participation.set_recipients(participation, [other_user.id]) + + assert participation.recipients |> length() == 2 + assert user in participation.recipients + assert other_user in participation.recipients + end end diff --git a/test/conversation_test.exs b/test/conversation_test.exs index aa193e0d4..4e36494f8 100644 --- a/test/conversation_test.exs +++ b/test/conversation_test.exs @@ -11,14 +11,8 @@ defmodule Pleroma.ConversationTest do import Pleroma.Factory - setup_all do - config_path = [:instance, :federating] - initial_setting = Pleroma.Config.get(config_path) - - Pleroma.Config.put(config_path, true) - on_exit(fn -> Pleroma.Config.put(config_path, initial_setting) end) - - :ok + clear_config_all([:instance, :federating]) do + Pleroma.Config.put([:instance, :federating], true) end test "it goes through old direct conversations" do diff --git a/test/emails/admin_email_test.exs b/test/emails/admin_email_test.exs index 4bf54b0c2..9e83c73c6 100644 --- a/test/emails/admin_email_test.exs +++ b/test/emails/admin_email_test.exs @@ -24,7 +24,6 @@ test "build report email" do assert res.to == [{to_user.name, to_user.email}] assert res.from == {config[:name], config[:notify_email]} - assert res.reply_to == {reporter.name, reporter.email} assert res.subject == "#{config[:name]} Report" assert res.html_body == @@ -34,4 +33,17 @@ test "build report email" do status_url }\">#{status_url}\n \n

\n\n" end + + test "it works when the reporter is a remote user without email" do + config = Pleroma.Config.get(:instance) + to_user = insert(:user) + reporter = insert(:user, email: nil, local: false) + account = insert(:user) + + res = + AdminEmail.report(to_user, reporter, account, [%{name: "Test", id: "12"}], "Test comment") + + assert res.to == [{to_user.name, to_user.email}] + assert res.from == {config[:name], config[:notify_email]} + end end diff --git a/test/emails/mailer_test.exs b/test/emails/mailer_test.exs index 450bb09c7..ae5effb7a 100644 --- a/test/emails/mailer_test.exs +++ b/test/emails/mailer_test.exs @@ -15,11 +15,7 @@ defmodule Pleroma.Emails.MailerTest do to: [{"Test User", "user1@example.com"}] } - setup do - value = Pleroma.Config.get([Pleroma.Emails.Mailer, :enabled]) - on_exit(fn -> Pleroma.Config.put([Pleroma.Emails.Mailer, :enabled], value) end) - :ok - end + clear_config([Pleroma.Emails.Mailer, :enabled]) test "not send email when mailer is disabled" do Pleroma.Config.put([Pleroma.Emails.Mailer, :enabled], false) diff --git a/test/fixtures/mastodon-update.json b/test/fixtures/mastodon-update.json index f6713fea5..dbf8b6dff 100644 --- a/test/fixtures/mastodon-update.json +++ b/test/fixtures/mastodon-update.json @@ -1,10 +1,10 @@ -{ - "type": "Update", - "object": { - "url": "http://mastodon.example.org/@gargron", - "type": "Person", - "summary": "

Some bio

", - "publicKey": { +{ + "type": "Update", + "object": { + "url": "http://mastodon.example.org/@gargron", + "type": "Person", + "summary": "

Some bio

", + "publicKey": { "publicKeyPem": "-----BEGIN PUBLIC KEY-----\nMIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIBCgKCAQEA0gs3VnQf6am3R+CeBV4H\nlfI1HZTNRIBHgvFszRZkCERbRgEWMu+P+I6/7GJC5H5jhVQ60z4MmXcyHOGmYMK/\n5XyuHQz7V2Ssu1AxLfRN5Biq1ayb0+DT/E7QxNXDJPqSTnstZ6C7zKH/uAETqg3l\nBonjCQWyds+IYbQYxf5Sp3yhvQ80lMwHML3DaNCMlXWLoOnrOX5/yK5+dedesg2\n/HIvGk+HEt36vm6hoH7bwPuEkgA++ACqwjXRe5Mta7i3eilHxFaF8XIrJFARV0t\nqOu4GID/jG6oA+swIWndGrtR2QRJIt9QIBFfK3HG5M0koZbY1eTqwNFRHFL3xaD\nUQIDAQAB\n-----END PUBLIC KEY-----\n", "owner": "http://mastodon.example.org/users/gargron", "id": "http://mastodon.example.org/users/gargron#main-key" @@ -20,7 +20,27 @@ "endpoints": { "sharedInbox": "http://mastodon.example.org/inbox" }, - "icon":{"type":"Image","mediaType":"image/jpeg","url":"https://cd.niu.moe/accounts/avatars/000/033/323/original/fd7f8ae0b3ffedc9.jpeg"},"image":{"type":"Image","mediaType":"image/png","url":"https://cd.niu.moe/accounts/headers/000/033/323/original/850b3448fa5fd477.png"} + "attachment": [{ + "type": "PropertyValue", + "name": "foo", + "value": "updated" + }, + { + "type": "PropertyValue", + "name": "foo1", + "value": "updated" + } + ], + "icon": { + "type": "Image", + "mediaType": "image/jpeg", + "url": "https://cd.niu.moe/accounts/avatars/000/033/323/original/fd7f8ae0b3ffedc9.jpeg" + }, + "image": { + "type": "Image", + "mediaType": "image/png", + "url": "https://cd.niu.moe/accounts/headers/000/033/323/original/850b3448fa5fd477.png" + } }, "id": "http://mastodon.example.org/users/gargron#updates/1519563538", "actor": "http://mastodon.example.org/users/gargron", diff --git a/test/fixtures/nypd-facial-recognition-children-teenagers.html b/test/fixtures/nypd-facial-recognition-children-teenagers.html new file mode 100644 index 000000000..5702c4484 --- /dev/null +++ b/test/fixtures/nypd-facial-recognition-children-teenagers.html @@ -0,0 +1,227 @@ + + + + She Was Arrested at 14. Then Her Photo Went to a Facial Recognition Database. - The New York Times + + + + + + + + + + + + + + + + + + + + + +

Advertisement

She Was Arrested at 14. Then Her Photo Went to a Facial Recognition Database.

With little oversight, the N.Y.P.D. has been using powerful surveillance technology on photos of children and teenagers.

Image
CreditCreditSarah Blesener for The New York Times
Joseph GoldsteinAli Watkins

[What you need to know to start the day: Get New York Today in your inbox.]

The New York Police Department has been loading thousands of arrest photos of children and teenagers into a facial recognition database despite evidence the technology has a higher risk of false matches in younger faces.

For about four years, internal records show, the department has used the technology to compare crime scene images with its collection of juvenile mug shots, the photos that are taken at an arrest. Most of the photos are of teenagers, largely 13 to 16 years old, but children as young as 11 have been included.

Elected officials and civil rights groups said the disclosure that the city was deploying a powerful surveillance tool on adolescents — whose privacy seems sacrosanct and whose status is protected in the criminal justice system — was a striking example of the Police Department’s ability to adopt advancing technology with little public scrutiny.

Several members of the City Council as well as a range of civil liberties groups said they were unaware of the policy until they were contacted by The New York Times.

Police Department officials defended the decision, saying it was just the latest evolution of a longstanding policing technique: using arrest photos to identify suspects.

“I don’t think this is any secret decision that’s made behind closed doors,” the city’s chief of detectives, Dermot F. Shea, said in an interview. “This is just process, and making sure we’re doing everything to fight crime.”

Other cities have begun to debate whether law enforcement should use facial recognition, which relies on an algorithm to quickly pore through images and suggest matches. In May, San Francisco blocked city agencies, including the police, from using the tool amid unease about potential government abuse. Detroit is facing public resistance to a technology that has been shown to have lower accuracy with people with darker skin.

In New York, the state Education Department recently told the Lockport, N.Y., school district to delay a plan to use facial recognition on students, citing privacy concerns.

“At the end of the day, it should be banned — no young people,” said Councilman Donovan Richards, a Queens Democrat who heads the Public Safety Committee, which oversees the Police Department.

The department said its legal bureau had approved using facial recognition on juveniles. The algorithm may suggest a lead, but detectives would not make an arrest based solely on that, Chief Shea said.

Image
CreditChang W. Lee/The New York Times

Still, facial recognition has not been widely tested on children. Most algorithms are taught to “think” based on adult faces, and there is growing evidence that they do not work as well on children.

The National Institute of Standards and Technology, which is part of the Commerce Department and evaluates facial recognition algorithms for accuracy, recently found the vast majority of more than 100 facial recognition algorithms had a higher rate of mistaken matches among children. The error rate was most pronounced in young children but was also seen in those aged 10 to 16.

Aging poses another problem: The appearance of children and adolescents can change drastically as bones stretch and shift, altering the underlying facial structure.

“I would use extreme caution in using those algorithms,” said Karl Ricanek Jr., a computer science professor and co-founder of the Face Aging Group at the University of North Carolina-Wilmington.

Technology that can match an image of a younger teenager to a recent arrest photo may be less effective at finding the same person even one or two years later, he said.

“The systems do not have the capacity to understand the dynamic changes that occur to a child’s face,” Dr. Ricanek said.

Idemia and DataWorks Plus, the two companies that provide facial recognition software to the Police Department, did not respond to requests for comment.

The New York Police Department can take arrest photos of minors as young as 11 who are charged with a felony, depending on the severity of the charge.

And in many cases, the department keeps the photos for years, making facial recognition comparisons to what may have effectively become outdated images. There are photos of 5,500 individuals in the juvenile database, 4,100 of whom are no longer 16 or under, the department said. Teenagers 17 and older are considered adults in the criminal justice system.

Police officials declined to provide statistics on how often their facial recognition systems provide false matches, or to explain how they evaluate the system’s effectiveness.

“We are comfortable with this technology because it has proved to be a valuable investigative method,” Chief Shea said. Facial recognition has helped lead to thousands of arrests of both adults and juveniles, the department has said.

Mayor Bill de Blasio had been aware the department was using the technology on minors, said Freddi Goldstein, a spokeswoman for the mayor.

She said the Police Department followed “strict guidelines” in applying the technology and City Hall monitored the agency’s compliance with the policies.

The Times learned details of the department’s use of facial recognition by reviewing documents posted online earlier this year by Clare Garvie, a senior associate at the Center on Privacy and Technology at Georgetown Law. Ms. Garvie received the documents as part of an open records lawsuit.

It could not be determined whether other large police departments used facial recognition with juveniles because very few have written policies governing the use of the technology, Ms. Garvie said.

New York detectives rely on a vast network of surveillance cameras throughout the city to provide images of people believed to have committed a crime. Since 2011, the department has had a dedicated unit of officers who use facial recognition to compare those images against millions of photos, usually mug shots. The software proposes matches, which have led to thousands of arrests, the department said.

By 2013, top police officials were meeting to discuss including juveniles in the program, the documents reviewed by The Times showed.

The documents showed that the juvenile database had been integrated into the system by 2015.

“We have these photos. It makes sense,” Chief Shea said in the interview.

State law requires that arrest photos be destroyed if the case is resolved in the juvenile’s favor, or if the child is found to have committed only a misdemeanor, rather than a felony. The photos also must be destroyed if a person reaches age 21 without a criminal record.

When children are charged with crimes, the court system usually takes some steps to prevent their acts from defining them in later years. Children who are 16 and under, for instance, are generally sent to Family Court, where records are not public.

Yet including their photos in a facial recognition database runs the risk that an imperfect algorithm identifies them as possible suspects in later crimes, civil rights advocates said. A mistaken match could lead investigators to focus on the wrong person from the outset, they said.

“It’s very disturbing to know that no matter what I’m doing at that moment, someone might be scanning my picture to try to find someone who committed a crime,” said Bailey, a 17-year-old Brooklyn girl who had admitted guilt in Family Court to a group attack that happened when she was 14. She said she was present at the attack but did not participate.

Bailey, who asked that she be identified only by her last name because she did not want her juvenile arrest to be public, has not been arrested again and is now a student at John Jay College of Criminal Justice.

Recent studies indicate that people of color, as well as children and women, have a greater risk of misidentification than their counterparts, said Joy Buolamwini, the founder of the Algorithmic Justice League and graduate researcher at the M.I.T. Media Lab, who has examined how human biases are built into artificial intelligence.

The racial disparities in the juvenile justice system are stark: In New York, black and Latino juveniles were charged with crimes at far higher rates than whites in 2017, the most recent year for which numbers were available. Black juveniles outnumbered white juveniles more than 15 to 1.

“If the facial recognition algorithm has a negative bias toward a black population, that will get magnified more toward children,” Dr. Ricanek said, adding that in terms of diminished accuracy, “you’re now putting yourself in unknown territory.”

Opinion | James O’Neill
How Facial Recognition Makes You Safer
Opinion | Jim Shultz
Spying on Children Won’t Keep Them Safe

Joseph Goldstein writes about policing and the criminal justice system. He has been a reporter at The Times since 2011, and is based in New York. He also worked for a year in the Kabul bureau, reporting on Afghanistan. @JoeKGoldstein

Ali Watkins is a reporter on the Metro Desk, covering courts and social services. Previously, she covered national security in Washington for The Times, BuzzFeed and McClatchy Newspapers. @AliWatkins

A version of this article appears in print on , Section A, Page 1 of the New York edition with the headline: In New York, Police Computers Scan Faces, Some as Young as 11. Order Reprints | Today’s Paper | Subscribe

Advertisement

+ + + + + + + + + + +
+ +
+ + + + \ No newline at end of file diff --git a/test/fixtures/nypd-facial-recognition-children-teenagers2.html b/test/fixtures/nypd-facial-recognition-children-teenagers2.html new file mode 100644 index 000000000..ae8b26aff --- /dev/null +++ b/test/fixtures/nypd-facial-recognition-children-teenagers2.html @@ -0,0 +1,226 @@ + + + + She Was Arrested at 14. Then Her Photo Went to a Facial Recognition Database. - The New York Times + + + + + + + + + + + + + + + + + + + + +

Advertisement

She Was Arrested at 14. Then Her Photo Went to a Facial Recognition Database.

With little oversight, the N.Y.P.D. has been using powerful surveillance technology on photos of children and teenagers.

Image
CreditCreditSarah Blesener for The New York Times
Joseph GoldsteinAli Watkins

[What you need to know to start the day: Get New York Today in your inbox.]

The New York Police Department has been loading thousands of arrest photos of children and teenagers into a facial recognition database despite evidence the technology has a higher risk of false matches in younger faces.

For about four years, internal records show, the department has used the technology to compare crime scene images with its collection of juvenile mug shots, the photos that are taken at an arrest. Most of the photos are of teenagers, largely 13 to 16 years old, but children as young as 11 have been included.

Elected officials and civil rights groups said the disclosure that the city was deploying a powerful surveillance tool on adolescents — whose privacy seems sacrosanct and whose status is protected in the criminal justice system — was a striking example of the Police Department’s ability to adopt advancing technology with little public scrutiny.

Several members of the City Council as well as a range of civil liberties groups said they were unaware of the policy until they were contacted by The New York Times.

Police Department officials defended the decision, saying it was just the latest evolution of a longstanding policing technique: using arrest photos to identify suspects.

“I don’t think this is any secret decision that’s made behind closed doors,” the city’s chief of detectives, Dermot F. Shea, said in an interview. “This is just process, and making sure we’re doing everything to fight crime.”

Other cities have begun to debate whether law enforcement should use facial recognition, which relies on an algorithm to quickly pore through images and suggest matches. In May, San Francisco blocked city agencies, including the police, from using the tool amid unease about potential government abuse. Detroit is facing public resistance to a technology that has been shown to have lower accuracy with people with darker skin.

In New York, the state Education Department recently told the Lockport, N.Y., school district to delay a plan to use facial recognition on students, citing privacy concerns.

“At the end of the day, it should be banned — no young people,” said Councilman Donovan Richards, a Queens Democrat who heads the Public Safety Committee, which oversees the Police Department.

The department said its legal bureau had approved using facial recognition on juveniles. The algorithm may suggest a lead, but detectives would not make an arrest based solely on that, Chief Shea said.

Image
CreditChang W. Lee/The New York Times

Still, facial recognition has not been widely tested on children. Most algorithms are taught to “think” based on adult faces, and there is growing evidence that they do not work as well on children.

The National Institute of Standards and Technology, which is part of the Commerce Department and evaluates facial recognition algorithms for accuracy, recently found the vast majority of more than 100 facial recognition algorithms had a higher rate of mistaken matches among children. The error rate was most pronounced in young children but was also seen in those aged 10 to 16.

Aging poses another problem: The appearance of children and adolescents can change drastically as bones stretch and shift, altering the underlying facial structure.

“I would use extreme caution in using those algorithms,” said Karl Ricanek Jr., a computer science professor and co-founder of the Face Aging Group at the University of North Carolina-Wilmington.

Technology that can match an image of a younger teenager to a recent arrest photo may be less effective at finding the same person even one or two years later, he said.

“The systems do not have the capacity to understand the dynamic changes that occur to a child’s face,” Dr. Ricanek said.

Idemia and DataWorks Plus, the two companies that provide facial recognition software to the Police Department, did not respond to requests for comment.

The New York Police Department can take arrest photos of minors as young as 11 who are charged with a felony, depending on the severity of the charge.

And in many cases, the department keeps the photos for years, making facial recognition comparisons to what may have effectively become outdated images. There are photos of 5,500 individuals in the juvenile database, 4,100 of whom are no longer 16 or under, the department said. Teenagers 17 and older are considered adults in the criminal justice system.

Police officials declined to provide statistics on how often their facial recognition systems provide false matches, or to explain how they evaluate the system’s effectiveness.

“We are comfortable with this technology because it has proved to be a valuable investigative method,” Chief Shea said. Facial recognition has helped lead to thousands of arrests of both adults and juveniles, the department has said.

Mayor Bill de Blasio had been aware the department was using the technology on minors, said Freddi Goldstein, a spokeswoman for the mayor.

She said the Police Department followed “strict guidelines” in applying the technology and City Hall monitored the agency’s compliance with the policies.

The Times learned details of the department’s use of facial recognition by reviewing documents posted online earlier this year by Clare Garvie, a senior associate at the Center on Privacy and Technology at Georgetown Law. Ms. Garvie received the documents as part of an open records lawsuit.

It could not be determined whether other large police departments used facial recognition with juveniles because very few have written policies governing the use of the technology, Ms. Garvie said.

New York detectives rely on a vast network of surveillance cameras throughout the city to provide images of people believed to have committed a crime. Since 2011, the department has had a dedicated unit of officers who use facial recognition to compare those images against millions of photos, usually mug shots. The software proposes matches, which have led to thousands of arrests, the department said.

By 2013, top police officials were meeting to discuss including juveniles in the program, the documents reviewed by The Times showed.

The documents showed that the juvenile database had been integrated into the system by 2015.

“We have these photos. It makes sense,” Chief Shea said in the interview.

State law requires that arrest photos be destroyed if the case is resolved in the juvenile’s favor, or if the child is found to have committed only a misdemeanor, rather than a felony. The photos also must be destroyed if a person reaches age 21 without a criminal record.

When children are charged with crimes, the court system usually takes some steps to prevent their acts from defining them in later years. Children who are 16 and under, for instance, are generally sent to Family Court, where records are not public.

Yet including their photos in a facial recognition database runs the risk that an imperfect algorithm identifies them as possible suspects in later crimes, civil rights advocates said. A mistaken match could lead investigators to focus on the wrong person from the outset, they said.

“It’s very disturbing to know that no matter what I’m doing at that moment, someone might be scanning my picture to try to find someone who committed a crime,” said Bailey, a 17-year-old Brooklyn girl who had admitted guilt in Family Court to a group attack that happened when she was 14. She said she was present at the attack but did not participate.

Bailey, who asked that she be identified only by her last name because she did not want her juvenile arrest to be public, has not been arrested again and is now a student at John Jay College of Criminal Justice.

Recent studies indicate that people of color, as well as children and women, have a greater risk of misidentification than their counterparts, said Joy Buolamwini, the founder of the Algorithmic Justice League and graduate researcher at the M.I.T. Media Lab, who has examined how human biases are built into artificial intelligence.

The racial disparities in the juvenile justice system are stark: In New York, black and Latino juveniles were charged with crimes at far higher rates than whites in 2017, the most recent year for which numbers were available. Black juveniles outnumbered white juveniles more than 15 to 1.

“If the facial recognition algorithm has a negative bias toward a black population, that will get magnified more toward children,” Dr. Ricanek said, adding that in terms of diminished accuracy, “you’re now putting yourself in unknown territory.”

Opinion | James O’Neill
How Facial Recognition Makes You Safer
Opinion | Jim Shultz
Spying on Children Won’t Keep Them Safe

Joseph Goldstein writes about policing and the criminal justice system. He has been a reporter at The Times since 2011, and is based in New York. He also worked for a year in the Kabul bureau, reporting on Afghanistan. @JoeKGoldstein

Ali Watkins is a reporter on the Metro Desk, covering courts and social services. Previously, she covered national security in Washington for The Times, BuzzFeed and McClatchy Newspapers. @AliWatkins

A version of this article appears in print on , Section A, Page 1 of the New York edition with the headline: In New York, Police Computers Scan Faces, Some as Young as 11. Order Reprints | Today’s Paper | Subscribe

Advertisement

+ + + + + + + + + + +
+ +
+ + + + diff --git a/test/fixtures/nypd-facial-recognition-children-teenagers3.html b/test/fixtures/nypd-facial-recognition-children-teenagers3.html new file mode 100644 index 000000000..53454d23e --- /dev/null +++ b/test/fixtures/nypd-facial-recognition-children-teenagers3.html @@ -0,0 +1,227 @@ + + + + She Was Arrested at 14. Then Her Photo Went to a Facial Recognition Database. - The New York Times + + + + + + + + + + + + + + + + + + + + + +

Advertisement

She Was Arrested at 14. Then Her Photo Went to a Facial Recognition Database.

With little oversight, the N.Y.P.D. has been using powerful surveillance technology on photos of children and teenagers.

Image
CreditCreditSarah Blesener for The New York Times
Joseph GoldsteinAli Watkins

[What you need to know to start the day: Get New York Today in your inbox.]

The New York Police Department has been loading thousands of arrest photos of children and teenagers into a facial recognition database despite evidence the technology has a higher risk of false matches in younger faces.

For about four years, internal records show, the department has used the technology to compare crime scene images with its collection of juvenile mug shots, the photos that are taken at an arrest. Most of the photos are of teenagers, largely 13 to 16 years old, but children as young as 11 have been included.

Elected officials and civil rights groups said the disclosure that the city was deploying a powerful surveillance tool on adolescents — whose privacy seems sacrosanct and whose status is protected in the criminal justice system — was a striking example of the Police Department’s ability to adopt advancing technology with little public scrutiny.

Several members of the City Council as well as a range of civil liberties groups said they were unaware of the policy until they were contacted by The New York Times.

Police Department officials defended the decision, saying it was just the latest evolution of a longstanding policing technique: using arrest photos to identify suspects.

“I don’t think this is any secret decision that’s made behind closed doors,” the city’s chief of detectives, Dermot F. Shea, said in an interview. “This is just process, and making sure we’re doing everything to fight crime.”

Other cities have begun to debate whether law enforcement should use facial recognition, which relies on an algorithm to quickly pore through images and suggest matches. In May, San Francisco blocked city agencies, including the police, from using the tool amid unease about potential government abuse. Detroit is facing public resistance to a technology that has been shown to have lower accuracy with people with darker skin.

In New York, the state Education Department recently told the Lockport, N.Y., school district to delay a plan to use facial recognition on students, citing privacy concerns.

“At the end of the day, it should be banned — no young people,” said Councilman Donovan Richards, a Queens Democrat who heads the Public Safety Committee, which oversees the Police Department.

The department said its legal bureau had approved using facial recognition on juveniles. The algorithm may suggest a lead, but detectives would not make an arrest based solely on that, Chief Shea said.

Image
CreditChang W. Lee/The New York Times

Still, facial recognition has not been widely tested on children. Most algorithms are taught to “think” based on adult faces, and there is growing evidence that they do not work as well on children.

The National Institute of Standards and Technology, which is part of the Commerce Department and evaluates facial recognition algorithms for accuracy, recently found the vast majority of more than 100 facial recognition algorithms had a higher rate of mistaken matches among children. The error rate was most pronounced in young children but was also seen in those aged 10 to 16.

Aging poses another problem: The appearance of children and adolescents can change drastically as bones stretch and shift, altering the underlying facial structure.

“I would use extreme caution in using those algorithms,” said Karl Ricanek Jr., a computer science professor and co-founder of the Face Aging Group at the University of North Carolina-Wilmington.

Technology that can match an image of a younger teenager to a recent arrest photo may be less effective at finding the same person even one or two years later, he said.

“The systems do not have the capacity to understand the dynamic changes that occur to a child’s face,” Dr. Ricanek said.

Idemia and DataWorks Plus, the two companies that provide facial recognition software to the Police Department, did not respond to requests for comment.

The New York Police Department can take arrest photos of minors as young as 11 who are charged with a felony, depending on the severity of the charge.

And in many cases, the department keeps the photos for years, making facial recognition comparisons to what may have effectively become outdated images. There are photos of 5,500 individuals in the juvenile database, 4,100 of whom are no longer 16 or under, the department said. Teenagers 17 and older are considered adults in the criminal justice system.

Police officials declined to provide statistics on how often their facial recognition systems provide false matches, or to explain how they evaluate the system’s effectiveness.

“We are comfortable with this technology because it has proved to be a valuable investigative method,” Chief Shea said. Facial recognition has helped lead to thousands of arrests of both adults and juveniles, the department has said.

Mayor Bill de Blasio had been aware the department was using the technology on minors, said Freddi Goldstein, a spokeswoman for the mayor.

She said the Police Department followed “strict guidelines” in applying the technology and City Hall monitored the agency’s compliance with the policies.

The Times learned details of the department’s use of facial recognition by reviewing documents posted online earlier this year by Clare Garvie, a senior associate at the Center on Privacy and Technology at Georgetown Law. Ms. Garvie received the documents as part of an open records lawsuit.

It could not be determined whether other large police departments used facial recognition with juveniles because very few have written policies governing the use of the technology, Ms. Garvie said.

New York detectives rely on a vast network of surveillance cameras throughout the city to provide images of people believed to have committed a crime. Since 2011, the department has had a dedicated unit of officers who use facial recognition to compare those images against millions of photos, usually mug shots. The software proposes matches, which have led to thousands of arrests, the department said.

By 2013, top police officials were meeting to discuss including juveniles in the program, the documents reviewed by The Times showed.

The documents showed that the juvenile database had been integrated into the system by 2015.

“We have these photos. It makes sense,” Chief Shea said in the interview.

State law requires that arrest photos be destroyed if the case is resolved in the juvenile’s favor, or if the child is found to have committed only a misdemeanor, rather than a felony. The photos also must be destroyed if a person reaches age 21 without a criminal record.

When children are charged with crimes, the court system usually takes some steps to prevent their acts from defining them in later years. Children who are 16 and under, for instance, are generally sent to Family Court, where records are not public.

Yet including their photos in a facial recognition database runs the risk that an imperfect algorithm identifies them as possible suspects in later crimes, civil rights advocates said. A mistaken match could lead investigators to focus on the wrong person from the outset, they said.

“It’s very disturbing to know that no matter what I’m doing at that moment, someone might be scanning my picture to try to find someone who committed a crime,” said Bailey, a 17-year-old Brooklyn girl who had admitted guilt in Family Court to a group attack that happened when she was 14. She said she was present at the attack but did not participate.

Bailey, who asked that she be identified only by her last name because she did not want her juvenile arrest to be public, has not been arrested again and is now a student at John Jay College of Criminal Justice.

Recent studies indicate that people of color, as well as children and women, have a greater risk of misidentification than their counterparts, said Joy Buolamwini, the founder of the Algorithmic Justice League and graduate researcher at the M.I.T. Media Lab, who has examined how human biases are built into artificial intelligence.

The racial disparities in the juvenile justice system are stark: In New York, black and Latino juveniles were charged with crimes at far higher rates than whites in 2017, the most recent year for which numbers were available. Black juveniles outnumbered white juveniles more than 15 to 1.

“If the facial recognition algorithm has a negative bias toward a black population, that will get magnified more toward children,” Dr. Ricanek said, adding that in terms of diminished accuracy, “you’re now putting yourself in unknown territory.”

Opinion | James O’Neill
How Facial Recognition Makes You Safer
Opinion | Jim Shultz
Spying on Children Won’t Keep Them Safe

Joseph Goldstein writes about policing and the criminal justice system. He has been a reporter at The Times since 2011, and is based in New York. He also worked for a year in the Kabul bureau, reporting on Afghanistan. @JoeKGoldstein

Ali Watkins is a reporter on the Metro Desk, covering courts and social services. Previously, she covered national security in Washington for The Times, BuzzFeed and McClatchy Newspapers. @AliWatkins

A version of this article appears in print on , Section A, Page 1 of the New York edition with the headline: In New York, Police Computers Scan Faces, Some as Young as 11. Order Reprints | Today’s Paper | Subscribe

Advertisement

+ + + + + + + + + + +
+ +
+ + + + diff --git a/test/fixtures/tesla_mock/admin@mastdon.example.org.json b/test/fixtures/tesla_mock/admin@mastdon.example.org.json index c297e4349..8159dc20a 100644 --- a/test/fixtures/tesla_mock/admin@mastdon.example.org.json +++ b/test/fixtures/tesla_mock/admin@mastdon.example.org.json @@ -1 +1,54 @@ -{"@context":["https://www.w3.org/ns/activitystreams","https://w3id.org/security/v1",{"manuallyApprovesFollowers":"as:manuallyApprovesFollowers","sensitive":"as:sensitive","movedTo":"as:movedTo","Hashtag":"as:Hashtag","ostatus":"http://ostatus.org#","atomUri":"ostatus:atomUri","inReplyToAtomUri":"ostatus:inReplyToAtomUri","conversation":"ostatus:conversation","toot":"http://joinmastodon.org/ns#","Emoji":"toot:Emoji"}],"id":"http://mastodon.example.org/users/admin","type":"Person","following":"http://mastodon.example.org/users/admin/following","followers":"http://mastodon.example.org/users/admin/followers","inbox":"http://mastodon.example.org/users/admin/inbox","outbox":"http://mastodon.example.org/users/admin/outbox","preferredUsername":"admin","name":null,"summary":"\u003cp\u003e\u003c/p\u003e","url":"http://mastodon.example.org/@admin","manuallyApprovesFollowers":false,"publicKey":{"id":"http://mastodon.example.org/users/admin#main-key","owner":"http://mastodon.example.org/users/admin","publicKeyPem":"-----BEGIN PUBLIC KEY-----\nMIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIBCgKCAQEAtc4Tir+3ADhSNF6VKrtW\nOU32T01w7V0yshmQei38YyiVwVvFu8XOP6ACchkdxbJ+C9mZud8qWaRJKVbFTMUG\nNX4+6Q+FobyuKrwN7CEwhDALZtaN2IPbaPd6uG1B7QhWorrY+yFa8f2TBM3BxnUy\nI4T+bMIZIEYG7KtljCBoQXuTQmGtuffO0UwJksidg2ffCF5Q+K//JfQagJ3UzrR+\nZXbKMJdAw4bCVJYs4Z5EhHYBwQWiXCyMGTd7BGlmMkY6Av7ZqHKC/owp3/0EWDNz\nNqF09Wcpr3y3e8nA10X40MJqp/wR+1xtxp+YGbq/Cj5hZGBG7etFOmIpVBrDOhry\nBwIDAQAB\n-----END PUBLIC KEY-----\n"},"endpoints":{"sharedInbox":"http://mastodon.example.org/inbox"},"icon":{"type":"Image","mediaType":"image/jpeg","url":"https://cdn.niu.moe/accounts/avatars/000/033/323/original/fd7f8ae0b3ffedc9.jpeg"},"image":{"type":"Image","mediaType":"image/png","url":"https://cdn.niu.moe/accounts/headers/000/033/323/original/850b3448fa5fd477.png"}} +{ + "@context": ["https://www.w3.org/ns/activitystreams", "https://w3id.org/security/v1", { + "manuallyApprovesFollowers": "as:manuallyApprovesFollowers", + "sensitive": "as:sensitive", + "movedTo": "as:movedTo", + "Hashtag": "as:Hashtag", + "ostatus": "http://ostatus.org#", + "atomUri": "ostatus:atomUri", + "inReplyToAtomUri": "ostatus:inReplyToAtomUri", + "conversation": "ostatus:conversation", + "toot": "http://joinmastodon.org/ns#", + "Emoji": "toot:Emoji" + }], + "id": "http://mastodon.example.org/users/admin", + "type": "Person", + "following": "http://mastodon.example.org/users/admin/following", + "followers": "http://mastodon.example.org/users/admin/followers", + "inbox": "http://mastodon.example.org/users/admin/inbox", + "outbox": "http://mastodon.example.org/users/admin/outbox", + "preferredUsername": "admin", + "name": null, + "summary": "\u003cp\u003e\u003c/p\u003e", + "url": "http://mastodon.example.org/@admin", + "manuallyApprovesFollowers": false, + "publicKey": { + "id": "http://mastodon.example.org/users/admin#main-key", + "owner": "http://mastodon.example.org/users/admin", + "publicKeyPem": "-----BEGIN PUBLIC KEY-----\nMIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIBCgKCAQEAtc4Tir+3ADhSNF6VKrtW\nOU32T01w7V0yshmQei38YyiVwVvFu8XOP6ACchkdxbJ+C9mZud8qWaRJKVbFTMUG\nNX4+6Q+FobyuKrwN7CEwhDALZtaN2IPbaPd6uG1B7QhWorrY+yFa8f2TBM3BxnUy\nI4T+bMIZIEYG7KtljCBoQXuTQmGtuffO0UwJksidg2ffCF5Q+K//JfQagJ3UzrR+\nZXbKMJdAw4bCVJYs4Z5EhHYBwQWiXCyMGTd7BGlmMkY6Av7ZqHKC/owp3/0EWDNz\nNqF09Wcpr3y3e8nA10X40MJqp/wR+1xtxp+YGbq/Cj5hZGBG7etFOmIpVBrDOhry\nBwIDAQAB\n-----END PUBLIC KEY-----\n" + }, + "attachment": [{ + "type": "PropertyValue", + "name": "foo", + "value": "bar" + }, + { + "type": "PropertyValue", + "name": "foo1", + "value": "bar1" + } + ], + "endpoints": { + "sharedInbox": "http://mastodon.example.org/inbox" + }, + "icon": { + "type": "Image", + "mediaType": "image/jpeg", + "url": "https://cdn.niu.moe/accounts/avatars/000/033/323/original/fd7f8ae0b3ffedc9.jpeg" + }, + "image": { + "type": "Image", + "mediaType": "image/png", + "url": "https://cdn.niu.moe/accounts/headers/000/033/323/original/850b3448fa5fd477.png" + } +} diff --git a/test/fixtures/tesla_mock/kpherox@mstdn.jp.xml b/test/fixtures/tesla_mock/kpherox@mstdn.jp.xml new file mode 100644 index 000000000..2ec134eaa --- /dev/null +++ b/test/fixtures/tesla_mock/kpherox@mstdn.jp.xml @@ -0,0 +1,10 @@ + + + acct:kPherox@mstdn.jp + https://mstdn.jp/@kPherox + https://mstdn.jp/users/kPherox + + + + + diff --git a/test/fixtures/tesla_mock/wedistribute-article.json b/test/fixtures/tesla_mock/wedistribute-article.json new file mode 100644 index 000000000..39dc1b982 --- /dev/null +++ b/test/fixtures/tesla_mock/wedistribute-article.json @@ -0,0 +1,18 @@ +{ + "@context": [ + "https://www.w3.org/ns/activitystreams" + ], + "type": "Article", + "name": "The end is near: Mastodon plans to drop OStatus support", + "content": "\n

The days of OStatus are numbered. The venerable protocol has served as a glue between many different types of servers since the early days of the Fediverse, connecting StatusNet (now GNU Social) to Friendica, Hubzilla, Mastodon, and Pleroma.

\n\n\n\n

Now that many fediverse platforms support ActivityPub as a successor protocol, Mastodon appears to be drawing a line in the sand. In a Patreon update, Eugen Rochko writes:

\n\n\n\n

...OStatus...has overstayed its welcome in the code...and now that most of the network uses ActivityPub, it's time for it to go.

Eugen Rochko, Mastodon creator
\n\n\n\n

The pull request to remove Pubsubhubbub and Salmon, two of the main components of OStatus, has already been merged into Mastodon's master branch.

\n\n\n\n

Some projects will be left in the dark as a side effect of this. GNU Social and PostActiv, for example, both only communicate using OStatus. While some discussion exists regarding adopting ActivityPub for GNU Social, and a plugin is in development, it hasn't been formally adopted yet. We just hope that the Free Software Foundation's instance gets updated in time!

\n", + "summary": "One of the largest platforms in the federated social web is dropping the protocol that it started with.", + "attributedTo": "https://wedistribute.org/wp-json/pterotype/v1/actor/-blog", + "url": "https://wedistribute.org/2019/07/mastodon-drops-ostatus/", + "to": [ + "https://www.w3.org/ns/activitystreams#Public", + "https://wedistribute.org/wp-json/pterotype/v1/actor/-blog/followers" + ], + "id": "https://wedistribute.org/wp-json/pterotype/v1/object/85810", + "likes": "https://wedistribute.org/wp-json/pterotype/v1/object/85810/likes", + "shares": "https://wedistribute.org/wp-json/pterotype/v1/object/85810/shares" +} diff --git a/test/fixtures/tesla_mock/wedistribute-user.json b/test/fixtures/tesla_mock/wedistribute-user.json new file mode 100644 index 000000000..fe2a15703 --- /dev/null +++ b/test/fixtures/tesla_mock/wedistribute-user.json @@ -0,0 +1,31 @@ +{ + "@context": [ + "https://www.w3.org/ns/activitystreams", + "https://w3id.org/security/v1", + { + "manuallyApprovesFollowers": "as:manuallyApprovesFollowers" + } + ], + "type": "Organization", + "id": "https://wedistribute.org/wp-json/pterotype/v1/actor/-blog", + "following": "https://wedistribute.org/wp-json/pterotype/v1/actor/-blog/following", + "followers": "https://wedistribute.org/wp-json/pterotype/v1/actor/-blog/followers", + "liked": "https://wedistribute.org/wp-json/pterotype/v1/actor/-blog/liked", + "inbox": "https://wedistribute.org/wp-json/pterotype/v1/actor/-blog/inbox", + "outbox": "https://wedistribute.org/wp-json/pterotype/v1/actor/-blog/outbox", + "name": "We Distribute", + "preferredUsername": "blog", + "summary": "

Connecting many threads in the federated web. We Distribute is an independent publication dedicated to the fediverse, decentralization, P2P technologies, and Free Software!

", + "url": "https://wedistribute.org/", + "publicKey": { + "id": "https://wedistribute.org/wp-json/pterotype/v1/actor/-blog#publicKey", + "owner": "https://wedistribute.org/wp-json/pterotype/v1/actor/-blog", + "publicKeyPem": "-----BEGIN PUBLIC KEY-----\r\nMIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIBCgKCAQEA1bmUJ+y8PS8JFVi0KugN\r\nFl4pLvLog3V2lsV9ftmCXpveB/WJx66Tr1fQLsU3qYvQFc8UPGWD52zV4RENR1SN\r\nx0O6T2f97KUbRM+Ckow7Jyjtssgl+Mqq8UBZQ/+H8I/1Vpvt5E5hUykhFgwzx9qg\r\nzoIA3OK7alOpQbSoKXo0QcOh6yTRUnMSRMJAgUoZJzzXI/FmH/DtKr7ziQ1T2KWs\r\nVs8mWnTb/OlCxiheLuMlmJNMF+lPyVthvMIxF6Z5gV9d5QAmASSCI628e6uH2EUF\r\nDEEF5jo+Z5ffeNv28953lrnM+VB/wTjl3tYA+zCQeAmUPksX3E+YkXGxj+4rxBAY\r\n8wIDAQAB\r\n-----END PUBLIC KEY-----" + }, + "manuallyApprovesFollowers": false, + "icon": { + "url": "https://wedistribute.org/wp-content/uploads/2019/02/b067de423757a538.png", + "type": "Image", + "mediaType": "image/png" + } +} diff --git a/test/fixtures/users_mock/masto_closed_followers_page.json b/test/fixtures/users_mock/masto_closed_followers_page.json new file mode 100644 index 000000000..04ab0c4d3 --- /dev/null +++ b/test/fixtures/users_mock/masto_closed_followers_page.json @@ -0,0 +1 @@ +{"@context":"https://www.w3.org/ns/activitystreams","id":"http://localhost:4001/users/masto_closed/followers?page=1","type":"OrderedCollectionPage","totalItems":437,"next":"http://localhost:4001/users/masto_closed/followers?page=2","partOf":"http://localhost:4001/users/masto_closed/followers","orderedItems":["https://testing.uguu.ltd/users/rin","https://patch.cx/users/rin","https://letsalllovela.in/users/xoxo","https://pleroma.site/users/crushv","https://aria.company/users/boris","https://kawen.space/users/crushv","https://freespeech.host/users/cvcvcv","https://pleroma.site/users/picpub","https://pixelfed.social/users/nosleep","https://boopsnoot.gq/users/5c1896d162f7d337f90492a3","https://pikachu.rocks/users/waifu","https://royal.crablettesare.life/users/crablettes"]} diff --git a/test/fixtures/users_mock/masto_closed_following_page.json b/test/fixtures/users_mock/masto_closed_following_page.json new file mode 100644 index 000000000..8d8324699 --- /dev/null +++ b/test/fixtures/users_mock/masto_closed_following_page.json @@ -0,0 +1 @@ +{"@context":"https://www.w3.org/ns/activitystreams","id":"http://localhost:4001/users/masto_closed/following?page=1","type":"OrderedCollectionPage","totalItems":152,"next":"http://localhost:4001/users/masto_closed/following?page=2","partOf":"http://localhost:4001/users/masto_closed/following","orderedItems":["https://testing.uguu.ltd/users/rin","https://patch.cx/users/rin","https://letsalllovela.in/users/xoxo","https://pleroma.site/users/crushv","https://aria.company/users/boris","https://kawen.space/users/crushv","https://freespeech.host/users/cvcvcv","https://pleroma.site/users/picpub","https://pixelfed.social/users/nosleep","https://boopsnoot.gq/users/5c1896d162f7d337f90492a3","https://pikachu.rocks/users/waifu","https://royal.crablettesare.life/users/crablettes"]} diff --git a/test/flake_id_test.exs b/test/flake_id_test.exs index ca2338041..85ed5bbdf 100644 --- a/test/flake_id_test.exs +++ b/test/flake_id_test.exs @@ -39,4 +39,9 @@ test "ecto type behaviour" do assert dump(flake_s) == {:ok, flake} assert dump(flake) == {:ok, flake} end + + test "is_flake_id?/1" do + assert is_flake_id?("9eoozpwTul5mjSEDRI") + refute is_flake_id?("http://example.com/activities/3ebbadd1-eb14-4e20-8118-b6f79c0c7b0b") + end end diff --git a/test/http/request_builder_test.exs b/test/http/request_builder_test.exs index 7febe84c5..170ca916f 100644 --- a/test/http/request_builder_test.exs +++ b/test/http/request_builder_test.exs @@ -4,21 +4,19 @@ defmodule Pleroma.HTTP.RequestBuilderTest do use ExUnit.Case, async: true + use Pleroma.Tests.Helpers alias Pleroma.HTTP.RequestBuilder describe "headers/2" do + clear_config([:http, :send_user_agent]) + test "don't send pleroma user agent" do assert RequestBuilder.headers(%{}, []) == %{headers: []} end test "send pleroma user agent" do - send = Pleroma.Config.get([:http, :send_user_agent]) Pleroma.Config.put([:http, :send_user_agent], true) - on_exit(fn -> - Pleroma.Config.put([:http, :send_user_agent], send) - end) - assert RequestBuilder.headers(%{}, []) == %{ headers: [{"User-Agent", Pleroma.Application.user_agent()}] } diff --git a/test/notification_test.exs b/test/notification_test.exs index 28f8df49d..80ea2a085 100644 --- a/test/notification_test.exs +++ b/test/notification_test.exs @@ -4,13 +4,15 @@ defmodule Pleroma.NotificationTest do use Pleroma.DataCase + + import Pleroma.Factory + alias Pleroma.Notification alias Pleroma.User alias Pleroma.Web.ActivityPub.Transmogrifier alias Pleroma.Web.CommonAPI alias Pleroma.Web.Streamer alias Pleroma.Web.TwitterAPI.TwitterAPI - import Pleroma.Factory describe "create_notifications" do test "notifies someone when they are directly addressed" do @@ -352,6 +354,51 @@ test "it sets all notifications as read up to a specified notification ID" do end end + describe "for_user_since/2" do + defp days_ago(days) do + NaiveDateTime.add( + NaiveDateTime.truncate(NaiveDateTime.utc_now(), :second), + -days * 60 * 60 * 24, + :second + ) + end + + test "Returns recent notifications" do + user1 = insert(:user) + user2 = insert(:user) + + Enum.each(0..10, fn i -> + {:ok, _activity} = + CommonAPI.post(user1, %{ + "status" => "hey ##{i} @#{user2.nickname}!" + }) + end) + + {old, new} = Enum.split(Notification.for_user(user2), 5) + + Enum.each(old, fn notification -> + notification + |> cast(%{updated_at: days_ago(10)}, [:updated_at]) + |> Pleroma.Repo.update!() + end) + + recent_notifications_ids = + user2 + |> Notification.for_user_since( + NaiveDateTime.add(NaiveDateTime.utc_now(), -5 * 86_400, :second) + ) + |> Enum.map(& &1.id) + + Enum.each(old, fn %{id: id} -> + refute id in recent_notifications_ids + end) + + Enum.each(new, fn %{id: id} -> + assert id in recent_notifications_ids + end) + end + end + describe "notification target determination" do test "it sends notifications to addressed users in new messages" do user = insert(:user) @@ -564,6 +611,64 @@ test "replying to a deleted post without tagging does not generate a notificatio assert Enum.empty?(Notification.for_user(user)) end + + test "notifications are deleted if a local user is deleted" do + user = insert(:user) + other_user = insert(:user) + + {:ok, _activity} = + CommonAPI.post(user, %{"status" => "hi @#{other_user.nickname}", "visibility" => "direct"}) + + refute Enum.empty?(Notification.for_user(other_user)) + + User.delete(user) + + assert Enum.empty?(Notification.for_user(other_user)) + end + + test "notifications are deleted if a remote user is deleted" do + remote_user = insert(:user) + local_user = insert(:user) + + dm_message = %{ + "@context" => "https://www.w3.org/ns/activitystreams", + "type" => "Create", + "actor" => remote_user.ap_id, + "id" => remote_user.ap_id <> "/activities/test", + "to" => [local_user.ap_id], + "cc" => [], + "object" => %{ + "type" => "Note", + "content" => "Hello!", + "tag" => [ + %{ + "type" => "Mention", + "href" => local_user.ap_id, + "name" => "@#{local_user.nickname}" + } + ], + "to" => [local_user.ap_id], + "cc" => [], + "attributedTo" => remote_user.ap_id + } + } + + {:ok, _dm_activity} = Transmogrifier.handle_incoming(dm_message) + + refute Enum.empty?(Notification.for_user(local_user)) + + delete_user_message = %{ + "@context" => "https://www.w3.org/ns/activitystreams", + "id" => remote_user.ap_id <> "/activities/delete", + "actor" => remote_user.ap_id, + "type" => "Delete", + "object" => remote_user.ap_id + } + + {:ok, _delete_activity} = Transmogrifier.handle_incoming(delete_user_message) + + assert Enum.empty?(Notification.for_user(local_user)) + end end describe "for_user" do diff --git a/test/object/fetcher_test.exs b/test/object/fetcher_test.exs index 482252cff..895a73d2c 100644 --- a/test/object/fetcher_test.exs +++ b/test/object/fetcher_test.exs @@ -110,6 +110,13 @@ test "it can fetch peertube videos" do assert object end + test "it can fetch wedistribute articles" do + {:ok, object} = + Fetcher.fetch_object_from_id("https://wedistribute.org/wp-json/pterotype/v1/object/85810") + + assert object + end + test "all objects with fake directions are rejected by the object fetcher" do assert {:error, _} = Fetcher.fetch_and_contain_remote_object_from_id( @@ -152,32 +159,28 @@ test "it can refetch pruned objects" do end describe "signed fetches" do + clear_config([:activitypub, :sign_object_fetches]) + test_with_mock "it signs fetches when configured to do so", Pleroma.Signature, [:passthrough], [] do - option = Pleroma.Config.get([:activitypub, :sign_object_fetches]) Pleroma.Config.put([:activitypub, :sign_object_fetches], true) Fetcher.fetch_object_from_id("http://mastodon.example.org/@admin/99541947525187367") assert called(Pleroma.Signature.sign(:_, :_)) - - Pleroma.Config.put([:activitypub, :sign_object_fetches], option) end test_with_mock "it doesn't sign fetches when not configured to do so", Pleroma.Signature, [:passthrough], [] do - option = Pleroma.Config.get([:activitypub, :sign_object_fetches]) Pleroma.Config.put([:activitypub, :sign_object_fetches], false) Fetcher.fetch_object_from_id("http://mastodon.example.org/@admin/99541947525187367") refute called(Pleroma.Signature.sign(:_, :_)) - - Pleroma.Config.put([:activitypub, :sign_object_fetches], option) end end end diff --git a/test/plugs/ensure_public_or_authenticated_plug_test.exs b/test/plugs/ensure_public_or_authenticated_plug_test.exs index ce5d77ff7..d45662a2a 100644 --- a/test/plugs/ensure_public_or_authenticated_plug_test.exs +++ b/test/plugs/ensure_public_or_authenticated_plug_test.exs @@ -9,8 +9,10 @@ defmodule Pleroma.Plugs.EnsurePublicOrAuthenticatedPlugTest do alias Pleroma.Plugs.EnsurePublicOrAuthenticatedPlug alias Pleroma.User + clear_config([:instance, :public]) + test "it halts if not public and no user is assigned", %{conn: conn} do - set_public_to(false) + Config.put([:instance, :public], false) conn = conn @@ -21,7 +23,7 @@ test "it halts if not public and no user is assigned", %{conn: conn} do end test "it continues if public", %{conn: conn} do - set_public_to(true) + Config.put([:instance, :public], true) ret_conn = conn @@ -31,7 +33,7 @@ test "it continues if public", %{conn: conn} do end test "it continues if a user is assigned, even if not public", %{conn: conn} do - set_public_to(false) + Config.put([:instance, :public], false) conn = conn @@ -43,13 +45,4 @@ test "it continues if a user is assigned, even if not public", %{conn: conn} do assert ret_conn == conn end - - defp set_public_to(value) do - orig = Config.get!([:instance, :public]) - Config.put([:instance, :public], value) - - on_exit(fn -> - Config.put([:instance, :public], orig) - end) - end end diff --git a/test/plugs/http_security_plug_test.exs b/test/plugs/http_security_plug_test.exs index 7dfd50c1f..7a2835e3d 100644 --- a/test/plugs/http_security_plug_test.exs +++ b/test/plugs/http_security_plug_test.exs @@ -7,17 +7,12 @@ defmodule Pleroma.Web.Plugs.HTTPSecurityPlugTest do alias Pleroma.Config alias Plug.Conn + clear_config([:http_securiy, :enabled]) + clear_config([:http_security, :sts]) + describe "http security enabled" do setup do - enabled = Config.get([:http_securiy, :enabled]) - Config.put([:http_security, :enabled], true) - - on_exit(fn -> - Config.put([:http_security, :enabled], enabled) - end) - - :ok end test "it sends CSP headers when enabled", %{conn: conn} do @@ -81,14 +76,8 @@ test "it sends `report-to` & `report-uri` CSP response headers" do end test "it does not send CSP headers when disabled", %{conn: conn} do - enabled = Config.get([:http_securiy, :enabled]) - Config.put([:http_security, :enabled], false) - on_exit(fn -> - Config.put([:http_security, :enabled], enabled) - end) - conn = get(conn, "/api/v1/instance") assert Conn.get_resp_header(conn, "x-xss-protection") == [] diff --git a/test/plugs/instance_static_test.exs b/test/plugs/instance_static_test.exs index e2dcfa3d8..6aabc45a4 100644 --- a/test/plugs/instance_static_test.exs +++ b/test/plugs/instance_static_test.exs @@ -8,14 +8,12 @@ defmodule Pleroma.Web.RuntimeStaticPlugTest do @dir "test/tmp/instance_static" setup do - static_dir = Pleroma.Config.get([:instance, :static_dir]) - Pleroma.Config.put([:instance, :static_dir], @dir) File.mkdir_p!(@dir) + on_exit(fn -> File.rm_rf(@dir) end) + end - on_exit(fn -> - Pleroma.Config.put([:instance, :static_dir], static_dir) - File.rm_rf(@dir) - end) + clear_config([:instance, :static_dir]) do + Pleroma.Config.put([:instance, :static_dir], @dir) end test "overrides index" do diff --git a/test/plugs/set_format_plug_test.exs b/test/plugs/set_format_plug_test.exs new file mode 100644 index 000000000..bb21956bb --- /dev/null +++ b/test/plugs/set_format_plug_test.exs @@ -0,0 +1,38 @@ +# Pleroma: A lightweight social networking server +# Copyright © 2017-2018 Pleroma Authors +# SPDX-License-Identifier: AGPL-3.0-only + +defmodule Pleroma.Plugs.SetFormatPlugTest do + use ExUnit.Case, async: true + use Plug.Test + + alias Pleroma.Plugs.SetFormatPlug + + test "set format from params" do + conn = + :get + |> conn("/cofe?_format=json") + |> SetFormatPlug.call([]) + + assert %{format: "json"} == conn.assigns + end + + test "set format from header" do + conn = + :get + |> conn("/cofe") + |> put_private(:phoenix_format, "xml") + |> SetFormatPlug.call([]) + + assert %{format: "xml"} == conn.assigns + end + + test "doesn't set format" do + conn = + :get + |> conn("/cofe") + |> SetFormatPlug.call([]) + + refute conn.assigns[:format] + end +end diff --git a/test/reverse_proxy_test.exs b/test/reverse_proxy_test.exs index f4b7d6add..3a83c4c48 100644 --- a/test/reverse_proxy_test.exs +++ b/test/reverse_proxy_test.exs @@ -108,11 +108,11 @@ defp stream_mock(invokes, with_close? \\ false) do end end - test "max_body_size returns error if streaming body more than that option", %{conn: conn} do + test "max_body_length returns error if streaming body more than that option", %{conn: conn} do stream_mock(3, true) assert capture_log(fn -> - ReverseProxy.call(conn, "/stream-bytes/50", max_body_size: 30) + ReverseProxy.call(conn, "/stream-bytes/50", max_body_length: 30) end) =~ "[warn] Elixir.Pleroma.ReverseProxy request to /stream-bytes/50 failed while reading/chunking: :body_too_large" end diff --git a/test/support/builders/user_builder.ex b/test/support/builders/user_builder.ex index f58e1b0ad..6da16f71a 100644 --- a/test/support/builders/user_builder.ex +++ b/test/support/builders/user_builder.ex @@ -9,7 +9,8 @@ def build(data \\ %{}) do nickname: "testname", password_hash: Comeonin.Pbkdf2.hashpwsalt("test"), bio: "A tester.", - ap_id: "some id" + ap_id: "some id", + last_digest_emailed_at: NaiveDateTime.truncate(NaiveDateTime.utc_now(), :second) } Map.merge(user, data) diff --git a/test/support/factory.ex b/test/support/factory.ex index 7a2ddcada..62d1de717 100644 --- a/test/support/factory.ex +++ b/test/support/factory.ex @@ -31,7 +31,8 @@ def user_factory do nickname: sequence(:nickname, &"nick#{&1}"), password_hash: Comeonin.Pbkdf2.hashpwsalt("test"), bio: sequence(:bio, &"Tester Number #{&1}"), - info: %{} + info: %{}, + last_digest_emailed_at: NaiveDateTime.utc_now() } %{ @@ -201,8 +202,8 @@ def announce_activity_factory(attrs \\ %{}) do } end - def like_activity_factory do - note_activity = insert(:note_activity) + def like_activity_factory(attrs \\ %{}) do + note_activity = attrs[:note_activity] || insert(:note_activity) object = Object.normalize(note_activity) user = insert(:user) diff --git a/test/support/helpers.ex b/test/support/helpers.ex index 1a92be065..a601b3ec8 100644 --- a/test/support/helpers.ex +++ b/test/support/helpers.ex @@ -7,8 +7,52 @@ defmodule Pleroma.Tests.Helpers do Helpers for use in tests. """ + defmacro clear_config(config_path) do + quote do + clear_config(unquote(config_path)) do + end + end + end + + defmacro clear_config(config_path, do: yield) do + quote do + setup do + initial_setting = Pleroma.Config.get(unquote(config_path)) + unquote(yield) + on_exit(fn -> Pleroma.Config.put(unquote(config_path), initial_setting) end) + :ok + end + end + end + + defmacro clear_config_all(config_path) do + quote do + clear_config_all(unquote(config_path)) do + end + end + end + + defmacro clear_config_all(config_path, do: yield) do + quote do + setup_all do + initial_setting = Pleroma.Config.get(unquote(config_path)) + unquote(yield) + on_exit(fn -> Pleroma.Config.put(unquote(config_path), initial_setting) end) + :ok + end + end + end + defmacro __using__(_opts) do quote do + import Pleroma.Tests.Helpers, + only: [ + clear_config: 1, + clear_config: 2, + clear_config_all: 1, + clear_config_all: 2 + ] + def collect_ids(collection) do collection |> Enum.map(& &1.id) @@ -30,6 +74,15 @@ def render_json(view, template, assigns) do |> Poison.encode!() |> Poison.decode!() end + + defmacro guards_config(config_path) do + quote do + initial_setting = Pleroma.Config.get(config_path) + + Pleroma.Config.put(config_path, true) + on_exit(fn -> Pleroma.Config.put(config_path, initial_setting) end) + end + end end end end diff --git a/test/support/http_request_mock.ex b/test/support/http_request_mock.ex index 7811f7807..3adb5ba3b 100644 --- a/test/support/http_request_mock.ex +++ b/test/support/http_request_mock.ex @@ -51,6 +51,10 @@ def get("https://mastodon.social/users/emelie", _, _, _) do }} end + def get("https://mastodon.social/users/not_found", _, _, _) do + {:ok, %Tesla.Env{status: 404}} + end + def get("https://mastodon.sdf.org/users/rinpatch", _, _, _) do {:ok, %Tesla.Env{ @@ -301,6 +305,22 @@ def get("https://baptiste.gelez.xyz/~/PlumeDevelopment/this-month-in-plume-june- }} end + def get("https://wedistribute.org/wp-json/pterotype/v1/object/85810", _, _, _) do + {:ok, + %Tesla.Env{ + status: 200, + body: File.read!("test/fixtures/tesla_mock/wedistribute-article.json") + }} + end + + def get("https://wedistribute.org/wp-json/pterotype/v1/actor/-blog", _, _, _) do + {:ok, + %Tesla.Env{ + status: 200, + body: File.read!("test/fixtures/tesla_mock/wedistribute-user.json") + }} + end + def get("http://mastodon.example.org/users/admin", _, _, Accept: "application/activity+json") do {:ok, %Tesla.Env{ @@ -614,6 +634,15 @@ def get( }} end + def get( + "https://social.heldscal.la/.well-known/webfinger?resource=invalid_content@social.heldscal.la", + _, + _, + Accept: "application/xrd+xml,application/jrd+json" + ) do + {:ok, %Tesla.Env{status: 200, body: ""}} + end + def get("http://framatube.org/.well-known/host-meta", _, _, _) do {:ok, %Tesla.Env{ @@ -767,6 +796,14 @@ def get("http://localhost:4001/users/masto_closed/followers", _, _, _) do }} end + def get("http://localhost:4001/users/masto_closed/followers?page=1", _, _, _) do + {:ok, + %Tesla.Env{ + status: 200, + body: File.read!("test/fixtures/users_mock/masto_closed_followers_page.json") + }} + end + def get("http://localhost:4001/users/masto_closed/following", _, _, _) do {:ok, %Tesla.Env{ @@ -775,6 +812,14 @@ def get("http://localhost:4001/users/masto_closed/following", _, _, _) do }} end + def get("http://localhost:4001/users/masto_closed/following?page=1", _, _, _) do + {:ok, + %Tesla.Env{ + status: 200, + body: File.read!("test/fixtures/users_mock/masto_closed_following_page.json") + }} + end + def get("http://localhost:4001/users/fuser2/followers", _, _, _) do {:ok, %Tesla.Env{ @@ -915,6 +960,14 @@ def get("https://info.pleroma.site/activity3.json", _, _, _) do {:ok, %Tesla.Env{status: 404, body: ""}} end + def get("https://mstdn.jp/.well-known/webfinger?resource=acct:kpherox@mstdn.jp", _, _, _) do + {:ok, + %Tesla.Env{ + status: 200, + body: File.read!("test/fixtures/tesla_mock/kpherox@mstdn.jp.xml") + }} + end + def get(url, query, body, headers) do {:error, "Not implemented the mock response for get #{inspect(url)}, #{query}, #{inspect(body)}, #{ diff --git a/test/support/mrf_module_mock.ex b/test/support/mrf_module_mock.ex new file mode 100644 index 000000000..12c7e22bc --- /dev/null +++ b/test/support/mrf_module_mock.ex @@ -0,0 +1,13 @@ +# Pleroma: A lightweight social networking server +# Copyright © 2017-2018 Pleroma Authors +# SPDX-License-Identifier: AGPL-3.0-only + +defmodule MRFModuleMock do + @behaviour Pleroma.Web.ActivityPub.MRF + + @impl true + def filter(message), do: {:ok, message} + + @impl true + def describe, do: {:ok, %{mrf_module_mock: "some config data"}} +end diff --git a/test/tasks/config_test.exs b/test/tasks/config_test.exs index a9b79eb5b..9cd47380c 100644 --- a/test/tasks/config_test.exs +++ b/test/tasks/config_test.exs @@ -11,21 +11,20 @@ defmodule Mix.Tasks.Pleroma.ConfigTest do Mix.shell(Mix.Shell.Process) temp_file = "config/temp.exported_from_db.secret.exs" - dynamic = Pleroma.Config.get([:instance, :dynamic_configuration]) - - Pleroma.Config.put([:instance, :dynamic_configuration], true) - on_exit(fn -> Mix.shell(Mix.Shell.IO) Application.delete_env(:pleroma, :first_setting) Application.delete_env(:pleroma, :second_setting) - Pleroma.Config.put([:instance, :dynamic_configuration], dynamic) :ok = File.rm(temp_file) end) {:ok, temp_file: temp_file} end + clear_config_all([:instance, :dynamic_configuration]) do + Pleroma.Config.put([:instance, :dynamic_configuration], true) + end + test "settings are migrated to db" do assert Repo.all(Config) == [] diff --git a/test/tasks/database_test.exs b/test/tasks/database_test.exs index 579130b05..a9925c361 100644 --- a/test/tasks/database_test.exs +++ b/test/tasks/database_test.exs @@ -3,8 +3,12 @@ # SPDX-License-Identifier: AGPL-3.0-only defmodule Mix.Tasks.Pleroma.DatabaseTest do + alias Pleroma.Activity + alias Pleroma.Object alias Pleroma.Repo alias Pleroma.User + alias Pleroma.Web.CommonAPI + use Pleroma.DataCase import Pleroma.Factory @@ -19,6 +23,52 @@ defmodule Mix.Tasks.Pleroma.DatabaseTest do :ok end + describe "running remove_embedded_objects" do + test "it replaces objects with references" do + user = insert(:user) + {:ok, activity} = CommonAPI.post(user, %{"status" => "test"}) + new_data = Map.put(activity.data, "object", activity.object.data) + + {:ok, activity} = + activity + |> Activity.change(%{data: new_data}) + |> Repo.update() + + assert is_map(activity.data["object"]) + + Mix.Tasks.Pleroma.Database.run(["remove_embedded_objects"]) + + activity = Activity.get_by_id_with_object(activity.id) + assert is_binary(activity.data["object"]) + end + end + + describe "prune_objects" do + test "it prunes old objects from the database" do + insert(:note) + deadline = Pleroma.Config.get([:instance, :remote_post_retention_days]) + 1 + + date = + Timex.now() + |> Timex.shift(days: -deadline) + |> Timex.to_naive_datetime() + |> NaiveDateTime.truncate(:second) + + %{id: id} = + :note + |> insert() + |> Ecto.Changeset.change(%{inserted_at: date}) + |> Repo.update!() + + assert length(Repo.all(Object)) == 2 + + Mix.Tasks.Pleroma.Database.run(["prune_objects"]) + + assert length(Repo.all(Object)) == 1 + refute Object.get_by_id(id) + end + end + describe "running update_users_following_followers_counts" do test "following and followers count are updated" do [user, user2] = insert_pair(:user) @@ -46,4 +96,37 @@ test "following and followers count are updated" do assert user.info.follower_count == 0 end end + + describe "running fix_likes_collections" do + test "it turns OrderedCollection likes into empty arrays" do + [user, user2] = insert_pair(:user) + + {:ok, %{id: id, object: object}} = CommonAPI.post(user, %{"status" => "test"}) + {:ok, %{object: object2}} = CommonAPI.post(user, %{"status" => "test test"}) + + CommonAPI.favorite(id, user2) + + likes = %{ + "first" => + "http://mastodon.example.org/objects/dbdbc507-52c8-490d-9b7c-1e1d52e5c132/likes?page=1", + "id" => "http://mastodon.example.org/objects/dbdbc507-52c8-490d-9b7c-1e1d52e5c132/likes", + "totalItems" => 3, + "type" => "OrderedCollection" + } + + new_data = Map.put(object2.data, "likes", likes) + + object2 + |> Ecto.Changeset.change(%{data: new_data}) + |> Repo.update() + + assert length(Object.get_by_id(object.id).data["likes"]) == 1 + assert is_map(Object.get_by_id(object2.id).data["likes"]) + + assert :ok == Mix.Tasks.Pleroma.Database.run(["fix_likes_collections"]) + + assert length(Object.get_by_id(object.id).data["likes"]) == 1 + assert Enum.empty?(Object.get_by_id(object2.id).data["likes"]) + end + end end diff --git a/test/tasks/digest_test.exs b/test/tasks/digest_test.exs new file mode 100644 index 000000000..4bfa1fb93 --- /dev/null +++ b/test/tasks/digest_test.exs @@ -0,0 +1,51 @@ +defmodule Mix.Tasks.Pleroma.DigestTest do + use Pleroma.DataCase + + import Pleroma.Factory + import Swoosh.TestAssertions + + alias Pleroma.Web.CommonAPI + + setup_all do + Mix.shell(Mix.Shell.Process) + + on_exit(fn -> + Mix.shell(Mix.Shell.IO) + end) + + :ok + end + + describe "pleroma.digest test" do + test "Sends digest to the given user" do + user1 = insert(:user) + user2 = insert(:user) + + Enum.each(0..10, fn i -> + {:ok, _activity} = + CommonAPI.post(user1, %{ + "status" => "hey ##{i} @#{user2.nickname}!" + }) + end) + + yesterday = + NaiveDateTime.add( + NaiveDateTime.truncate(NaiveDateTime.utc_now(), :second), + -60 * 60 * 24, + :second + ) + + {:ok, yesterday_date} = Timex.format(yesterday, "%F", :strftime) + + :ok = Mix.Tasks.Pleroma.Digest.run(["test", user2.nickname, yesterday_date]) + + assert_receive {:mix_shell, :info, [message]} + assert message =~ "Digest email have been sent" + + assert_email_sent( + to: {user2.name, user2.email}, + html_body: ~r/here is what you've missed!/i + ) + end + end +end diff --git a/test/tasks/relay_test.exs b/test/tasks/relay_test.exs index 9d260da3e..0d341c8d6 100644 --- a/test/tasks/relay_test.exs +++ b/test/tasks/relay_test.exs @@ -69,4 +69,27 @@ test "relay is unfollowed" do assert undo_activity.data["object"] == cancelled_activity.data end end + + describe "mix pleroma.relay list" do + test "Prints relay subscription list" do + :ok = Mix.Tasks.Pleroma.Relay.run(["list"]) + + refute_receive {:mix_shell, :info, _} + + Pleroma.Web.ActivityPub.Relay.get_actor() + |> Ecto.Changeset.change( + following: [ + "http://test-app.com/user/test1", + "http://test-app.com/user/test1", + "http://test-app-42.com/user/test1" + ] + ) + |> Pleroma.User.update_and_set_cache() + + :ok = Mix.Tasks.Pleroma.Relay.run(["list"]) + + assert_receive {:mix_shell, :info, ["test-app.com"]} + assert_receive {:mix_shell, :info, ["test-app-42.com"]} + end + end end diff --git a/test/tasks/robots_txt_test.exs b/test/tasks/robots_txt_test.exs index 78a3f17b4..917df2675 100644 --- a/test/tasks/robots_txt_test.exs +++ b/test/tasks/robots_txt_test.exs @@ -4,17 +4,17 @@ defmodule Mix.Tasks.Pleroma.RobotsTxtTest do use ExUnit.Case + use Pleroma.Tests.Helpers alias Mix.Tasks.Pleroma.RobotsTxt + clear_config([:instance, :static_dir]) + test "creates new dir" do path = "test/fixtures/new_dir/" file_path = path <> "robots.txt" - - static_dir = Pleroma.Config.get([:instance, :static_dir]) Pleroma.Config.put([:instance, :static_dir], path) on_exit(fn -> - Pleroma.Config.put([:instance, :static_dir], static_dir) {:ok, ["test/fixtures/new_dir/", "test/fixtures/new_dir/robots.txt"]} = File.rm_rf(path) end) @@ -29,11 +29,9 @@ test "creates new dir" do test "to existance folder" do path = "test/fixtures/" file_path = path <> "robots.txt" - static_dir = Pleroma.Config.get([:instance, :static_dir]) Pleroma.Config.put([:instance, :static_dir], path) on_exit(fn -> - Pleroma.Config.put([:instance, :static_dir], static_dir) :ok = File.rm(file_path) end) diff --git a/test/upload/filter/anonymize_filename_test.exs b/test/upload/filter/anonymize_filename_test.exs index a31b38ab1..6b33e7395 100644 --- a/test/upload/filter/anonymize_filename_test.exs +++ b/test/upload/filter/anonymize_filename_test.exs @@ -9,12 +9,6 @@ defmodule Pleroma.Upload.Filter.AnonymizeFilenameTest do alias Pleroma.Upload setup do - custom_filename = Config.get([Upload.Filter.AnonymizeFilename, :text]) - - on_exit(fn -> - Config.put([Upload.Filter.AnonymizeFilename, :text], custom_filename) - end) - upload_file = %Upload{ name: "an… image.jpg", content_type: "image/jpg", @@ -24,6 +18,8 @@ defmodule Pleroma.Upload.Filter.AnonymizeFilenameTest do %{upload_file: upload_file} end + clear_config([Pleroma.Upload.Filter.AnonymizeFilename, :text]) + test "it replaces filename on pre-defined text", %{upload_file: upload_file} do Config.put([Upload.Filter.AnonymizeFilename, :text], "custom-file.png") {:ok, %Upload{name: name}} = Upload.Filter.AnonymizeFilename.filter(upload_file) diff --git a/test/upload/filter/dedupe_test.exs b/test/upload/filter/dedupe_test.exs index fddd594dc..3de94dc20 100644 --- a/test/upload/filter/dedupe_test.exs +++ b/test/upload/filter/dedupe_test.exs @@ -25,7 +25,7 @@ test "adds shasum" do assert { :ok, - %Pleroma.Upload{id: @shasum, path: "#{@shasum}.jpg"} + %Pleroma.Upload{id: @shasum, path: @shasum <> ".jpg"} } = Dedupe.filter(upload) end end diff --git a/test/upload/filter/mogrify_test.exs b/test/upload/filter/mogrify_test.exs index c301440fd..210320d30 100644 --- a/test/upload/filter/mogrify_test.exs +++ b/test/upload/filter/mogrify_test.exs @@ -10,13 +10,7 @@ defmodule Pleroma.Upload.Filter.MogrifyTest do alias Pleroma.Upload alias Pleroma.Upload.Filter - setup do - filter = Config.get([Filter.Mogrify, :args]) - - on_exit(fn -> - Config.put([Filter.Mogrify, :args], filter) - end) - end + clear_config([Filter.Mogrify, :args]) test "apply mogrify filter" do Config.put([Filter.Mogrify, :args], [{"tint", "40"}]) diff --git a/test/upload/filter_test.exs b/test/upload/filter_test.exs index 640cd7107..03887c06a 100644 --- a/test/upload/filter_test.exs +++ b/test/upload/filter_test.exs @@ -8,13 +8,7 @@ defmodule Pleroma.Upload.FilterTest do alias Pleroma.Config alias Pleroma.Upload.Filter - setup do - custom_filename = Config.get([Pleroma.Upload.Filter.AnonymizeFilename, :text]) - - on_exit(fn -> - Config.put([Pleroma.Upload.Filter.AnonymizeFilename, :text], custom_filename) - end) - end + clear_config([Pleroma.Upload.Filter.AnonymizeFilename, :text]) test "applies filters" do Config.put([Pleroma.Upload.Filter.AnonymizeFilename, :text], "custom-file.png") diff --git a/test/upload_test.exs b/test/upload_test.exs index 32c6977d1..6721fe82e 100644 --- a/test/upload_test.exs +++ b/test/upload_test.exs @@ -122,24 +122,6 @@ test "returns a media url" do assert String.starts_with?(url, Pleroma.Web.base_url() <> "/media/") end - test "returns a media url with configured base_url" do - base_url = "https://cache.pleroma.social" - - File.cp!("test/fixtures/image.jpg", "test/fixtures/image_tmp.jpg") - - file = %Plug.Upload{ - content_type: "image/jpg", - path: Path.absname("test/fixtures/image_tmp.jpg"), - filename: "image.jpg" - } - - {:ok, data} = Upload.store(file, base_url: base_url) - - assert %{"url" => [%{"href" => url}]} = data - - assert String.starts_with?(url, base_url <> "/media/") - end - test "copies the file to the configured folder with deduping" do File.cp!("test/fixtures/image.jpg", "test/fixtures/image_tmp.jpg") @@ -266,4 +248,28 @@ test "escapes reserved uri characters" do "%3A%3F%23%5B%5D%40%21%24%26%5C%27%28%29%2A%2B%2C%3B%3D.jpg" end end + + describe "Setting a custom base_url for uploaded media" do + clear_config([Pleroma.Upload, :base_url]) do + Pleroma.Config.put([Pleroma.Upload, :base_url], "https://cache.pleroma.social") + end + + test "returns a media url with configured base_url" do + base_url = Pleroma.Config.get([Pleroma.Upload, :base_url]) + + File.cp!("test/fixtures/image.jpg", "test/fixtures/image_tmp.jpg") + + file = %Plug.Upload{ + content_type: "image/jpg", + path: Path.absname("test/fixtures/image_tmp.jpg"), + filename: "image.jpg" + } + + {:ok, data} = Upload.store(file, base_url: base_url) + + assert %{"url" => [%{"href" => url}]} = data + + refute String.starts_with?(url, base_url <> "/media/") + end + end end diff --git a/test/uploaders/local_test.exs b/test/uploaders/local_test.exs new file mode 100644 index 000000000..fc442d0f1 --- /dev/null +++ b/test/uploaders/local_test.exs @@ -0,0 +1,32 @@ +# Pleroma: A lightweight social networking server +# Copyright © 2017-2019 Pleroma Authors +# SPDX-License-Identifier: AGPL-3.0-only + +defmodule Pleroma.Uploaders.LocalTest do + use Pleroma.DataCase + alias Pleroma.Uploaders.Local + + describe "get_file/1" do + test "it returns path to local folder for files" do + assert Local.get_file("") == {:ok, {:static_dir, "test/uploads"}} + end + end + + describe "put_file/1" do + test "put file to local folder" do + file_path = "local_upload/files/image.jpg" + + file = %Pleroma.Upload{ + name: "image.jpg", + content_type: "image/jpg", + path: file_path, + tempfile: Path.absname("test/fixtures/image_tmp.jpg") + } + + assert Local.put_file(file) == :ok + + assert Path.join([Local.upload_path(), file_path]) + |> File.exists?() + end + end +end diff --git a/test/uploaders/mdii_test.exs b/test/uploaders/mdii_test.exs new file mode 100644 index 000000000..d432d40f0 --- /dev/null +++ b/test/uploaders/mdii_test.exs @@ -0,0 +1,50 @@ +# Pleroma: A lightweight social networking server +# Copyright © 2017-2019 Pleroma Authors +# SPDX-License-Identifier: AGPL-3.0-only + +defmodule Pleroma.Uploaders.MDIITest do + use Pleroma.DataCase + alias Pleroma.Uploaders.MDII + import Tesla.Mock + + describe "get_file/1" do + test "it returns path to local folder for files" do + assert MDII.get_file("") == {:ok, {:static_dir, "test/uploads"}} + end + end + + describe "put_file/1" do + setup do + file_upload = %Pleroma.Upload{ + name: "mdii-image.jpg", + content_type: "image/jpg", + path: "test_folder/mdii-image.jpg", + tempfile: Path.absname("test/fixtures/image_tmp.jpg") + } + + [file_upload: file_upload] + end + + test "save file", %{file_upload: file_upload} do + mock(fn + %{method: :post, url: "https://mdii.sakura.ne.jp/mdii-post.cgi?jpg"} -> + %Tesla.Env{status: 200, body: "mdii-image"} + end) + + assert MDII.put_file(file_upload) == + {:ok, {:url, "https://mdii.sakura.ne.jp/mdii-image.jpg"}} + end + + test "save file to local if MDII isn`t available", %{file_upload: file_upload} do + mock(fn + %{method: :post, url: "https://mdii.sakura.ne.jp/mdii-post.cgi?jpg"} -> + %Tesla.Env{status: 500} + end) + + assert MDII.put_file(file_upload) == :ok + + assert Path.join([Pleroma.Uploaders.Local.upload_path(), file_upload.path]) + |> File.exists?() + end + end +end diff --git a/test/uploaders/s3_test.exs b/test/uploaders/s3_test.exs new file mode 100644 index 000000000..171316340 --- /dev/null +++ b/test/uploaders/s3_test.exs @@ -0,0 +1,82 @@ +# Pleroma: A lightweight social networking server +# Copyright © 2017-2019 Pleroma Authors +# SPDX-License-Identifier: AGPL-3.0-only + +defmodule Pleroma.Uploaders.S3Test do + use Pleroma.DataCase + + alias Pleroma.Config + alias Pleroma.Uploaders.S3 + + import Mock + import ExUnit.CaptureLog + + clear_config([Pleroma.Uploaders.S3]) do + Config.put([Pleroma.Uploaders.S3], + bucket: "test_bucket", + public_endpoint: "https://s3.amazonaws.com" + ) + end + + describe "get_file/1" do + test "it returns path to local folder for files" do + assert S3.get_file("test_image.jpg") == { + :ok, + {:url, "https://s3.amazonaws.com/test_bucket/test_image.jpg"} + } + end + + test "it returns path without bucket when truncated_namespace set to ''" do + Config.put([Pleroma.Uploaders.S3], + bucket: "test_bucket", + public_endpoint: "https://s3.amazonaws.com", + truncated_namespace: "" + ) + + assert S3.get_file("test_image.jpg") == { + :ok, + {:url, "https://s3.amazonaws.com/test_image.jpg"} + } + end + + test "it returns path with bucket namespace when namespace is set" do + Config.put([Pleroma.Uploaders.S3], + bucket: "test_bucket", + public_endpoint: "https://s3.amazonaws.com", + bucket_namespace: "family" + ) + + assert S3.get_file("test_image.jpg") == { + :ok, + {:url, "https://s3.amazonaws.com/family:test_bucket/test_image.jpg"} + } + end + end + + describe "put_file/1" do + setup do + file_upload = %Pleroma.Upload{ + name: "image-tet.jpg", + content_type: "image/jpg", + path: "test_folder/image-tet.jpg", + tempfile: Path.absname("test/fixtures/image_tmp.jpg") + } + + [file_upload: file_upload] + end + + test "save file", %{file_upload: file_upload} do + with_mock ExAws, request: fn _ -> {:ok, :ok} end do + assert S3.put_file(file_upload) == {:ok, {:file, "test_folder/image-tet.jpg"}} + end + end + + test "returns error", %{file_upload: file_upload} do + with_mock ExAws, request: fn _ -> {:error, "S3 Upload failed"} end do + assert capture_log(fn -> + assert S3.put_file(file_upload) == {:error, "S3 Upload failed"} + end) =~ "Elixir.Pleroma.Uploaders.S3: {:error, \"S3 Upload failed\"}" + end + end + end +end diff --git a/test/user_info_test.exs b/test/user_info_test.exs new file mode 100644 index 000000000..2d795594e --- /dev/null +++ b/test/user_info_test.exs @@ -0,0 +1,24 @@ +defmodule Pleroma.UserInfoTest do + alias Pleroma.Repo + alias Pleroma.User.Info + + use Pleroma.DataCase + + import Pleroma.Factory + + describe "update_email_notifications/2" do + setup do + user = insert(:user, %{info: %{email_notifications: %{"digest" => true}}}) + + {:ok, user: user} + end + + test "Notifications are updated", %{user: user} do + true = user.info.email_notifications["digest"] + changeset = Info.update_email_notifications(user.info, %{"digest" => false}) + assert changeset.valid? + {:ok, result} = Ecto.Changeset.apply_action(changeset, :insert) + assert result.email_notifications["digest"] == false + end + end +end diff --git a/test/user_search_test.exs b/test/user_search_test.exs index 4de6c82a5..48ce973ad 100644 --- a/test/user_search_test.exs +++ b/test/user_search_test.exs @@ -193,7 +193,14 @@ test "works with URIs" do user = User.get_cached_by_ap_id("http://mastodon.example.org/users/admin") assert length(results) == 1 - assert user == result |> Map.put(:search_rank, nil) |> Map.put(:search_type, nil) + + expected = + result + |> Map.put(:search_rank, nil) + |> Map.put(:search_type, nil) + |> Map.put(:last_digest_emailed_at, nil) + + assert user == expected end test "excludes a blocked users from search result" do diff --git a/test/user_test.exs b/test/user_test.exs index 8a7b7537f..661ffc0b3 100644 --- a/test/user_test.exs +++ b/test/user_test.exs @@ -21,6 +21,8 @@ defmodule Pleroma.UserTest do :ok end + clear_config([:instance, :account_activation_required]) + describe "when tags are nil" do test "tagging a user" do user = insert(:user, %{tags: nil}) @@ -90,6 +92,17 @@ test "doesn't return already accepted or duplicate follow requests" do assert activity end + test "clears follow requests when requester is blocked" do + followed = insert(:user, %{info: %{locked: true}}) + follower = insert(:user) + + CommonAPI.follow(follower, followed) + assert {:ok, [_activity]} = User.get_follow_requests(followed) + + {:ok, _follower} = User.block(followed, follower) + assert {:ok, []} = User.get_follow_requests(followed) + end + test "follow_all follows mutliple users" do user = insert(:user) followed_zero = insert(:user) @@ -192,24 +205,64 @@ test "local users do not automatically follow local locked accounts" do # assert websub # end - test "unfollow takes a user and another user" do - followed = insert(:user) - user = insert(:user, %{following: [User.ap_followers(followed)]}) + describe "unfollow/2" do + setup do + setting = Pleroma.Config.get([:instance, :external_user_synchronization]) - {:ok, user, _activity} = User.unfollow(user, followed) + on_exit(fn -> + Pleroma.Config.put([:instance, :external_user_synchronization], setting) + end) - user = User.get_cached_by_id(user.id) + :ok + end - assert user.following == [] - end + test "unfollow with syncronizes external user" do + Pleroma.Config.put([:instance, :external_user_synchronization], true) - test "unfollow doesn't unfollow yourself" do - user = insert(:user) + followed = + insert(:user, + nickname: "fuser1", + follower_address: "http://localhost:4001/users/fuser1/followers", + following_address: "http://localhost:4001/users/fuser1/following", + ap_id: "http://localhost:4001/users/fuser1" + ) - {:error, _} = User.unfollow(user, user) + user = + insert(:user, %{ + local: false, + nickname: "fuser2", + ap_id: "http://localhost:4001/users/fuser2", + follower_address: "http://localhost:4001/users/fuser2/followers", + following_address: "http://localhost:4001/users/fuser2/following", + following: [User.ap_followers(followed)] + }) - user = User.get_cached_by_id(user.id) - assert user.following == [user.ap_id] + {:ok, user, _activity} = User.unfollow(user, followed) + + user = User.get_cached_by_id(user.id) + + assert user.following == [] + end + + test "unfollow takes a user and another user" do + followed = insert(:user) + user = insert(:user, %{following: [User.ap_followers(followed)]}) + + {:ok, user, _activity} = User.unfollow(user, followed) + + user = User.get_cached_by_id(user.id) + + assert user.following == [] + end + + test "unfollow doesn't unfollow yourself" do + user = insert(:user) + + {:error, _} = User.unfollow(user, user) + + user = User.get_cached_by_id(user.id) + assert user.following == [user.ap_id] + end end test "test if a user is following another user" do @@ -236,6 +289,9 @@ test "fetches correct profile for nickname beginning with number" do password_confirmation: "test", email: "email@example.com" } + clear_config([:instance, :autofollowed_nicknames]) + clear_config([:instance, :welcome_message]) + clear_config([:instance, :welcome_user_nickname]) test "it autofollows accounts that are set for it" do user = insert(:user) @@ -252,8 +308,6 @@ test "it autofollows accounts that are set for it" do assert User.following?(registered_user, user) refute User.following?(registered_user, remote_user) - - Pleroma.Config.put([:instance, :autofollowed_nicknames], []) end test "it sends a welcome message if it is set" do @@ -269,9 +323,6 @@ test "it sends a welcome message if it is set" do assert registered_user.ap_id in activity.recipients assert Object.normalize(activity).data["content"] =~ "cool site" assert activity.actor == welcome_user.ap_id - - Pleroma.Config.put([:instance, :welcome_user_nickname], nil) - Pleroma.Config.put([:instance, :welcome_message], nil) end test "it requires an email, name, nickname and password, bio is optional" do @@ -337,15 +388,8 @@ test "it ensures info is not nil" do email: "email@example.com" } - setup do - setting = Pleroma.Config.get([:instance, :account_activation_required]) - - unless setting do - Pleroma.Config.put([:instance, :account_activation_required], true) - on_exit(fn -> Pleroma.Config.put([:instance, :account_activation_required], setting) end) - end - - :ok + clear_config([:instance, :account_activation_required]) do + Pleroma.Config.put([:instance, :account_activation_required], true) end test "it creates unconfirmed user" do @@ -497,6 +541,9 @@ test "returns an ap_followers link for a user" do avatar: %{some: "avatar"} } + clear_config([:instance, :user_bio_length]) + clear_config([:instance, :user_name_length]) + test "it confirms validity" do cs = User.remote_user_creation(@valid_remote) assert cs.valid? @@ -525,7 +572,10 @@ test "it has required fields" do end test "it restricts some sizes" do - [bio: 5000, name: 100] + bio_limit = Pleroma.Config.get([:instance, :user_bio_length], 5000) + name_limit = Pleroma.Config.get([:instance, :user_name_length], 100) + + [bio: bio_limit, name: name_limit] |> Enum.each(fn {field, size} -> string = String.pad_leading(".", size) cs = User.remote_user_creation(Map.put(@valid_remote, field, string)) @@ -989,6 +1039,8 @@ test "hide a user's statuses from timelines and notifications" do [user: user] end + clear_config([:instance, :federating]) + test ".delete_user_activities deletes all create activities", %{user: user} do {:ok, activity} = CommonAPI.post(user, %{"status" => "2hu"}) @@ -998,6 +1050,13 @@ test ".delete_user_activities deletes all create activities", %{user: user} do refute Activity.get_by_id(activity.id) end + test "it deletes deactivated user" do + {:ok, user} = insert(:user, info: %{deactivated: true}) |> User.set_cache() + + assert {:ok, _} = User.delete(user) + refute User.get_by_id(user.id) + end + test "it deletes a user, all follow relationships and all activities", %{user: user} do follower = insert(:user) {:ok, follower} = User.follow(follower, user) @@ -1039,9 +1098,7 @@ test "it deletes a user, all follow relationships and all activities", %{user: u Pleroma.Web.ActivityPub.Publisher, [:passthrough], [] do - config_path = [:instance, :federating] - initial_setting = Pleroma.Config.get(config_path) - Pleroma.Config.put(config_path, true) + Pleroma.Config.put([:instance, :federating], true) {:ok, follower} = User.get_or_fetch_by_ap_id("http://mastodon.example.org/users/admin") {:ok, _} = User.follow(follower, user) @@ -1053,8 +1110,6 @@ test "it deletes a user, all follow relationships and all activities", %{user: u inbox: "http://mastodon.example.org/inbox" }) ) - - Pleroma.Config.put(config_path, initial_setting) end end @@ -1120,8 +1175,6 @@ test "auth_active?/1 works correctly" do refute User.auth_active?(local_user) assert User.auth_active?(confirmed_user) assert User.auth_active?(remote_user) - - Pleroma.Config.put([:instance, :account_activation_required], false) end describe "superuser?/1" do @@ -1166,8 +1219,6 @@ test "returns false when the account is unauthenticated and auth is required" do other_user = insert(:user, local: true) refute User.visible_for?(user, other_user) - - Pleroma.Config.put([:instance, :account_activation_required], false) end test "returns true when the account is unauthenticated and auth is not required" do @@ -1184,8 +1235,6 @@ test "returns true when the account is unauthenticated and being viewed by a pri other_user = insert(:user, local: true, info: %{is_admin: true}) assert User.visible_for?(user, other_user) - - Pleroma.Config.put([:instance, :account_activation_required], false) end end @@ -1237,6 +1286,109 @@ test "follower count is updated when a follower is blocked" do assert Map.get(user_show, "followers_count") == 2 end + describe "list_inactive_users_query/1" do + defp days_ago(days) do + NaiveDateTime.add( + NaiveDateTime.truncate(NaiveDateTime.utc_now(), :second), + -days * 60 * 60 * 24, + :second + ) + end + + test "Users are inactive by default" do + total = 10 + + users = + Enum.map(1..total, fn _ -> + insert(:user, last_digest_emailed_at: days_ago(20), info: %{deactivated: false}) + end) + + inactive_users_ids = + Pleroma.User.list_inactive_users_query() + |> Pleroma.Repo.all() + |> Enum.map(& &1.id) + + Enum.each(users, fn user -> + assert user.id in inactive_users_ids + end) + end + + test "Only includes users who has no recent activity" do + total = 10 + + users = + Enum.map(1..total, fn _ -> + insert(:user, last_digest_emailed_at: days_ago(20), info: %{deactivated: false}) + end) + + {inactive, active} = Enum.split(users, trunc(total / 2)) + + Enum.map(active, fn user -> + to = Enum.random(users -- [user]) + + {:ok, _} = + Pleroma.Web.TwitterAPI.TwitterAPI.create_status(user, %{ + "status" => "hey @#{to.nickname}" + }) + end) + + inactive_users_ids = + Pleroma.User.list_inactive_users_query() + |> Pleroma.Repo.all() + |> Enum.map(& &1.id) + + Enum.each(active, fn user -> + refute user.id in inactive_users_ids + end) + + Enum.each(inactive, fn user -> + assert user.id in inactive_users_ids + end) + end + + test "Only includes users with no read notifications" do + total = 10 + + users = + Enum.map(1..total, fn _ -> + insert(:user, last_digest_emailed_at: days_ago(20), info: %{deactivated: false}) + end) + + [sender | recipients] = users + {inactive, active} = Enum.split(recipients, trunc(total / 2)) + + Enum.each(recipients, fn to -> + {:ok, _} = + Pleroma.Web.TwitterAPI.TwitterAPI.create_status(sender, %{ + "status" => "hey @#{to.nickname}" + }) + + {:ok, _} = + Pleroma.Web.TwitterAPI.TwitterAPI.create_status(sender, %{ + "status" => "hey again @#{to.nickname}" + }) + end) + + Enum.each(active, fn user -> + [n1, _n2] = Pleroma.Notification.for_user(user) + {:ok, _} = Pleroma.Notification.read_one(user, n1.id) + end) + + inactive_users_ids = + Pleroma.User.list_inactive_users_query() + |> Pleroma.Repo.all() + |> Enum.map(& &1.id) + + Enum.each(active, fn user -> + refute user.id in inactive_users_ids + end) + + Enum.each(inactive, fn user -> + assert user.id in inactive_users_ids + end) + end + end + describe "toggle_confirmation/1" do test "if user is confirmed" do user = insert(:user, info: %{confirmation_pending: false}) @@ -1369,4 +1521,99 @@ test "user with internal-prefixed nickname returns true" do assert User.is_internal_user?(user) end end + + describe "update_and_set_cache/1" do + test "returns error when user is stale instead Ecto.StaleEntryError" do + user = insert(:user) + + changeset = Ecto.Changeset.change(user, bio: "test") + + Repo.delete(user) + + assert {:error, %Ecto.Changeset{errors: [id: {"is stale", [stale: true]}], valid?: false}} = + User.update_and_set_cache(changeset) + end + + test "performs update cache if user updated" do + user = insert(:user) + assert {:ok, nil} = Cachex.get(:user_cache, "ap_id:#{user.ap_id}") + + changeset = Ecto.Changeset.change(user, bio: "test-bio") + + assert {:ok, %User{bio: "test-bio"} = user} = User.update_and_set_cache(changeset) + assert {:ok, user} = Cachex.get(:user_cache, "ap_id:#{user.ap_id}") + assert %User{bio: "test-bio"} = User.get_cached_by_ap_id(user.ap_id) + end + end + + describe "following/followers synchronization" do + clear_config([:instance, :external_user_synchronization]) + + test "updates the counters normally on following/getting a follow when disabled" do + Pleroma.Config.put([:instance, :external_user_synchronization], false) + user = insert(:user) + + other_user = + insert(:user, + local: false, + follower_address: "http://localhost:4001/users/masto_closed/followers", + following_address: "http://localhost:4001/users/masto_closed/following", + info: %{ap_enabled: true} + ) + + assert User.user_info(other_user).following_count == 0 + assert User.user_info(other_user).follower_count == 0 + + {:ok, user} = Pleroma.User.follow(user, other_user) + other_user = Pleroma.User.get_by_id(other_user.id) + + assert User.user_info(user).following_count == 1 + assert User.user_info(other_user).follower_count == 1 + end + + test "syncronizes the counters with the remote instance for the followed when enabled" do + Pleroma.Config.put([:instance, :external_user_synchronization], false) + + user = insert(:user) + + other_user = + insert(:user, + local: false, + follower_address: "http://localhost:4001/users/masto_closed/followers", + following_address: "http://localhost:4001/users/masto_closed/following", + info: %{ap_enabled: true} + ) + + assert User.user_info(other_user).following_count == 0 + assert User.user_info(other_user).follower_count == 0 + + Pleroma.Config.put([:instance, :external_user_synchronization], true) + {:ok, _user} = User.follow(user, other_user) + other_user = User.get_by_id(other_user.id) + + assert User.user_info(other_user).follower_count == 437 + end + + test "syncronizes the counters with the remote instance for the follower when enabled" do + Pleroma.Config.put([:instance, :external_user_synchronization], false) + + user = insert(:user) + + other_user = + insert(:user, + local: false, + follower_address: "http://localhost:4001/users/masto_closed/followers", + following_address: "http://localhost:4001/users/masto_closed/following", + info: %{ap_enabled: true} + ) + + assert User.user_info(other_user).following_count == 0 + assert User.user_info(other_user).follower_count == 0 + + Pleroma.Config.put([:instance, :external_user_synchronization], true) + {:ok, other_user} = User.follow(other_user, user) + + assert User.user_info(other_user).following_count == 152 + end + end end diff --git a/test/web/activity_pub/activity_pub_controller_test.exs b/test/web/activity_pub/activity_pub_controller_test.exs index 40344f17e..77f5e39fa 100644 --- a/test/web/activity_pub/activity_pub_controller_test.exs +++ b/test/web/activity_pub/activity_pub_controller_test.exs @@ -16,17 +16,16 @@ defmodule Pleroma.Web.ActivityPub.ActivityPubControllerTest do setup_all do Tesla.Mock.mock_global(fn env -> apply(HttpRequestMock, :request, [env]) end) - - config_path = [:instance, :federating] - initial_setting = Pleroma.Config.get(config_path) - - Pleroma.Config.put(config_path, true) - on_exit(fn -> Pleroma.Config.put(config_path, initial_setting) end) - :ok end + clear_config_all([:instance, :federating], + do: Pleroma.Config.put([:instance, :federating], true) + ) + describe "/relay" do + clear_config([:instance, :allow_relay]) + test "with the relay active, it returns the relay user", %{conn: conn} do res = conn @@ -43,8 +42,6 @@ test "with the relay disabled, it returns 404", %{conn: conn} do |> get(activity_pub_path(conn, :relay)) |> json_response(404) |> assert - - Pleroma.Config.put([:instance, :allow_relay], true) end end @@ -180,18 +177,65 @@ test "it returns 404 for tombstone objects", %{conn: conn} do end describe "/object/:uuid/likes" do - test "it returns the like activities in a collection", %{conn: conn} do + setup do like = insert(:like_activity) like_object_ap_id = Object.normalize(like).data["id"] - uuid = String.split(like_object_ap_id, "/") |> List.last() + uuid = + like_object_ap_id + |> String.split("/") + |> List.last() + + [id: like.data["id"], uuid: uuid] + end + + test "it returns the like activities in a collection", %{conn: conn, id: id, uuid: uuid} do result = conn |> put_req_header("accept", "application/activity+json") |> get("/objects/#{uuid}/likes") |> json_response(200) - assert List.first(result["first"]["orderedItems"])["id"] == like.data["id"] + assert List.first(result["first"]["orderedItems"])["id"] == id + assert result["type"] == "OrderedCollection" + assert result["totalItems"] == 1 + refute result["first"]["next"] + end + + test "it does not crash when page number is exceeded total pages", %{conn: conn, uuid: uuid} do + result = + conn + |> put_req_header("accept", "application/activity+json") + |> get("/objects/#{uuid}/likes?page=2") + |> json_response(200) + + assert result["type"] == "OrderedCollectionPage" + assert result["totalItems"] == 1 + refute result["next"] + assert Enum.empty?(result["orderedItems"]) + end + + test "it contains the next key when likes count is more than 10", %{conn: conn} do + note = insert(:note_activity) + insert_list(11, :like_activity, note_activity: note) + + uuid = + note + |> Object.normalize() + |> Map.get(:data) + |> Map.get("id") + |> String.split("/") + |> List.last() + + result = + conn + |> put_req_header("accept", "application/activity+json") + |> get("/objects/#{uuid}/likes?page=1") + |> json_response(200) + + assert result["totalItems"] == 11 + assert length(result["orderedItems"]) == 10 + assert result["next"] end end diff --git a/test/web/activity_pub/activity_pub_test.exs b/test/web/activity_pub/activity_pub_test.exs index 1c0b274cb..1515f4eb6 100644 --- a/test/web/activity_pub/activity_pub_test.exs +++ b/test/web/activity_pub/activity_pub_test.exs @@ -538,6 +538,29 @@ test "doesn't return muted activities" do assert Enum.member?(activities, activity_one) end + test "doesn't return thread muted activities" do + user = insert(:user) + _activity_one = insert(:note_activity) + note_two = insert(:note, data: %{"context" => "suya.."}) + activity_two = insert(:note_activity, note: note_two) + + {:ok, _activity_two} = CommonAPI.add_mute(user, activity_two) + + assert [_activity_one] = ActivityPub.fetch_activities([], %{"muting_user" => user}) + end + + test "returns thread muted activities when with_muted is set" do + user = insert(:user) + _activity_one = insert(:note_activity) + note_two = insert(:note, data: %{"context" => "suya.."}) + activity_two = insert(:note_activity, note: note_two) + + {:ok, _activity_two} = CommonAPI.add_mute(user, activity_two) + + assert [_activity_two, _activity_one] = + ActivityPub.fetch_activities([], %{"muting_user" => user, "with_muted" => true}) + end + test "does include announces on request" do activity_three = insert(:note_activity) user = insert(:user) @@ -677,14 +700,8 @@ test "adds a like activity to the db" do assert object.data["likes"] == [user.ap_id] assert object.data["like_count"] == 1 - [note_activity] = Activity.get_all_create_by_object_ap_id(object.data["id"]) - assert note_activity.data["object"]["like_count"] == 1 - {:ok, _like_activity, object} = ActivityPub.like(user_two, object) assert object.data["like_count"] == 2 - - [note_activity] = Activity.get_all_create_by_object_ap_id(object.data["id"]) - assert note_activity.data["object"]["like_count"] == 2 end end @@ -1128,4 +1145,65 @@ test "fetches only public posts for other users" do assert result.id == activity.id end end + + describe "fetch_follow_information_for_user" do + test "syncronizes following/followers counters" do + user = + insert(:user, + local: false, + follower_address: "http://localhost:4001/users/fuser2/followers", + following_address: "http://localhost:4001/users/fuser2/following" + ) + + {:ok, info} = ActivityPub.fetch_follow_information_for_user(user) + assert info.follower_count == 527 + assert info.following_count == 267 + end + + test "detects hidden followers" do + mock(fn env -> + case env.url do + "http://localhost:4001/users/masto_closed/followers?page=1" -> + %Tesla.Env{status: 403, body: ""} + + _ -> + apply(HttpRequestMock, :request, [env]) + end + end) + + user = + insert(:user, + local: false, + follower_address: "http://localhost:4001/users/masto_closed/followers", + following_address: "http://localhost:4001/users/masto_closed/following" + ) + + {:ok, info} = ActivityPub.fetch_follow_information_for_user(user) + assert info.hide_followers == true + assert info.hide_follows == false + end + + test "detects hidden follows" do + mock(fn env -> + case env.url do + "http://localhost:4001/users/masto_closed/following?page=1" -> + %Tesla.Env{status: 403, body: ""} + + _ -> + apply(HttpRequestMock, :request, [env]) + end + end) + + user = + insert(:user, + local: false, + follower_address: "http://localhost:4001/users/masto_closed/followers", + following_address: "http://localhost:4001/users/masto_closed/following" + ) + + {:ok, info} = ActivityPub.fetch_follow_information_for_user(user) + assert info.hide_followers == false + assert info.hide_follows == true + end + end end diff --git a/test/web/activity_pub/mrf/mrf_test.exs b/test/web/activity_pub/mrf/mrf_test.exs index a9cdf5317..04709df17 100644 --- a/test/web/activity_pub/mrf/mrf_test.exs +++ b/test/web/activity_pub/mrf/mrf_test.exs @@ -1,11 +1,12 @@ defmodule Pleroma.Web.ActivityPub.MRFTest do use ExUnit.Case, async: true + use Pleroma.Tests.Helpers alias Pleroma.Web.ActivityPub.MRF test "subdomains_regex/1" do assert MRF.subdomains_regex(["unsafe.tld", "*.unsafe.tld"]) == [ - ~r/^unsafe.tld$/, - ~r/^(.*\.)*unsafe.tld$/ + ~r/^unsafe.tld$/i, + ~r/^(.*\.)*unsafe.tld$/i ] end @@ -13,7 +14,7 @@ test "subdomains_regex/1" do test "common domains" do regexes = MRF.subdomains_regex(["unsafe.tld", "unsafe2.tld"]) - assert regexes == [~r/^unsafe.tld$/, ~r/^unsafe2.tld$/] + assert regexes == [~r/^unsafe.tld$/i, ~r/^unsafe2.tld$/i] assert MRF.subdomain_match?(regexes, "unsafe.tld") assert MRF.subdomain_match?(regexes, "unsafe2.tld") @@ -24,7 +25,7 @@ test "common domains" do test "wildcard domains with one subdomain" do regexes = MRF.subdomains_regex(["*.unsafe.tld"]) - assert regexes == [~r/^(.*\.)*unsafe.tld$/] + assert regexes == [~r/^(.*\.)*unsafe.tld$/i] assert MRF.subdomain_match?(regexes, "unsafe.tld") assert MRF.subdomain_match?(regexes, "sub.unsafe.tld") @@ -35,12 +36,51 @@ test "wildcard domains with one subdomain" do test "wildcard domains with two subdomains" do regexes = MRF.subdomains_regex(["*.unsafe.tld"]) - assert regexes == [~r/^(.*\.)*unsafe.tld$/] + assert regexes == [~r/^(.*\.)*unsafe.tld$/i] assert MRF.subdomain_match?(regexes, "unsafe.tld") assert MRF.subdomain_match?(regexes, "sub.sub.unsafe.tld") refute MRF.subdomain_match?(regexes, "sub.anotherunsafe.tld") refute MRF.subdomain_match?(regexes, "sub.unsafe.tldanother") end + + test "matches are case-insensitive" do + regexes = MRF.subdomains_regex(["UnSafe.TLD", "UnSAFE2.Tld"]) + + assert regexes == [~r/^UnSafe.TLD$/i, ~r/^UnSAFE2.Tld$/i] + + assert MRF.subdomain_match?(regexes, "UNSAFE.TLD") + assert MRF.subdomain_match?(regexes, "UNSAFE2.TLD") + assert MRF.subdomain_match?(regexes, "unsafe.tld") + assert MRF.subdomain_match?(regexes, "unsafe2.tld") + + refute MRF.subdomain_match?(regexes, "EXAMPLE.COM") + refute MRF.subdomain_match?(regexes, "example.com") + end + end + + describe "describe/0" do + clear_config([:instance, :rewrite_policy]) + + test "it works as expected with noop policy" do + expected = %{ + mrf_policies: ["NoOpPolicy"], + exclusions: false + } + + {:ok, ^expected} = MRF.describe() + end + + test "it works as expected with mock policy" do + Pleroma.Config.put([:instance, :rewrite_policy], [MRFModuleMock]) + + expected = %{ + mrf_policies: ["MRFModuleMock"], + mrf_module_mock: "some config data", + exclusions: false + } + + {:ok, ^expected} = MRF.describe() + end end end diff --git a/test/web/activity_pub/mrf/reject_non_public_test.exs b/test/web/activity_pub/mrf/reject_non_public_test.exs index fdf6b245e..fc1d190bb 100644 --- a/test/web/activity_pub/mrf/reject_non_public_test.exs +++ b/test/web/activity_pub/mrf/reject_non_public_test.exs @@ -8,12 +8,7 @@ defmodule Pleroma.Web.ActivityPub.MRF.RejectNonPublicTest do alias Pleroma.Web.ActivityPub.MRF.RejectNonPublic - setup do - policy = Pleroma.Config.get([:mrf_rejectnonpublic]) - on_exit(fn -> Pleroma.Config.put([:mrf_rejectnonpublic], policy) end) - - :ok - end + clear_config([:mrf_rejectnonpublic]) describe "public message" do test "it's allowed when address is public" do diff --git a/test/web/activity_pub/mrf/simple_policy_test.exs b/test/web/activity_pub/mrf/simple_policy_test.exs index 8e86d2219..7203b27da 100644 --- a/test/web/activity_pub/mrf/simple_policy_test.exs +++ b/test/web/activity_pub/mrf/simple_policy_test.exs @@ -8,9 +8,7 @@ defmodule Pleroma.Web.ActivityPub.MRF.SimplePolicyTest do alias Pleroma.Config alias Pleroma.Web.ActivityPub.MRF.SimplePolicy - setup do - orig = Config.get!(:mrf_simple) - + clear_config([:mrf_simple]) do Config.put(:mrf_simple, media_removal: [], media_nsfw: [], @@ -21,10 +19,6 @@ defmodule Pleroma.Web.ActivityPub.MRF.SimplePolicyTest do avatar_removal: [], banner_removal: [] ) - - on_exit(fn -> - Config.put(:mrf_simple, orig) - end) end describe "when :media_removal" do diff --git a/test/web/activity_pub/mrf/user_allowlist_policy_test.exs b/test/web/activity_pub/mrf/user_allowlist_policy_test.exs index 6519e2398..72084c0fd 100644 --- a/test/web/activity_pub/mrf/user_allowlist_policy_test.exs +++ b/test/web/activity_pub/mrf/user_allowlist_policy_test.exs @@ -7,12 +7,7 @@ defmodule Pleroma.Web.ActivityPub.MRF.UserAllowListPolicyTest do alias Pleroma.Web.ActivityPub.MRF.UserAllowListPolicy - setup do - policy = Pleroma.Config.get([:mrf_user_allowlist]) || [] - on_exit(fn -> Pleroma.Config.put([:mrf_user_allowlist], policy) end) - - :ok - end + clear_config([:mrf_user_allowlist, :localhost]) test "pass filter if allow list is empty" do actor = insert(:user) diff --git a/test/web/activity_pub/mrf/vocabulary_policy_test.exs b/test/web/activity_pub/mrf/vocabulary_policy_test.exs new file mode 100644 index 000000000..38309f9f1 --- /dev/null +++ b/test/web/activity_pub/mrf/vocabulary_policy_test.exs @@ -0,0 +1,106 @@ +# Pleroma: A lightweight social networking server +# Copyright © 2017-2019 Pleroma Authors +# SPDX-License-Identifier: AGPL-3.0-only + +defmodule Pleroma.Web.ActivityPub.MRF.VocabularyPolicyTest do + use Pleroma.DataCase + + alias Pleroma.Web.ActivityPub.MRF.VocabularyPolicy + + describe "accept" do + clear_config([:mrf_vocabulary, :accept]) + + test "it accepts based on parent activity type" do + Pleroma.Config.put([:mrf_vocabulary, :accept], ["Like"]) + + message = %{ + "type" => "Like", + "object" => "whatever" + } + + {:ok, ^message} = VocabularyPolicy.filter(message) + end + + test "it accepts based on child object type" do + Pleroma.Config.put([:mrf_vocabulary, :accept], ["Create", "Note"]) + + message = %{ + "type" => "Create", + "object" => %{ + "type" => "Note", + "content" => "whatever" + } + } + + {:ok, ^message} = VocabularyPolicy.filter(message) + end + + test "it does not accept disallowed child objects" do + Pleroma.Config.put([:mrf_vocabulary, :accept], ["Create", "Note"]) + + message = %{ + "type" => "Create", + "object" => %{ + "type" => "Article", + "content" => "whatever" + } + } + + {:reject, nil} = VocabularyPolicy.filter(message) + end + + test "it does not accept disallowed parent types" do + Pleroma.Config.put([:mrf_vocabulary, :accept], ["Announce", "Note"]) + + message = %{ + "type" => "Create", + "object" => %{ + "type" => "Note", + "content" => "whatever" + } + } + + {:reject, nil} = VocabularyPolicy.filter(message) + end + end + + describe "reject" do + clear_config([:mrf_vocabulary, :reject]) + + test "it rejects based on parent activity type" do + Pleroma.Config.put([:mrf_vocabulary, :reject], ["Like"]) + + message = %{ + "type" => "Like", + "object" => "whatever" + } + + {:reject, nil} = VocabularyPolicy.filter(message) + end + + test "it rejects based on child object type" do + Pleroma.Config.put([:mrf_vocabulary, :reject], ["Note"]) + + message = %{ + "type" => "Create", + "object" => %{ + "type" => "Note", + "content" => "whatever" + } + } + + {:reject, nil} = VocabularyPolicy.filter(message) + end + + test "it passes through objects that aren't disallowed" do + Pleroma.Config.put([:mrf_vocabulary, :reject], ["Like"]) + + message = %{ + "type" => "Announce", + "object" => "whatever" + } + + {:ok, ^message} = VocabularyPolicy.filter(message) + end + end +end diff --git a/test/web/activity_pub/relay_test.exs b/test/web/activity_pub/relay_test.exs index 21a63c493..e10b808f7 100644 --- a/test/web/activity_pub/relay_test.exs +++ b/test/web/activity_pub/relay_test.exs @@ -5,11 +5,71 @@ defmodule Pleroma.Web.ActivityPub.RelayTest do use Pleroma.DataCase + alias Pleroma.Activity + alias Pleroma.Object + alias Pleroma.Web.ActivityPub.ActivityPub alias Pleroma.Web.ActivityPub.Relay + import Pleroma.Factory + test "gets an actor for the relay" do user = Relay.get_actor() + assert user.ap_id == "#{Pleroma.Web.Endpoint.url()}/relay" + end - assert user.ap_id =~ "/relay" + describe "follow/1" do + test "returns errors when user not found" do + assert Relay.follow("test-ap-id") == {:error, "Could not fetch by AP id"} + end + + test "returns activity" do + user = insert(:user) + service_actor = Relay.get_actor() + assert {:ok, %Activity{} = activity} = Relay.follow(user.ap_id) + assert activity.actor == "#{Pleroma.Web.Endpoint.url()}/relay" + assert user.ap_id in activity.recipients + assert activity.data["type"] == "Follow" + assert activity.data["actor"] == service_actor.ap_id + assert activity.data["object"] == user.ap_id + end + end + + describe "unfollow/1" do + test "returns errors when user not found" do + assert Relay.unfollow("test-ap-id") == {:error, "Could not fetch by AP id"} + end + + test "returns activity" do + user = insert(:user) + service_actor = Relay.get_actor() + ActivityPub.follow(service_actor, user) + assert {:ok, %Activity{} = activity} = Relay.unfollow(user.ap_id) + assert activity.actor == "#{Pleroma.Web.Endpoint.url()}/relay" + assert user.ap_id in activity.recipients + assert activity.data["type"] == "Undo" + assert activity.data["actor"] == service_actor.ap_id + assert activity.data["to"] == [user.ap_id] + end + end + + describe "publish/1" do + test "returns error when activity not `Create` type" do + activity = insert(:like_activity) + assert Relay.publish(activity) == {:error, "Not implemented"} + end + + test "returns error when activity not public" do + activity = insert(:direct_note_activity) + assert Relay.publish(activity) == {:error, false} + end + + test "returns announce activity" do + service_actor = Relay.get_actor() + note = insert(:note_activity) + assert {:ok, %Activity{} = activity, %Object{} = obj} = Relay.publish(note) + assert activity.data["type"] == "Announce" + assert activity.data["actor"] == service_actor.ap_id + assert activity.data["object"] == obj.data["id"] + end end end diff --git a/test/web/activity_pub/transmogrifier_test.exs b/test/web/activity_pub/transmogrifier_test.exs index a1f5f6e36..629c76c97 100644 --- a/test/web/activity_pub/transmogrifier_test.exs +++ b/test/web/activity_pub/transmogrifier_test.exs @@ -24,6 +24,8 @@ defmodule Pleroma.Web.ActivityPub.TransmogrifierTest do :ok end + clear_config([:instance, :max_remote_account_fields]) + describe "handle_incoming" do test "it ignores an incoming notice if we already have it" do activity = insert(:note_activity) @@ -450,6 +452,27 @@ test "it ensures that address fields become lists" do assert !is_nil(data["cc"]) end + test "it strips internal likes" do + data = + File.read!("test/fixtures/mastodon-post-activity.json") + |> Poison.decode!() + + likes = %{ + "first" => + "http://mastodon.example.org/objects/dbdbc507-52c8-490d-9b7c-1e1d52e5c132/likes?page=1", + "id" => "http://mastodon.example.org/objects/dbdbc507-52c8-490d-9b7c-1e1d52e5c132/likes", + "totalItems" => 3, + "type" => "OrderedCollection" + } + + object = Map.put(data["object"], "likes", likes) + data = Map.put(data, "object", object) + + {:ok, %Activity{object: object}} = Transmogrifier.handle_incoming(data) + + refute Map.has_key?(object.data, "likes") + end + test "it works for incoming update activities" do data = File.read!("test/fixtures/mastodon-post-activity.json") |> Poison.decode!() @@ -488,6 +511,60 @@ test "it works for incoming update activities" do assert user.bio == "

Some bio

" end + test "it works with custom profile fields" do + {:ok, activity} = + "test/fixtures/mastodon-post-activity.json" + |> File.read!() + |> Poison.decode!() + |> Transmogrifier.handle_incoming() + + user = User.get_cached_by_ap_id(activity.actor) + + assert User.Info.fields(user.info) == [ + %{"name" => "foo", "value" => "bar"}, + %{"name" => "foo1", "value" => "bar1"} + ] + + update_data = File.read!("test/fixtures/mastodon-update.json") |> Poison.decode!() + + object = + update_data["object"] + |> Map.put("actor", user.ap_id) + |> Map.put("id", user.ap_id) + + update_data = + update_data + |> Map.put("actor", user.ap_id) + |> Map.put("object", object) + + {:ok, _update_activity} = Transmogrifier.handle_incoming(update_data) + + user = User.get_cached_by_ap_id(user.ap_id) + + assert User.Info.fields(user.info) == [ + %{"name" => "foo", "value" => "updated"}, + %{"name" => "foo1", "value" => "updated"} + ] + + Pleroma.Config.put([:instance, :max_remote_account_fields], 2) + + update_data = + put_in(update_data, ["object", "attachment"], [ + %{"name" => "foo", "type" => "PropertyValue", "value" => "bar"}, + %{"name" => "foo11", "type" => "PropertyValue", "value" => "bar11"}, + %{"name" => "foo22", "type" => "PropertyValue", "value" => "bar22"} + ]) + + {:ok, _} = Transmogrifier.handle_incoming(update_data) + + user = User.get_cached_by_ap_id(user.ap_id) + + assert User.Info.fields(user.info) == [ + %{"name" => "foo", "value" => "updated"}, + %{"name" => "foo1", "value" => "updated"} + ] + end + test "it works for incoming update activities which lock the account" do data = File.read!("test/fixtures/mastodon-post-activity.json") |> Poison.decode!() @@ -1061,14 +1138,7 @@ test "it strips internal fields of article" do assert is_nil(modified["object"]["announcements"]) assert is_nil(modified["object"]["announcement_count"]) assert is_nil(modified["object"]["context_id"]) - end - - test "it adds like collection to object" do - activity = insert(:note_activity) - {:ok, modified} = Transmogrifier.prepare_outgoing(activity.data) - - assert modified["object"]["likes"]["type"] == "OrderedCollection" - assert modified["object"]["likes"]["totalItems"] == 0 + assert is_nil(modified["object"]["likes"]) end test "the directMessage flag is present" do @@ -1373,32 +1443,4 @@ test "removes recipient's follower collection from cc", %{user: user} do refute recipient.follower_address in fixed_object["to"] end end - - test "update_following_followers_counters/1" do - user1 = - insert(:user, - local: false, - follower_address: "http://localhost:4001/users/masto_closed/followers", - following_address: "http://localhost:4001/users/masto_closed/following" - ) - - user2 = - insert(:user, - local: false, - follower_address: "http://localhost:4001/users/fuser2/followers", - following_address: "http://localhost:4001/users/fuser2/following" - ) - - Transmogrifier.update_following_followers_counters(user1) - Transmogrifier.update_following_followers_counters(user2) - - %{follower_count: followers, following_count: following} = User.get_cached_user_info(user1) - assert followers == 437 - assert following == 152 - - %{follower_count: followers, following_count: following} = User.get_cached_user_info(user2) - - assert followers == 527 - assert following == 267 - end end diff --git a/test/web/activity_pub/views/user_view_test.exs b/test/web/activity_pub/views/user_view_test.exs index 86254117f..fb7fd9e79 100644 --- a/test/web/activity_pub/views/user_view_test.exs +++ b/test/web/activity_pub/views/user_view_test.exs @@ -22,6 +22,21 @@ test "Renders a user, including the public key" do assert String.contains?(result["publicKey"]["publicKeyPem"], "BEGIN PUBLIC KEY") end + test "Renders profile fields" do + fields = [ + %{"name" => "foo", "value" => "bar"} + ] + + {:ok, user} = + insert(:user) + |> User.upgrade_changeset(%{info: %{fields: fields}}) + |> User.update_and_set_cache() + + assert %{ + "attachment" => [%{"name" => "foo", "type" => "PropertyValue", "value" => "bar"}] + } = UserView.render("user.json", %{user: user}) + end + test "Does not add an avatar image if the user hasn't set one" do user = insert(:user) {:ok, user} = User.ensure_keys_present(user) diff --git a/test/web/admin_api/admin_api_controller_test.exs b/test/web/admin_api/admin_api_controller_test.exs index 6dda4ae51..844cd0732 100644 --- a/test/web/admin_api/admin_api_controller_test.exs +++ b/test/web/admin_api/admin_api_controller_test.exs @@ -294,20 +294,17 @@ test "returns 403 when requested by a non-admin", %{conn: conn} do describe "POST /api/pleroma/admin/email_invite, with valid config" do setup do - registrations_open = Pleroma.Config.get([:instance, :registrations_open]) - invites_enabled = Pleroma.Config.get([:instance, :invites_enabled]) - Pleroma.Config.put([:instance, :registrations_open], false) - Pleroma.Config.put([:instance, :invites_enabled], true) - - on_exit(fn -> - Pleroma.Config.put([:instance, :registrations_open], registrations_open) - Pleroma.Config.put([:instance, :invites_enabled], invites_enabled) - :ok - end) - [user: insert(:user, info: %{is_admin: true})] end + clear_config([:instance, :registrations_open]) do + Pleroma.Config.put([:instance, :registrations_open], false) + end + + clear_config([:instance, :invites_enabled]) do + Pleroma.Config.put([:instance, :invites_enabled], true) + end + test "sends invitation and returns 204", %{conn: conn, user: user} do recipient_email = "foo@bar.com" recipient_name = "J. D." @@ -360,18 +357,13 @@ test "it returns 403 if requested by a non-admin", %{conn: conn} do [user: insert(:user, info: %{is_admin: true})] end + clear_config([:instance, :registrations_open]) + clear_config([:instance, :invites_enabled]) + test "it returns 500 if `invites_enabled` is not enabled", %{conn: conn, user: user} do - registrations_open = Pleroma.Config.get([:instance, :registrations_open]) - invites_enabled = Pleroma.Config.get([:instance, :invites_enabled]) Pleroma.Config.put([:instance, :registrations_open], false) Pleroma.Config.put([:instance, :invites_enabled], false) - on_exit(fn -> - Pleroma.Config.put([:instance, :registrations_open], registrations_open) - Pleroma.Config.put([:instance, :invites_enabled], invites_enabled) - :ok - end) - conn = conn |> assign(:user, user) @@ -381,17 +373,9 @@ test "it returns 500 if `invites_enabled` is not enabled", %{conn: conn, user: u end test "it returns 500 if `registrations_open` is enabled", %{conn: conn, user: user} do - registrations_open = Pleroma.Config.get([:instance, :registrations_open]) - invites_enabled = Pleroma.Config.get([:instance, :invites_enabled]) Pleroma.Config.put([:instance, :registrations_open], true) Pleroma.Config.put([:instance, :invites_enabled], true) - on_exit(fn -> - Pleroma.Config.put([:instance, :registrations_open], registrations_open) - Pleroma.Config.put([:instance, :invites_enabled], invites_enabled) - :ok - end) - conn = conn |> assign(:user, user) @@ -1402,17 +1386,13 @@ test "with settings in db", %{conn: conn} do :ok = File.rm(temp_file) end) - dynamic = Pleroma.Config.get([:instance, :dynamic_configuration]) - - Pleroma.Config.put([:instance, :dynamic_configuration], true) - - on_exit(fn -> - Pleroma.Config.put([:instance, :dynamic_configuration], dynamic) - end) - %{conn: assign(conn, :user, admin)} end + clear_config([:instance, :dynamic_configuration]) do + Pleroma.Config.put([:instance, :dynamic_configuration], true) + end + test "create new config setting in db", %{conn: conn} do conn = post(conn, "/api/pleroma/admin/config", %{ @@ -1914,6 +1894,74 @@ test "queues key as atom", %{conn: conn} do ] } end + + test "delete part of settings by atom subkeys", %{conn: conn} do + config = + insert(:config, + key: "keyaa1", + value: :erlang.term_to_binary(subkey1: "val1", subkey2: "val2", subkey3: "val3") + ) + + conn = + post(conn, "/api/pleroma/admin/config", %{ + configs: [ + %{ + group: config.group, + key: config.key, + subkeys: [":subkey1", ":subkey3"], + delete: "true" + } + ] + }) + + assert( + json_response(conn, 200) == %{ + "configs" => [ + %{ + "group" => "pleroma", + "key" => "keyaa1", + "value" => [%{"tuple" => [":subkey2", "val2"]}] + } + ] + } + ) + end + end + + describe "config mix tasks run" do + setup %{conn: conn} do + admin = insert(:user, info: %{is_admin: true}) + + temp_file = "config/test.exported_from_db.secret.exs" + + Mix.shell(Mix.Shell.Quiet) + + on_exit(fn -> + Mix.shell(Mix.Shell.IO) + :ok = File.rm(temp_file) + end) + + %{conn: assign(conn, :user, admin), admin: admin} + end + + clear_config([:instance, :dynamic_configuration]) do + Pleroma.Config.put([:instance, :dynamic_configuration], true) + end + + test "transfer settings to DB and to file", %{conn: conn, admin: admin} do + assert Pleroma.Repo.all(Pleroma.Web.AdminAPI.Config) == [] + conn = get(conn, "/api/pleroma/admin/config/migrate_to_db") + assert json_response(conn, 200) == %{} + assert Pleroma.Repo.all(Pleroma.Web.AdminAPI.Config) > 0 + + conn = + build_conn() + |> assign(:user, admin) + |> get("/api/pleroma/admin/config/migrate_from_db") + + assert json_response(conn, 200) == %{} + assert Pleroma.Repo.all(Pleroma.Web.AdminAPI.Config) == [] + end end describe "GET /api/pleroma/admin/users/:nickname/statuses" do diff --git a/test/web/common_api/common_api_test.exs b/test/web/common_api/common_api_test.exs index 210314a4a..5fda91438 100644 --- a/test/web/common_api/common_api_test.exs +++ b/test/web/common_api/common_api_test.exs @@ -5,18 +5,66 @@ defmodule Pleroma.Web.CommonAPITest do use Pleroma.DataCase alias Pleroma.Activity + alias Pleroma.Conversation.Participation alias Pleroma.Object alias Pleroma.User alias Pleroma.Web.ActivityPub.ActivityPub + alias Pleroma.Web.ActivityPub.Visibility alias Pleroma.Web.CommonAPI import Pleroma.Factory + clear_config([:instance, :safe_dm_mentions]) + clear_config([:instance, :limit]) + clear_config([:instance, :max_pinned_statuses]) + + test "when replying to a conversation / participation, it will set the correct context id even if no explicit reply_to is given" do + user = insert(:user) + {:ok, activity} = CommonAPI.post(user, %{"status" => ".", "visibility" => "direct"}) + + [participation] = Participation.for_user(user) + + {:ok, convo_reply} = + CommonAPI.post(user, %{"status" => ".", "in_reply_to_conversation_id" => participation.id}) + + assert Visibility.is_direct?(convo_reply) + + assert activity.data["context"] == convo_reply.data["context"] + end + + test "when replying to a conversation / participation, it only mentions the recipients explicitly declared in the participation" do + har = insert(:user) + jafnhar = insert(:user) + tridi = insert(:user) + + {:ok, activity} = + CommonAPI.post(har, %{ + "status" => "@#{jafnhar.nickname} hey", + "visibility" => "direct" + }) + + assert har.ap_id in activity.recipients + assert jafnhar.ap_id in activity.recipients + + [participation] = Participation.for_user(har) + + {:ok, activity} = + CommonAPI.post(har, %{ + "status" => "I don't really like @#{tridi.nickname}", + "visibility" => "direct", + "in_reply_to_status_id" => activity.id, + "in_reply_to_conversation_id" => participation.id + }) + + assert har.ap_id in activity.recipients + assert jafnhar.ap_id in activity.recipients + refute tridi.ap_id in activity.recipients + end + test "with the safe_dm_mention option set, it does not mention people beyond the initial tags" do har = insert(:user) jafnhar = insert(:user) tridi = insert(:user) - option = Pleroma.Config.get([:instance, :safe_dm_mentions]) Pleroma.Config.put([:instance, :safe_dm_mentions], true) {:ok, activity} = @@ -27,7 +75,6 @@ test "with the safe_dm_mention option set, it does not mention people beyond the refute tridi.ap_id in activity.recipients assert jafnhar.ap_id in activity.recipients - Pleroma.Config.put([:instance, :safe_dm_mentions], option) end test "it de-duplicates tags" do @@ -150,15 +197,12 @@ test "it returns error when status is empty and no attachments" do end test "it returns error when character limit is exceeded" do - limit = Pleroma.Config.get([:instance, :limit]) Pleroma.Config.put([:instance, :limit], 5) user = insert(:user) assert {:error, "The status is over the character limit"} = CommonAPI.post(user, %{"status" => "foobar"}) - - Pleroma.Config.put([:instance, :limit], limit) end test "it can handle activities that expire" do diff --git a/test/web/common_api/common_api_utils_test.exs b/test/web/common_api/common_api_utils_test.exs index af320f31f..c281dd1f1 100644 --- a/test/web/common_api/common_api_utils_test.exs +++ b/test/web/common_api/common_api_utils_test.exs @@ -239,7 +239,7 @@ test "for public posts, not a reply" do mentioned_user = insert(:user) mentions = [mentioned_user.ap_id] - {to, cc} = Utils.get_to_and_cc(user, mentions, nil, "public") + {to, cc} = Utils.get_to_and_cc(user, mentions, nil, "public", nil) assert length(to) == 2 assert length(cc) == 1 @@ -256,7 +256,7 @@ test "for public posts, a reply" do {:ok, activity} = CommonAPI.post(third_user, %{"status" => "uguu"}) mentions = [mentioned_user.ap_id] - {to, cc} = Utils.get_to_and_cc(user, mentions, activity, "public") + {to, cc} = Utils.get_to_and_cc(user, mentions, activity, "public", nil) assert length(to) == 3 assert length(cc) == 1 @@ -272,7 +272,7 @@ test "for unlisted posts, not a reply" do mentioned_user = insert(:user) mentions = [mentioned_user.ap_id] - {to, cc} = Utils.get_to_and_cc(user, mentions, nil, "unlisted") + {to, cc} = Utils.get_to_and_cc(user, mentions, nil, "unlisted", nil) assert length(to) == 2 assert length(cc) == 1 @@ -289,7 +289,7 @@ test "for unlisted posts, a reply" do {:ok, activity} = CommonAPI.post(third_user, %{"status" => "uguu"}) mentions = [mentioned_user.ap_id] - {to, cc} = Utils.get_to_and_cc(user, mentions, activity, "unlisted") + {to, cc} = Utils.get_to_and_cc(user, mentions, activity, "unlisted", nil) assert length(to) == 3 assert length(cc) == 1 @@ -305,8 +305,7 @@ test "for private posts, not a reply" do mentioned_user = insert(:user) mentions = [mentioned_user.ap_id] - {to, cc} = Utils.get_to_and_cc(user, mentions, nil, "private") - + {to, cc} = Utils.get_to_and_cc(user, mentions, nil, "private", nil) assert length(to) == 2 assert length(cc) == 0 @@ -321,7 +320,7 @@ test "for private posts, a reply" do {:ok, activity} = CommonAPI.post(third_user, %{"status" => "uguu"}) mentions = [mentioned_user.ap_id] - {to, cc} = Utils.get_to_and_cc(user, mentions, activity, "private") + {to, cc} = Utils.get_to_and_cc(user, mentions, activity, "private", nil) assert length(to) == 3 assert length(cc) == 0 @@ -336,7 +335,7 @@ test "for direct posts, not a reply" do mentioned_user = insert(:user) mentions = [mentioned_user.ap_id] - {to, cc} = Utils.get_to_and_cc(user, mentions, nil, "direct") + {to, cc} = Utils.get_to_and_cc(user, mentions, nil, "direct", nil) assert length(to) == 1 assert length(cc) == 0 @@ -351,7 +350,7 @@ test "for direct posts, a reply" do {:ok, activity} = CommonAPI.post(third_user, %{"status" => "uguu"}) mentions = [mentioned_user.ap_id] - {to, cc} = Utils.get_to_and_cc(user, mentions, activity, "direct") + {to, cc} = Utils.get_to_and_cc(user, mentions, activity, "direct", nil) assert length(to) == 2 assert length(cc) == 0 @@ -360,4 +359,242 @@ test "for direct posts, a reply" do assert third_user.ap_id in to end end + + describe "get_by_id_or_ap_id/1" do + test "get activity by id" do + activity = insert(:note_activity) + %Pleroma.Activity{} = note = Utils.get_by_id_or_ap_id(activity.id) + assert note.id == activity.id + end + + test "get activity by ap_id" do + activity = insert(:note_activity) + %Pleroma.Activity{} = note = Utils.get_by_id_or_ap_id(activity.data["object"]) + assert note.id == activity.id + end + + test "get activity by object when type isn't `Create` " do + activity = insert(:like_activity) + %Pleroma.Activity{} = like = Utils.get_by_id_or_ap_id(activity.id) + assert like.data["object"] == activity.data["object"] + end + end + + describe "to_master_date/1" do + test "removes microseconds from date (NaiveDateTime)" do + assert Utils.to_masto_date(~N[2015-01-23 23:50:07.123]) == "2015-01-23T23:50:07.000Z" + end + + test "removes microseconds from date (String)" do + assert Utils.to_masto_date("2015-01-23T23:50:07.123Z") == "2015-01-23T23:50:07.000Z" + end + + test "returns empty string when date invalid" do + assert Utils.to_masto_date("2015-01?23T23:50:07.123Z") == "" + end + end + + describe "conversation_id_to_context/1" do + test "returns id" do + object = insert(:note) + assert Utils.conversation_id_to_context(object.id) == object.data["id"] + end + + test "returns error if object not found" do + assert Utils.conversation_id_to_context("123") == {:error, "No such conversation"} + end + end + + describe "maybe_notify_mentioned_recipients/2" do + test "returns recipients when activity is not `Create`" do + activity = insert(:like_activity) + assert Utils.maybe_notify_mentioned_recipients(["test"], activity) == ["test"] + end + + test "returns recipients from tag" do + user = insert(:user) + + object = + insert(:note, + user: user, + data: %{ + "tag" => [ + %{"type" => "Hashtag"}, + "", + %{"type" => "Mention", "href" => "https://testing.pleroma.lol/users/lain"}, + %{"type" => "Mention", "href" => "https://shitposter.club/user/5381"}, + %{"type" => "Mention", "href" => "https://shitposter.club/user/5381"} + ] + } + ) + + activity = insert(:note_activity, user: user, note: object) + + assert Utils.maybe_notify_mentioned_recipients(["test"], activity) == [ + "test", + "https://testing.pleroma.lol/users/lain", + "https://shitposter.club/user/5381" + ] + end + + test "returns recipients when object is map" do + user = insert(:user) + object = insert(:note, user: user) + + activity = + insert(:note_activity, + user: user, + note: object, + data_attrs: %{ + "object" => %{ + "tag" => [ + %{"type" => "Hashtag"}, + "", + %{"type" => "Mention", "href" => "https://testing.pleroma.lol/users/lain"}, + %{"type" => "Mention", "href" => "https://shitposter.club/user/5381"}, + %{"type" => "Mention", "href" => "https://shitposter.club/user/5381"} + ] + } + } + ) + + Pleroma.Repo.delete(object) + + assert Utils.maybe_notify_mentioned_recipients(["test"], activity) == [ + "test", + "https://testing.pleroma.lol/users/lain", + "https://shitposter.club/user/5381" + ] + end + + test "returns recipients when object not found" do + user = insert(:user) + object = insert(:note, user: user) + + activity = insert(:note_activity, user: user, note: object) + Pleroma.Repo.delete(object) + + assert Utils.maybe_notify_mentioned_recipients(["test-test"], activity) == [ + "test-test" + ] + end + end + + describe "attachments_from_ids_descs/2" do + test "returns [] when attachment ids is empty" do + assert Utils.attachments_from_ids_descs([], "{}") == [] + end + + test "returns list attachments with desc" do + object = insert(:note) + desc = Jason.encode!(%{object.id => "test-desc"}) + + assert Utils.attachments_from_ids_descs(["#{object.id}", "34"], desc) == [ + Map.merge(object.data, %{"name" => "test-desc"}) + ] + end + end + + describe "attachments_from_ids/1" do + test "returns attachments with descs" do + object = insert(:note) + desc = Jason.encode!(%{object.id => "test-desc"}) + + assert Utils.attachments_from_ids(%{ + "media_ids" => ["#{object.id}"], + "descriptions" => desc + }) == [ + Map.merge(object.data, %{"name" => "test-desc"}) + ] + end + + test "returns attachments without descs" do + object = insert(:note) + assert Utils.attachments_from_ids(%{"media_ids" => ["#{object.id}"]}) == [object.data] + end + + test "returns [] when not pass media_ids" do + assert Utils.attachments_from_ids(%{}) == [] + end + end + + describe "maybe_add_list_data/3" do + test "adds list params when found user list" do + user = insert(:user) + {:ok, %Pleroma.List{} = list} = Pleroma.List.create("title", user) + + assert Utils.maybe_add_list_data(%{additional: %{}, object: %{}}, user, {:list, list.id}) == + %{ + additional: %{"bcc" => [list.ap_id], "listMessage" => list.ap_id}, + object: %{"listMessage" => list.ap_id} + } + end + + test "returns original params when list not found" do + user = insert(:user) + {:ok, %Pleroma.List{} = list} = Pleroma.List.create("title", insert(:user)) + + assert Utils.maybe_add_list_data(%{additional: %{}, object: %{}}, user, {:list, list.id}) == + %{additional: %{}, object: %{}} + end + end + + describe "make_note_data/11" do + test "returns note data" do + user = insert(:user) + note = insert(:note) + user2 = insert(:user) + user3 = insert(:user) + + assert Utils.make_note_data( + user.ap_id, + [user2.ap_id], + "2hu", + "

This is :moominmamma: note

", + [], + note.id, + [name: "jimm"], + "test summary", + [user3.ap_id], + false, + %{"custom_tag" => "test"} + ) == %{ + "actor" => user.ap_id, + "attachment" => [], + "cc" => [user3.ap_id], + "content" => "

This is :moominmamma: note

", + "context" => "2hu", + "sensitive" => false, + "summary" => "test summary", + "tag" => ["jimm"], + "to" => [user2.ap_id], + "type" => "Note", + "custom_tag" => "test" + } + end + end + + describe "maybe_add_attachments/3" do + test "returns parsed results when no_links is true" do + assert Utils.maybe_add_attachments( + {"test", [], ["tags"]}, + [], + true + ) == {"test", [], ["tags"]} + end + + test "adds attachments to parsed results" do + attachment = %{"url" => [%{"href" => "SakuraPM.png"}]} + + assert Utils.maybe_add_attachments( + {"test", [], ["tags"]}, + [attachment], + false + ) == { + "test
SakuraPM.png", + [], + ["tags"] + } + end + end end diff --git a/test/web/digest_email_worker_test.exs b/test/web/digest_email_worker_test.exs new file mode 100644 index 000000000..15002330f --- /dev/null +++ b/test/web/digest_email_worker_test.exs @@ -0,0 +1,31 @@ +# Pleroma: A lightweight social networking server +# Copyright © 2017-2019 Pleroma Authors +# SPDX-License-Identifier: AGPL-3.0-only + +defmodule Pleroma.DigestEmailWorkerTest do + use Pleroma.DataCase + import Pleroma.Factory + + alias Pleroma.DigestEmailWorker + alias Pleroma.User + alias Pleroma.Web.CommonAPI + + test "it sends digest emails" do + user = insert(:user) + + date = + Timex.now() + |> Timex.shift(days: -10) + |> Timex.to_naive_datetime() + + user2 = insert(:user, last_digest_emailed_at: date) + User.switch_email_notifications(user2, "digest", true) + CommonAPI.post(user, %{"status" => "hey @#{user2.nickname}!"}) + + DigestEmailWorker.perform() + + assert_received {:email, email} + assert email.to == [{user2.name, user2.email}] + assert email.subject == "Your digest from #{Pleroma.Config.get(:instance)[:name]}" + end +end diff --git a/test/web/fallback_test.exs b/test/web/fallback_test.exs index cc78b3ae1..c13db9526 100644 --- a/test/web/fallback_test.exs +++ b/test/web/fallback_test.exs @@ -30,6 +30,10 @@ test "GET /api*path", %{conn: conn} do |> json_response(404) == %{"error" => "Not implemented"} end + test "GET /pleroma/admin -> /pleroma/admin/", %{conn: conn} do + assert redirected_to(get(conn, "/pleroma/admin")) =~ "/pleroma/admin/" + end + test "GET /*path", %{conn: conn} do assert conn |> get("/foo") diff --git a/test/web/federator_test.exs b/test/web/federator_test.exs index 6e143eee4..09e54533f 100644 --- a/test/web/federator_test.exs +++ b/test/web/federator_test.exs @@ -13,15 +13,17 @@ defmodule Pleroma.Web.FederatorTest do setup_all do Tesla.Mock.mock_global(fn env -> apply(HttpRequestMock, :request, [env]) end) - config_path = [:instance, :federating] - initial_setting = Pleroma.Config.get(config_path) - - Pleroma.Config.put(config_path, true) - on_exit(fn -> Pleroma.Config.put(config_path, initial_setting) end) - :ok end + clear_config_all([:instance, :federating]) do + Pleroma.Config.put([:instance, :federating], true) + end + + clear_config([:instance, :allow_relay]) + clear_config([:instance, :rewrite_policy]) + clear_config([:mrf_keyword]) + describe "Publisher.perform" do test "call `perform` with unknown task" do assert { @@ -67,8 +69,6 @@ test "with relays deactivated, it does not publish to the relay", %{ end refute_received :relay_publish - - Pleroma.Config.put([:instance, :allow_relay], true) end end @@ -229,5 +229,20 @@ test "rejects incoming AP docs with incorrect origin" do :error = Federator.incoming_ap_doc(params) end + + test "it does not crash if MRF rejects the post" do + Pleroma.Config.put([:mrf_keyword, :reject], ["lain"]) + + Pleroma.Config.put( + [:instance, :rewrite_policy], + Pleroma.Web.ActivityPub.MRF.KeywordPolicy + ) + + params = + File.read!("test/fixtures/mastodon-post-activity.json") + |> Poison.decode!() + + assert Federator.incoming_ap_doc(params) == :error + end end end diff --git a/test/web/instances/instance_test.exs b/test/web/instances/instance_test.exs index d28730994..3fd011fd3 100644 --- a/test/web/instances/instance_test.exs +++ b/test/web/instances/instance_test.exs @@ -10,14 +10,8 @@ defmodule Pleroma.Instances.InstanceTest do import Pleroma.Factory - setup_all do - config_path = [:instance, :federation_reachability_timeout_days] - initial_setting = Pleroma.Config.get(config_path) - - Pleroma.Config.put(config_path, 1) - on_exit(fn -> Pleroma.Config.put(config_path, initial_setting) end) - - :ok + clear_config_all([:instance, :federation_reachability_timeout_days]) do + Pleroma.Config.put([:instance, :federation_reachability_timeout_days], 1) end describe "set_reachable/1" do diff --git a/test/web/instances/instances_test.exs b/test/web/instances/instances_test.exs index f0d84edea..dea8e2aea 100644 --- a/test/web/instances/instances_test.exs +++ b/test/web/instances/instances_test.exs @@ -7,14 +7,8 @@ defmodule Pleroma.InstancesTest do use Pleroma.DataCase - setup_all do - config_path = [:instance, :federation_reachability_timeout_days] - initial_setting = Pleroma.Config.get(config_path) - - Pleroma.Config.put(config_path, 1) - on_exit(fn -> Pleroma.Config.put(config_path, initial_setting) end) - - :ok + clear_config_all([:instance, :federation_reachability_timeout_days]) do + Pleroma.Config.put([:instance, :federation_reachability_timeout_days], 1) end describe "reachable?/1" do diff --git a/test/web/mastodon_api/account_view_test.exs b/test/web/mastodon_api/account_view_test.exs index fa44d35cc..1d8b28339 100644 --- a/test/web/mastodon_api/account_view_test.exs +++ b/test/web/mastodon_api/account_view_test.exs @@ -67,7 +67,8 @@ test "Represent a user account" do source: %{ note: "valid html", sensitive: false, - pleroma: %{} + pleroma: %{}, + fields: [] }, pleroma: %{ background_image: "https://example.com/images/asuka_hospital.png", @@ -134,7 +135,8 @@ test "Represent a Service(bot) account" do source: %{ note: user.bio, sensitive: false, - pleroma: %{} + pleroma: %{}, + fields: [] }, pleroma: %{ background_image: nil, @@ -231,6 +233,16 @@ test "represent a relationship for the blocking and blocked user" do AccountView.render("relationship.json", %{user: user, target: other_user}) end + test "represent a relationship for the user blocking a domain" do + user = insert(:user) + other_user = insert(:user, ap_id: "https://bad.site/users/other_user") + + {:ok, user} = User.block_domain(user, "bad.site") + + assert %{domain_blocking: true, blocking: false} = + AccountView.render("relationship.json", %{user: user, target: other_user}) + end + test "represent a relationship for the user with a pending follow request" do user = insert(:user) other_user = insert(:user, %{info: %User.Info{locked: true}}) @@ -294,7 +306,8 @@ test "represent an embedded relationship" do source: %{ note: user.bio, sensitive: false, - pleroma: %{} + pleroma: %{}, + fields: [] }, pleroma: %{ background_image: nil, @@ -346,4 +359,31 @@ test "sanitizes display names" do result = AccountView.render("account.json", %{user: user}) refute result.display_name == " username " end + + describe "hiding follows/following" do + test "shows when follows/following are hidden and sets follower/following count to 0" do + user = insert(:user, info: %{hide_followers: true, hide_follows: true}) + other_user = insert(:user) + {:ok, user, other_user, _activity} = CommonAPI.follow(user, other_user) + {:ok, _other_user, user, _activity} = CommonAPI.follow(other_user, user) + + assert %{ + followers_count: 0, + following_count: 0, + pleroma: %{hide_follows: true, hide_followers: true} + } = AccountView.render("account.json", %{user: user}) + end + + test "shows actual follower/following count to the account owner" do + user = insert(:user, info: %{hide_followers: true, hide_follows: true}) + other_user = insert(:user) + {:ok, user, other_user, _activity} = CommonAPI.follow(user, other_user) + {:ok, _other_user, user, _activity} = CommonAPI.follow(other_user, user) + + assert %{ + followers_count: 1, + following_count: 1 + } = AccountView.render("account.json", %{user: user, for: user}) + end + end end diff --git a/test/web/mastodon_api/conversation_view_test.exs b/test/web/mastodon_api/conversation_view_test.exs new file mode 100644 index 000000000..a2a880705 --- /dev/null +++ b/test/web/mastodon_api/conversation_view_test.exs @@ -0,0 +1,34 @@ +# Pleroma: A lightweight social networking server +# Copyright © 2017-2019 Pleroma Authors +# SPDX-License-Identifier: AGPL-3.0-only + +defmodule Pleroma.Web.MastodonAPI.ConversationViewTest do + use Pleroma.DataCase + + alias Pleroma.Conversation.Participation + alias Pleroma.Web.CommonAPI + alias Pleroma.Web.MastodonAPI.ConversationView + + import Pleroma.Factory + + test "represents a Mastodon Conversation entity" do + user = insert(:user) + other_user = insert(:user) + + {:ok, activity} = + CommonAPI.post(user, %{"status" => "hey @#{other_user.nickname}", "visibility" => "direct"}) + + [participation] = Participation.for_user_with_last_activity_id(user) + + assert participation + + conversation = + ConversationView.render("participation.json", %{participation: participation, for: user}) + + assert conversation.id == participation.id |> to_string() + assert conversation.last_status.id == activity.id + + assert [account] = conversation.accounts + assert account.id == other_user.id + end +end diff --git a/test/web/mastodon_api/mastodon_api_controller/update_credentials_test.exs b/test/web/mastodon_api/mastodon_api_controller/update_credentials_test.exs index 71d0c8af8..87ee82050 100644 --- a/test/web/mastodon_api/mastodon_api_controller/update_credentials_test.exs +++ b/test/web/mastodon_api/mastodon_api_controller/update_credentials_test.exs @@ -9,6 +9,7 @@ defmodule Pleroma.Web.MastodonAPI.MastodonAPIController.UpdateCredentialsTest do use Pleroma.Web.ConnCase import Pleroma.Factory + clear_config([:instance, :max_account_fields]) describe "updating credentials" do test "sets user settings in a generic way", %{conn: conn} do @@ -300,5 +301,69 @@ test "updates profile emojos", %{conn: conn} do assert user["display_name"] == name assert [%{"shortcode" => "blank"}, %{"shortcode" => "firefox"}] = user["emojis"] end + + test "update fields", %{conn: conn} do + user = insert(:user) + + fields = [ + %{"name" => "foo", "value" => ""}, + %{"name" => "link", "value" => "cofe.io"} + ] + + account = + conn + |> assign(:user, user) + |> patch("/api/v1/accounts/update_credentials", %{"fields" => fields}) + |> json_response(200) + + assert account["fields"] == [ + %{"name" => "foo", "value" => "bar"}, + %{"name" => "link", "value" => "cofe.io"} + ] + + assert account["source"]["fields"] == [ + %{ + "name" => "foo", + "value" => "" + }, + %{"name" => "link", "value" => "cofe.io"} + ] + + name_limit = Pleroma.Config.get([:instance, :account_field_name_length]) + value_limit = Pleroma.Config.get([:instance, :account_field_value_length]) + + long_value = Enum.map(0..value_limit, fn _ -> "x" end) |> Enum.join() + + fields = [%{"name" => "foo", "value" => long_value}] + + assert %{"error" => "Invalid request"} == + conn + |> assign(:user, user) + |> patch("/api/v1/accounts/update_credentials", %{"fields" => fields}) + |> json_response(403) + + long_name = Enum.map(0..name_limit, fn _ -> "x" end) |> Enum.join() + + fields = [%{"name" => long_name, "value" => "bar"}] + + assert %{"error" => "Invalid request"} == + conn + |> assign(:user, user) + |> patch("/api/v1/accounts/update_credentials", %{"fields" => fields}) + |> json_response(403) + + Pleroma.Config.put([:instance, :max_account_fields], 1) + + fields = [ + %{"name" => "foo", "value" => "bar"}, + %{"name" => "link", "value" => "cofe.io"} + ] + + assert %{"error" => "Invalid request"} == + conn + |> assign(:user, user) + |> patch("/api/v1/accounts/update_credentials", %{"fields" => fields}) + |> json_response(403) + end end end diff --git a/test/web/mastodon_api/mastodon_api_controller_test.exs b/test/web/mastodon_api/mastodon_api_controller_test.exs index fbe0ab375..c05c39db6 100644 --- a/test/web/mastodon_api/mastodon_api_controller_test.exs +++ b/test/web/mastodon_api/mastodon_api_controller_test.exs @@ -8,6 +8,7 @@ defmodule Pleroma.Web.MastodonAPI.MastodonAPIControllerTest do alias Ecto.Changeset alias Pleroma.Activity alias Pleroma.ActivityExpiration + alias Pleroma.Config alias Pleroma.Notification alias Pleroma.Object alias Pleroma.Repo @@ -33,6 +34,9 @@ defmodule Pleroma.Web.MastodonAPI.MastodonAPIControllerTest do :ok end + clear_config([:instance, :public]) + clear_config([:rich_media, :enabled]) + test "the home timeline", %{conn: conn} do user = insert(:user) following = insert(:user) @@ -86,12 +90,7 @@ test "the public timeline", %{conn: conn} do end test "the public timeline when public is set to false", %{conn: conn} do - public = Pleroma.Config.get([:instance, :public]) - Pleroma.Config.put([:instance, :public], false) - - on_exit(fn -> - Pleroma.Config.put([:instance, :public], public) - end) + Config.put([:instance, :public], false) assert conn |> get("/api/v1/timelines/public", %{"local" => "False"}) @@ -274,7 +273,7 @@ test "posting a fake status", %{conn: conn} do end test "posting a status with OGP link preview", %{conn: conn} do - Pleroma.Config.put([:rich_media, :enabled], true) + Config.put([:rich_media, :enabled], true) conn = conn @@ -284,7 +283,6 @@ test "posting a status with OGP link preview", %{conn: conn} do assert %{"id" => id, "card" => %{"title" => "The Rock"}} = json_response(conn, 200) assert Activity.get_by_id(id) - Pleroma.Config.put([:rich_media, :enabled], false) end test "posting a direct status", %{conn: conn} do @@ -328,7 +326,7 @@ test "posting a poll", %{conn: conn} do test "option limit is enforced", %{conn: conn} do user = insert(:user) - limit = Pleroma.Config.get([:instance, :poll_limits, :max_options]) + limit = Config.get([:instance, :poll_limits, :max_options]) conn = conn @@ -344,7 +342,7 @@ test "option limit is enforced", %{conn: conn} do test "option character limit is enforced", %{conn: conn} do user = insert(:user) - limit = Pleroma.Config.get([:instance, :poll_limits, :max_option_chars]) + limit = Config.get([:instance, :poll_limits, :max_option_chars]) conn = conn @@ -363,7 +361,7 @@ test "option character limit is enforced", %{conn: conn} do test "minimal date limit is enforced", %{conn: conn} do user = insert(:user) - limit = Pleroma.Config.get([:instance, :poll_limits, :min_expiration]) + limit = Config.get([:instance, :poll_limits, :min_expiration]) conn = conn @@ -382,7 +380,7 @@ test "minimal date limit is enforced", %{conn: conn} do test "maximum date limit is enforced", %{conn: conn} do user = insert(:user) - limit = Pleroma.Config.get([:instance, :poll_limits, :max_expiration]) + limit = Config.get([:instance, :poll_limits, :max_expiration]) conn = conn @@ -1657,14 +1655,6 @@ test "returns the relationships for the current user", %{conn: conn} do describe "media upload" do setup do - upload_config = Pleroma.Config.get([Pleroma.Upload]) - proxy_config = Pleroma.Config.get([:media_proxy]) - - on_exit(fn -> - Pleroma.Config.put([Pleroma.Upload], upload_config) - Pleroma.Config.put([:media_proxy], proxy_config) - end) - user = insert(:user) conn = @@ -1680,6 +1670,9 @@ test "returns the relationships for the current user", %{conn: conn} do [conn: conn, image: image] end + clear_config([:media_proxy]) + clear_config([Pleroma.Upload]) + test "returns uploaded image", %{conn: conn, image: image} do desc = "Description of the image" @@ -1695,40 +1688,6 @@ test "returns uploaded image", %{conn: conn, image: image} do object = Repo.get(Object, media["id"]) assert object.data["actor"] == User.ap_id(conn.assigns[:user]) end - - test "returns proxied url when media proxy is enabled", %{conn: conn, image: image} do - Pleroma.Config.put([Pleroma.Upload, :base_url], "https://media.pleroma.social") - - proxy_url = "https://cache.pleroma.social" - Pleroma.Config.put([:media_proxy, :enabled], true) - Pleroma.Config.put([:media_proxy, :base_url], proxy_url) - - media = - conn - |> post("/api/v1/media", %{"file" => image}) - |> json_response(:ok) - - assert String.starts_with?(media["url"], proxy_url) - end - - test "returns media url when proxy is enabled but media url is whitelisted", %{ - conn: conn, - image: image - } do - media_url = "https://media.pleroma.social" - Pleroma.Config.put([Pleroma.Upload, :base_url], media_url) - - Pleroma.Config.put([:media_proxy, :enabled], true) - Pleroma.Config.put([:media_proxy, :base_url], "https://cache.pleroma.social") - Pleroma.Config.put([:media_proxy, :whitelist], ["media.pleroma.social"]) - - media = - conn - |> post("/api/v1/media", %{"file" => image}) - |> json_response(:ok) - - assert String.starts_with?(media["url"], media_url) - end end describe "locked accounts" do @@ -2639,7 +2598,7 @@ test "get instance information", %{conn: conn} do conn = get(conn, "/api/v1/instance") assert result = json_response(conn, 200) - email = Pleroma.Config.get([:instance, :email]) + email = Config.get([:instance, :email]) # Note: not checking for "max_toot_chars" since it's optional assert %{ "uri" => _, @@ -2681,7 +2640,7 @@ test "get instance stats", %{conn: conn} do |> Changeset.put_embed(:info, info_change) |> User.update_and_set_cache() - Pleroma.Stats.update_stats() + Pleroma.Stats.force_update() conn = get(conn, "/api/v1/instance") @@ -2699,7 +2658,7 @@ test "get peers", %{conn: conn} do insert(:user, %{local: false, nickname: "u@peer1.com"}) insert(:user, %{local: false, nickname: "u@peer2.com"}) - Pleroma.Stats.update_stats() + Pleroma.Stats.force_update() conn = get(conn, "/api/v1/instance/peers") @@ -2724,14 +2683,16 @@ test "put settings", %{conn: conn} do describe "pinned statuses" do setup do - Pleroma.Config.put([:instance, :max_pinned_statuses], 1) - user = insert(:user) {:ok, activity} = CommonAPI.post(user, %{"status" => "HI!!!"}) [user: user, activity: activity] end + clear_config([:instance, :max_pinned_statuses]) do + Config.put([:instance, :max_pinned_statuses], 1) + end + test "returns pinned statuses", %{conn: conn, user: user, activity: activity} do {:ok, _} = CommonAPI.pin(activity.id, user) @@ -2824,11 +2785,7 @@ test "max pinned statuses", %{conn: conn, user: user, activity: activity_one} do describe "cards" do setup do - Pleroma.Config.put([:rich_media, :enabled], true) - - on_exit(fn -> - Pleroma.Config.put([:rich_media, :enabled], false) - end) + Config.put([:rich_media, :enabled], true) user = insert(:user) %{user: user} @@ -2959,8 +2916,10 @@ test "bookmarks" do describe "conversation muting" do setup do + post_user = insert(:user) user = insert(:user) - {:ok, activity} = CommonAPI.post(user, %{"status" => "HIE"}) + + {:ok, activity} = CommonAPI.post(post_user, %{"status" => "HIE"}) [user: user, activity: activity] end @@ -3053,7 +3012,7 @@ test "comment must be up to the size specified in the config", %{ reporter: reporter, target_user: target_user } do - max_size = Pleroma.Config.get([:instance, :max_report_comment_size], 1000) + max_size = Config.get([:instance, :max_report_comment_size], 1000) comment = String.pad_trailing("a", max_size + 1, "a") error = %{"error" => "Comment must be up to #{max_size} characters"} @@ -3178,6 +3137,18 @@ test "redirects not logged-in users to the login page", %{conn: conn, path: path assert redirected_to(conn) == "/web/login" end + test "redirects not logged-in users to the login page on private instances", %{ + conn: conn, + path: path + } do + Config.put([:instance, :public], false) + + conn = get(conn, path) + + assert conn.status == 302 + assert redirected_to(conn) == "/web/login" + end + test "does not redirect logged in users to the login page", %{conn: conn, path: path} do token = insert(:oauth_token) @@ -3917,8 +3888,8 @@ test "it sends an email to user", %{user: user} do token_record = Repo.get_by(Pleroma.PasswordResetToken, user_id: user.id) email = Pleroma.Emails.UserEmail.password_reset_email(user, token_record.token) - notify_email = Pleroma.Config.get([:instance, :notify_email]) - instance_name = Pleroma.Config.get([:instance, :name]) + notify_email = Config.get([:instance, :notify_email]) + instance_name = Config.get([:instance, :name]) assert_email_sent( from: {instance_name, notify_email}, @@ -3947,4 +3918,121 @@ test "it returns 400 when user is not local", %{conn: conn, user: user} do assert conn.resp_body == "" end end + + describe "POST /api/v1/pleroma/accounts/confirmation_resend" do + setup do + user = insert(:user) + info_change = User.Info.confirmation_changeset(user.info, need_confirmation: true) + + {:ok, user} = + user + |> Changeset.change() + |> Changeset.put_embed(:info, info_change) + |> Repo.update() + + assert user.info.confirmation_pending + + [user: user] + end + + clear_config([:instance, :account_activation_required]) do + Config.put([:instance, :account_activation_required], true) + end + + test "resend account confirmation email", %{conn: conn, user: user} do + conn + |> assign(:user, user) + |> post("/api/v1/pleroma/accounts/confirmation_resend?email=#{user.email}") + |> json_response(:no_content) + + email = Pleroma.Emails.UserEmail.account_confirmation_email(user) + notify_email = Config.get([:instance, :notify_email]) + instance_name = Config.get([:instance, :name]) + + assert_email_sent( + from: {instance_name, notify_email}, + to: {user.name, user.email}, + html_body: email.html_body + ) + end + end + + describe "GET /api/v1/suggestions" do + setup do + user = insert(:user) + other_user = insert(:user) + host = Config.get([Pleroma.Web.Endpoint, :url, :host]) + url500 = "http://test500?#{host}&#{user.nickname}" + url200 = "http://test200?#{host}&#{user.nickname}" + + mock(fn + %{method: :get, url: ^url500} -> + %Tesla.Env{status: 500, body: "bad request"} + + %{method: :get, url: ^url200} -> + %Tesla.Env{ + status: 200, + body: + ~s([{"acct":"yj455","avatar":"https://social.heldscal.la/avatar/201.jpeg","avatar_static":"https://social.heldscal.la/avatar/s/201.jpeg"}, {"acct":"#{ + other_user.ap_id + }","avatar":"https://social.heldscal.la/avatar/202.jpeg","avatar_static":"https://social.heldscal.la/avatar/s/202.jpeg"}]) + } + end) + + [user: user, other_user: other_user] + end + + clear_config(:suggestions) + + test "returns empty result when suggestions disabled", %{conn: conn, user: user} do + Config.put([:suggestions, :enabled], false) + + res = + conn + |> assign(:user, user) + |> get("/api/v1/suggestions") + |> json_response(200) + + assert res == [] + end + + test "returns error", %{conn: conn, user: user} do + Config.put([:suggestions, :enabled], true) + Config.put([:suggestions, :third_party_engine], "http://test500?{{host}}&{{user}}") + + res = + conn + |> assign(:user, user) + |> get("/api/v1/suggestions") + |> json_response(500) + + assert res == "Something went wrong" + end + + test "returns suggestions", %{conn: conn, user: user, other_user: other_user} do + Config.put([:suggestions, :enabled], true) + Config.put([:suggestions, :third_party_engine], "http://test200?{{host}}&{{user}}") + + res = + conn + |> assign(:user, user) + |> get("/api/v1/suggestions") + |> json_response(200) + + assert res == [ + %{ + "acct" => "yj455", + "avatar" => "https://social.heldscal.la/avatar/201.jpeg", + "avatar_static" => "https://social.heldscal.la/avatar/s/201.jpeg", + "id" => 0 + }, + %{ + "acct" => other_user.ap_id, + "avatar" => "https://social.heldscal.la/avatar/202.jpeg", + "avatar_static" => "https://social.heldscal.la/avatar/s/202.jpeg", + "id" => other_user.id + } + ] + end + end end diff --git a/test/web/mastodon_api/mastodon_api_test.exs b/test/web/mastodon_api/mastodon_api_test.exs new file mode 100644 index 000000000..b4c0427c9 --- /dev/null +++ b/test/web/mastodon_api/mastodon_api_test.exs @@ -0,0 +1,103 @@ +# Pleroma: A lightweight social networking server +# Copyright © 2017-2019 Pleroma Authors +# SPDX-License-Identifier: AGPL-3.0-only + +defmodule Pleroma.Web.MastodonAPI.MastodonAPITest do + use Pleroma.Web.ConnCase + + alias Pleroma.Notification + alias Pleroma.ScheduledActivity + alias Pleroma.User + alias Pleroma.Web.MastodonAPI.MastodonAPI + alias Pleroma.Web.TwitterAPI.TwitterAPI + + import Pleroma.Factory + + describe "follow/3" do + test "returns error when user deactivated" do + follower = insert(:user) + user = insert(:user, local: true, info: %{deactivated: true}) + {:error, error} = MastodonAPI.follow(follower, user) + assert error == "Could not follow user: You are deactivated." + end + + test "following for user" do + follower = insert(:user) + user = insert(:user) + {:ok, follower} = MastodonAPI.follow(follower, user) + assert User.following?(follower, user) + end + + test "returns ok if user already followed" do + follower = insert(:user) + user = insert(:user) + {:ok, follower} = User.follow(follower, user) + {:ok, follower} = MastodonAPI.follow(follower, refresh_record(user)) + assert User.following?(follower, user) + end + end + + describe "get_followers/2" do + test "returns user followers" do + follower1_user = insert(:user) + follower2_user = insert(:user) + user = insert(:user) + {:ok, _follower1_user} = User.follow(follower1_user, user) + {:ok, follower2_user} = User.follow(follower2_user, user) + + assert MastodonAPI.get_followers(user, %{"limit" => 1}) == [follower2_user] + end + end + + describe "get_friends/2" do + test "returns user friends" do + user = insert(:user) + followed_one = insert(:user) + followed_two = insert(:user) + followed_three = insert(:user) + + {:ok, user} = User.follow(user, followed_one) + {:ok, user} = User.follow(user, followed_two) + {:ok, user} = User.follow(user, followed_three) + res = MastodonAPI.get_friends(user) + + assert length(res) == 3 + assert Enum.member?(res, refresh_record(followed_three)) + assert Enum.member?(res, refresh_record(followed_two)) + assert Enum.member?(res, refresh_record(followed_one)) + end + end + + describe "get_notifications/2" do + test "returns notifications for user" do + user = insert(:user) + subscriber = insert(:user) + + User.subscribe(subscriber, user) + + {:ok, status} = TwitterAPI.create_status(user, %{"status" => "Akariiiin"}) + {:ok, status1} = TwitterAPI.create_status(user, %{"status" => "Magi"}) + {:ok, [notification]} = Notification.create_notifications(status) + {:ok, [notification1]} = Notification.create_notifications(status1) + res = MastodonAPI.get_notifications(subscriber) + + assert Enum.member?(Enum.map(res, & &1.id), notification.id) + assert Enum.member?(Enum.map(res, & &1.id), notification1.id) + end + end + + describe "get_scheduled_activities/2" do + test "returns user scheduled activities" do + user = insert(:user) + + today = + NaiveDateTime.utc_now() + |> NaiveDateTime.add(:timer.minutes(6), :millisecond) + |> NaiveDateTime.to_iso8601() + + attrs = %{params: %{}, scheduled_at: today} + {:ok, schedule} = ScheduledActivity.create(user, attrs) + assert MastodonAPI.get_scheduled_activities(user) == [schedule] + end + end +end diff --git a/test/web/mastodon_api/search_controller_test.exs b/test/web/mastodon_api/search_controller_test.exs index 043b96c14..49c79ff0a 100644 --- a/test/web/mastodon_api/search_controller_test.exs +++ b/test/web/mastodon_api/search_controller_test.exs @@ -95,6 +95,18 @@ test "account search", %{conn: conn} do assert user_three.nickname in result_ids end + + test "returns account if query contains a space", %{conn: conn} do + user = insert(:user, %{nickname: "shp@shitposter.club"}) + + results = + conn + |> assign(:user, user) + |> get("/api/v1/accounts/search", %{"q" => "shp@shitposter.club xxx "}) + |> json_response(200) + + assert length(results) == 1 + end end describe ".search" do diff --git a/test/web/mastodon_api/status_view_test.exs b/test/web/mastodon_api/status_view_test.exs index 073c69659..1b6beb6d2 100644 --- a/test/web/mastodon_api/status_view_test.exs +++ b/test/web/mastodon_api/status_view_test.exs @@ -23,6 +23,21 @@ defmodule Pleroma.Web.MastodonAPI.StatusViewTest do :ok end + test "returns the direct conversation id when given the `with_conversation_id` option" do + user = insert(:user) + + {:ok, activity} = CommonAPI.post(user, %{"status" => "Hey @shp!", "visibility" => "direct"}) + + status = + StatusView.render("status.json", + activity: activity, + with_direct_conversation_id: true, + for: user + ) + + assert status[:pleroma][:direct_conversation_id] + end + test "returns a temporary ap_id based user for activities missing db users" do user = insert(:user) @@ -134,7 +149,8 @@ test "a note activity" do in_reply_to_account_acct: nil, content: %{"text/plain" => HtmlSanitizeEx.strip_tags(object_data["content"])}, spoiler_text: %{"text/plain" => HtmlSanitizeEx.strip_tags(object_data["summary"])}, - expires_at: nil + expires_at: nil, + direct_conversation_id: nil } } @@ -301,6 +317,16 @@ test "attachments" do assert %{id: "2"} = StatusView.render("attachment.json", %{attachment: object}) end + test "put the url advertised in the Activity in to the url attribute" do + id = "https://wedistribute.org/wp-json/pterotype/v1/object/85810" + [activity] = Activity.search(nil, id) + + status = StatusView.render("status.json", %{activity: activity}) + + assert status.uri == id + assert status.url == "https://wedistribute.org/2019/07/mastodon-drops-ostatus/" + end + test "a reblog" do user = insert(:user) activity = insert(:note_activity) diff --git a/test/web/media_proxy/media_proxy_test.exs b/test/web/media_proxy/media_proxy_test.exs index edbbf9b66..79699cac5 100644 --- a/test/web/media_proxy/media_proxy_test.exs +++ b/test/web/media_proxy/media_proxy_test.exs @@ -4,14 +4,11 @@ defmodule Pleroma.Web.MediaProxyTest do use ExUnit.Case + use Pleroma.Tests.Helpers import Pleroma.Web.MediaProxy alias Pleroma.Web.MediaProxy.MediaProxyController - setup do - enabled = Pleroma.Config.get([:media_proxy, :enabled]) - on_exit(fn -> Pleroma.Config.put([:media_proxy, :enabled], enabled) end) - :ok - end + clear_config([:media_proxy, :enabled]) describe "when enabled" do setup do @@ -171,21 +168,6 @@ test "preserve unicode characters" do encoded = url(url) assert decode_result(encoded) == url end - - test "does not change whitelisted urls" do - upload_config = Pleroma.Config.get([Pleroma.Upload]) - media_url = "https://media.pleroma.social" - Pleroma.Config.put([Pleroma.Upload, :base_url], media_url) - Pleroma.Config.put([:media_proxy, :whitelist], ["media.pleroma.social"]) - Pleroma.Config.put([:media_proxy, :base_url], "https://cache.pleroma.social") - - url = "#{media_url}/static/logo.png" - encoded = url(url) - - assert String.starts_with?(encoded, media_url) - - Pleroma.Config.put([Pleroma.Upload], upload_config) - end end describe "when disabled" do @@ -215,12 +197,43 @@ defp decode_result(encoded) do decoded end - test "mediaproxy whitelist" do - Pleroma.Config.put([:media_proxy, :enabled], true) - Pleroma.Config.put([:media_proxy, :whitelist], ["google.com", "feld.me"]) - url = "https://feld.me/foo.png" + describe "whitelist" do + setup do + Pleroma.Config.put([:media_proxy, :enabled], true) + :ok + end - unencoded = url(url) - assert unencoded == url + test "mediaproxy whitelist" do + Pleroma.Config.put([:media_proxy, :whitelist], ["google.com", "feld.me"]) + url = "https://feld.me/foo.png" + + unencoded = url(url) + assert unencoded == url + end + + test "does not change whitelisted urls" do + Pleroma.Config.put([:media_proxy, :whitelist], ["mycdn.akamai.com"]) + Pleroma.Config.put([:media_proxy, :base_url], "https://cache.pleroma.social") + + media_url = "https://mycdn.akamai.com" + + url = "#{media_url}/static/logo.png" + encoded = url(url) + + assert String.starts_with?(encoded, media_url) + end + + test "ensure Pleroma.Upload base_url is always whitelisted" do + upload_config = Pleroma.Config.get([Pleroma.Upload]) + media_url = "https://media.pleroma.social" + Pleroma.Config.put([Pleroma.Upload, :base_url], media_url) + + url = "#{media_url}/static/logo.png" + encoded = url(url) + + assert String.starts_with?(encoded, media_url) + + Pleroma.Config.put([Pleroma.Upload], upload_config) + end end end diff --git a/test/web/node_info_test.exs b/test/web/node_info_test.exs index d7f848bfa..f6147c286 100644 --- a/test/web/node_info_test.exs +++ b/test/web/node_info_test.exs @@ -85,6 +85,9 @@ test "it returns the safe_dm_mentions feature if enabled", %{conn: conn} do end test "it shows MRF transparency data if enabled", %{conn: conn} do + config = Pleroma.Config.get([:instance, :rewrite_policy]) + Pleroma.Config.put([:instance, :rewrite_policy], [Pleroma.Web.ActivityPub.MRF.SimplePolicy]) + option = Pleroma.Config.get([:instance, :mrf_transparency]) Pleroma.Config.put([:instance, :mrf_transparency], true) @@ -98,11 +101,15 @@ test "it shows MRF transparency data if enabled", %{conn: conn} do assert response["metadata"]["federation"]["mrf_simple"] == simple_config + Pleroma.Config.put([:instance, :rewrite_policy], config) Pleroma.Config.put([:instance, :mrf_transparency], option) Pleroma.Config.put(:mrf_simple, %{}) end test "it performs exclusions from MRF transparency data if configured", %{conn: conn} do + config = Pleroma.Config.get([:instance, :rewrite_policy]) + Pleroma.Config.put([:instance, :rewrite_policy], [Pleroma.Web.ActivityPub.MRF.SimplePolicy]) + option = Pleroma.Config.get([:instance, :mrf_transparency]) Pleroma.Config.put([:instance, :mrf_transparency], true) @@ -122,6 +129,7 @@ test "it performs exclusions from MRF transparency data if configured", %{conn: assert response["metadata"]["federation"]["mrf_simple"] == expected_config assert response["metadata"]["federation"]["exclusions"] == true + Pleroma.Config.put([:instance, :rewrite_policy], config) Pleroma.Config.put([:instance, :mrf_transparency], option) Pleroma.Config.put([:instance, :mrf_transparency_exclusions], exclusions) Pleroma.Config.put(:mrf_simple, %{}) diff --git a/test/web/oauth/ldap_authorization_test.exs b/test/web/oauth/ldap_authorization_test.exs index 0eb191c76..1cbe133b7 100644 --- a/test/web/oauth/ldap_authorization_test.exs +++ b/test/web/oauth/ldap_authorization_test.exs @@ -12,21 +12,12 @@ defmodule Pleroma.Web.OAuth.LDAPAuthorizationTest do @skip if !Code.ensure_loaded?(:eldap), do: :skip - setup_all do - ldap_authenticator = - Pleroma.Config.get(Pleroma.Web.Auth.Authenticator, Pleroma.Web.Auth.PleromaAuthenticator) - - ldap_enabled = Pleroma.Config.get([:ldap, :enabled]) - - on_exit(fn -> - Pleroma.Config.put(Pleroma.Web.Auth.Authenticator, ldap_authenticator) - Pleroma.Config.put([:ldap, :enabled], ldap_enabled) - end) - - Pleroma.Config.put(Pleroma.Web.Auth.Authenticator, Pleroma.Web.Auth.LDAPAuthenticator) + clear_config_all([:ldap, :enabled]) do Pleroma.Config.put([:ldap, :enabled], true) + end - :ok + clear_config_all(Pleroma.Web.Auth.Authenticator) do + Pleroma.Config.put(Pleroma.Web.Auth.Authenticator, Pleroma.Web.Auth.LDAPAuthenticator) end @tag @skip diff --git a/test/web/oauth/oauth_controller_test.exs b/test/web/oauth/oauth_controller_test.exs index 92e156347..b492c7794 100644 --- a/test/web/oauth/oauth_controller_test.exs +++ b/test/web/oauth/oauth_controller_test.exs @@ -11,23 +11,15 @@ defmodule Pleroma.Web.OAuth.OAuthControllerTest do alias Pleroma.Web.OAuth.OAuthController alias Pleroma.Web.OAuth.Token - @oauth_config_path [:oauth2, :issue_new_refresh_token] @session_opts [ store: :cookie, key: "_test", signing_salt: "cooldude" ] + clear_config_all([:instance, :account_activation_required]) describe "in OAuth consumer mode, " do setup do - oauth_consumer_strategies_path = [:auth, :oauth_consumer_strategies] - oauth_consumer_strategies = Pleroma.Config.get(oauth_consumer_strategies_path) - Pleroma.Config.put(oauth_consumer_strategies_path, ~w(twitter facebook)) - - on_exit(fn -> - Pleroma.Config.put(oauth_consumer_strategies_path, oauth_consumer_strategies) - end) - [ app: insert(:oauth_app), conn: @@ -37,6 +29,13 @@ defmodule Pleroma.Web.OAuth.OAuthControllerTest do ] end + clear_config([:auth, :oauth_consumer_strategies]) do + Pleroma.Config.put( + [:auth, :oauth_consumer_strategies], + ~w(twitter facebook) + ) + end + test "GET /oauth/authorize renders auth forms, including OAuth consumer form", %{ app: app, conn: conn @@ -775,12 +774,7 @@ test "rejects token exchange with invalid client credentials" do end test "rejects token exchange for valid credentials belonging to unconfirmed user and confirmation is required" do - setting = Pleroma.Config.get([:instance, :account_activation_required]) - - unless setting do - Pleroma.Config.put([:instance, :account_activation_required], true) - on_exit(fn -> Pleroma.Config.put([:instance, :account_activation_required], setting) end) - end + Pleroma.Config.put([:instance, :account_activation_required], true) password = "testpassword" user = insert(:user, password_hash: Comeonin.Pbkdf2.hashpwsalt(password)) @@ -857,16 +851,10 @@ test "rejects an invalid authorization code" do end describe "POST /oauth/token - refresh token" do - setup do - oauth_token_config = Pleroma.Config.get(@oauth_config_path) - - on_exit(fn -> - Pleroma.Config.get(@oauth_config_path, oauth_token_config) - end) - end + clear_config([:oauth2, :issue_new_refresh_token]) test "issues a new access token with keep fresh token" do - Pleroma.Config.put(@oauth_config_path, true) + Pleroma.Config.put([:oauth2, :issue_new_refresh_token], true) user = insert(:user) app = insert(:oauth_app, scopes: ["read", "write"]) @@ -906,7 +894,7 @@ test "issues a new access token with keep fresh token" do end test "issues a new access token with new fresh token" do - Pleroma.Config.put(@oauth_config_path, false) + Pleroma.Config.put([:oauth2, :issue_new_refresh_token], false) user = insert(:user) app = insert(:oauth_app, scopes: ["read", "write"]) diff --git a/test/web/ostatus/ostatus_controller_test.exs b/test/web/ostatus/ostatus_controller_test.exs index bb7648bdd..095ae7041 100644 --- a/test/web/ostatus/ostatus_controller_test.exs +++ b/test/web/ostatus/ostatus_controller_test.exs @@ -15,16 +15,13 @@ defmodule Pleroma.Web.OStatus.OStatusControllerTest do setup_all do Tesla.Mock.mock_global(fn env -> apply(HttpRequestMock, :request, [env]) end) - - config_path = [:instance, :federating] - initial_setting = Pleroma.Config.get(config_path) - - Pleroma.Config.put(config_path, true) - on_exit(fn -> Pleroma.Config.put(config_path, initial_setting) end) - :ok end + clear_config_all([:instance, :federating]) do + Pleroma.Config.put([:instance, :federating], true) + end + describe "salmon_incoming" do test "decodes a salmon", %{conn: conn} do user = insert(:user) @@ -101,160 +98,538 @@ test "returns 404 for a missing feed", %{conn: conn} do assert response(conn, 404) end - test "gets an object", %{conn: conn} do - note_activity = insert(:note_activity) - object = Object.normalize(note_activity) - user = User.get_cached_by_ap_id(note_activity.data["actor"]) - [_, uuid] = hd(Regex.scan(~r/.+\/([\w-]+)$/, object.data["id"])) - url = "/objects/#{uuid}" + describe "GET object/2" do + test "gets an object", %{conn: conn} do + note_activity = insert(:note_activity) + object = Object.normalize(note_activity) + user = User.get_cached_by_ap_id(note_activity.data["actor"]) + [_, uuid] = hd(Regex.scan(~r/.+\/([\w-]+)$/, object.data["id"])) + url = "/objects/#{uuid}" + + conn = + conn + |> put_req_header("accept", "application/xml") + |> get(url) + + expected = + ActivityRepresenter.to_simple_form(note_activity, user, true) + |> ActivityRepresenter.wrap_with_entry() + |> :xmerl.export_simple(:xmerl_xml) + |> to_string + + assert response(conn, 200) == expected + end + + test "redirects to /notice/id for html format", %{conn: conn} do + note_activity = insert(:note_activity) + object = Object.normalize(note_activity) + [_, uuid] = hd(Regex.scan(~r/.+\/([\w-]+)$/, object.data["id"])) + url = "/objects/#{uuid}" + + conn = + conn + |> put_req_header("accept", "text/html") + |> get(url) + + assert redirected_to(conn) == "/notice/#{note_activity.id}" + end + + test "500s when user not found", %{conn: conn} do + note_activity = insert(:note_activity) + object = Object.normalize(note_activity) + user = User.get_cached_by_ap_id(note_activity.data["actor"]) + User.invalidate_cache(user) + Pleroma.Repo.delete(user) + [_, uuid] = hd(Regex.scan(~r/.+\/([\w-]+)$/, object.data["id"])) + url = "/objects/#{uuid}" + + conn = + conn + |> put_req_header("accept", "application/xml") + |> get(url) + + assert response(conn, 500) == ~S({"error":"Something went wrong"}) + end + + test "404s on private objects", %{conn: conn} do + note_activity = insert(:direct_note_activity) + object = Object.normalize(note_activity) + [_, uuid] = hd(Regex.scan(~r/.+\/([\w-]+)$/, object.data["id"])) + + conn + |> get("/objects/#{uuid}") + |> response(404) + end + + test "404s on nonexisting objects", %{conn: conn} do + conn + |> get("/objects/123") + |> response(404) + end + end + + describe "GET activity/2" do + test "gets an activity in xml format", %{conn: conn} do + note_activity = insert(:note_activity) + [_, uuid] = hd(Regex.scan(~r/.+\/([\w-]+)$/, note_activity.data["id"])) - conn = conn |> put_req_header("accept", "application/xml") - |> get(url) + |> get("/activities/#{uuid}") + |> response(200) + end - expected = - ActivityRepresenter.to_simple_form(note_activity, user, true) - |> ActivityRepresenter.wrap_with_entry() - |> :xmerl.export_simple(:xmerl_xml) - |> to_string + test "redirects to /notice/id for html format", %{conn: conn} do + note_activity = insert(:note_activity) + [_, uuid] = hd(Regex.scan(~r/.+\/([\w-]+)$/, note_activity.data["id"])) - assert response(conn, 200) == expected + conn = + conn + |> put_req_header("accept", "text/html") + |> get("/activities/#{uuid}") + + assert redirected_to(conn) == "/notice/#{note_activity.id}" + end + + test "505s when user not found", %{conn: conn} do + note_activity = insert(:note_activity) + [_, uuid] = hd(Regex.scan(~r/.+\/([\w-]+)$/, note_activity.data["id"])) + user = User.get_cached_by_ap_id(note_activity.data["actor"]) + User.invalidate_cache(user) + Pleroma.Repo.delete(user) + + conn = + conn + |> put_req_header("accept", "text/html") + |> get("/activities/#{uuid}") + + assert response(conn, 500) == ~S({"error":"Something went wrong"}) + end + + test "404s on deleted objects", %{conn: conn} do + note_activity = insert(:note_activity) + object = Object.normalize(note_activity) + [_, uuid] = hd(Regex.scan(~r/.+\/([\w-]+)$/, object.data["id"])) + + conn + |> put_req_header("accept", "application/xml") + |> get("/objects/#{uuid}") + |> response(200) + + Object.delete(object) + + conn + |> put_req_header("accept", "application/xml") + |> get("/objects/#{uuid}") + |> response(404) + end + + test "404s on private activities", %{conn: conn} do + note_activity = insert(:direct_note_activity) + [_, uuid] = hd(Regex.scan(~r/.+\/([\w-]+)$/, note_activity.data["id"])) + + conn + |> get("/activities/#{uuid}") + |> response(404) + end + + test "404s on nonexistent activities", %{conn: conn} do + conn + |> get("/activities/123") + |> response(404) + end + + test "gets an activity in AS2 format", %{conn: conn} do + note_activity = insert(:note_activity) + [_, uuid] = hd(Regex.scan(~r/.+\/([\w-]+)$/, note_activity.data["id"])) + url = "/activities/#{uuid}" + + conn = + conn + |> put_req_header("accept", "application/activity+json") + |> get(url) + + assert json_response(conn, 200) + end end - test "404s on private objects", %{conn: conn} do - note_activity = insert(:direct_note_activity) - object = Object.normalize(note_activity) - [_, uuid] = hd(Regex.scan(~r/.+\/([\w-]+)$/, object.data["id"])) + describe "GET notice/2" do + test "gets a notice in xml format", %{conn: conn} do + note_activity = insert(:note_activity) - conn - |> get("/objects/#{uuid}") - |> response(404) - end + conn + |> get("/notice/#{note_activity.id}") + |> response(200) + end - test "404s on nonexisting objects", %{conn: conn} do - conn - |> get("/objects/123") - |> response(404) - end + test "gets a notice in AS2 format", %{conn: conn} do + note_activity = insert(:note_activity) - test "gets an activity in xml format", %{conn: conn} do - note_activity = insert(:note_activity) - [_, uuid] = hd(Regex.scan(~r/.+\/([\w-]+)$/, note_activity.data["id"])) - - conn - |> put_req_header("accept", "application/xml") - |> get("/activities/#{uuid}") - |> response(200) - end - - test "404s on deleted objects", %{conn: conn} do - note_activity = insert(:note_activity) - object = Object.normalize(note_activity) - [_, uuid] = hd(Regex.scan(~r/.+\/([\w-]+)$/, object.data["id"])) - - conn - |> put_req_header("accept", "application/xml") - |> get("/objects/#{uuid}") - |> response(200) - - Object.delete(object) - - conn - |> put_req_header("accept", "application/xml") - |> get("/objects/#{uuid}") - |> response(404) - end - - test "404s on private activities", %{conn: conn} do - note_activity = insert(:direct_note_activity) - [_, uuid] = hd(Regex.scan(~r/.+\/([\w-]+)$/, note_activity.data["id"])) - - conn - |> get("/activities/#{uuid}") - |> response(404) - end - - test "404s on nonexistent activities", %{conn: conn} do - conn - |> get("/activities/123") - |> response(404) - end - - test "gets a notice in xml format", %{conn: conn} do - note_activity = insert(:note_activity) - - conn - |> get("/notice/#{note_activity.id}") - |> response(200) - end - - test "gets a notice in AS2 format", %{conn: conn} do - note_activity = insert(:note_activity) - - conn - |> put_req_header("accept", "application/activity+json") - |> get("/notice/#{note_activity.id}") - |> json_response(200) - end - - test "only gets a notice in AS2 format for Create messages", %{conn: conn} do - note_activity = insert(:note_activity) - url = "/notice/#{note_activity.id}" - - conn = conn |> put_req_header("accept", "application/activity+json") - |> get(url) + |> get("/notice/#{note_activity.id}") + |> json_response(200) + end - assert json_response(conn, 200) + test "500s when actor not found", %{conn: conn} do + note_activity = insert(:note_activity) + user = User.get_cached_by_ap_id(note_activity.data["actor"]) + User.invalidate_cache(user) + Pleroma.Repo.delete(user) - user = insert(:user) + conn = + conn + |> get("/notice/#{note_activity.id}") - {:ok, like_activity, _} = CommonAPI.favorite(note_activity.id, user) - url = "/notice/#{like_activity.id}" + assert response(conn, 500) == ~S({"error":"Something went wrong"}) + end - assert like_activity.data["type"] == "Like" + test "only gets a notice in AS2 format for Create messages", %{conn: conn} do + note_activity = insert(:note_activity) + url = "/notice/#{note_activity.id}" - conn = - build_conn() - |> put_req_header("accept", "application/activity+json") - |> get(url) + conn = + conn + |> put_req_header("accept", "application/activity+json") + |> get(url) - assert response(conn, 404) + assert json_response(conn, 200) + + user = insert(:user) + + {:ok, like_activity, _} = CommonAPI.favorite(note_activity.id, user) + url = "/notice/#{like_activity.id}" + + assert like_activity.data["type"] == "Like" + + conn = + build_conn() + |> put_req_header("accept", "application/activity+json") + |> get(url) + + assert response(conn, 404) + end + + test "render html for redirect for html format", %{conn: conn} do + note_activity = insert(:note_activity) + + resp = + conn + |> put_req_header("accept", "text/html") + |> get("/notice/#{note_activity.id}") + |> response(200) + + assert resp =~ + "" + + user = insert(:user) + + {:ok, like_activity, _} = CommonAPI.favorite(note_activity.id, user) + + assert like_activity.data["type"] == "Like" + + resp = + conn + |> put_req_header("accept", "text/html") + |> get("/notice/#{like_activity.id}") + |> response(200) + + assert resp =~ "" + end + + test "404s a private notice", %{conn: conn} do + note_activity = insert(:direct_note_activity) + url = "/notice/#{note_activity.id}" + + conn = + conn + |> get(url) + + assert response(conn, 404) + end + + test "404s a nonexisting notice", %{conn: conn} do + url = "/notice/123" + + conn = + conn + |> get(url) + + assert response(conn, 404) + end end - test "gets an activity in AS2 format", %{conn: conn} do - note_activity = insert(:note_activity) - [_, uuid] = hd(Regex.scan(~r/.+\/([\w-]+)$/, note_activity.data["id"])) - url = "/activities/#{uuid}" + describe "feed_redirect" do + test "undefined format. it redirects to feed", %{conn: conn} do + note_activity = insert(:note_activity) + user = User.get_cached_by_ap_id(note_activity.data["actor"]) - conn = - conn - |> put_req_header("accept", "application/activity+json") - |> get(url) + response = + conn + |> put_req_header("accept", "application/xml") + |> get("/users/#{user.nickname}") + |> response(302) - assert json_response(conn, 200) + assert response == + "You are being redirected." + end + + test "undefined format. it returns error when user not found", %{conn: conn} do + response = + conn + |> put_req_header("accept", "application/xml") + |> get("/users/jimm") + |> response(404) + + assert response == ~S({"error":"Not found"}) + end + + test "activity+json format. it redirects on actual feed of user", %{conn: conn} do + note_activity = insert(:note_activity) + user = User.get_cached_by_ap_id(note_activity.data["actor"]) + + response = + conn + |> put_req_header("accept", "application/activity+json") + |> get("/users/#{user.nickname}") + |> json_response(200) + + assert response["endpoints"] == %{ + "oauthAuthorizationEndpoint" => "#{Pleroma.Web.base_url()}/oauth/authorize", + "oauthRegistrationEndpoint" => "#{Pleroma.Web.base_url()}/api/v1/apps", + "oauthTokenEndpoint" => "#{Pleroma.Web.base_url()}/oauth/token", + "sharedInbox" => "#{Pleroma.Web.base_url()}/inbox" + } + + assert response["@context"] == [ + "https://www.w3.org/ns/activitystreams", + "http://localhost:4001/schemas/litepub-0.1.jsonld", + %{"@language" => "und"} + ] + + assert Map.take(response, [ + "followers", + "following", + "id", + "inbox", + "manuallyApprovesFollowers", + "name", + "outbox", + "preferredUsername", + "summary", + "tag", + "type", + "url" + ]) == %{ + "followers" => "#{Pleroma.Web.base_url()}/users/#{user.nickname}/followers", + "following" => "#{Pleroma.Web.base_url()}/users/#{user.nickname}/following", + "id" => "#{Pleroma.Web.base_url()}/users/#{user.nickname}", + "inbox" => "#{Pleroma.Web.base_url()}/users/#{user.nickname}/inbox", + "manuallyApprovesFollowers" => false, + "name" => user.name, + "outbox" => "#{Pleroma.Web.base_url()}/users/#{user.nickname}/outbox", + "preferredUsername" => user.nickname, + "summary" => user.bio, + "tag" => [], + "type" => "Person", + "url" => "#{Pleroma.Web.base_url()}/users/#{user.nickname}" + } + end + + test "activity+json format. it returns error whe use not found", %{conn: conn} do + response = + conn + |> put_req_header("accept", "application/activity+json") + |> get("/users/jimm") + |> json_response(404) + + assert response == "Not found" + end + + test "json format. it redirects on actual feed of user", %{conn: conn} do + note_activity = insert(:note_activity) + user = User.get_cached_by_ap_id(note_activity.data["actor"]) + + response = + conn + |> put_req_header("accept", "application/json") + |> get("/users/#{user.nickname}") + |> json_response(200) + + assert response["endpoints"] == %{ + "oauthAuthorizationEndpoint" => "#{Pleroma.Web.base_url()}/oauth/authorize", + "oauthRegistrationEndpoint" => "#{Pleroma.Web.base_url()}/api/v1/apps", + "oauthTokenEndpoint" => "#{Pleroma.Web.base_url()}/oauth/token", + "sharedInbox" => "#{Pleroma.Web.base_url()}/inbox" + } + + assert response["@context"] == [ + "https://www.w3.org/ns/activitystreams", + "http://localhost:4001/schemas/litepub-0.1.jsonld", + %{"@language" => "und"} + ] + + assert Map.take(response, [ + "followers", + "following", + "id", + "inbox", + "manuallyApprovesFollowers", + "name", + "outbox", + "preferredUsername", + "summary", + "tag", + "type", + "url" + ]) == %{ + "followers" => "#{Pleroma.Web.base_url()}/users/#{user.nickname}/followers", + "following" => "#{Pleroma.Web.base_url()}/users/#{user.nickname}/following", + "id" => "#{Pleroma.Web.base_url()}/users/#{user.nickname}", + "inbox" => "#{Pleroma.Web.base_url()}/users/#{user.nickname}/inbox", + "manuallyApprovesFollowers" => false, + "name" => user.name, + "outbox" => "#{Pleroma.Web.base_url()}/users/#{user.nickname}/outbox", + "preferredUsername" => user.nickname, + "summary" => user.bio, + "tag" => [], + "type" => "Person", + "url" => "#{Pleroma.Web.base_url()}/users/#{user.nickname}" + } + end + + test "json format. it returns error whe use not found", %{conn: conn} do + response = + conn + |> put_req_header("accept", "application/json") + |> get("/users/jimm") + |> json_response(404) + + assert response == "Not found" + end + + test "html format. it redirects on actual feed of user", %{conn: conn} do + note_activity = insert(:note_activity) + user = User.get_cached_by_ap_id(note_activity.data["actor"]) + + response = + conn + |> get("/users/#{user.nickname}") + |> response(200) + + assert response == + Fallback.RedirectController.redirector_with_meta( + conn, + %{user: user} + ).resp_body + end + + test "html format. it returns error when user not found", %{conn: conn} do + response = + conn + |> get("/users/jimm") + |> json_response(404) + + assert response == %{"error" => "Not found"} + end end - test "404s a private notice", %{conn: conn} do - note_activity = insert(:direct_note_activity) - url = "/notice/#{note_activity.id}" + describe "GET /notice/:id/embed_player" do + test "render embed player", %{conn: conn} do + note_activity = insert(:note_activity) + object = Pleroma.Object.normalize(note_activity) - conn = - conn - |> get(url) + object_data = + Map.put(object.data, "attachment", [ + %{ + "url" => [ + %{ + "href" => + "https://peertube.moe/static/webseed/df5f464b-be8d-46fb-ad81-2d4c2d1630e3-480.mp4", + "mediaType" => "video/mp4", + "type" => "Link" + } + ] + } + ]) - assert response(conn, 404) - end + object + |> Ecto.Changeset.change(data: object_data) + |> Pleroma.Repo.update() - test "404s a nonexisting notice", %{conn: conn} do - url = "/notice/123" + conn = + conn + |> get("/notice/#{note_activity.id}/embed_player") - conn = - conn - |> get(url) + assert Plug.Conn.get_resp_header(conn, "x-frame-options") == ["ALLOW"] - assert response(conn, 404) + assert Plug.Conn.get_resp_header( + conn, + "content-security-policy" + ) == [ + "default-src 'none';style-src 'self' 'unsafe-inline';img-src 'self' data: https:; media-src 'self' https:;" + ] + + assert response(conn, 200) =~ + "" + end + + test "404s when activity isn't create", %{conn: conn} do + note_activity = insert(:note_activity, data_attrs: %{"type" => "Like"}) + + assert conn + |> get("/notice/#{note_activity.id}/embed_player") + |> response(404) + end + + test "404s when activity is direct message", %{conn: conn} do + note_activity = insert(:note_activity, data_attrs: %{"directMessage" => true}) + + assert conn + |> get("/notice/#{note_activity.id}/embed_player") + |> response(404) + end + + test "404s when attachment is empty", %{conn: conn} do + note_activity = insert(:note_activity) + object = Pleroma.Object.normalize(note_activity) + object_data = Map.put(object.data, "attachment", []) + + object + |> Ecto.Changeset.change(data: object_data) + |> Pleroma.Repo.update() + + assert conn + |> get("/notice/#{note_activity.id}/embed_player") + |> response(404) + end + + test "404s when attachment isn't audio or video", %{conn: conn} do + note_activity = insert(:note_activity) + object = Pleroma.Object.normalize(note_activity) + + object_data = + Map.put(object.data, "attachment", [ + %{ + "url" => [ + %{ + "href" => "https://peertube.moe/static/webseed/480.jpg", + "mediaType" => "image/jpg", + "type" => "Link" + } + ] + } + ]) + + object + |> Ecto.Changeset.change(data: object_data) + |> Pleroma.Repo.update() + + assert conn + |> get("/notice/#{note_activity.id}/embed_player") + |> response(404) + end end end diff --git a/test/web/ostatus/ostatus_test.exs b/test/web/ostatus/ostatus_test.exs index 4e8f3a0fc..803a97695 100644 --- a/test/web/ostatus/ostatus_test.exs +++ b/test/web/ostatus/ostatus_test.exs @@ -199,7 +199,7 @@ test "handle incoming retweets - GS, subscription - local message" do assert retweeted_activity.data["type"] == "Create" assert retweeted_activity.data["actor"] == user.ap_id assert retweeted_activity.local - assert retweeted_activity.data["object"]["announcement_count"] == 1 + assert Object.normalize(retweeted_activity).data["announcement_count"] == 1 end test "handle incoming retweets - Mastodon, salmon" do @@ -326,6 +326,14 @@ test "handle incoming follows" do assert User.following?(follower, followed) end + test "refuse following over OStatus if the followed's account is locked" do + incoming = File.read!("test/fixtures/follow.xml") + _user = insert(:user, info: %{locked: true}, ap_id: "https://pawoo.net/users/pekorino") + + {:ok, [{:error, "It's not possible to follow locked accounts over OStatus"}]} = + OStatus.handle_incoming(incoming) + end + test "handle incoming unfollows with existing follow" do incoming_follow = File.read!("test/fixtures/follow.xml") {:ok, [_activity]} = OStatus.handle_incoming(incoming_follow) @@ -426,7 +434,7 @@ test "find_or_make_user sets all the nessary input fields" do } end - test "find_make_or_update_user takes an author element and returns an updated user" do + test "find_make_or_update_actor takes an author element and returns an updated user" do uri = "https://social.heldscal.la/user/23211" {:ok, user} = OStatus.find_or_make_user(uri) @@ -439,14 +447,56 @@ test "find_make_or_update_user takes an author element and returns an updated us doc = XML.parse_document(File.read!("test/fixtures/23211.atom")) [author] = :xmerl_xpath.string('//author[1]', doc) - {:ok, user} = OStatus.find_make_or_update_user(author) + {:ok, user} = OStatus.find_make_or_update_actor(author) assert user.avatar["type"] == "Image" assert user.name == old_name assert user.bio == old_bio - {:ok, user_again} = OStatus.find_make_or_update_user(author) + {:ok, user_again} = OStatus.find_make_or_update_actor(author) assert user_again == user end + + test "find_or_make_user disallows protocol downgrade" do + user = insert(:user, %{local: true}) + {:ok, user} = OStatus.find_or_make_user(user.ap_id) + + assert User.ap_enabled?(user) + + user = + insert(:user, %{ + ap_id: "https://social.heldscal.la/user/23211", + info: %{ap_enabled: true}, + local: false + }) + + assert User.ap_enabled?(user) + + {:ok, user} = OStatus.find_or_make_user(user.ap_id) + assert User.ap_enabled?(user) + end + + test "find_make_or_update_actor disallows protocol downgrade" do + user = insert(:user, %{local: true}) + {:ok, user} = OStatus.find_or_make_user(user.ap_id) + + assert User.ap_enabled?(user) + + user = + insert(:user, %{ + ap_id: "https://social.heldscal.la/user/23211", + info: %{ap_enabled: true}, + local: false + }) + + assert User.ap_enabled?(user) + + {:ok, user} = OStatus.find_or_make_user(user.ap_id) + assert User.ap_enabled?(user) + + doc = XML.parse_document(File.read!("test/fixtures/23211.atom")) + [author] = :xmerl_xpath.string('//author[1]', doc) + {:error, :invalid_protocol} = OStatus.find_make_or_update_actor(author) + end end describe "gathering user info from a user id" do diff --git a/test/web/pleroma_api/pleroma_api_controller_test.exs b/test/web/pleroma_api/pleroma_api_controller_test.exs new file mode 100644 index 000000000..ed6b79727 --- /dev/null +++ b/test/web/pleroma_api/pleroma_api_controller_test.exs @@ -0,0 +1,94 @@ +# Pleroma: A lightweight social networking server +# Copyright © 2017-2019 Pleroma Authors +# SPDX-License-Identifier: AGPL-3.0-only + +defmodule Pleroma.Web.PleromaAPI.PleromaAPIControllerTest do + use Pleroma.Web.ConnCase + + alias Pleroma.Conversation.Participation + alias Pleroma.Repo + alias Pleroma.Web.CommonAPI + + import Pleroma.Factory + + test "/api/v1/pleroma/conversations/:id", %{conn: conn} do + user = insert(:user) + other_user = insert(:user) + + {:ok, _activity} = + CommonAPI.post(user, %{"status" => "Hi @#{other_user.nickname}!", "visibility" => "direct"}) + + [participation] = Participation.for_user(other_user) + + result = + conn + |> assign(:user, other_user) + |> get("/api/v1/pleroma/conversations/#{participation.id}") + |> json_response(200) + + assert result["id"] == participation.id |> to_string() + end + + test "/api/v1/pleroma/conversations/:id/statuses", %{conn: conn} do + user = insert(:user) + other_user = insert(:user) + third_user = insert(:user) + + {:ok, _activity} = + CommonAPI.post(user, %{"status" => "Hi @#{third_user.nickname}!", "visibility" => "direct"}) + + {:ok, activity} = + CommonAPI.post(user, %{"status" => "Hi @#{other_user.nickname}!", "visibility" => "direct"}) + + [participation] = Participation.for_user(other_user) + + {:ok, activity_two} = + CommonAPI.post(other_user, %{ + "status" => "Hi!", + "in_reply_to_status_id" => activity.id, + "in_reply_to_conversation_id" => participation.id + }) + + result = + conn + |> assign(:user, other_user) + |> get("/api/v1/pleroma/conversations/#{participation.id}/statuses") + |> json_response(200) + + assert length(result) == 2 + + id_one = activity.id + id_two = activity_two.id + assert [%{"id" => ^id_one}, %{"id" => ^id_two}] = result + end + + test "PATCH /api/v1/pleroma/conversations/:id", %{conn: conn} do + user = insert(:user) + other_user = insert(:user) + + {:ok, _activity} = CommonAPI.post(user, %{"status" => "Hi", "visibility" => "direct"}) + + [participation] = Participation.for_user(user) + + participation = Repo.preload(participation, :recipients) + + assert [user] == participation.recipients + assert other_user not in participation.recipients + + result = + conn + |> assign(:user, user) + |> patch("/api/v1/pleroma/conversations/#{participation.id}", %{ + "recipients" => [user.id, other_user.id] + }) + |> json_response(200) + + assert result["id"] == participation.id |> to_string + + [participation] = Participation.for_user(user) + participation = Repo.preload(participation, :recipients) + + assert user in participation.recipients + assert other_user in participation.recipients + end +end diff --git a/test/web/plugs/federating_plug_test.exs b/test/web/plugs/federating_plug_test.exs index c01e01124..bb2e1687a 100644 --- a/test/web/plugs/federating_plug_test.exs +++ b/test/web/plugs/federating_plug_test.exs @@ -4,15 +4,7 @@ defmodule Pleroma.Web.FederatingPlugTest do use Pleroma.Web.ConnCase - - setup_all do - config_path = [:instance, :federating] - initial_setting = Pleroma.Config.get(config_path) - - on_exit(fn -> Pleroma.Config.put(config_path, initial_setting) end) - - :ok - end + clear_config_all([:instance, :federating]) test "returns and halt the conn when federating is disabled" do Pleroma.Config.put([:instance, :federating], false) diff --git a/test/web/push/impl_test.exs b/test/web/push/impl_test.exs index 1e948086a..e2f89f40a 100644 --- a/test/web/push/impl_test.exs +++ b/test/web/push/impl_test.exs @@ -124,8 +124,7 @@ test "renders body for follow activity" do {:ok, _, _, activity} = CommonAPI.follow(user, other_user) object = Object.normalize(activity) - assert Impl.format_body(%{activity: activity}, user, object) == - "@Bob has followed you" + assert Impl.format_body(%{activity: activity}, user, object) == "@Bob has followed you" end test "renders body for announce activity" do @@ -156,7 +155,6 @@ test "renders body for like activity" do {:ok, activity, _} = CommonAPI.favorite(activity.id, user) object = Object.normalize(activity) - assert Impl.format_body(%{activity: activity}, user, object) == - "@Bob has favorited your post" + assert Impl.format_body(%{activity: activity}, user, object) == "@Bob has favorited your post" end end diff --git a/test/web/rich_media/aws_signed_url_test.exs b/test/web/rich_media/aws_signed_url_test.exs index 122787bc2..a3a50cbb1 100644 --- a/test/web/rich_media/aws_signed_url_test.exs +++ b/test/web/rich_media/aws_signed_url_test.exs @@ -60,7 +60,8 @@ test "s3 signed url is parsed and correct ttl is set for rich media" do {:ok, cache_ttl} = Cachex.ttl(:rich_media_cache, url) # as there is delay in setting and pulling the data from cache we ignore 1 second - assert_in_delta(valid_till * 1000, cache_ttl, 1000) + # make it 2 seconds for flakyness + assert_in_delta(valid_till * 1000, cache_ttl, 2000) end defp construct_s3_url(timestamp, valid_till) do diff --git a/test/web/rich_media/helpers_test.exs b/test/web/rich_media/helpers_test.exs index 92198f3d9..48884319d 100644 --- a/test/web/rich_media/helpers_test.exs +++ b/test/web/rich_media/helpers_test.exs @@ -15,12 +15,12 @@ defmodule Pleroma.Web.RichMedia.HelpersTest do setup do mock(fn env -> apply(HttpRequestMock, :request, [env]) end) - rich_media = Config.get([:rich_media, :enabled]) - on_exit(fn -> Config.put([:rich_media, :enabled], rich_media) end) :ok end + clear_config([:rich_media, :enabled]) + test "refuses to crawl incomplete URLs" do user = insert(:user) diff --git a/test/web/rich_media/parsers/twitter_card_test.exs b/test/web/rich_media/parsers/twitter_card_test.exs new file mode 100644 index 000000000..f8e1c9b40 --- /dev/null +++ b/test/web/rich_media/parsers/twitter_card_test.exs @@ -0,0 +1,69 @@ +# Pleroma: A lightweight social networking server +# Copyright © 2017-2019 Pleroma Authors +# SPDX-License-Identifier: AGPL-3.0-only + +defmodule Pleroma.Web.RichMedia.Parsers.TwitterCardTest do + use ExUnit.Case, async: true + alias Pleroma.Web.RichMedia.Parsers.TwitterCard + + test "returns error when html not contains twitter card" do + assert TwitterCard.parse("", %{}) == {:error, "No twitter card metadata found"} + end + + test "parses twitter card with only name attributes" do + html = File.read!("test/fixtures/nypd-facial-recognition-children-teenagers3.html") + + assert TwitterCard.parse(html, %{}) == + {:ok, + %{ + "app:id:googleplay": "com.nytimes.android", + "app:name:googleplay": "NYTimes", + "app:url:googleplay": "nytimes://reader/id/100000006583622", + site: nil, + title: + "She Was Arrested at 14. Then Her Photo Went to a Facial Recognition Database. - The New York Times" + }} + end + + test "parses twitter card with only property attributes" do + html = File.read!("test/fixtures/nypd-facial-recognition-children-teenagers2.html") + + assert TwitterCard.parse(html, %{}) == + {:ok, + %{ + card: "summary_large_image", + description: + "With little oversight, the N.Y.P.D. has been using powerful surveillance technology on photos of children and teenagers.", + image: + "https://static01.nyt.com/images/2019/08/01/nyregion/01nypd-juveniles-promo/01nypd-juveniles-promo-videoSixteenByNineJumbo1600.jpg", + "image:alt": "", + title: + "She Was Arrested at 14. Then Her Photo Went to a Facial Recognition Database.", + url: + "https://www.nytimes.com/2019/08/01/nyregion/nypd-facial-recognition-children-teenagers.html" + }} + end + + test "parses twitter card with name & property attributes" do + html = File.read!("test/fixtures/nypd-facial-recognition-children-teenagers.html") + + assert TwitterCard.parse(html, %{}) == + {:ok, + %{ + "app:id:googleplay": "com.nytimes.android", + "app:name:googleplay": "NYTimes", + "app:url:googleplay": "nytimes://reader/id/100000006583622", + card: "summary_large_image", + description: + "With little oversight, the N.Y.P.D. has been using powerful surveillance technology on photos of children and teenagers.", + image: + "https://static01.nyt.com/images/2019/08/01/nyregion/01nypd-juveniles-promo/01nypd-juveniles-promo-videoSixteenByNineJumbo1600.jpg", + "image:alt": "", + site: nil, + title: + "She Was Arrested at 14. Then Her Photo Went to a Facial Recognition Database.", + url: + "https://www.nytimes.com/2019/08/01/nyregion/nypd-facial-recognition-children-teenagers.html" + }} + end +end diff --git a/test/web/streamer_test.exs b/test/web/streamer_test.exs index 8f56e7486..96fa7645f 100644 --- a/test/web/streamer_test.exs +++ b/test/web/streamer_test.exs @@ -11,15 +11,7 @@ defmodule Pleroma.Web.StreamerTest do alias Pleroma.Web.Streamer import Pleroma.Factory - setup do - skip_thread_containment = Pleroma.Config.get([:instance, :skip_thread_containment]) - - on_exit(fn -> - Pleroma.Config.put([:instance, :skip_thread_containment], skip_thread_containment) - end) - - :ok - end + clear_config_all([:instance, :skip_thread_containment]) describe "user streams" do setup do @@ -103,6 +95,25 @@ test "it doesn't send notify to the 'user:notification' stream when a thread is Streamer.stream("user:notification", notif) Task.await(task) end + + test "it doesn't send notify to the 'user:notification' stream' when a domain is blocked", %{ + user: user + } do + user2 = insert(:user, %{ap_id: "https://hecking-lewd-place.com/user/meanie"}) + task = Task.async(fn -> refute_receive {:text, _}, 4_000 end) + + Streamer.add_socket( + "user:notification", + %{transport_pid: task.pid, assigns: %{user: user}} + ) + + {:ok, user} = User.block_domain(user, "hecking-lewd-place.com") + {:ok, activity} = CommonAPI.post(user, %{"status" => "super hot take"}) + {:ok, notif, _} = CommonAPI.favorite(activity.id, user2) + + Streamer.stream("user:notification", notif) + Task.await(task) + end end test "it sends to public" do @@ -395,6 +406,26 @@ test "it doesn't send muted reblogs" do Task.await(task) end + test "it doesn't send posts from muted threads" do + user = insert(:user) + user2 = insert(:user) + {:ok, user2, user, _activity} = CommonAPI.follow(user2, user) + + {:ok, activity} = CommonAPI.post(user, %{"status" => "super hot take"}) + + {:ok, activity} = CommonAPI.add_mute(user2, activity) + + task = Task.async(fn -> refute_receive {:text, _}, 4_000 end) + + Streamer.add_socket( + "user", + %{transport_pid: task.pid, assigns: %{user: user2}} + ) + + Streamer.stream("user", activity) + Task.await(task) + end + describe "direct streams" do setup do GenServer.start(Streamer, %{}, name: Streamer) diff --git a/test/web/twitter_api/twitter_api_controller_test.exs b/test/web/twitter_api/twitter_api_controller_test.exs index 8bb8aa36d..8ef14b4c5 100644 --- a/test/web/twitter_api/twitter_api_controller_test.exs +++ b/test/web/twitter_api/twitter_api_controller_test.exs @@ -151,6 +151,7 @@ test "with credentials", %{conn: conn, user: user} do describe "GET /statuses/public_timeline.json" do setup [:valid_user] + clear_config([:instance, :public]) test "returns statuses", %{conn: conn} do user = insert(:user) @@ -173,8 +174,6 @@ test "returns 403 to unauthenticated request when the instance is not public", % conn |> get("/api/statuses/public_timeline.json") |> json_response(403) - - Pleroma.Config.put([:instance, :public], true) end test "returns 200 to authenticated request when the instance is not public", @@ -185,8 +184,6 @@ test "returns 200 to authenticated request when the instance is not public", |> with_credentials(user.nickname, "test") |> get("/api/statuses/public_timeline.json") |> json_response(200) - - Pleroma.Config.put([:instance, :public], true) end test "returns 200 to unauthenticated request when the instance is public", %{conn: conn} do @@ -220,6 +217,7 @@ test "returns 200 to authenticated request when the instance is public", describe "GET /statuses/public_and_external_timeline.json" do setup [:valid_user] + clear_config([:instance, :public]) test "returns 403 to unauthenticated request when the instance is not public", %{conn: conn} do Pleroma.Config.put([:instance, :public], false) @@ -227,8 +225,6 @@ test "returns 403 to unauthenticated request when the instance is not public", % conn |> get("/api/statuses/public_and_external_timeline.json") |> json_response(403) - - Pleroma.Config.put([:instance, :public], true) end test "returns 200 to authenticated request when the instance is not public", @@ -239,8 +235,6 @@ test "returns 200 to authenticated request when the instance is not public", |> with_credentials(user.nickname, "test") |> get("/api/statuses/public_and_external_timeline.json") |> json_response(200) - - Pleroma.Config.put([:instance, :public], true) end test "returns 200 to unauthenticated request when the instance is public", %{conn: conn} do @@ -1176,13 +1170,6 @@ test "it returns 500 if token is invalid", %{conn: conn, user: user} do describe "POST /api/account/resend_confirmation_email" do setup do - setting = Pleroma.Config.get([:instance, :account_activation_required]) - - unless setting do - Pleroma.Config.put([:instance, :account_activation_required], true) - on_exit(fn -> Pleroma.Config.put([:instance, :account_activation_required], setting) end) - end - user = insert(:user) info_change = User.Info.confirmation_changeset(user.info, need_confirmation: true) @@ -1197,6 +1184,10 @@ test "it returns 500 if token is invalid", %{conn: conn, user: user} do [user: user] end + clear_config([:instance, :account_activation_required]) do + Pleroma.Config.put([:instance, :account_activation_required], true) + end + test "it returns 204 No Content", %{conn: conn, user: user} do conn |> assign(:user, user) diff --git a/test/web/twitter_api/util_controller_test.exs b/test/web/twitter_api/util_controller_test.exs index 3d699e1df..fe4ffdb59 100644 --- a/test/web/twitter_api/util_controller_test.exs +++ b/test/web/twitter_api/util_controller_test.exs @@ -17,6 +17,10 @@ defmodule Pleroma.Web.TwitterAPI.UtilControllerTest do :ok end + clear_config([:instance]) + clear_config([:frontend_configurations, :pleroma_fe]) + clear_config([:user, :deny_follow_blocked]) + describe "POST /api/pleroma/follow_import" do test "it returns HTTP 200", %{conn: conn} do user1 = insert(:user) @@ -31,6 +35,35 @@ test "it returns HTTP 200", %{conn: conn} do assert response == "job started" end + test "it imports follow lists from file", %{conn: conn} do + user1 = insert(:user) + user2 = insert(:user) + + with_mocks([ + {File, [], + read!: fn "follow_list.txt" -> + "Account address,Show boosts\n#{user2.ap_id},true" + end}, + {PleromaJobQueue, [:passthrough], []} + ]) do + response = + conn + |> assign(:user, user1) + |> post("/api/pleroma/follow_import", %{"list" => %Plug.Upload{path: "follow_list.txt"}}) + |> json_response(:ok) + + assert called( + PleromaJobQueue.enqueue( + :background, + User, + [:follow_import, user1, [user2.ap_id]] + ) + ) + + assert response == "job started" + end + end + test "it imports new-style mastodon follow lists", %{conn: conn} do user1 = insert(:user) user2 = insert(:user) @@ -79,6 +112,33 @@ test "it returns HTTP 200", %{conn: conn} do assert response == "job started" end + + test "it imports blocks users from file", %{conn: conn} do + user1 = insert(:user) + user2 = insert(:user) + user3 = insert(:user) + + with_mocks([ + {File, [], read!: fn "blocks_list.txt" -> "#{user2.ap_id} #{user3.ap_id}" end}, + {PleromaJobQueue, [:passthrough], []} + ]) do + response = + conn + |> assign(:user, user1) + |> post("/api/pleroma/blocks_import", %{"list" => %Plug.Upload{path: "blocks_list.txt"}}) + |> json_response(:ok) + + assert called( + PleromaJobQueue.enqueue( + :background, + User, + [:blocks_import, user1, [user2.ap_id, user3.ap_id]] + ) + ) + + assert response == "job started" + end + end end describe "POST /api/pleroma/notifications/read" do @@ -98,6 +158,18 @@ test "it marks a single notification as read", %{conn: conn} do assert Repo.get(Notification, notification1.id).seen refute Repo.get(Notification, notification2.id).seen end + + test "it returns error when notification not found", %{conn: conn} do + user1 = insert(:user) + + response = + conn + |> assign(:user, user1) + |> post("/api/pleroma/notifications/read", %{"id" => "22222222222222"}) + |> json_response(403) + + assert response == %{"error" => "Cannot get notification"} + end end describe "PUT /api/pleroma/notification_settings" do @@ -123,9 +195,64 @@ test "it updates notification settings", %{conn: conn} do end end - describe "GET /api/statusnet/config.json" do + describe "GET /api/statusnet/config" do + test "it returns config in xml format", %{conn: conn} do + instance = Pleroma.Config.get(:instance) + + response = + conn + |> put_req_header("accept", "application/xml") + |> get("/api/statusnet/config") + |> response(:ok) + + assert response == + "\n\n#{Keyword.get(instance, :name)}\n#{ + Pleroma.Web.base_url() + }\n#{Keyword.get(instance, :limit)}\n#{ + !Keyword.get(instance, :registrations_open) + }\n\n\n" + end + + test "it returns config in json format", %{conn: conn} do + instance = Pleroma.Config.get(:instance) + Pleroma.Config.put([:instance, :managed_config], true) + Pleroma.Config.put([:instance, :registrations_open], false) + Pleroma.Config.put([:instance, :invites_enabled], true) + Pleroma.Config.put([:instance, :public], false) + Pleroma.Config.put([:frontend_configurations, :pleroma_fe], %{theme: "asuka-hospital"}) + + response = + conn + |> put_req_header("accept", "application/json") + |> get("/api/statusnet/config") + |> json_response(:ok) + + expected_data = %{ + "site" => %{ + "accountActivationRequired" => "0", + "closed" => "1", + "description" => Keyword.get(instance, :description), + "invitesEnabled" => "1", + "name" => Keyword.get(instance, :name), + "pleromafe" => %{"theme" => "asuka-hospital"}, + "private" => "1", + "safeDMMentionsEnabled" => "0", + "server" => Pleroma.Web.base_url(), + "textlimit" => to_string(Keyword.get(instance, :limit)), + "uploadlimit" => %{ + "avatarlimit" => to_string(Keyword.get(instance, :avatar_upload_limit)), + "backgroundlimit" => to_string(Keyword.get(instance, :background_upload_limit)), + "bannerlimit" => to_string(Keyword.get(instance, :banner_upload_limit)), + "uploadlimit" => to_string(Keyword.get(instance, :upload_limit)) + }, + "vapidPublicKey" => Keyword.get(Pleroma.Web.Push.vapid_config(), :public_key) + } + } + + assert response == expected_data + end + test "returns the state of safe_dm_mentions flag", %{conn: conn} do - option = Pleroma.Config.get([:instance, :safe_dm_mentions]) Pleroma.Config.put([:instance, :safe_dm_mentions], true) response = @@ -143,8 +270,6 @@ test "returns the state of safe_dm_mentions flag", %{conn: conn} do |> json_response(:ok) assert response["site"]["safeDMMentionsEnabled"] == "0" - - Pleroma.Config.put([:instance, :safe_dm_mentions], option) end test "it returns the managed config", %{conn: conn} do @@ -210,7 +335,7 @@ test "returns json with custom emoji with tags", %{conn: conn} do end end - describe "GET /ostatus_subscribe?acct=...." do + describe "GET /ostatus_subscribe - remote_follow/2" do test "adds status to pleroma instance if the `acct` is a status", %{conn: conn} do conn = get( @@ -230,18 +355,176 @@ test "show follow account page if the `acct` is a account link", %{conn: conn} d assert html_response(response, 200) =~ "Log in to follow" end + + test "show follow page if the `acct` is a account link", %{conn: conn} do + user = insert(:user) + + response = + conn + |> assign(:user, user) + |> get("/ostatus_subscribe?acct=https://mastodon.social/users/emelie") + + assert html_response(response, 200) =~ "Remote follow" + end + + test "show follow page with error when user cannot fecth by `acct` link", %{conn: conn} do + user = insert(:user) + + response = + conn + |> assign(:user, user) + |> get("/ostatus_subscribe?acct=https://mastodon.social/users/not_found") + + assert html_response(response, 200) =~ "Error fetching user" + end + end + + describe "POST /ostatus_subscribe - do_remote_follow/2 with assigned user " do + test "follows user", %{conn: conn} do + user = insert(:user) + user2 = insert(:user) + + response = + conn + |> assign(:user, user) + |> post("/ostatus_subscribe", %{"user" => %{"id" => user2.id}}) + |> response(200) + + assert response =~ "Account followed!" + assert user2.follower_address in refresh_record(user).following + end + + test "returns error when user is deactivated", %{conn: conn} do + user = insert(:user, info: %{deactivated: true}) + user2 = insert(:user) + + response = + conn + |> assign(:user, user) + |> post("/ostatus_subscribe", %{"user" => %{"id" => user2.id}}) + |> response(200) + + assert response =~ "Error following account" + end + + test "returns error when user is blocked", %{conn: conn} do + Pleroma.Config.put([:user, :deny_follow_blocked], true) + user = insert(:user) + user2 = insert(:user) + + {:ok, _user} = Pleroma.User.block(user2, user) + + response = + conn + |> assign(:user, user) + |> post("/ostatus_subscribe", %{"user" => %{"id" => user2.id}}) + |> response(200) + + assert response =~ "Error following account" + end + + test "returns error when followee not found", %{conn: conn} do + user = insert(:user) + + response = + conn + |> assign(:user, user) + |> post("/ostatus_subscribe", %{"user" => %{"id" => "jimm"}}) + |> response(200) + + assert response =~ "Error following account" + end + + test "returns success result when user already in followers", %{conn: conn} do + user = insert(:user) + user2 = insert(:user) + {:ok, _, _, _} = CommonAPI.follow(user, user2) + + response = + conn + |> assign(:user, refresh_record(user)) + |> post("/ostatus_subscribe", %{"user" => %{"id" => user2.id}}) + |> response(200) + + assert response =~ "Account followed!" + end + end + + describe "POST /ostatus_subscribe - do_remote_follow/2 without assigned user " do + test "follows", %{conn: conn} do + user = insert(:user) + user2 = insert(:user) + + response = + conn + |> post("/ostatus_subscribe", %{ + "authorization" => %{"name" => user.nickname, "password" => "test", "id" => user2.id} + }) + |> response(200) + + assert response =~ "Account followed!" + assert user2.follower_address in refresh_record(user).following + end + + test "returns error when followee not found", %{conn: conn} do + user = insert(:user) + + response = + conn + |> post("/ostatus_subscribe", %{ + "authorization" => %{"name" => user.nickname, "password" => "test", "id" => "jimm"} + }) + |> response(200) + + assert response =~ "Error following account" + end + + test "returns error when login invalid", %{conn: conn} do + user = insert(:user) + + response = + conn + |> post("/ostatus_subscribe", %{ + "authorization" => %{"name" => "jimm", "password" => "test", "id" => user.id} + }) + |> response(200) + + assert response =~ "Wrong username or password" + end + + test "returns error when password invalid", %{conn: conn} do + user = insert(:user) + user2 = insert(:user) + + response = + conn + |> post("/ostatus_subscribe", %{ + "authorization" => %{"name" => user.nickname, "password" => "42", "id" => user2.id} + }) + |> response(200) + + assert response =~ "Wrong username or password" + end + + test "returns error when user is blocked", %{conn: conn} do + Pleroma.Config.put([:user, :deny_follow_blocked], true) + user = insert(:user) + user2 = insert(:user) + {:ok, _user} = Pleroma.User.block(user2, user) + + response = + conn + |> post("/ostatus_subscribe", %{ + "authorization" => %{"name" => user.nickname, "password" => "test", "id" => user2.id} + }) + |> response(200) + + assert response =~ "Error following account" + end end describe "GET /api/pleroma/healthcheck" do - setup do - config_healthcheck = Pleroma.Config.get([:instance, :healthcheck]) - - on_exit(fn -> - Pleroma.Config.put([:instance, :healthcheck], config_healthcheck) - end) - - :ok - end + clear_config([:instance, :healthcheck]) test "returns 503 when healthcheck disabled", %{conn: conn} do Pleroma.Config.put([:instance, :healthcheck], false) @@ -311,5 +594,104 @@ test "it returns HTTP 200", %{conn: conn} do assert user.info.deactivated == true end + + test "it returns returns when password invalid", %{conn: conn} do + user = insert(:user) + + response = + conn + |> assign(:user, user) + |> post("/api/pleroma/disable_account", %{"password" => "test1"}) + |> json_response(:ok) + + assert response == %{"error" => "Invalid password."} + user = User.get_cached_by_id(user.id) + + refute user.info.deactivated + end + end + + describe "GET /api/statusnet/version" do + test "it returns version in xml format", %{conn: conn} do + response = + conn + |> put_req_header("accept", "application/xml") + |> get("/api/statusnet/version") + |> response(:ok) + + assert response == "#{Pleroma.Application.named_version()}" + end + + test "it returns version in json format", %{conn: conn} do + response = + conn + |> put_req_header("accept", "application/json") + |> get("/api/statusnet/version") + |> json_response(:ok) + + assert response == "#{Pleroma.Application.named_version()}" + end + end + + describe "POST /main/ostatus - remote_subscribe/2" do + test "renders subscribe form", %{conn: conn} do + user = insert(:user) + + response = + conn + |> post("/main/ostatus", %{"nickname" => user.nickname, "profile" => ""}) + |> response(:ok) + + refute response =~ "Could not find user" + assert response =~ "Remotely follow #{user.nickname}" + end + + test "renders subscribe form with error when user not found", %{conn: conn} do + response = + conn + |> post("/main/ostatus", %{"nickname" => "nickname", "profile" => ""}) + |> response(:ok) + + assert response =~ "Could not find user" + refute response =~ "Remotely follow" + end + + test "it redirect to webfinger url", %{conn: conn} do + user = insert(:user) + user2 = insert(:user, ap_id: "shp@social.heldscal.la") + + conn = + conn + |> post("/main/ostatus", %{ + "user" => %{"nickname" => user.nickname, "profile" => user2.ap_id} + }) + + assert redirected_to(conn) == + "https://social.heldscal.la/main/ostatussub?profile=#{user.ap_id}" + end + + test "it renders form with error when use not found", %{conn: conn} do + user2 = insert(:user, ap_id: "shp@social.heldscal.la") + + response = + conn + |> post("/main/ostatus", %{"user" => %{"nickname" => "jimm", "profile" => user2.ap_id}}) + |> response(:ok) + + assert response =~ "Something went wrong." + end + end + + test "it returns new captcha", %{conn: conn} do + with_mock Pleroma.Captcha, + new: fn -> "test_captcha" end do + resp = + conn + |> get("/api/pleroma/captcha") + |> response(200) + + assert resp == "\"test_captcha\"" + assert called(Pleroma.Captcha.new()) + end end end diff --git a/test/web/web_finger/web_finger_controller_test.exs b/test/web/web_finger/web_finger_controller_test.exs index a14ed3126..e23086b2a 100644 --- a/test/web/web_finger/web_finger_controller_test.exs +++ b/test/web/web_finger/web_finger_controller_test.exs @@ -10,15 +10,26 @@ defmodule Pleroma.Web.WebFinger.WebFingerControllerTest do setup do mock(fn env -> apply(HttpRequestMock, :request, [env]) end) - - config_path = [:instance, :federating] - initial_setting = Pleroma.Config.get(config_path) - - Pleroma.Config.put(config_path, true) - on_exit(fn -> Pleroma.Config.put(config_path, initial_setting) end) :ok end + clear_config_all([:instance, :federating]) do + Pleroma.Config.put([:instance, :federating], true) + end + + test "GET host-meta" do + response = + build_conn() + |> get("/.well-known/host-meta") + + assert response.status == 200 + + assert response.resp_body == + ~s() + end + test "Webfinger JRD" do user = insert(:user) @@ -30,6 +41,16 @@ test "Webfinger JRD" do assert json_response(response, 200)["subject"] == "acct:#{user.nickname}@localhost" end + test "it returns 404 when user isn't found (JSON)" do + result = + build_conn() + |> put_req_header("accept", "application/jrd+json") + |> get("/.well-known/webfinger?resource=acct:jimm@localhost") + |> json_response(404) + + assert result == "Couldn't find user" + end + test "Webfinger XML" do user = insert(:user) @@ -41,6 +62,26 @@ test "Webfinger XML" do assert response(response, 200) end + test "it returns 404 when user isn't found (XML)" do + result = + build_conn() + |> put_req_header("accept", "application/xrd+xml") + |> get("/.well-known/webfinger?resource=acct:jimm@localhost") + |> response(404) + + assert result == "Couldn't find user" + end + + test "Sends a 404 when invalid format" do + user = insert(:user) + + assert_raise Phoenix.NotAcceptableError, fn -> + build_conn() + |> put_req_header("accept", "text/html") + |> get("/.well-known/webfinger?resource=acct:#{user.nickname}@localhost") + end + end + test "Sends a 400 when resource param is missing" do response = build_conn() diff --git a/test/web/web_finger/web_finger_test.exs b/test/web/web_finger/web_finger_test.exs index 0578b4b8e..8fdb9adea 100644 --- a/test/web/web_finger/web_finger_test.exs +++ b/test/web/web_finger/web_finger_test.exs @@ -40,6 +40,11 @@ test "works for ap_ids" do end describe "fingering" do + test "returns error when fails parse xml or json" do + user = "invalid_content@social.heldscal.la" + assert {:error, %Jason.DecodeError{}} = WebFinger.finger(user) + end + test "returns the info for an OStatus user" do user = "shp@social.heldscal.la" @@ -81,6 +86,20 @@ test "returns the correctly for json ostatus users" do assert data["subscribe_address"] == "https://gnusocial.de/main/ostatussub?profile={uri}" end + test "it work for AP-only user" do + user = "kpherox@mstdn.jp" + + {:ok, data} = WebFinger.finger(user) + + assert data["magic_key"] == nil + assert data["salmon"] == nil + + assert data["topic"] == "https://mstdn.jp/users/kPherox.atom" + assert data["subject"] == "acct:kPherox@mstdn.jp" + assert data["ap_id"] == "https://mstdn.jp/users/kPherox" + assert data["subscribe_address"] == "https://mstdn.jp/authorize_interaction?acct={uri}" + end + test "it works for friendica" do user = "lain@squeet.me" diff --git a/test/web/websub/websub_controller_test.exs b/test/web/websub/websub_controller_test.exs index aa7262beb..59cacbe68 100644 --- a/test/web/websub/websub_controller_test.exs +++ b/test/web/websub/websub_controller_test.exs @@ -9,14 +9,8 @@ defmodule Pleroma.Web.Websub.WebsubControllerTest do alias Pleroma.Web.Websub alias Pleroma.Web.Websub.WebsubClientSubscription - setup_all do - config_path = [:instance, :federating] - initial_setting = Pleroma.Config.get(config_path) - - Pleroma.Config.put(config_path, true) - on_exit(fn -> Pleroma.Config.put(config_path, initial_setting) end) - - :ok + clear_config_all([:instance, :federating]) do + Pleroma.Config.put([:instance, :federating], true) end test "websub subscription request", %{conn: conn} do