AkkomaGang/akkoma
AkkomaGang/akkoma
15
65
Fork 92
Code Issues 178 Pull requests 25 Projects 2 Releases 14 Packages Wiki Activity

Resolve conflicts

Browse source
This commit is contained in:
Roman Chvanikov 2019-06-07 01:20:50 +03:00
commit 01fe5abad1
30 changed files with 1016 additions and 380 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

3
CHANGELOG.md
View file

@ -7,6 +7,7 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/).
### Added ### Added
- Digest email for inactive users - Digest email for inactive users
- Add a generic settings store for frontends / clients to use. - Add a generic settings store for frontends / clients to use.
- Explicit addressing option for posting.
- Optional SSH access mode. (Needs `erlang-ssh` package on some distributions). - Optional SSH access mode. (Needs `erlang-ssh` package on some distributions).
- [MongooseIM](https://github.com/esl/MongooseIM) http authentication support. - [MongooseIM](https://github.com/esl/MongooseIM) http authentication support.
- LDAP authentication - LDAP authentication
@ -28,6 +29,7 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/).
- Configuration: Media proxy `whitelist` option - Configuration: Media proxy `whitelist` option
- Configuration: `report_uri` option - Configuration: `report_uri` option
- Configuration: `email_notifications` option - Configuration: `email_notifications` option
- Configuration: `limit_unauthenticated_to_local_content` option
- Pleroma API: User subscriptions - Pleroma API: User subscriptions
- Pleroma API: Healthcheck endpoint - Pleroma API: Healthcheck endpoint
- Pleroma API: `/api/v1/pleroma/mascot` per-user frontend mascot configuration endpoints - Pleroma API: `/api/v1/pleroma/mascot` per-user frontend mascot configuration endpoints
@ -91,6 +93,7 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/).
- Respond with a 404 Not implemented JSON error message when requested API is not implemented - Respond with a 404 Not implemented JSON error message when requested API is not implemented
### Fixed ### Fixed
- Follow requests don't get 'stuck' anymore.
- Added an FTS index on objects. Running `vacuum analyze` and setting a larger `work_mem` is recommended. - Added an FTS index on objects. Running `vacuum analyze` and setting a larger `work_mem` is recommended.
- Followers counter not being updated when a follower is blocked - Followers counter not being updated when a follower is blocked
- Deactivated users being able to request an access token - Deactivated users being able to request an access token

3
config/config.exs
View file

@ -244,7 +244,8 @@
safe_dm_mentions: false, safe_dm_mentions: false,
healthcheck: false, healthcheck: false,
remote_post_retention_days: 90, remote_post_retention_days: 90,
skip_thread_containment: false skip_thread_containment: false,
limit_unauthenticated_to_local_content: true
config :pleroma, :app_account_creation, enabled: true, max_requests: 25, interval: 1800 config :pleroma, :app_account_creation, enabled: true, max_requests: 25, interval: 1800

2
config/test.exs
View file

@ -17,6 +17,8 @@
# Print only warnings and errors during test # Print only warnings and errors during test
config :logger, level: :warn config :logger, level: :warn
config :pleroma, :auth, oauth_consumer_strategies: []
config :pleroma, Pleroma.Upload, filters: [], link_name: false config :pleroma, Pleroma.Upload, filters: [], link_name: false
config :pleroma, Pleroma.Uploaders.Local, uploads: "test/uploads" config :pleroma, Pleroma.Uploaders.Local, uploads: "test/uploads"

1
docs/api/differences_in_mastoapi_responses.md
View file

@ -70,6 +70,7 @@ Additional parameters can be added to the JSON body/Form data:
- `preview`: boolean, if set to `true` the post won't be actually posted, but the status entitiy would still be rendered back. This could be useful for previewing rich text/custom emoji, for example. - `preview`: boolean, if set to `true` the post won't be actually posted, but the status entitiy would still be rendered back. This could be useful for previewing rich text/custom emoji, for example.
- `content_type`: string, contain the MIME type of the status, it is transformed into HTML by the backend. You can get the list of the supported MIME types with the nodeinfo endpoint. - `content_type`: string, contain the MIME type of the status, it is transformed into HTML by the backend. You can get the list of the supported MIME types with the nodeinfo endpoint.
- `to`: A list of nicknames (like `lain@soykaf.club` or `lain` on the local server) that will be used to determine who is going to be addressed by this post. Using this will disable the implicit addressing by mentioned names in the `status` body, only the people in the `to` list will be addressed. The normal rules for for post visibility are not affected by this and will still apply.
## PATCH `/api/v1/update_credentials` ## PATCH `/api/v1/update_credentials`

29
docs/config.md
View file

@ -108,10 +108,11 @@ config :pleroma, Pleroma.Emails.Mailer,
* `welcome_message`: A message that will be send to a newly registered users as a direct message. * `welcome_message`: A message that will be send to a newly registered users as a direct message.
* `welcome_user_nickname`: The nickname of the local user that sends the welcome message. * `welcome_user_nickname`: The nickname of the local user that sends the welcome message.
* `max_report_comment_size`: The maximum size of the report comment (Default: `1000`) * `max_report_comment_size`: The maximum size of the report comment (Default: `1000`)
* `safe_dm_mentions`: If set to true, only mentions at the beginning of a post will be used to address people in direct messages. This is to prevent accidental mentioning of people when talking about them (e.g. "@friend hey i really don't like @enemy"). (Default: `false`) * `safe_dm_mentions`: If set to true, only mentions at the beginning of a post will be used to address people in direct messages. This is to prevent accidental mentioning of people when talking about them (e.g. "@friend hey i really don't like @enemy"). Default: `false`.
* `healthcheck`: if set to true, system data will be shown on ``/api/pleroma/healthcheck``. * `healthcheck`: If set to true, system data will be shown on ``/api/pleroma/healthcheck``.
* `remote_post_retention_days`: the default amount of days to retain remote posts when pruning the database * `remote_post_retention_days`: The default amount of days to retain remote posts when pruning the database.
* `skip_thread_containment`: Skip filter out broken threads. the default is `false`. * `skip_thread_containment`: Skip filter out broken threads. The default is `false`.
* `limit_unauthenticated_to_local_content`: Limit unauthenticated users to search for local statutes and users only. The default is `true`.
## :app_account_creation ## :app_account_creation
REST API for creating an account settings REST API for creating an account settings
@ -514,7 +515,7 @@ Authentication / authorization settings.
* `auth_template`: authentication form template. By default it's `show.html` which corresponds to `lib/pleroma/web/templates/o_auth/o_auth/show.html.eex`. * `auth_template`: authentication form template. By default it's `show.html` which corresponds to `lib/pleroma/web/templates/o_auth/o_auth/show.html.eex`.
* `oauth_consumer_template`: OAuth consumer mode authentication form template. By default it's `consumer.html` which corresponds to `lib/pleroma/web/templates/o_auth/o_auth/consumer.html.eex`. * `oauth_consumer_template`: OAuth consumer mode authentication form template. By default it's `consumer.html` which corresponds to `lib/pleroma/web/templates/o_auth/o_auth/consumer.html.eex`.
* `oauth_consumer_strategies`: the list of enabled OAuth consumer strategies; by default it's set by OAUTH_CONSUMER_STRATEGIES environment variable. Each entry in this space-delimited string should be of format `<strategy>` or `<strategy>:<dependency>` (e.g. `twitter` or `keycloak:ueberauth_keycloak_strategy` in case dependency is named differently than `ueberauth_<strategy>`). * `oauth_consumer_strategies`: the list of enabled OAuth consumer strategies; by default it's set by `OAUTH_CONSUMER_STRATEGIES` environment variable. Each entry in this space-delimited string should be of format `<strategy>` or `<strategy>:<dependency>` (e.g. `twitter` or `keycloak:ueberauth_keycloak_strategy` in case dependency is named differently than `ueberauth_<strategy>`).
## :email_notifications ## :email_notifications
@ -579,6 +580,24 @@ config :ueberauth, Ueberauth,
providers: [ providers: [
microsoft: {Ueberauth.Strategy.Microsoft, [callback_params: []]} microsoft: {Ueberauth.Strategy.Microsoft, [callback_params: []]}
] ]
# Keycloak
# Note: make sure to add `keycloak:ueberauth_keycloak_strategy` entry to `OAUTH_CONSUMER_STRATEGIES` environment variable
keycloak_url = "https://publicly-reachable-keycloak-instance.org:8080"
config :ueberauth, Ueberauth.Strategy.Keycloak.OAuth,
client_id: System.get_env("KEYCLOAK_CLIENT_ID"),
client_secret: System.get_env("KEYCLOAK_CLIENT_SECRET"),
site: keycloak_url,
authorize_url: "#{keycloak_url}/auth/realms/master/protocol/openid-connect/auth",
token_url: "#{keycloak_url}/auth/realms/master/protocol/openid-connect/token",
userinfo_url: "#{keycloak_url}/auth/realms/master/protocol/openid-connect/userinfo",
token_method: :post
config :ueberauth, Ueberauth,
providers: [
keycloak: {Ueberauth.Strategy.Keycloak, [uid_field: :email]}
]
``` ```
## OAuth 2.0 provider - :oauth2 ## OAuth 2.0 provider - :oauth2

2
lib/mix/tasks/benchmark.ex
View file

@ -7,7 +7,7 @@ def run(["search"]) do
Benchee.run(%{ Benchee.run(%{
"search" => fn -> "search" => fn ->
Pleroma.Web.MastodonAPI.MastodonAPIController.status_search(nil, "cofe") Pleroma.Activity.search(nil, "cofe")
end end
}) })
end end

2
lib/pleroma/activity.ex
View file

@ -343,4 +343,6 @@ def restrict_deactivated_users(query) do
) )
) )
end end
defdelegate search(user, query), to: Pleroma.Activity.Search
end end

81
lib/pleroma/activity/search.ex Normal file
View file

@ -0,0 +1,81 @@
# Pleroma: A lightweight social networking server
# Copyright © 2017-2019 Pleroma Authors <https://pleroma.social/>
# SPDX-License-Identifier: AGPL-3.0-only
defmodule Pleroma.Activity.Search do
alias Pleroma.Activity
alias Pleroma.Object.Fetcher
alias Pleroma.Repo
alias Pleroma.User
alias Pleroma.Web.ActivityPub.Visibility
import Ecto.Query
def search(user, search_query) do
index_type = if Pleroma.Config.get([:database, :rum_enabled]), do: :rum, else: :gin
Activity
|> Activity.with_preloaded_object()
|> Activity.restrict_deactivated_users()
|> restrict_public()
|> query_with(index_type, search_query)
|> maybe_restrict_local(user)
|> Repo.all()
|> maybe_fetch(user, search_query)
end
defp restrict_public(q) do
from([a, o] in q,
where: fragment("?->>'type' = 'Create'", a.data),
where: "https://www.w3.org/ns/activitystreams#Public" in a.recipients,
limit: 40
)
end
defp query_with(q, :gin, search_query) do
from([a, o] in q,
where:
fragment(
"to_tsvector('english', ?->>'content') @@ plainto_tsquery('english', ?)",
o.data,
^search_query
),
order_by: [desc: :id]
)
end
defp query_with(q, :rum, search_query) do
from([a, o] in q,
where:
fragment(
"? @@ plainto_tsquery('english', ?)",
o.fts_content,
^search_query
),
order_by: [fragment("? <=> now()::date", o.inserted_at)]
)
end
# users can search everything
defp maybe_restrict_local(q, %User{}), do: q
# unauthenticated users can only search local activities
defp maybe_restrict_local(q, _) do
if Pleroma.Config.get([:instance, :limit_unauthenticated_to_local_content], true) do
where(q, local: true)
else
q
end
end
defp maybe_fetch(activities, user, search_query) do
with true <- Regex.match?(~r/https?:/, search_query),
{:ok, object} <- Fetcher.fetch_object_from_id(search_query),
%Activity{} = activity <- Activity.get_create_by_object_ap_id(object.data["id"]),
true <- Visibility.visible_for_user?(activity, user) do
activities ++ [activity]
else
_ -> activities
end
end
end

132
lib/pleroma/application.ex
View file

@ -4,7 +4,6 @@
defmodule Pleroma.Application do defmodule Pleroma.Application do
use Application use Application
import Supervisor.Spec
@name Mix.Project.config()[:name] @name Mix.Project.config()[:name]
@version Mix.Project.config()[:version] @version Mix.Project.config()[:version]
@ -31,21 +30,25 @@ def start(_type, _args) do
children = children =
[ [
# Start the Ecto repository # Start the Ecto repository
supervisor(Pleroma.Repo, []), %{id: Pleroma.Repo, start: {Pleroma.Repo, :start_link, []}, type: :supervisor},
worker(Pleroma.Emoji, []), %{id: Pleroma.Emoji, start: {Pleroma.Emoji, :start_link, []}},
worker(Pleroma.Captcha, []), %{id: Pleroma.Captcha, start: {Pleroma.Captcha, :start_link, []}},
worker( %{
Cachex, id: :cachex_used_captcha_cache,
start:
{Cachex, :start_link,
[ [
:used_captcha_cache, :used_captcha_cache,
[ [
ttl_interval: :timer.seconds(Pleroma.Config.get!([Pleroma.Captcha, :seconds_valid])) ttl_interval:
:timer.seconds(Pleroma.Config.get!([Pleroma.Captcha, :seconds_valid]))
] ]
], ]}
id: :cachex_used_captcha_cache },
), %{
worker( id: :cachex_user,
Cachex, start:
{Cachex, :start_link,
[ [
:user_cache, :user_cache,
[ [
@ -53,11 +56,12 @@ def start(_type, _args) do
ttl_interval: 1000, ttl_interval: 1000,
limit: 2500 limit: 2500
] ]
], ]}
id: :cachex_user },
), %{
worker( id: :cachex_object,
Cachex, start:
{Cachex, :start_link,
[ [
:object_cache, :object_cache,
[ [
@ -65,32 +69,35 @@ def start(_type, _args) do
ttl_interval: 1000, ttl_interval: 1000,
limit: 2500 limit: 2500
] ]
], ]}
id: :cachex_object },
), %{
worker( id: :cachex_rich_media,
Cachex, start:
{Cachex, :start_link,
[ [
:rich_media_cache, :rich_media_cache,
[ [
default_ttl: :timer.minutes(120), default_ttl: :timer.minutes(120),
limit: 5000 limit: 5000
] ]
], ]}
id: :cachex_rich_media },
), %{
worker( id: :cachex_scrubber,
Cachex, start:
{Cachex, :start_link,
[ [
:scrubber_cache, :scrubber_cache,
[ [
limit: 2500 limit: 2500
] ]
], ]}
id: :cachex_scrubber },
), %{
worker( id: :cachex_idem,
Cachex, start:
{Cachex, :start_link,
[ [
:idempotency_cache, :idempotency_cache,
[ [
@ -101,27 +108,53 @@ def start(_type, _args) do
), ),
limit: 2500 limit: 2500
] ]
], ]}
id: :cachex_idem },
), %{id: Pleroma.FlakeId, start: {Pleroma.FlakeId, :start_link, []}},
worker(Pleroma.FlakeId, []), %{
worker(Pleroma.ScheduledActivityWorker, []), id: Pleroma.ScheduledActivityWorker,
worker(Pleroma.QuantumScheduler, []) start: {Pleroma.ScheduledActivityWorker, :start_link, []}
},
%{
id: Pleroma.QuantumScheduler,
start: {Pleroma.QuantumScheduler, :start_link, []}
}
] ++ ] ++
hackney_pool_children() ++ hackney_pool_children() ++
[ [
worker(Pleroma.Web.Federator.RetryQueue, []), %{
worker(Pleroma.Web.OAuth.Token.CleanWorker, []), id: Pleroma.Web.Federator.RetryQueue,
worker(Pleroma.Stats, []), start: {Pleroma.Web.Federator.RetryQueue, :start_link, []}
worker(Task, [&Pleroma.Web.Push.init/0], restart: :temporary, id: :web_push_init), },
worker(Task, [&Pleroma.Web.Federator.init/0], restart: :temporary, id: :federator_init) %{
id: Pleroma.Web.OAuth.Token.CleanWorker,
start: {Pleroma.Web.OAuth.Token.CleanWorker, :start_link, []}
},
%{
id: Pleroma.Stats,
start: {Pleroma.Stats, :start_link, []}
},
%{
id: :web_push_init,
start: {Task, :start_link, [&Pleroma.Web.Push.init/0]},
restart: :temporary
},
%{
id: :federator_init,
start: {Task, :start_link, [&Pleroma.Web.Federator.init/0]},
restart: :temporary
}
] ++ ] ++
streamer_child() ++ streamer_child() ++
chat_child() ++ chat_child() ++
[ [
# Start the endpoint when the application starts # Start the endpoint when the application starts
supervisor(Pleroma.Web.Endpoint, []), %{
worker(Pleroma.Gopher.Server, []) id: Pleroma.Web.Endpoint,
start: {Pleroma.Web.Endpoint, :start_link, []},
type: :supervisor
},
%{id: Pleroma.Gopher.Server, start: {Pleroma.Gopher.Server, :start_link, []}}
] ]
# See http://elixir-lang.org/docs/stable/elixir/Supervisor.html # See http://elixir-lang.org/docs/stable/elixir/Supervisor.html
@ -172,12 +205,17 @@ defp streamer_child, do: []
defp chat_child, do: [] defp chat_child, do: []
else else
defp streamer_child do defp streamer_child do
[worker(Pleroma.Web.Streamer, [])] [%{id: Pleroma.Web.Streamer, start: {Pleroma.Web.Streamer, :start_link, []}}]
end end
defp chat_child do defp chat_child do
if Pleroma.Config.get([:chat, :enabled]) do if Pleroma.Config.get([:chat, :enabled]) do
[worker(Pleroma.Web.ChatChannel.ChatChannelState, [])] [
%{
id: Pleroma.Web.ChatChannel.ChatChannelState,
start: {Pleroma.Web.ChatChannel.ChatChannelState, :start_link, []}
}
]
else else
[] []
end end

138
lib/pleroma/user.ex
View file

@ -325,14 +325,6 @@ def maybe_direct_follow(%User{} = follower, %User{} = followed) do
end end
end end
def maybe_follow(%User{} = follower, %User{info: _info} = followed) do
if not following?(follower, followed) do
follow(follower, followed)
else
{:ok, follower}
end
end
@doc "A mass follow for local users. Respects blocks in both directions but does not create activities." @doc "A mass follow for local users. Respects blocks in both directions but does not create activities."
@spec follow_all(User.t(), list(User.t())) :: {atom(), User.t()} @spec follow_all(User.t(), list(User.t())) :: {atom(), User.t()}
def follow_all(follower, followeds) do def follow_all(follower, followeds) do
@ -371,8 +363,8 @@ def follow(%User{} = follower, %User{info: info} = followed) do
ap_followers = followed.follower_address ap_followers = followed.follower_address
cond do cond do
following?(follower, followed) or info.deactivated -> info.deactivated ->
{:error, "Could not follow user: #{followed.nickname} is already on your list."} {:error, "Could not follow user: You are deactivated."}
deny_follow_blocked and blocks?(followed, follower) -> deny_follow_blocked and blocks?(followed, follower) ->
{:error, "Could not follow user: #{followed.nickname} blocked you."} {:error, "Could not follow user: #{followed.nickname} blocked you."}
@ -736,122 +728,6 @@ def get_recipients_from_activity(%Activity{recipients: to}) do
|> Repo.all() |> Repo.all()
end end
def search(query, resolve \\ false, for_user \\ nil) do
# Strip the beginning @ off if there is a query
query = String.trim_leading(query, "@")
if resolve, do: get_or_fetch(query)
{:ok, results} =
Repo.transaction(fn ->
Ecto.Adapters.SQL.query(Repo, "select set_limit(0.25)", [])
Repo.all(search_query(query, for_user))
end)
results
end
def search_query(query, for_user) do
fts_subquery = fts_search_subquery(query)
trigram_subquery = trigram_search_subquery(query)
union_query = from(s in trigram_subquery, union_all: ^fts_subquery)
distinct_query = from(s in subquery(union_query), order_by: s.search_type, distinct: s.id)
from(s in subquery(boost_search_rank_query(distinct_query, for_user)),
order_by: [desc: s.search_rank],
limit: 40
)
end
defp boost_search_rank_query(query, nil), do: query
defp boost_search_rank_query(query, for_user) do
friends_ids = get_friends_ids(for_user)
followers_ids = get_followers_ids(for_user)
from(u in subquery(query),
select_merge: %{
search_rank:
fragment(
"""
CASE WHEN (?) THEN (?) * 1.3
WHEN (?) THEN (?) * 1.2
WHEN (?) THEN (?) * 1.1
ELSE (?) END
""",
u.id in ^friends_ids and u.id in ^followers_ids,
u.search_rank,
u.id in ^friends_ids,
u.search_rank,
u.id in ^followers_ids,
u.search_rank,
u.search_rank
)
}
)
end
defp fts_search_subquery(term, query \\ User) do
processed_query =
term
|> String.replace(~r/\W+/, " ")
|> String.trim()
|> String.split()
|> Enum.map(&(&1 <> ":*"))
|> Enum.join(" | ")
from(
u in query,
select_merge: %{
search_type: ^0,
search_rank:
fragment(
"""
ts_rank_cd(
setweight(to_tsvector('simple', regexp_replace(?, '\\W', ' ', 'g')), 'A') ||
setweight(to_tsvector('simple', regexp_replace(coalesce(?, ''), '\\W', ' ', 'g')), 'B'),
to_tsquery('simple', ?),
32
)
""",
u.nickname,
u.name,
^processed_query
)
},
where:
fragment(
"""
(setweight(to_tsvector('simple', regexp_replace(?, '\\W', ' ', 'g')), 'A') ||
setweight(to_tsvector('simple', regexp_replace(coalesce(?, ''), '\\W', ' ', 'g')), 'B')) @@ to_tsquery('simple', ?)
""",
u.nickname,
u.name,
^processed_query
)
)
|> restrict_deactivated()
end
defp trigram_search_subquery(term) do
from(
u in User,
select_merge: %{
# ^1 gives 'Postgrex expected a binary, got 1' for some weird reason
search_type: fragment("?", 1),
search_rank:
fragment(
"similarity(?, trim(? || ' ' || coalesce(?, '')))",
^term,
u.nickname,
u.name
)
},
where: fragment("trim(? || ' ' || coalesce(?, '')) % ?", u.nickname, u.name, ^term)
)
|> restrict_deactivated()
end
def mute(muter, %User{ap_id: ap_id}) do def mute(muter, %User{ap_id: ap_id}) do
info_cng = info_cng =
muter.info muter.info
@ -1516,4 +1392,14 @@ def ensure_keys_present(user) do
update_and_set_cache(cng) update_and_set_cache(cng)
end end
end end
def get_ap_ids_by_nicknames(nicknames) do
from(u in User,
where: u.nickname in ^nicknames,
select: u.ap_id
)
|> Repo.all()
end
defdelegate search(query, opts \\ []), to: User.Search
end end

166
lib/pleroma/user/search.ex Normal file
View file

@ -0,0 +1,166 @@
# Pleroma: A lightweight social networking server
# Copyright © 2017-2019 Pleroma Authors <https://pleroma.social/>
# SPDX-License-Identifier: AGPL-3.0-only
defmodule Pleroma.User.Search do
alias Pleroma.Repo
alias Pleroma.User
import Ecto.Query
def search(query, opts \\ []) do
resolve = Keyword.get(opts, :resolve, false)
for_user = Keyword.get(opts, :for_user)
# Strip the beginning @ off if there is a query
query = String.trim_leading(query, "@")
maybe_resolve(resolve, for_user, query)
{:ok, results} =
Repo.transaction(fn ->
Ecto.Adapters.SQL.query(Repo, "select set_limit(0.25)", [])
query
|> search_query(for_user)
|> Repo.all()
end)
results
end
defp maybe_resolve(true, %User{}, query) do
User.get_or_fetch(query)
end
defp maybe_resolve(true, _, query) do
unless restrict_local?(), do: User.get_or_fetch(query)
end
defp maybe_resolve(_, _, _), do: :noop
defp search_query(query, for_user) do
query
|> union_query()
|> distinct_query()
|> boost_search_rank_query(for_user)
|> subquery()
|> order_by(desc: :search_rank)
|> limit(20)
|> maybe_restrict_local(for_user)
end
defp restrict_local? do
Pleroma.Config.get([:instance, :limit_unauthenticated_to_local_content], true)
end
defp union_query(query) do
fts_subquery = fts_search_subquery(query)
trigram_subquery = trigram_search_subquery(query)
from(s in trigram_subquery, union_all: ^fts_subquery)
end
defp distinct_query(q) do
from(s in subquery(q), order_by: s.search_type, distinct: s.id)
end
# unauthenticated users can only search local activities
defp maybe_restrict_local(q, %User{}), do: q
defp maybe_restrict_local(q, _) do
if restrict_local?() do
where(q, [u], u.local == true)
else
q
end
end
defp boost_search_rank_query(query, nil), do: query
defp boost_search_rank_query(query, for_user) do
friends_ids = User.get_friends_ids(for_user)
followers_ids = User.get_followers_ids(for_user)
from(u in subquery(query),
select_merge: %{
search_rank:
fragment(
"""
CASE WHEN (?) THEN 0.5 + (?) * 1.3
WHEN (?) THEN 0.5 + (?) * 1.2
WHEN (?) THEN (?) * 1.1
ELSE (?) END
""",
u.id in ^friends_ids and u.id in ^followers_ids,
u.search_rank,
u.id in ^friends_ids,
u.search_rank,
u.id in ^followers_ids,
u.search_rank,
u.search_rank
)
}
)
end
defp fts_search_subquery(term, query \\ User) do
processed_query =
term
|> String.replace(~r/\W+/, " ")
|> String.trim()
|> String.split()
|> Enum.map(&(&1 <> ":*"))
|> Enum.join(" | ")
from(
u in query,
select_merge: %{
search_type: ^0,
search_rank:
fragment(
"""
ts_rank_cd(
setweight(to_tsvector('simple', regexp_replace(?, '\\W', ' ', 'g')), 'A') ||
setweight(to_tsvector('simple', regexp_replace(coalesce(?, ''), '\\W', ' ', 'g')), 'B'),
to_tsquery('simple', ?),
32
)
""",
u.nickname,
u.name,
^processed_query
)
},
where:
fragment(
"""
(setweight(to_tsvector('simple', regexp_replace(?, '\\W', ' ', 'g')), 'A') ||
setweight(to_tsvector('simple', regexp_replace(coalesce(?, ''), '\\W', ' ', 'g')), 'B')) @@ to_tsquery('simple', ?)
""",
u.nickname,
u.name,
^processed_query
)
)
|> User.restrict_deactivated()
end
defp trigram_search_subquery(term) do
from(
u in User,
select_merge: %{
# ^1 gives 'Postgrex expected a binary, got 1' for some weird reason
search_type: fragment("?", 1),
search_rank:
fragment(
"similarity(?, trim(? || ' ' || coalesce(?, '')))",
^term,
u.nickname,
u.name
)
},
where: fragment("trim(? || ' ' || coalesce(?, '')) % ?", u.nickname, u.name, ^term)
)
|> User.restrict_deactivated()
end
end

25
lib/pleroma/web/activity_pub/transmogrifier.ex
View file

@ -458,10 +458,12 @@ def handle_incoming(
{:ok, %User{} = follower} <- User.get_or_fetch_by_ap_id(follower), {:ok, %User{} = follower} <- User.get_or_fetch_by_ap_id(follower),
{:ok, activity} <- ActivityPub.follow(follower, followed, id, false) do {:ok, activity} <- ActivityPub.follow(follower, followed, id, false) do
with deny_follow_blocked <- Pleroma.Config.get([:user, :deny_follow_blocked]), with deny_follow_blocked <- Pleroma.Config.get([:user, :deny_follow_blocked]),
{:user_blocked, false} <- {_, false} <-
{:user_blocked, User.blocks?(followed, follower) && deny_follow_blocked}, {:user_blocked, User.blocks?(followed, follower) && deny_follow_blocked},
{:user_locked, false} <- {:user_locked, User.locked?(followed)}, {_, false} <- {:user_locked, User.locked?(followed)},
{:follow, {:ok, follower}} <- {:follow, User.follow(follower, followed)} do {_, {:ok, follower}} <- {:follow, User.follow(follower, followed)},
{_, {:ok, _}} <-
{:follow_state_update, Utils.update_follow_state_for_all(activity, "accept")} do
ActivityPub.accept(%{ ActivityPub.accept(%{
to: [follower.ap_id], to: [follower.ap_id],
actor: followed, actor: followed,
@ -470,7 +472,7 @@ def handle_incoming(
}) })
else else
{:user_blocked, true} -> {:user_blocked, true} ->
{:ok, _} = Utils.update_follow_state(activity, "reject") {:ok, _} = Utils.update_follow_state_for_all(activity, "reject")
ActivityPub.reject(%{ ActivityPub.reject(%{
to: [follower.ap_id], to: [follower.ap_id],
@ -480,7 +482,7 @@ def handle_incoming(
}) })
{:follow, {:error, _}} -> {:follow, {:error, _}} ->
{:ok, _} = Utils.update_follow_state(activity, "reject") {:ok, _} = Utils.update_follow_state_for_all(activity, "reject")
ActivityPub.reject(%{ ActivityPub.reject(%{
to: [follower.ap_id], to: [follower.ap_id],
@ -506,21 +508,16 @@ def handle_incoming(
with actor <- Containment.get_actor(data), with actor <- Containment.get_actor(data),
{:ok, %User{} = followed} <- User.get_or_fetch_by_ap_id(actor), {:ok, %User{} = followed} <- User.get_or_fetch_by_ap_id(actor),
{:ok, follow_activity} <- get_follow_activity(follow_object, followed), {:ok, follow_activity} <- get_follow_activity(follow_object, followed),
{:ok, follow_activity} <- Utils.update_follow_state(follow_activity, "accept"), {:ok, follow_activity} <- Utils.update_follow_state_for_all(follow_activity, "accept"),
%User{local: true} = follower <- User.get_cached_by_ap_id(follow_activity.data["actor"]), %User{local: true} = follower <- User.get_cached_by_ap_id(follow_activity.data["actor"]),
{:ok, activity} <- {:ok, _follower} = User.follow(follower, followed) do
ActivityPub.accept(%{ ActivityPub.accept(%{
to: follow_activity.data["to"], to: follow_activity.data["to"],
type: "Accept", type: "Accept",
actor: followed, actor: followed,
object: follow_activity.data["id"], object: follow_activity.data["id"],
local: false local: false
}) do })
if not User.following?(follower, followed) do
{:ok, _follower} = User.follow(follower, followed)
end
{:ok, activity}
else else
_e -> :error _e -> :error
end end
@ -532,7 +529,7 @@ def handle_incoming(
with actor <- Containment.get_actor(data), with actor <- Containment.get_actor(data),
{:ok, %User{} = followed} <- User.get_or_fetch_by_ap_id(actor), {:ok, %User{} = followed} <- User.get_or_fetch_by_ap_id(actor),
{:ok, follow_activity} <- get_follow_activity(follow_object, followed), {:ok, follow_activity} <- get_follow_activity(follow_object, followed),
{:ok, follow_activity} <- Utils.update_follow_state(follow_activity, "reject"), {:ok, follow_activity} <- Utils.update_follow_state_for_all(follow_activity, "reject"),
%User{local: true} = follower <- User.get_cached_by_ap_id(follow_activity.data["actor"]), %User{local: true} = follower <- User.get_cached_by_ap_id(follow_activity.data["actor"]),
{:ok, activity} <- {:ok, activity} <-
ActivityPub.reject(%{ ActivityPub.reject(%{

4
lib/pleroma/web/activity_pub/utils.ex
View file

@ -376,8 +376,8 @@ def remove_like_from_object(%Activity{data: %{"actor" => actor}}, object) do
@doc """ @doc """
Updates a follow activity's state (for locked accounts). Updates a follow activity's state (for locked accounts).
""" """
def update_follow_state( def update_follow_state_for_all(
%Activity{data: %{"actor" => actor, "object" => object, "state" => "pending"}} = activity, %Activity{data: %{"actor" => actor, "object" => object}} = activity,
state state
) do ) do
try do try do

11
lib/pleroma/web/auth/pleroma_authenticator.ex
View file

@ -24,6 +24,14 @@ def get_user(%Plug.Conn{} = conn) do
end end
end end
@doc """
Gets or creates Pleroma.Registration record from Ueberauth assigns.
Note: some strategies (like `keycloak`) might need extra configuration to fill `uid` from callback response —
see [`docs/config.md`](docs/config.md).
"""
def get_registration(%Plug.Conn{assigns: %{ueberauth_auth: %{uid: nil}}}),
do: {:error, :missing_uid}
def get_registration(%Plug.Conn{ def get_registration(%Plug.Conn{
assigns: %{ueberauth_auth: %{provider: provider, uid: uid} = auth} assigns: %{ueberauth_auth: %{provider: provider, uid: uid} = auth}
}) do }) do
@ -51,9 +59,10 @@ def get_registration(%Plug.Conn{
def get_registration(%Plug.Conn{} = _conn), do: {:error, :missing_credentials} def get_registration(%Plug.Conn{} = _conn), do: {:error, :missing_credentials}
@doc "Creates Pleroma.User record basing on params and Pleroma.Registration record."
def create_from_registration( def create_from_registration(
%Plug.Conn{params: %{"authorization" => registration_attrs}}, %Plug.Conn{params: %{"authorization" => registration_attrs}},
registration %Registration{} = registration
) do ) do
nickname = value([registration_attrs["nickname"], Registration.nickname(registration)]) nickname = value([registration_attrs["nickname"], Registration.nickname(registration)])
email = value([registration_attrs["email"], Registration.email(registration)]) email = value([registration_attrs["email"], Registration.email(registration)])

10
lib/pleroma/web/common_api/common_api.ex
View file

@ -35,9 +35,9 @@ def unfollow(follower, unfollowed) do
end end
def accept_follow_request(follower, followed) do def accept_follow_request(follower, followed) do
with {:ok, follower} <- User.maybe_follow(follower, followed), with {:ok, follower} <- User.follow(follower, followed),
%Activity{} = follow_activity <- Utils.fetch_latest_follow(follower, followed), %Activity{} = follow_activity <- Utils.fetch_latest_follow(follower, followed),
{:ok, follow_activity} <- Utils.update_follow_state(follow_activity, "accept"), {:ok, follow_activity} <- Utils.update_follow_state_for_all(follow_activity, "accept"),
{:ok, _activity} <- {:ok, _activity} <-
ActivityPub.accept(%{ ActivityPub.accept(%{
to: [follower.ap_id], to: [follower.ap_id],
@ -51,7 +51,7 @@ def accept_follow_request(follower, followed) do
def reject_follow_request(follower, followed) do def reject_follow_request(follower, followed) do
with %Activity{} = follow_activity <- Utils.fetch_latest_follow(follower, followed), with %Activity{} = follow_activity <- Utils.fetch_latest_follow(follower, followed),
{:ok, follow_activity} <- Utils.update_follow_state(follow_activity, "reject"), {:ok, follow_activity} <- Utils.update_follow_state_for_all(follow_activity, "reject"),
{:ok, _activity} <- {:ok, _activity} <-
ActivityPub.reject(%{ ActivityPub.reject(%{
to: [follower.ap_id], to: [follower.ap_id],
@ -204,8 +204,10 @@ def post(user, %{"status" => status} = data) do
data, data,
visibility visibility
), ),
mentioned_users <- for({_, mentioned_user} <- mentions, do: mentioned_user.ap_id),
addressed_users <- get_addressed_users(mentioned_users, data["to"]),
{poll, poll_emoji} <- make_poll_data(data), {poll, poll_emoji} <- make_poll_data(data),
{to, cc} <- to_for_user_and_mentions(user, mentions, in_reply_to, visibility), {to, cc} <- get_to_and_cc(user, addressed_users, in_reply_to, visibility),
context <- make_context(in_reply_to), context <- make_context(in_reply_to),
cw <- data["spoiler_text"] || "", cw <- data["spoiler_text"] || "",
sensitive <- data["sensitive"] || Enum.member?(tags, {"#nsfw", "nsfw"}), sensitive <- data["sensitive"] || Enum.member?(tags, {"#nsfw", "nsfw"}),

24
lib/pleroma/web/common_api/utils.ex
View file

@ -61,9 +61,9 @@ def attachments_from_ids_descs(ids, descs_str) do
end) end)
end end
def to_for_user_and_mentions(user, mentions, inReplyTo, "public") do @spec get_to_and_cc(User.t(), list(String.t()), Activity.t() | nil, String.t()) ::
mentioned_users = Enum.map(mentions, fn {_, %{ap_id: ap_id}} -> ap_id end) {list(String.t()), list(String.t())}
def get_to_and_cc(user, mentioned_users, inReplyTo, "public") do
to = ["https://www.w3.org/ns/activitystreams#Public" | mentioned_users] to = ["https://www.w3.org/ns/activitystreams#Public" | mentioned_users]
cc = [user.follower_address] cc = [user.follower_address]
@ -74,9 +74,7 @@ def to_for_user_and_mentions(user, mentions, inReplyTo, "public") do
end end
end end
def to_for_user_and_mentions(user, mentions, inReplyTo, "unlisted") do def get_to_and_cc(user, mentioned_users, inReplyTo, "unlisted") do
mentioned_users = Enum.map(mentions, fn {_, %{ap_id: ap_id}} -> ap_id end)
to = [user.follower_address | mentioned_users] to = [user.follower_address | mentioned_users]
cc = ["https://www.w3.org/ns/activitystreams#Public"] cc = ["https://www.w3.org/ns/activitystreams#Public"]
@ -87,14 +85,12 @@ def to_for_user_and_mentions(user, mentions, inReplyTo, "unlisted") do
end end
end end
def to_for_user_and_mentions(user, mentions, inReplyTo, "private") do def get_to_and_cc(user, mentioned_users, inReplyTo, "private") do
{to, cc} = to_for_user_and_mentions(user, mentions, inReplyTo, "direct") {to, cc} = get_to_and_cc(user, mentioned_users, inReplyTo, "direct")
{[user.follower_address | to], cc} {[user.follower_address | to], cc}
end end
def to_for_user_and_mentions(_user, mentions, inReplyTo, "direct") do def get_to_and_cc(_user, mentioned_users, inReplyTo, "direct") do
mentioned_users = Enum.map(mentions, fn {_, %{ap_id: ap_id}} -> ap_id end)
if inReplyTo do if inReplyTo do
{Enum.uniq([inReplyTo.data["actor"] | mentioned_users]), []} {Enum.uniq([inReplyTo.data["actor"] | mentioned_users]), []}
else else
@ -102,6 +98,12 @@ def to_for_user_and_mentions(_user, mentions, inReplyTo, "direct") do
end end
end end
def get_addressed_users(_, to) when is_list(to) do
User.get_ap_ids_by_nicknames(to)
end
def get_addressed_users(mentioned_users, _), do: mentioned_users
def make_poll_data(%{"poll" => %{"options" => options, "expires_in" => expires_in}} = data) def make_poll_data(%{"poll" => %{"options" => options, "expires_in" => expires_in}} = data)
when is_list(options) do when is_list(options) do
%{max_expiration: max_expiration, min_expiration: min_expiration} = %{max_expiration: max_expiration, min_expiration: min_expiration} =

61
lib/pleroma/web/mastodon_api/mastodon_api_controller.ex
View file

@ -14,7 +14,6 @@ defmodule Pleroma.Web.MastodonAPI.MastodonAPIController do
alias Pleroma.HTTP alias Pleroma.HTTP
alias Pleroma.Notification alias Pleroma.Notification
alias Pleroma.Object alias Pleroma.Object
alias Pleroma.Object.Fetcher
alias Pleroma.Pagination alias Pleroma.Pagination
alias Pleroma.Repo alias Pleroma.Repo
alias Pleroma.ScheduledActivity alias Pleroma.ScheduledActivity
@ -1125,64 +1124,9 @@ def unsubscribe(%{assigns: %{user: user}} = conn, %{"id" => id}) do
end end
end end
def status_search_query_with_gin(q, query) do
from([a, o] in q,
where:
fragment(
"to_tsvector('english', ?->>'content') @@ plainto_tsquery('english', ?)",
o.data,
^query
),
order_by: [desc: :id]
)
end
def status_search_query_with_rum(q, query) do
from([a, o] in q,
where:
fragment(
"? @@ plainto_tsquery('english', ?)",
o.fts_content,
^query
),
order_by: [fragment("? <=> now()::date", o.inserted_at)]
)
end
def status_search(user, query) do
fetched =
if Regex.match?(~r/https?:/, query) do
with {:ok, object} <- Fetcher.fetch_object_from_id(query),
%Activity{} = activity <- Activity.get_create_by_object_ap_id(object.data["id"]),
true <- Visibility.visible_for_user?(activity, user) do
[activity]
else
_e -> []
end
end || []
q =
from([a, o] in Activity.with_preloaded_object(Activity),
where: fragment("?->>'type' = 'Create'", a.data),
where: "https://www.w3.org/ns/activitystreams#Public" in a.recipients,
limit: 40
)
q =
if Pleroma.Config.get([:database, :rum_enabled]) do
status_search_query_with_rum(q, query)
else
status_search_query_with_gin(q, query)
end
Repo.all(q) ++ fetched
end
def search2(%{assigns: %{user: user}} = conn, %{"q" => query} = params) do def search2(%{assigns: %{user: user}} = conn, %{"q" => query} = params) do
accounts = User.search(query, resolve: params["resolve"] == "true", for_user: user) accounts = User.search(query, resolve: params["resolve"] == "true", for_user: user)
statuses = Activity.search(user, query)
statuses = status_search(user, query)
tags_path = Web.base_url() <> "/tag/" tags_path = Web.base_url() <> "/tag/"
tags = tags =
@ -1205,8 +1149,7 @@ def search2(%{assigns: %{user: user}} = conn, %{"q" => query} = params) do
def search(%{assigns: %{user: user}} = conn, %{"q" => query} = params) do def search(%{assigns: %{user: user}} = conn, %{"q" => query} = params) do
accounts = User.search(query, resolve: params["resolve"] == "true", for_user: user) accounts = User.search(query, resolve: params["resolve"] == "true", for_user: user)
statuses = Activity.search(user, query)
statuses = status_search(user, query)
tags = tags =
query query

1
lib/pleroma/web/nodeinfo/nodeinfo_controller.ex
View file

@ -98,6 +98,7 @@ def raw_nodeinfo do
"mastodon_api", "mastodon_api",
"mastodon_api_streaming", "mastodon_api_streaming",
"polls", "polls",
"pleroma_explicit_addressing",
if Config.get([:media_proxy, :enabled]) do if Config.get([:media_proxy, :enabled]) do
"media_proxy" "media_proxy"
end, end,

6
lib/pleroma/web/oauth/oauth_controller.ex
View file

@ -17,6 +17,8 @@ defmodule Pleroma.Web.OAuth.OAuthController do
alias Pleroma.Web.OAuth.Token.Strategy.Revoke, as: RevokeToken alias Pleroma.Web.OAuth.Token.Strategy.Revoke, as: RevokeToken
alias Pleroma.Web.OAuth.Scopes alias Pleroma.Web.OAuth.Scopes
require Logger
if Pleroma.Config.oauth_consumer_enabled?(), do: plug(Ueberauth) if Pleroma.Config.oauth_consumer_enabled?(), do: plug(Ueberauth)
plug(:fetch_session) plug(:fetch_session)
@ -318,7 +320,9 @@ def callback(conn, params) do
|> registration_details(%{"authorization" => registration_params}) |> registration_details(%{"authorization" => registration_params})
end end
else else
_ -> error ->
Logger.debug(inspect(["OAUTH_ERROR", error, conn.assigns]))
conn conn
|> put_flash(:error, "Failed to set up user account.") |> put_flash(:error, "Failed to set up user account.")
|> redirect(external: redirect_uri(conn, params["redirect_uri"])) |> redirect(external: redirect_uri(conn, params["redirect_uri"]))

7
priv/repo/migrations/20190603115238_add_index_on_activities_local.exs Normal file
View file

@ -0,0 +1,7 @@
defmodule Pleroma.Repo.Migrations.AddIndexOnActivitiesLocal do
use Ecto.Migration
def change do
create(index("activities", [:local]))
end
end

54
test/activity_test.exs
View file

@ -99,4 +99,58 @@ test "when association is not loaded" do
assert Activity.get_bookmark(queried_activity, user) == bookmark assert Activity.get_bookmark(queried_activity, user) == bookmark
end end
end end
describe "search" do
setup do
Tesla.Mock.mock_global(fn env -> apply(HttpRequestMock, :request, [env]) end)
user = insert(:user)
params = %{
"@context" => "https://www.w3.org/ns/activitystreams",
"actor" => "http://mastodon.example.org/users/admin",
"type" => "Create",
"id" => "http://mastodon.example.org/users/admin/activities/1",
"object" => %{
"type" => "Note",
"content" => "find me!",
"id" => "http://mastodon.example.org/users/admin/objects/1",
"attributedTo" => "http://mastodon.example.org/users/admin"
},
"to" => ["https://www.w3.org/ns/activitystreams#Public"]
}
{:ok, local_activity} = Pleroma.Web.CommonAPI.post(user, %{"status" => "find me!"})
{:ok, remote_activity} = Pleroma.Web.Federator.incoming_ap_doc(params)
%{local_activity: local_activity, remote_activity: remote_activity, user: user}
end
test "find local and remote statuses for authenticated users", %{
local_activity: local_activity,
remote_activity: remote_activity,
user: user
} do
activities = Enum.sort_by(Activity.search(user, "find me"), & &1.id)
assert [^local_activity, ^remote_activity] = activities
end
test "find only local statuses for unauthenticated users", %{local_activity: local_activity} do
assert [^local_activity] = Activity.search(nil, "find me")
end
test "find all statuses for unauthenticated users when `limit_unauthenticated_to_local_content` is `false`",
%{
local_activity: local_activity,
remote_activity: remote_activity
} do
Pleroma.Config.put([:instance, :limit_unauthenticated_to_local_content], false)
activities = Enum.sort_by(Activity.search(nil, "find me"), & &1.id)
assert [^local_activity, ^remote_activity] = activities
Pleroma.Config.put([:instance, :limit_unauthenticated_to_local_content], true)
end
end
end end

21
test/tasks/user_test.exs
View file

@ -366,4 +366,25 @@ test "user is not confirmed" do
refute user.info.confirmation_token refute user.info.confirmation_token
end end
end end
describe "search" do
test "it returns users matching" do
user = insert(:user)
moon = insert(:user, nickname: "moon", name: "fediverse expert moon")
moot = insert(:user, nickname: "moot")
kawen = insert(:user, nickname: "kawen", name: "fediverse expert moon")
{:ok, user} = User.follow(user, kawen)
assert [moon.id, kawen.id] == User.Search.search("moon") |> Enum.map(& &1.id)
res = User.search("moo") |> Enum.map(& &1.id)
assert moon.id in res
assert moot.id in res
assert kawen.id in res
assert [moon.id, kawen.id] == User.Search.search("moon fediverse") |> Enum.map(& &1.id)
assert [kawen.id, moon.id] ==
User.Search.search("moon fediverse", for_user: user) |> Enum.map(& &1.id)
end
end
end end

65
test/user_test.exs
View file

@ -75,7 +75,7 @@ test "doesn't return already accepted or duplicate follow requests" do
Pleroma.Web.TwitterAPI.TwitterAPI.follow(pending_follower, %{"user_id" => locked.id}) Pleroma.Web.TwitterAPI.TwitterAPI.follow(pending_follower, %{"user_id" => locked.id})
Pleroma.Web.TwitterAPI.TwitterAPI.follow(pending_follower, %{"user_id" => locked.id}) Pleroma.Web.TwitterAPI.TwitterAPI.follow(pending_follower, %{"user_id" => locked.id})
Pleroma.Web.TwitterAPI.TwitterAPI.follow(accepted_follower, %{"user_id" => locked.id}) Pleroma.Web.TwitterAPI.TwitterAPI.follow(accepted_follower, %{"user_id" => locked.id})
User.maybe_follow(accepted_follower, locked) User.follow(accepted_follower, locked)
assert {:ok, [activity]} = User.get_follow_requests(locked) assert {:ok, [activity]} = User.get_follow_requests(locked)
assert activity assert activity
@ -1055,7 +1055,7 @@ test "finds users, ranking by similarity" do
u3 = insert(:user, %{name: "ebn", nickname: "lain@mastodon.social"}) u3 = insert(:user, %{name: "ebn", nickname: "lain@mastodon.social"})
u4 = insert(:user, %{nickname: "lain@pleroma.soykaf.com"}) u4 = insert(:user, %{nickname: "lain@pleroma.soykaf.com"})
assert [u4.id, u3.id, u1.id] == Enum.map(User.search("lain@ple"), & &1.id) assert [u4.id, u3.id, u1.id] == Enum.map(User.search("lain@ple", for_user: u1), & &1.id)
end end
test "finds users, handling misspelled requests" do test "finds users, handling misspelled requests" do
@ -1077,6 +1077,46 @@ test "finds users, boosting ranks of friends and followers" do
Enum.map(User.search("doe", resolve: false, for_user: u1), & &1.id) == [] Enum.map(User.search("doe", resolve: false, for_user: u1), & &1.id) == []
end end
test "find local and remote users for authenticated users" do
u1 = insert(:user, %{name: "lain"})
u2 = insert(:user, %{name: "ebn", nickname: "lain@mastodon.social", local: false})
u3 = insert(:user, %{nickname: "lain@pleroma.soykaf.com", local: false})
results =
"lain"
|> User.search(for_user: u1)
|> Enum.map(& &1.id)
|> Enum.sort()
assert [u1.id, u2.id, u3.id] == results
end
test "find only local users for unauthenticated users" do
%{id: id} = insert(:user, %{name: "lain"})
insert(:user, %{name: "ebn", nickname: "lain@mastodon.social", local: false})
insert(:user, %{nickname: "lain@pleroma.soykaf.com", local: false})
assert [%{id: ^id}] = User.search("lain")
end
test "find all users for unauthenticated users when `limit_unauthenticated_to_local_content` is `false`" do
Pleroma.Config.put([:instance, :limit_unauthenticated_to_local_content], false)
u1 = insert(:user, %{name: "lain"})
u2 = insert(:user, %{name: "ebn", nickname: "lain@mastodon.social", local: false})
u3 = insert(:user, %{nickname: "lain@pleroma.soykaf.com", local: false})
results =
"lain"
|> User.search()
|> Enum.map(& &1.id)
|> Enum.sort()
assert [u1.id, u2.id, u3.id] == results
Pleroma.Config.put([:instance, :limit_unauthenticated_to_local_content], true)
end
test "finds a user whose name is nil" do test "finds a user whose name is nil" do
_user = insert(:user, %{name: "notamatch", nickname: "testuser@pleroma.amplifie.red"}) _user = insert(:user, %{name: "notamatch", nickname: "testuser@pleroma.amplifie.red"})
user_two = insert(:user, %{name: nil, nickname: "lain@pleroma.soykaf.com"}) user_two = insert(:user, %{name: nil, nickname: "lain@pleroma.soykaf.com"})
@ -1097,7 +1137,12 @@ test "does not yield false-positive matches" do
end end
test "works with URIs" do test "works with URIs" do
[result] = User.search("http://mastodon.example.org/users/admin", resolve: true) user = insert(:user)
[result] =
User.search("http://mastodon.example.org/users/admin", resolve: true, for_user: user)
user = User.get_cached_by_ap_id("http://mastodon.example.org/users/admin")
expected = expected =
result result
@ -1105,8 +1150,6 @@ test "works with URIs" do
|> Map.put(:search_type, nil) |> Map.put(:search_type, nil)
|> Map.put(:last_digest_emailed_at, nil) |> Map.put(:last_digest_emailed_at, nil)
user = User.get_cached_by_ap_id("http://mastodon.example.org/users/admin")
assert user == expected assert user == expected
end end
end end
@ -1373,4 +1416,16 @@ test "it doesn't create keys if there already are some" do
assert user.info.keys == "xxx" assert user.info.keys == "xxx"
end end
end end
describe "get_ap_ids_by_nicknames" do
test "it returns a list of AP ids for a given set of nicknames" do
user = insert(:user)
user_two = insert(:user)
ap_ids = User.get_ap_ids_by_nicknames([user.nickname, user_two.nickname, "nonexistent"])
assert length(ap_ids) == 2
assert user.ap_id in ap_ids
assert user_two.ap_id in ap_ids
end
end
end end

143
test/web/activity_pub/transmogrifier/follow_handling_test.exs Normal file
View file

@ -0,0 +1,143 @@
# Pleroma: A lightweight social networking server
# Copyright © 2017-2018 Pleroma Authors <https://pleroma.social/>
# SPDX-License-Identifier: AGPL-3.0-only
defmodule Pleroma.Web.ActivityPub.Transmogrifier.FollowHandlingTest do
use Pleroma.DataCase
alias Pleroma.Activity
alias Pleroma.Repo
alias Pleroma.User
alias Pleroma.Web.ActivityPub.Transmogrifier
alias Pleroma.Web.ActivityPub.Utils
import Pleroma.Factory
import Ecto.Query
setup_all do
Tesla.Mock.mock_global(fn env -> apply(HttpRequestMock, :request, [env]) end)
:ok
end
describe "handle_incoming" do
test "it works for incoming follow requests" do
user = insert(:user)
data =
File.read!("test/fixtures/mastodon-follow-activity.json")
|> Poison.decode!()
|> Map.put("object", user.ap_id)
{:ok, %Activity{data: data, local: false} = activity} = Transmogrifier.handle_incoming(data)
assert data["actor"] == "http://mastodon.example.org/users/admin"
assert data["type"] == "Follow"
assert data["id"] == "http://mastodon.example.org/users/admin#follows/2"
activity = Repo.get(Activity, activity.id)
assert activity.data["state"] == "accept"
assert User.following?(User.get_cached_by_ap_id(data["actor"]), user)
end
test "with locked accounts, it does not create a follow or an accept" do
user = insert(:user, info: %{locked: true})
data =
File.read!("test/fixtures/mastodon-follow-activity.json")
|> Poison.decode!()
|> Map.put("object", user.ap_id)
{:ok, %Activity{data: data, local: false}} = Transmogrifier.handle_incoming(data)
assert data["state"] == "pending"
refute User.following?(User.get_cached_by_ap_id(data["actor"]), user)
accepts =
from(
a in Activity,
where: fragment("?->>'type' = ?", a.data, "Accept")
)
|> Repo.all()
assert length(accepts) == 0
end
test "it works for follow requests when you are already followed, creating a new accept activity" do
# This is important because the remote might have the wrong idea about the
# current follow status. This can lead to instance A thinking that x@A is
# followed by y@B, but B thinks they are not. In this case, the follow can
# never go through again because it will never get an Accept.
user = insert(:user)
data =
File.read!("test/fixtures/mastodon-follow-activity.json")
|> Poison.decode!()
|> Map.put("object", user.ap_id)
{:ok, %Activity{local: false}} = Transmogrifier.handle_incoming(data)
accepts =
from(
a in Activity,
where: fragment("?->>'type' = ?", a.data, "Accept")
)
|> Repo.all()
assert length(accepts) == 1
data =
File.read!("test/fixtures/mastodon-follow-activity.json")
|> Poison.decode!()
|> Map.put("id", String.replace(data["id"], "2", "3"))
|> Map.put("object", user.ap_id)
{:ok, %Activity{local: false}} = Transmogrifier.handle_incoming(data)
accepts =
from(
a in Activity,
where: fragment("?->>'type' = ?", a.data, "Accept")
)
|> Repo.all()
assert length(accepts) == 2
end
test "it rejects incoming follow requests from blocked users when deny_follow_blocked is enabled" do
Pleroma.Config.put([:user, :deny_follow_blocked], true)
user = insert(:user)
{:ok, target} = User.get_or_fetch("http://mastodon.example.org/users/admin")
{:ok, user} = User.block(user, target)
data =
File.read!("test/fixtures/mastodon-follow-activity.json")
|> Poison.decode!()
|> Map.put("object", user.ap_id)
{:ok, %Activity{data: %{"id" => id}}} = Transmogrifier.handle_incoming(data)
%Activity{} = activity = Activity.get_by_ap_id(id)
assert activity.data["state"] == "reject"
end
test "it works for incoming follow requests from hubzilla" do
user = insert(:user)
data =
File.read!("test/fixtures/hubzilla-follow-activity.json")
|> Poison.decode!()
|> Map.put("object", user.ap_id)
|> Utils.normalize_params()
{:ok, %Activity{data: data, local: false}} = Transmogrifier.handle_incoming(data)
assert data["actor"] == "https://hubzilla.example.org/channel/kaniini"
assert data["type"] == "Follow"
assert data["id"] == "https://hubzilla.example.org/channel/kaniini#follows/2"
assert User.following?(User.get_cached_by_ap_id(data["actor"]), user)
end
end
end

54
test/web/activity_pub/transmogrifier_test.exs
View file

@ -11,7 +11,6 @@ defmodule Pleroma.Web.ActivityPub.TransmogrifierTest do
alias Pleroma.User alias Pleroma.User
alias Pleroma.Web.ActivityPub.ActivityPub alias Pleroma.Web.ActivityPub.ActivityPub
alias Pleroma.Web.ActivityPub.Transmogrifier alias Pleroma.Web.ActivityPub.Transmogrifier
alias Pleroma.Web.ActivityPub.Utils
alias Pleroma.Web.OStatus alias Pleroma.Web.OStatus
alias Pleroma.Web.Websub.WebsubClientSubscription alias Pleroma.Web.Websub.WebsubClientSubscription
@ -248,59 +247,6 @@ test "it cleans up incoming notices which are not really DMs" do
assert object_data["cc"] == to assert object_data["cc"] == to
end end
test "it works for incoming follow requests" do
user = insert(:user)
data =
File.read!("test/fixtures/mastodon-follow-activity.json")
|> Poison.decode!()
|> Map.put("object", user.ap_id)
{:ok, %Activity{data: data, local: false}} = Transmogrifier.handle_incoming(data)
assert data["actor"] == "http://mastodon.example.org/users/admin"
assert data["type"] == "Follow"
assert data["id"] == "http://mastodon.example.org/users/admin#follows/2"
assert User.following?(User.get_cached_by_ap_id(data["actor"]), user)
end
test "it rejects incoming follow requests from blocked users when deny_follow_blocked is enabled" do
Pleroma.Config.put([:user, :deny_follow_blocked], true)
user = insert(:user)
{:ok, target} = User.get_or_fetch("http://mastodon.example.org/users/admin")
{:ok, user} = User.block(user, target)
data =
File.read!("test/fixtures/mastodon-follow-activity.json")
|> Poison.decode!()
|> Map.put("object", user.ap_id)
{:ok, %Activity{data: %{"id" => id}}} = Transmogrifier.handle_incoming(data)
%Activity{} = activity = Activity.get_by_ap_id(id)
assert activity.data["state"] == "reject"
end
test "it works for incoming follow requests from hubzilla" do
user = insert(:user)
data =
File.read!("test/fixtures/hubzilla-follow-activity.json")
|> Poison.decode!()
|> Map.put("object", user.ap_id)
|> Utils.normalize_params()
{:ok, %Activity{data: data, local: false}} = Transmogrifier.handle_incoming(data)
assert data["actor"] == "https://hubzilla.example.org/channel/kaniini"
assert data["type"] == "Follow"
assert data["id"] == "https://hubzilla.example.org/channel/kaniini#follows/2"
assert User.following?(User.get_cached_by_ap_id(data["actor"]), user)
end
test "it works for incoming likes" do test "it works for incoming likes" do
user = insert(:user) user = insert(:user)
{:ok, activity} = CommonAPI.post(user, %{"status" => "hello"}) {:ok, activity} = CommonAPI.post(user, %{"status" => "hello"})

48
test/web/activity_pub/utils_test.exs
View file

@ -2,6 +2,7 @@ defmodule Pleroma.Web.ActivityPub.UtilsTest do
use Pleroma.DataCase use Pleroma.DataCase
alias Pleroma.Activity alias Pleroma.Activity
alias Pleroma.Object alias Pleroma.Object
alias Pleroma.Repo
alias Pleroma.User alias Pleroma.User
alias Pleroma.Web.ActivityPub.ActivityPub alias Pleroma.Web.ActivityPub.ActivityPub
alias Pleroma.Web.ActivityPub.Utils alias Pleroma.Web.ActivityPub.Utils
@ -247,4 +248,51 @@ test "fetches only Create activities" do
assert fetched_vote.id == vote.id assert fetched_vote.id == vote.id
end end
end end
describe "update_follow_state_for_all/2" do
test "updates the state of all Follow activities with the same actor and object" do
user = insert(:user, info: %{locked: true})
follower = insert(:user)
{:ok, follow_activity} = ActivityPub.follow(follower, user)
{:ok, follow_activity_two} = ActivityPub.follow(follower, user)
data =
follow_activity_two.data
|> Map.put("state", "accept")
cng = Ecto.Changeset.change(follow_activity_two, data: data)
{:ok, follow_activity_two} = Repo.update(cng)
{:ok, follow_activity_two} =
Utils.update_follow_state_for_all(follow_activity_two, "accept")
assert Repo.get(Activity, follow_activity.id).data["state"] == "accept"
assert Repo.get(Activity, follow_activity_two.id).data["state"] == "accept"
end
end
describe "update_follow_state/2" do
test "updates the state of the given follow activity" do
user = insert(:user, info: %{locked: true})
follower = insert(:user)
{:ok, follow_activity} = ActivityPub.follow(follower, user)
{:ok, follow_activity_two} = ActivityPub.follow(follower, user)
data =
follow_activity_two.data
|> Map.put("state", "accept")
cng = Ecto.Changeset.change(follow_activity_two, data: data)
{:ok, follow_activity_two} = Repo.update(cng)
{:ok, follow_activity_two} = Utils.update_follow_state(follow_activity_two, "reject")
assert Repo.get(Activity, follow_activity.id).data["state"] == "pending"
assert Repo.get(Activity, follow_activity_two.id).data["state"] == "reject"
end
end
end end

62
test/web/common_api/common_api_test.exs
View file

@ -7,6 +7,7 @@ defmodule Pleroma.Web.CommonAPITest do
alias Pleroma.Activity alias Pleroma.Activity
alias Pleroma.Object alias Pleroma.Object
alias Pleroma.User alias Pleroma.User
alias Pleroma.Web.ActivityPub.ActivityPub
alias Pleroma.Web.CommonAPI alias Pleroma.Web.CommonAPI
import Pleroma.Factory import Pleroma.Factory
@ -56,6 +57,25 @@ test "it adds emoji when updating profiles" do
end end
describe "posting" do describe "posting" do
test "it supports explicit addressing" do
user = insert(:user)
user_two = insert(:user)
user_three = insert(:user)
user_four = insert(:user)
{:ok, activity} =
CommonAPI.post(user, %{
"status" =>
"Hey, I think @#{user_three.nickname} is ugly. @#{user_four.nickname} is alright though.",
"to" => [user_two.nickname, user_four.nickname, "nonexistent"]
})
assert user.ap_id in activity.recipients
assert user_two.ap_id in activity.recipients
assert user_four.ap_id in activity.recipients
refute user_three.ap_id in activity.recipients
end
test "it filters out obviously bad tags when accepting a post as HTML" do test "it filters out obviously bad tags when accepting a post as HTML" do
user = insert(:user) user = insert(:user)
@ -320,4 +340,46 @@ test "remove a reblog mute", %{muter: muter, muted: muted} do
assert User.showing_reblogs?(muter, muted) == true assert User.showing_reblogs?(muter, muted) == true
end end
end end
describe "accept_follow_request/2" do
test "after acceptance, it sets all existing pending follow request states to 'accept'" do
user = insert(:user, info: %{locked: true})
follower = insert(:user)
follower_two = insert(:user)
{:ok, follow_activity} = ActivityPub.follow(follower, user)
{:ok, follow_activity_two} = ActivityPub.follow(follower, user)
{:ok, follow_activity_three} = ActivityPub.follow(follower_two, user)
assert follow_activity.data["state"] == "pending"
assert follow_activity_two.data["state"] == "pending"
assert follow_activity_three.data["state"] == "pending"
{:ok, _follower} = CommonAPI.accept_follow_request(follower, user)
assert Repo.get(Activity, follow_activity.id).data["state"] == "accept"
assert Repo.get(Activity, follow_activity_two.id).data["state"] == "accept"
assert Repo.get(Activity, follow_activity_three.id).data["state"] == "pending"
end
test "after rejection, it sets all existing pending follow request states to 'reject'" do
user = insert(:user, info: %{locked: true})
follower = insert(:user)
follower_two = insert(:user)
{:ok, follow_activity} = ActivityPub.follow(follower, user)
{:ok, follow_activity_two} = ActivityPub.follow(follower, user)
{:ok, follow_activity_three} = ActivityPub.follow(follower_two, user)
assert follow_activity.data["state"] == "pending"
assert follow_activity_two.data["state"] == "pending"
assert follow_activity_three.data["state"] == "pending"
{:ok, _follower} = CommonAPI.reject_follow_request(follower, user)
assert Repo.get(Activity, follow_activity.id).data["state"] == "reject"
assert Repo.get(Activity, follow_activity_two.id).data["state"] == "reject"
assert Repo.get(Activity, follow_activity_three.id).data["state"] == "pending"
end
end
end end

133
test/web/common_api/common_api_utils_test.exs
View file

@ -5,10 +5,15 @@
defmodule Pleroma.Web.CommonAPI.UtilsTest do defmodule Pleroma.Web.CommonAPI.UtilsTest do
alias Pleroma.Builders.UserBuilder alias Pleroma.Builders.UserBuilder
alias Pleroma.Object alias Pleroma.Object
alias Pleroma.Web.CommonAPI
alias Pleroma.Web.CommonAPI.Utils alias Pleroma.Web.CommonAPI.Utils
alias Pleroma.Web.Endpoint alias Pleroma.Web.Endpoint
use Pleroma.DataCase use Pleroma.DataCase
import Pleroma.Factory
@public_address "https://www.w3.org/ns/activitystreams#Public"
test "it adds attachment links to a given text and attachment set" do test "it adds attachment links to a given text and attachment set" do
name = name =
"Sakura%20Mana%20%E2%80%93%20Turned%20on%20by%20a%20Senior%20OL%20with%20a%20Temptating%20Tight%20Skirt-s%20Full%20Hipline%20and%20Panty%20Shot-%20Beautiful%20Thick%20Thighs-%20and%20Erotic%20Ass-%20-2015-%20--%20Oppaitime%208-28-2017%206-50-33%20PM.png" "Sakura%20Mana%20%E2%80%93%20Turned%20on%20by%20a%20Senior%20OL%20with%20a%20Temptating%20Tight%20Skirt-s%20Full%20Hipline%20and%20Panty%20Shot-%20Beautiful%20Thick%20Thighs-%20and%20Erotic%20Ass-%20-2015-%20--%20Oppaitime%208-28-2017%206-50-33%20PM.png"
@ -214,4 +219,132 @@ test "when date is nil" do
assert Utils.date_to_asctime(nil) == expected assert Utils.date_to_asctime(nil) == expected
end end
end end
describe "get_to_and_cc" do
test "for public posts, not a reply" do
user = insert(:user)
mentioned_user = insert(:user)
mentions = [mentioned_user.ap_id]
{to, cc} = Utils.get_to_and_cc(user, mentions, nil, "public")
assert length(to) == 2
assert length(cc) == 1
assert @public_address in to
assert mentioned_user.ap_id in to
assert user.follower_address in cc
end
test "for public posts, a reply" do
user = insert(:user)
mentioned_user = insert(:user)
third_user = insert(:user)
{:ok, activity} = CommonAPI.post(third_user, %{"status" => "uguu"})
mentions = [mentioned_user.ap_id]
{to, cc} = Utils.get_to_and_cc(user, mentions, activity, "public")
assert length(to) == 3
assert length(cc) == 1
assert @public_address in to
assert mentioned_user.ap_id in to
assert third_user.ap_id in to
assert user.follower_address in cc
end
test "for unlisted posts, not a reply" do
user = insert(:user)
mentioned_user = insert(:user)
mentions = [mentioned_user.ap_id]
{to, cc} = Utils.get_to_and_cc(user, mentions, nil, "unlisted")
assert length(to) == 2
assert length(cc) == 1
assert @public_address in cc
assert mentioned_user.ap_id in to
assert user.follower_address in to
end
test "for unlisted posts, a reply" do
user = insert(:user)
mentioned_user = insert(:user)
third_user = insert(:user)
{:ok, activity} = CommonAPI.post(third_user, %{"status" => "uguu"})
mentions = [mentioned_user.ap_id]
{to, cc} = Utils.get_to_and_cc(user, mentions, activity, "unlisted")
assert length(to) == 3
assert length(cc) == 1
assert @public_address in cc
assert mentioned_user.ap_id in to
assert third_user.ap_id in to
assert user.follower_address in to
end
test "for private posts, not a reply" do
user = insert(:user)
mentioned_user = insert(:user)
mentions = [mentioned_user.ap_id]
{to, cc} = Utils.get_to_and_cc(user, mentions, nil, "private")
assert length(to) == 2
assert length(cc) == 0
assert mentioned_user.ap_id in to
assert user.follower_address in to
end
test "for private posts, a reply" do
user = insert(:user)
mentioned_user = insert(:user)
third_user = insert(:user)
{:ok, activity} = CommonAPI.post(third_user, %{"status" => "uguu"})
mentions = [mentioned_user.ap_id]
{to, cc} = Utils.get_to_and_cc(user, mentions, activity, "private")
assert length(to) == 3
assert length(cc) == 0
assert mentioned_user.ap_id in to
assert third_user.ap_id in to
assert user.follower_address in to
end
test "for direct posts, not a reply" do
user = insert(:user)
mentioned_user = insert(:user)
mentions = [mentioned_user.ap_id]
{to, cc} = Utils.get_to_and_cc(user, mentions, nil, "direct")
assert length(to) == 1
assert length(cc) == 0
assert mentioned_user.ap_id in to
end
test "for direct posts, a reply" do
user = insert(:user)
mentioned_user = insert(:user)
third_user = insert(:user)
{:ok, activity} = CommonAPI.post(third_user, %{"status" => "uguu"})
mentions = [mentioned_user.ap_id]
{to, cc} = Utils.get_to_and_cc(user, mentions, activity, "direct")
assert length(to) == 2
assert length(cc) == 0
assert mentioned_user.ap_id in to
assert third_user.ap_id in to
end
end
end end

12
test/web/mastodon_api/mastodon_api_controller_test.exs
View file

@ -2173,8 +2173,11 @@ test "search doesn't show statuses that it shouldn't", %{conn: conn} do
end end
test "search fetches remote accounts", %{conn: conn} do test "search fetches remote accounts", %{conn: conn} do
user = insert(:user)
conn = conn =
conn conn
|> assign(:user, user)
|> get("/api/v1/search", %{"q" => "shp@social.heldscal.la", "resolve" => "true"}) |> get("/api/v1/search", %{"q" => "shp@social.heldscal.la", "resolve" => "true"})
assert results = json_response(conn, 200) assert results = json_response(conn, 200)
@ -2182,6 +2185,15 @@ test "search fetches remote accounts", %{conn: conn} do
assert account["acct"] == "shp@social.heldscal.la" assert account["acct"] == "shp@social.heldscal.la"
end end
test "search doesn't fetch remote accounts if resolve is false", %{conn: conn} do
conn =
conn
|> get("/api/v1/search", %{"q" => "shp@social.heldscal.la", "resolve" => "false"})
assert results = json_response(conn, 200)
assert [] == results["accounts"]
end
test "returns the favorites of a user", %{conn: conn} do test "returns the favorites of a user", %{conn: conn} do
user = insert(:user) user = insert(:user)
other_user = insert(:user) other_user = insert(:user)

6
test/web/twitter_api/twitter_api_test.exs
View file

@ -116,8 +116,7 @@ test "Follow another user using user_id" do
{:ok, user, followed, _activity} = TwitterAPI.follow(user, %{"user_id" => followed.id}) {:ok, user, followed, _activity} = TwitterAPI.follow(user, %{"user_id" => followed.id})
assert User.ap_followers(followed) in user.following assert User.ap_followers(followed) in user.following
{:error, msg} = TwitterAPI.follow(user, %{"user_id" => followed.id}) {:ok, _, _, _} = TwitterAPI.follow(user, %{"user_id" => followed.id})
assert msg == "Could not follow user: #{followed.nickname} is already on your list."
end end
test "Follow another user using screen_name" do test "Follow another user using screen_name" do
@ -132,8 +131,7 @@ test "Follow another user using screen_name" do
followed = User.get_cached_by_ap_id(followed.ap_id) followed = User.get_cached_by_ap_id(followed.ap_id)
assert followed.info.follower_count == 1 assert followed.info.follower_count == 1
{:error, msg} = TwitterAPI.follow(user, %{"screen_name" => followed.nickname}) {:ok, _, _, _} = TwitterAPI.follow(user, %{"screen_name" => followed.nickname})
assert msg == "Could not follow user: #{followed.nickname} is already on your list."
end end
test "Unfollow another user using user_id" do test "Unfollow another user using user_id" do