)`.
* In controllers, `use Pleroma.Web, :controller` will result in `action/2` (see `Pleroma.Web.controller/0` for definition) be called prior to actual controller action, and it'll perform security / privacy checks before passing control to actual controller action.
@@ -16,7 +16,7 @@ This document contains notes and guidelines for Pleroma developers.
## [HTTP Basic Authentication](https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Authorization)
-* With HTTP Basic Auth, OAuth scopes check is _not_ performed for any action (since password is provided during the auth, requester is able to obtain a token with full permissions anyways). `Pleroma.Plugs.AuthenticationPlug` and `Pleroma.Plugs.LegacyAuthenticationPlug` both call `Pleroma.Plugs.OAuthScopesPlug.skip_plug(conn)` when password is provided.
+* With HTTP Basic Auth, OAuth scopes check is _not_ performed for any action (since password is provided during the auth, requester is able to obtain a token with full permissions anyways). `Pleroma.Web.Plugs.AuthenticationPlug` and `Pleroma.Web.Plugs.LegacyAuthenticationPlug` both call `Pleroma.Web.Plugs.OAuthScopesPlug.skip_plug(conn)` when password is provided.
## Auth-related configuration, OAuth consumer mode etc.
diff --git a/docs/installation/alpine_linux_en.md b/docs/installation/alpine_linux_en.md
index a5683f18c..62f2fb778 100644
--- a/docs/installation/alpine_linux_en.md
+++ b/docs/installation/alpine_linux_en.md
@@ -13,6 +13,7 @@ It assumes that you have administrative rights, either as root or a user with [s
* `erlang-parsetools`
* `erlang-xmerl`
* `git`
+* `file-dev`
* Development Tools
* `cmake`
@@ -20,6 +21,9 @@ It assumes that you have administrative rights, either as root or a user with [s
* `nginx` (preferred, example configs for other reverse proxies can be found in the repo)
* `certbot` (or any other ACME client for Let’s Encrypt certificates)
+* `ImageMagick`
+* `ffmpeg`
+* `exiftool`
### Prepare the system
@@ -29,7 +33,6 @@ It assumes that you have administrative rights, either as root or a user with [s
awk 'NR==2' /etc/apk/repositories | sed 's/main/community/' | tee -a /etc/apk/repositories
```
-
* Then update the system, if not already done:
```shell
@@ -40,7 +43,7 @@ sudo apk upgrade
* Install some tools, which are needed later:
```shell
-sudo apk add git build-base cmake
+sudo apk add git build-base cmake file-dev
```
### Install Elixir and Erlang
@@ -56,6 +59,7 @@ sudo apk add erlang erlang-runtime-tools erlang-xmerl elixir
```shell
sudo apk add erlang-eldap
```
+
### Install PostgreSQL
* Install Postgresql server:
@@ -76,6 +80,12 @@ sudo /etc/init.d/postgresql start
sudo rc-update add postgresql
```
+### Install media / graphics packages (optional, see [`docs/installation/optional/media_graphics_packages.md`](docs/installation/optional/media_graphics_packages.md))
+
+```shell
+sudo apk add ffmpeg imagemagick exiftool
+```
+
### Install PleromaBE
* Add a new system user for the Pleroma service:
diff --git a/docs/installation/arch_linux_en.md b/docs/installation/arch_linux_en.md
index 7fb69dd60..0eb6d2d5f 100644
--- a/docs/installation/arch_linux_en.md
+++ b/docs/installation/arch_linux_en.md
@@ -10,11 +10,15 @@ This guide will assume that you have administrative rights, either as root or a
* `git`
* `base-devel`
* `cmake`
+* `file`
#### Optional packages used in this guide
* `nginx` (preferred, example configs for other reverse proxies can be found in the repo)
* `certbot` (or any other ACME client for Let’s Encrypt certificates)
+* `ImageMagick`
+* `ffmpeg`
+* `exiftool`
### Prepare the system
@@ -27,7 +31,7 @@ sudo pacman -Syu
* Install some of the above mentioned programs:
```shell
-sudo pacman -S git base-devel elixir cmake
+sudo pacman -S git base-devel elixir cmake file
```
### Install PostgreSQL
@@ -52,6 +56,12 @@ sudo -iu postgres initdb -D /var/lib/postgres/data
sudo systemctl enable --now postgresql.service
```
+### Install media / graphics packages (optional, see [`docs/installation/optional/media_graphics_packages.md`](docs/installation/optional/media_graphics_packages.md))
+
+```shell
+sudo pacman -S ffmpeg imagemagick perl-image-exiftool
+```
+
### Install PleromaBE
* Add a new system user for the Pleroma service:
diff --git a/docs/installation/debian_based_en.md b/docs/installation/debian_based_en.md
index 60c2f47e5..b9fc4e112 100644
--- a/docs/installation/debian_based_en.md
+++ b/docs/installation/debian_based_en.md
@@ -10,6 +10,7 @@ This guide will assume you are on Debian Stretch. This guide should also work wi
* `elixir` (1.8+, Follow the guide to install from the Erlang Solutions repo or use [asdf](https://github.com/asdf-vm/asdf) as the pleroma user)
* `erlang-dev`
* `erlang-nox`
+* `libmagic-dev`
* `git`
* `build-essential`
* `cmake`
@@ -18,6 +19,9 @@ This guide will assume you are on Debian Stretch. This guide should also work wi
* `nginx` (preferred, example configs for other reverse proxies can be found in the repo)
* `certbot` (or any other ACME client for Let’s Encrypt certificates)
+* `ImageMagick`
+* `ffmpeg`
+* `exiftool`
### Prepare the system
@@ -31,7 +35,7 @@ sudo apt full-upgrade
* Install some of the above mentioned programs:
```shell
-sudo apt install git build-essential postgresql postgresql-contrib cmake
+sudo apt install git build-essential postgresql postgresql-contrib cmake libmagic-devel
```
### Install Elixir and Erlang
@@ -50,6 +54,12 @@ sudo apt update
sudo apt install elixir erlang-dev erlang-nox
```
+### Optional packages: [`docs/installation/optional/media_graphics_packages.md`](docs/installation/optional/media_graphics_packages.md)
+
+```shell
+sudo apt install imagemagick ffmpeg libimage-exiftool-perl
+```
+
### Install PleromaBE
* Add a new system user for the Pleroma service:
@@ -91,6 +101,7 @@ sudo -Hu pleroma mix deps.get
mv config/{generated_config.exs,prod.secret.exs}
```
+
* The previous command creates also the file `config/setup_db.psql`, with which you can create the database:
```shell
@@ -171,6 +182,7 @@ sudo cp /opt/pleroma/installation/pleroma.service /etc/systemd/system/pleroma.se
```
* Edit the service file and make sure that all paths fit your installation
+* Check that `EnvironmentFile` contains the correct path to the env file. Or generate the env file: `sudo -Hu pleroma mix pleroma.release_env gen`
* Enable and start `pleroma.service`:
```shell
diff --git a/docs/installation/debian_based_jp.md b/docs/installation/debian_based_jp.md
index c2dd840d3..94e22325c 100644
--- a/docs/installation/debian_based_jp.md
+++ b/docs/installation/debian_based_jp.md
@@ -17,11 +17,15 @@
- `git`
- `build-essential`
- `cmake`
+- `libmagic-dev`
#### このガイドで利用している追加パッケージ
- `nginx` (おすすめです。他のリバースプロキシを使う場合は、参考となる設定をこのリポジトリから探してください)
- `certbot` (または何らかのLet's Encrypt向けACMEクライアント)
+- `ImageMagick`
+- `ffmpeg`
+- `exiftool`
### システムを準備する
@@ -33,10 +37,9 @@ sudo apt full-upgrade
* 上記に挙げたパッケージをインストールしておきます。
```
-sudo apt install git build-essential postgresql postgresql-contrib cmake
+sudo apt install git build-essential postgresql postgresql-contrib cmake ffmpeg imagemagick libmagic-dev
```
-
### ElixirとErlangをインストールします
* Erlangのリポジトリをダウンロードおよびインストールします。
@@ -51,6 +54,12 @@ sudo apt update
sudo apt install elixir erlang-dev erlang-nox
```
+### オプションパッケージ: [`docs/installation/optional/media_graphics_packages.md`](docs/installation/optional/media_graphics_packages.md)
+
+```shell
+sudo apt install imagemagick ffmpeg libimage-exiftool-perl
+```
+
### Pleroma BE (バックエンド) をインストールします
* Pleroma用に新しいユーザーを作ります。
diff --git a/docs/installation/freebsd_en.md b/docs/installation/freebsd_en.md
index ca2575d9b..fdcb06c53 100644
--- a/docs/installation/freebsd_en.md
+++ b/docs/installation/freebsd_en.md
@@ -26,6 +26,12 @@ Setup the required services to automatically start at boot, using `sysrc(8)`.
# service postgresql start
```
+### Install media / graphics packages (optional, see [`docs/installation/optional/media_graphics_packages.md`](docs/installation/optional/media_graphics_packages.md))
+
+```shell
+# pkg install imagemagick ffmpeg p5-Image-ExifTool
+```
+
## Configuring Pleroma
Create a user for Pleroma:
diff --git a/docs/installation/gentoo_en.md b/docs/installation/gentoo_en.md
index 5a676380c..f2380ab72 100644
--- a/docs/installation/gentoo_en.md
+++ b/docs/installation/gentoo_en.md
@@ -29,12 +29,16 @@ Gentoo quite pointedly does not come with a cron daemon installed, and as such i
* `dev-lang/elixir`
* `dev-vcs/git`
* `dev-util/cmake`
+* `sys-apps/file`
#### Optional ebuilds used in this guide
* `www-servers/nginx` (preferred, example configs for other reverse proxies can be found in the repo)
* `app-crypt/certbot` (or any other ACME client for Let’s Encrypt certificates)
* `app-crypt/certbot-nginx` (nginx certbot plugin that allows use of the all-powerful `--nginx` flag on certbot)
+* `media-gfx/imagemagick`
+* `media-video/ffmpeg`
+* `media-libs/exiftool`
### Prepare the system
@@ -47,7 +51,7 @@ Gentoo quite pointedly does not come with a cron daemon installed, and as such i
* Emerge all required the required and suggested software in one go:
```shell
- # emerge --ask dev-db/postgresql dev-lang/elixir dev-vcs/git www-servers/nginx app-crypt/certbot app-crypt/certbot-nginx dev-util/cmake
+ # emerge --ask dev-db/postgresql dev-lang/elixir dev-vcs/git www-servers/nginx app-crypt/certbot app-crypt/certbot-nginx dev-util/cmake sys-apps/file
```
If you would not like to install the optional packages, remove them from this line.
@@ -87,6 +91,12 @@ If you do not plan to make any modifications to your Pleroma instance, cloning d
Not only does this make it much easier to deploy changes you make, as you can commit and pull from upstream and all that good stuff from the comfort of your local machine then simply `git pull` on your instance server when you're ready to deploy, it also ensures you are compliant with the Affero General Public Licence that Pleroma is licenced under, which stipulates that all network services provided with modified AGPL code must publish their changes on a publicly available internet service and for free. It also makes it much easier to ask for help from and provide help to your fellow Pleroma admins if your public repo always reflects what you are running because it is part of your deployment procedure.
+### Install media / graphics packages (optional, see [`docs/installation/optional/media_graphics_packages.md`](docs/installation/optional/media_graphics_packages.md))
+
+```shell
+# emerge --ask media-video/ffmpeg media-gfx/imagemagick media-libs/exiftool
+```
+
### Install PleromaBE
* Add a new system user for the Pleroma service and set up default directories:
diff --git a/docs/installation/netbsd_en.md b/docs/installation/netbsd_en.md
index 6ad0de2f6..d5fa04fdf 100644
--- a/docs/installation/netbsd_en.md
+++ b/docs/installation/netbsd_en.md
@@ -10,7 +10,7 @@ Pleroma uses.
The `mksh` shell is needed to run the Elixir `mix` script.
-`# pkgin install acmesh elixir git-base git-docs mksh nginx postgresql11-server postgresql11-client postgresql11-contrib sudo`
+`# pkgin install acmesh elixir git-base git-docs mksh nginx postgresql11-server postgresql11-client postgresql11-contrib sudo ffmpeg4 ImageMagick`
You can also build these packages using pkgsrc:
```
@@ -44,6 +44,10 @@ pgsql=YES
First, run `# /etc/rc.d/pgsql start`. Then, `$ sudo -Hu pgsql -g pgsql createdb`.
+### Install media / graphics packages (optional, see [`docs/installation/optional/media_graphics_packages.md`](docs/installation/optional/media_graphics_packages.md))
+
+`# pkgin install ImageMagick ffmpeg4 p5-Image-ExifTool`
+
## Configuring Pleroma
Create a user for Pleroma:
diff --git a/docs/installation/openbsd_en.md b/docs/installation/openbsd_en.md
index eee452845..8092ac379 100644
--- a/docs/installation/openbsd_en.md
+++ b/docs/installation/openbsd_en.md
@@ -10,20 +10,34 @@ The following packages need to be installed:
* elixir
* gmake
- * ImageMagick
* git
* postgresql-server
* postgresql-contrib
* cmake
+ * ffmpeg
+ * ImageMagick
To install them, run the following command (with doas or as root):
```
-pkg_add elixir gmake ImageMagick git postgresql-server postgresql-contrib cmake
+pkg_add elixir gmake git postgresql-server postgresql-contrib cmake ffmpeg ImageMagick
```
Pleroma requires a reverse proxy, OpenBSD has relayd in base (and is used in this guide) and packages/ports are available for nginx (www/nginx) and apache (www/apache-httpd). Independently of the reverse proxy, [acme-client(1)](https://man.openbsd.org/acme-client) can be used to get a certificate from Let's Encrypt.
+#### Optional software
+
+Per [`docs/installation/optional/media_graphics_packages.md`](docs/installation/optional/media_graphics_packages.md):
+ * ImageMagick
+ * ffmpeg
+ * exiftool
+
+To install the above:
+
+```
+pkg_add ImageMagick ffmpeg p5-Image-ExifTool
+```
+
#### Creating the pleroma user
Pleroma will be run by a dedicated user, \_pleroma. Before creating it, insert the following lines in login.conf:
```
diff --git a/docs/installation/openbsd_fi.md b/docs/installation/openbsd_fi.md
index b5b5056a9..01cf34ab4 100644
--- a/docs/installation/openbsd_fi.md
+++ b/docs/installation/openbsd_fi.md
@@ -16,7 +16,18 @@ Matrix-kanava #freenode_#pleroma:matrix.org ovat hyviä paikkoja löytää apua
Asenna tarvittava ohjelmisto:
-`# pkg_add git elixir gmake postgresql-server-10.3 postgresql-contrib-10.3 cmake`
+`# pkg_add git elixir gmake postgresql-server-10.3 postgresql-contrib-10.3 cmake ffmpeg ImageMagick`
+
+#### Optional software
+
+[`docs/installation/optional/media_graphics_packages.md`](docs/installation/optional/media_graphics_packages.md):
+ * ImageMagick
+ * ffmpeg
+ * exiftool
+
+Asenna tarvittava ohjelmisto:
+
+`# pkg_add ImageMagick ffmpeg p5-Image-ExifTool`
Luo postgresql-tietokanta:
diff --git a/docs/installation/optional/media_graphics_packages.md b/docs/installation/optional/media_graphics_packages.md
new file mode 100644
index 000000000..cb3d71188
--- /dev/null
+++ b/docs/installation/optional/media_graphics_packages.md
@@ -0,0 +1,32 @@
+# Optional software packages needed for specific functionality
+
+For specific Pleroma functionality (which is disabled by default) some or all of the below packages are required:
+ * `ImageMagic`
+ * `ffmpeg`
+ * `exiftool`
+
+Please refer to documentation in `docs/installation` on how to install them on specific OS.
+
+Note: the packages are not required with the current default settings of Pleroma.
+
+## `ImageMagick`
+
+`ImageMagick` is a set of tools to create, edit, compose, or convert bitmap images.
+
+It is required for the following Pleroma features:
+ * `Pleroma.Upload.Filters.Mogrify`, `Pleroma.Upload.Filters.Mogrifun` upload filters (related config: `Plaroma.Upload/filters` in `config/config.exs`)
+ * Media preview proxy for still images (related config: `media_preview_proxy/enabled` in `config/config.exs`)
+
+## `ffmpeg`
+
+`ffmpeg` is software to record, convert and stream audio and video.
+
+It is required for the following Pleroma features:
+ * Media preview proxy for videos (related config: `media_preview_proxy/enabled` in `config/config.exs`)
+
+## `exiftool`
+
+`exiftool` is media files metadata reader/writer.
+
+It is required for the following Pleroma features:
+ * `Pleroma.Upload.Filters.Exiftool` upload filter (related config: `Plaroma.Upload/filters` in `config/config.exs`)
diff --git a/docs/installation/otp_en.md b/docs/installation/otp_en.md
index b7e3bb2ac..98360bcf7 100644
--- a/docs/installation/otp_en.md
+++ b/docs/installation/otp_en.md
@@ -27,17 +27,37 @@ Other than things bundled in the OTP release Pleroma depends on:
* PostgreSQL (also utilizes extensions in postgresql-contrib)
* nginx (could be swapped with another reverse proxy but this guide covers only it)
* certbot (for Let's Encrypt certificates, could be swapped with another ACME client, but this guide covers only it)
+* libmagic/file
=== "Alpine"
```
echo "http://nl.alpinelinux.org/alpine/latest-stable/community" >> /etc/apk/repositories
apk update
- apk add curl unzip ncurses postgresql postgresql-contrib nginx certbot
+ apk add curl unzip ncurses postgresql postgresql-contrib nginx certbot file-dev
```
=== "Debian/Ubuntu"
```
- apt install curl unzip libncurses5 postgresql postgresql-contrib nginx certbot
+ apt install curl unzip libncurses5 postgresql postgresql-contrib nginx certbot libmagic-dev
+ ```
+
+### Installing optional packages
+
+Per [`docs/installation/optional/media_graphics_packages.md`](optional/media_graphics_packages.md):
+ * ImageMagick
+ * ffmpeg
+ * exiftool
+
+=== "Alpine"
+ ```
+ echo "http://nl.alpinelinux.org/alpine/latest-stable/community" >> /etc/apk/repositories
+ apk update
+ apk add imagemagick ffmpeg exiftool
+ ```
+
+=== "Debian/Ubuntu"
+ ```
+ apt install imagemagick ffmpeg libimage-exiftool-perl
```
## Setup
@@ -82,6 +102,8 @@ It is encouraged to check [Optimizing your PostgreSQL performance](../configurat
If you are using PostgreSQL 12 or higher, add this to your Ecto database configuration
```elixir
+#
+config :pleroma, Pleroma.Repo,
prepare: :named,
parameters: [
plan_cache_mode: "force_custom_plan"
@@ -127,6 +149,9 @@ chown -R pleroma /etc/pleroma
# Run the config generator
su pleroma -s $SHELL -lc "./bin/pleroma_ctl instance gen --output /etc/pleroma/config.exs --output-psql /tmp/setup_db.psql"
+# Run the environment file generator.
+su pleroma -s $SHELL -lc "./bin/pleroma_ctl release_env gen"
+
# Create the postgres database
su postgres -s $SHELL -lc "psql -f /tmp/setup_db.psql"
@@ -137,7 +162,7 @@ su pleroma -s $SHELL -lc "./bin/pleroma_ctl migrate"
# su pleroma -s $SHELL -lc "./bin/pleroma_ctl migrate --migrations-path priv/repo/optional_migrations/rum_indexing/"
# Start the instance to verify that everything is working as expected
-su pleroma -s $SHELL -lc "./bin/pleroma daemon"
+su pleroma -s $SHELL -lc "export $(cat /opt/pleroma/config/pleroma.env); ./bin/pleroma daemon"
# Wait for about 20 seconds and query the instance endpoint, if it shows your uri, name and email correctly, you are configured correctly
sleep 20 && curl http://localhost:4000/api/v1/instance
@@ -289,4 +314,3 @@ This will create an account withe the username of 'joeuser' with the email addre
## Questions
Questions about the installation or didn’t it work as it should be, ask in [#pleroma:matrix.org](https://matrix.heldscal.la/#/room/#freenode_#pleroma:matrix.org) or IRC Channel **#pleroma** on **Freenode**.
-
diff --git a/installation/init.d/pleroma b/installation/init.d/pleroma
index 384536f7e..e908cda1b 100755
--- a/installation/init.d/pleroma
+++ b/installation/init.d/pleroma
@@ -8,6 +8,7 @@ pidfile="/var/run/pleroma.pid"
directory=/opt/pleroma
healthcheck_delay=60
healthcheck_timer=30
+export $(cat /opt/pleroma/config/pleroma.env)
: ${pleroma_port:-4000}
diff --git a/installation/pleroma.nginx b/installation/pleroma.nginx
index d301ca615..d613befd2 100644
--- a/installation/pleroma.nginx
+++ b/installation/pleroma.nginx
@@ -9,6 +9,12 @@
proxy_cache_path /tmp/pleroma-media-cache levels=1:2 keys_zone=pleroma_media_cache:10m max_size=10g
inactive=720m use_temp_path=off;
+# this is explicitly IPv4 since Pleroma.Web.Endpoint binds on IPv4 only
+# and `localhost.` resolves to [::0] on some systems: see issue #930
+upstream phoenix {
+ server 127.0.0.1:4000 max_fails=5 fail_timeout=60s;
+}
+
server {
server_name example.tld;
@@ -63,19 +69,16 @@ server {
# the nginx default is 1m, not enough for large media uploads
client_max_body_size 16m;
+ ignore_invalid_headers off;
+
+ proxy_http_version 1.1;
+ proxy_set_header Upgrade $http_upgrade;
+ proxy_set_header Connection "upgrade";
+ proxy_set_header Host $http_host;
+ proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
location / {
- proxy_http_version 1.1;
- proxy_set_header Upgrade $http_upgrade;
- proxy_set_header Connection "upgrade";
- proxy_set_header Host $http_host;
- proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
-
- # this is explicitly IPv4 since Pleroma.Web.Endpoint binds on IPv4 only
- # and `localhost.` resolves to [::0] on some systems: see issue #930
- proxy_pass http://127.0.0.1:4000;
-
- client_max_body_size 16m;
+ proxy_pass http://phoenix;
}
location ~ ^/(media|proxy) {
@@ -83,12 +86,16 @@ server {
slice 1m;
proxy_cache_key $host$uri$is_args$args$slice_range;
proxy_set_header Range $slice_range;
- proxy_http_version 1.1;
proxy_cache_valid 200 206 301 304 1h;
proxy_cache_lock on;
proxy_ignore_client_abort on;
proxy_buffering on;
chunked_transfer_encoding on;
- proxy_pass http://127.0.0.1:4000;
+ proxy_pass http://phoenix;
+ }
+
+ location /api/fedsocket/v1 {
+ proxy_request_buffering off;
+ proxy_pass http://phoenix/api/fedsocket/v1;
}
}
diff --git a/installation/pleroma.service b/installation/pleroma.service
index 5dcbc1387..63e83ed6e 100644
--- a/installation/pleroma.service
+++ b/installation/pleroma.service
@@ -17,6 +17,8 @@ Environment="MIX_ENV=prod"
Environment="HOME=/var/lib/pleroma"
; Path to the folder containing the Pleroma installation.
WorkingDirectory=/opt/pleroma
+; Path to the environment file. the file contains RELEASE_COOKIE and etc
+EnvironmentFile=/opt/pleroma/config/pleroma.env
; Path to the Mix binary.
ExecStart=/usr/bin/mix phx.server
@@ -29,8 +31,6 @@ ProtectHome=true
ProtectSystem=full
; Sets up a new /dev mount for the process and only adds API pseudo devices like /dev/null, /dev/zero or /dev/random but not physical devices. Disabled by default because it may not work on devices like the Raspberry Pi.
PrivateDevices=false
-; Ensures that the service process and all its children can never gain new privileges through execve().
-NoNewPrivileges=true
; Drops the sysadmin capability from the daemon.
CapabilityBoundingSet=~CAP_SYS_ADMIN
diff --git a/installation/pleroma.vcl b/installation/pleroma.vcl
index 154747aa6..13dad784c 100644
--- a/installation/pleroma.vcl
+++ b/installation/pleroma.vcl
@@ -1,3 +1,4 @@
+# Recommended varnishncsa logging format: '%h %l %u %t "%m %{X-Forwarded-Proto}i://%{Host}i%U%q %H" %s %b "%{Referer}i" "%{User-agent}i"'
vcl 4.1;
import std;
@@ -14,8 +15,11 @@ acl purge {
sub vcl_recv {
# Redirect HTTP to HTTPS
if (std.port(server.ip) != 443) {
+ set req.http.X-Forwarded-Proto = "http";
set req.http.x-redir = "https://" + req.http.host + req.url;
return (synth(750, ""));
+ } else {
+ set req.http.X-Forwarded-Proto = "https";
}
# CHUNKED SUPPORT
@@ -105,7 +109,7 @@ sub vcl_hash {
sub vcl_backend_fetch {
# Be more lenient for slow servers on the fediverse
- if bereq.url ~ "^/proxy/" {
+ if (bereq.url ~ "^/proxy/") {
set bereq.first_byte_timeout = 300s;
}
diff --git a/lib/mix/tasks/pleroma/config.ex b/lib/mix/tasks/pleroma/config.ex
index 904c5a74b..18f99318d 100644
--- a/lib/mix/tasks/pleroma/config.ex
+++ b/lib/mix/tasks/pleroma/config.ex
@@ -32,7 +32,8 @@ def run(["migrate_from_db" | options]) do
@spec migrate_to_db(Path.t() | nil) :: any()
def migrate_to_db(file_path \\ nil) do
- if Pleroma.Config.get([:configurable_from_database]) do
+ with true <- Pleroma.Config.get([:configurable_from_database]),
+ :ok <- Pleroma.Config.DeprecationWarnings.warn() do
config_file =
if file_path do
file_path
@@ -46,7 +47,8 @@ def migrate_to_db(file_path \\ nil) do
do_migrate_to_db(config_file)
else
- migration_error()
+ :error -> deprecation_error()
+ _ -> migration_error()
end
end
@@ -120,6 +122,10 @@ defp migration_error do
)
end
+ defp deprecation_error do
+ shell_error("Migration is not allowed until all deprecation warnings have been resolved.")
+ end
+
if Code.ensure_loaded?(Config.Reader) do
defp config_header, do: "import Config\r\n\r\n"
defp read_file(config_file), do: Config.Reader.read_imports!(config_file)
diff --git a/lib/mix/tasks/pleroma/count_statuses.ex b/lib/mix/tasks/pleroma/count_statuses.ex
index e1e8195dd..8761d8f17 100644
--- a/lib/mix/tasks/pleroma/count_statuses.ex
+++ b/lib/mix/tasks/pleroma/count_statuses.ex
@@ -1,3 +1,7 @@
+# Pleroma: A lightweight social networking server
+# Copyright © 2017-2020 Pleroma Authors
+# SPDX-License-Identifier: AGPL-3.0-only
+
defmodule Mix.Tasks.Pleroma.CountStatuses do
@shortdoc "Re-counts statuses for all users"
diff --git a/lib/mix/tasks/pleroma/database.ex b/lib/mix/tasks/pleroma/database.ex
index 7d8f00b08..a01c36ece 100644
--- a/lib/mix/tasks/pleroma/database.ex
+++ b/lib/mix/tasks/pleroma/database.ex
@@ -99,7 +99,7 @@ def run(["fix_likes_collections"]) do
where: fragment("(?)->>'likes' is not null", object.data),
select: %{id: object.id, likes: fragment("(?)->>'likes'", object.data)}
)
- |> Pleroma.RepoStreamer.chunk_stream(100)
+ |> Pleroma.Repo.chunk_stream(100, :batches)
|> Stream.each(fn objects ->
ids =
objects
@@ -133,8 +133,7 @@ def run(["ensure_expiration"]) do
days = Pleroma.Config.get([:mrf_activity_expiration, :days], 365)
Pleroma.Activity
- |> join(:left, [a], u in assoc(a, :expiration))
- |> join(:inner, [a, _u], o in Object,
+ |> join(:inner, [a], o in Object,
on:
fragment(
"(?->>'id') = COALESCE((?)->'object'->> 'id', (?)->>'object')",
@@ -144,14 +143,20 @@ def run(["ensure_expiration"]) do
)
)
|> where(local: true)
- |> where([a, u], is_nil(u))
|> where([a], fragment("(? ->> 'type'::text) = 'Create'", a.data))
- |> where([_a, _u, o], fragment("?->>'type' = 'Note'", o.data))
- |> Pleroma.RepoStreamer.chunk_stream(100)
+ |> where([_a, o], fragment("?->>'type' = 'Note'", o.data))
+ |> Pleroma.Repo.chunk_stream(100, :batches)
|> Stream.each(fn activities ->
Enum.each(activities, fn activity ->
- expires_at = Timex.shift(activity.inserted_at, days: days)
- Pleroma.ActivityExpiration.create(activity, expires_at, false)
+ expires_at =
+ activity.inserted_at
+ |> DateTime.from_naive!("Etc/UTC")
+ |> Timex.shift(days: days)
+
+ Pleroma.Workers.PurgeExpiredActivity.enqueue(%{
+ activity_id: activity.id,
+ expires_at: expires_at
+ })
end)
end)
|> Stream.run()
diff --git a/lib/mix/tasks/pleroma/digest.ex b/lib/mix/tasks/pleroma/digest.ex
index 3595f912d..cac148b88 100644
--- a/lib/mix/tasks/pleroma/digest.ex
+++ b/lib/mix/tasks/pleroma/digest.ex
@@ -1,3 +1,7 @@
+# Pleroma: A lightweight social networking server
+# Copyright © 2017-2020 Pleroma Authors
+# SPDX-License-Identifier: AGPL-3.0-only
+
defmodule Mix.Tasks.Pleroma.Digest do
use Mix.Task
import Mix.Pleroma
diff --git a/lib/mix/tasks/pleroma/docs.ex b/lib/mix/tasks/pleroma/docs.ex
index 6088fc71d..ad5c37fc9 100644
--- a/lib/mix/tasks/pleroma/docs.ex
+++ b/lib/mix/tasks/pleroma/docs.ex
@@ -1,3 +1,7 @@
+# Pleroma: A lightweight social networking server
+# Copyright © 2017-2020 Pleroma Authors
+# SPDX-License-Identifier: AGPL-3.0-only
+
defmodule Mix.Tasks.Pleroma.Docs do
use Mix.Task
import Mix.Pleroma
diff --git a/lib/mix/tasks/pleroma/ecto/ecto.ex b/lib/mix/tasks/pleroma/ecto.ex
similarity index 100%
rename from lib/mix/tasks/pleroma/ecto/ecto.ex
rename to lib/mix/tasks/pleroma/ecto.ex
diff --git a/lib/mix/tasks/pleroma/email.ex b/lib/mix/tasks/pleroma/email.ex
index d3fac6ec8..bc5facc09 100644
--- a/lib/mix/tasks/pleroma/email.ex
+++ b/lib/mix/tasks/pleroma/email.ex
@@ -1,12 +1,16 @@
+# Pleroma: A lightweight social networking server
+# Copyright © 2017-2020 Pleroma Authors
+# SPDX-License-Identifier: AGPL-3.0-only
+
defmodule Mix.Tasks.Pleroma.Email do
use Mix.Task
import Mix.Pleroma
- @shortdoc "Simple Email test"
+ @shortdoc "Email administrative tasks"
@moduledoc File.read!("docs/administration/CLI_tasks/email.md")
def run(["test" | args]) do
- Mix.Pleroma.start_pleroma()
+ start_pleroma()
{options, [], []} =
OptionParser.parse(
@@ -21,4 +25,20 @@ def run(["test" | args]) do
shell_info("Test email has been sent to #{inspect(email.to)} from #{inspect(email.from)}")
end
+
+ def run(["resend_confirmation_emails"]) do
+ start_pleroma()
+
+ shell_info("Sending emails to all unconfirmed users")
+
+ Pleroma.User.Query.build(%{
+ local: true,
+ deactivated: false,
+ confirmation_pending: true,
+ invisible: false
+ })
+ |> Pleroma.Repo.chunk_stream(500)
+ |> Stream.each(&Pleroma.User.try_send_confirmation_email(&1))
+ |> Stream.run()
+ end
end
diff --git a/lib/mix/tasks/pleroma/emoji.ex b/lib/mix/tasks/pleroma/emoji.ex
index 8f52ee98d..1750373f9 100644
--- a/lib/mix/tasks/pleroma/emoji.ex
+++ b/lib/mix/tasks/pleroma/emoji.ex
@@ -183,7 +183,7 @@ def run(["gen-pack" | args]) do
IO.puts("Downloading the pack and generating SHA256")
- binary_archive = Tesla.get!(client(), src).body
+ {:ok, %{body: binary_archive}} = Pleroma.HTTP.get(src)
archive_sha = :crypto.hash(:sha256, binary_archive) |> Base.encode16()
IO.puts("SHA256 is #{archive_sha}")
@@ -252,7 +252,7 @@ defp fetch_and_decode!(from) do
end
defp fetch("http" <> _ = from) do
- with {:ok, %{body: body}} <- Tesla.get(client(), from) do
+ with {:ok, %{body: body}} <- Pleroma.HTTP.get(from) do
{:ok, body}
end
end
@@ -271,13 +271,5 @@ defp parse_global_opts(args) do
)
end
- defp client do
- middleware = [
- {Tesla.Middleware.FollowRedirects, [max_redirects: 3]}
- ]
-
- Tesla.client(middleware)
- end
-
defp default_manifest, do: Pleroma.Config.get!([:emoji, :default_manifest])
end
diff --git a/lib/mix/tasks/pleroma/instance.ex b/lib/mix/tasks/pleroma/instance.ex
index 91440b453..1915aacd9 100644
--- a/lib/mix/tasks/pleroma/instance.ex
+++ b/lib/mix/tasks/pleroma/instance.ex
@@ -33,7 +33,12 @@ def run(["gen" | rest]) do
uploads_dir: :string,
static_dir: :string,
listen_ip: :string,
- listen_port: :string
+ listen_port: :string,
+ strip_uploads: :string,
+ anonymize_uploads: :string,
+ dedupe_uploads: :string,
+ skip_release_env: :boolean,
+ release_env_file: :string
],
aliases: [
o: :output,
@@ -158,6 +163,30 @@ def run(["gen" | rest]) do
)
|> Path.expand()
+ strip_uploads =
+ get_option(
+ options,
+ :strip_uploads,
+ "Do you want to strip location (GPS) data from uploaded images? (y/n)",
+ "y"
+ ) === "y"
+
+ anonymize_uploads =
+ get_option(
+ options,
+ :anonymize_uploads,
+ "Do you want to anonymize the filenames of uploads? (y/n)",
+ "n"
+ ) === "y"
+
+ dedupe_uploads =
+ get_option(
+ options,
+ :dedupe_uploads,
+ "Do you want to deduplicate uploaded files? (y/n)",
+ "n"
+ ) === "y"
+
Config.put([:instance, :static_dir], static_dir)
secret = :crypto.strong_rand_bytes(64) |> Base.encode64() |> binary_part(0, 64)
@@ -188,7 +217,13 @@ def run(["gen" | rest]) do
uploads_dir: uploads_dir,
rum_enabled: rum_enabled,
listen_ip: listen_ip,
- listen_port: listen_port
+ listen_port: listen_port,
+ upload_filters:
+ upload_filters(%{
+ strip: strip_uploads,
+ anonymize: anonymize_uploads,
+ dedupe: dedupe_uploads
+ })
)
result_psql =
@@ -208,6 +243,24 @@ def run(["gen" | rest]) do
write_robots_txt(static_dir, indexable, template_dir)
+ if Keyword.get(options, :skip_release_env, false) do
+ shell_info("""
+ Release environment file is skip. Please generate the release env file before start.
+ `MIX_ENV=#{Mix.env()} mix pleroma.release_env gen`
+ """)
+ else
+ shell_info("Generation the environment file:")
+
+ release_env_args =
+ with path when not is_nil(path) <- Keyword.get(options, :release_env_file) do
+ ["gen", "--path", path]
+ else
+ _ -> ["gen"]
+ end
+
+ Mix.Tasks.Pleroma.ReleaseEnv.run(release_env_args)
+ end
+
shell_info(
"\n All files successfully written! Refer to the installation instructions for your platform for next steps."
)
@@ -247,4 +300,31 @@ defp write_robots_txt(static_dir, indexable, template_dir) do
File.write(robots_txt_path, robots_txt)
shell_info("Writing #{robots_txt_path}.")
end
+
+ defp upload_filters(filters) when is_map(filters) do
+ enabled_filters =
+ if filters.strip do
+ [Pleroma.Upload.Filter.ExifTool]
+ else
+ []
+ end
+
+ enabled_filters =
+ if filters.anonymize do
+ enabled_filters ++ [Pleroma.Upload.Filter.AnonymizeFilename]
+ else
+ enabled_filters
+ end
+
+ enabled_filters =
+ if filters.dedupe do
+ enabled_filters ++ [Pleroma.Upload.Filter.Dedupe]
+ else
+ enabled_filters
+ end
+
+ enabled_filters
+ end
+
+ defp upload_filters(_), do: []
end
diff --git a/lib/mix/tasks/pleroma/notification_settings.ex b/lib/mix/tasks/pleroma/notification_settings.ex
index 00f5ba7bf..f99275de1 100644
--- a/lib/mix/tasks/pleroma/notification_settings.ex
+++ b/lib/mix/tasks/pleroma/notification_settings.ex
@@ -1,3 +1,7 @@
+# Pleroma: A lightweight social networking server
+# Copyright © 2017-2020 Pleroma Authors
+# SPDX-License-Identifier: AGPL-3.0-only
+
defmodule Mix.Tasks.Pleroma.NotificationSettings do
@shortdoc "Enable&Disable privacy option for push notifications"
@moduledoc """
diff --git a/lib/mix/tasks/pleroma/relay.ex b/lib/mix/tasks/pleroma/relay.ex
index a6d8d6c1c..bb808ca47 100644
--- a/lib/mix/tasks/pleroma/relay.ex
+++ b/lib/mix/tasks/pleroma/relay.ex
@@ -21,10 +21,19 @@ def run(["follow", target]) do
end
end
- def run(["unfollow", target]) do
+ def run(["unfollow", target | rest]) do
start_pleroma()
- with {:ok, _activity} <- Relay.unfollow(target) do
+ {options, [], []} =
+ OptionParser.parse(
+ rest,
+ strict: [force: :boolean],
+ aliases: [f: :force]
+ )
+
+ force = Keyword.get(options, :force, false)
+
+ with {:ok, _activity} <- Relay.unfollow(target, %{force: force}) do
# put this task to sleep to allow the genserver to push out the messages
:timer.sleep(500)
else
diff --git a/lib/mix/tasks/pleroma/release_env.ex b/lib/mix/tasks/pleroma/release_env.ex
new file mode 100644
index 000000000..9da74ffcf
--- /dev/null
+++ b/lib/mix/tasks/pleroma/release_env.ex
@@ -0,0 +1,76 @@
+# Pleroma: A lightweight social networking server
+# Copyright © 2017-2020 Pleroma Authors
+# SPDX-License-Identifier: AGPL-3.0-only
+
+defmodule Mix.Tasks.Pleroma.ReleaseEnv do
+ use Mix.Task
+ import Mix.Pleroma
+
+ @shortdoc "Generate Pleroma environment file."
+ @moduledoc File.read!("docs/administration/CLI_tasks/release_environments.md")
+
+ def run(["gen" | rest]) do
+ {options, [], []} =
+ OptionParser.parse(
+ rest,
+ strict: [
+ force: :boolean,
+ path: :string
+ ],
+ aliases: [
+ p: :path,
+ f: :force
+ ]
+ )
+
+ file_path =
+ get_option(
+ options,
+ :path,
+ "Environment file path",
+ "./config/pleroma.env"
+ )
+
+ env_path = Path.expand(file_path)
+
+ proceed? =
+ if File.exists?(env_path) do
+ get_option(
+ options,
+ :force,
+ "Environment file already exists. Do you want to overwrite the #{env_path} file? (y/n)",
+ "n"
+ ) === "y"
+ else
+ true
+ end
+
+ if proceed? do
+ case do_generate(env_path) do
+ {:error, reason} ->
+ shell_error(
+ File.Error.message(%{action: "write to file", reason: reason, path: env_path})
+ )
+
+ _ ->
+ shell_info("\nThe file generated: #{env_path}.\n")
+
+ shell_info("""
+ WARNING: before start pleroma app please make sure to make the file read-only and non-modifiable.
+ Example:
+ chmod 0444 #{file_path}
+ chattr +i #{file_path}
+ """)
+ end
+ else
+ shell_info("\nThe file is exist. #{env_path}.\n")
+ end
+ end
+
+ def do_generate(path) do
+ content = "RELEASE_COOKIE=#{Base.encode32(:crypto.strong_rand_bytes(32))}"
+
+ File.mkdir_p!(Path.dirname(path))
+ File.write(path, content)
+ end
+end
diff --git a/lib/mix/tasks/pleroma/robotstxt.ex b/lib/mix/tasks/pleroma/robots_txt.ex
similarity index 100%
rename from lib/mix/tasks/pleroma/robotstxt.ex
rename to lib/mix/tasks/pleroma/robots_txt.ex
diff --git a/lib/mix/tasks/pleroma/user.ex b/lib/mix/tasks/pleroma/user.ex
index 01824aa18..a8d251411 100644
--- a/lib/mix/tasks/pleroma/user.ex
+++ b/lib/mix/tasks/pleroma/user.ex
@@ -179,7 +179,7 @@ def run(["deactivate_all_from_instance", instance]) do
start_pleroma()
Pleroma.User.Query.build(%{nickname: "@#{instance}"})
- |> Pleroma.RepoStreamer.chunk_stream(500)
+ |> Pleroma.Repo.chunk_stream(500, :batches)
|> Stream.each(fn users ->
users
|> Enum.each(fn user ->
@@ -196,17 +196,24 @@ def run(["set", nickname | rest]) do
OptionParser.parse(
rest,
strict: [
- moderator: :boolean,
admin: :boolean,
- locked: :boolean
+ confirmed: :boolean,
+ locked: :boolean,
+ moderator: :boolean
]
)
with %User{local: true} = user <- User.get_cached_by_nickname(nickname) do
user =
- case Keyword.get(options, :moderator) do
+ case Keyword.get(options, :admin) do
nil -> user
- value -> set_moderator(user, value)
+ value -> set_admin(user, value)
+ end
+
+ user =
+ case Keyword.get(options, :confirmed) do
+ nil -> user
+ value -> set_confirmed(user, value)
end
user =
@@ -216,9 +223,9 @@ def run(["set", nickname | rest]) do
end
_user =
- case Keyword.get(options, :admin) do
+ case Keyword.get(options, :moderator) do
nil -> user
- value -> set_admin(user, value)
+ value -> set_moderator(user, value)
end
else
_ ->
@@ -353,6 +360,42 @@ def run(["toggle_confirmed", nickname]) do
end
end
+ def run(["confirm_all"]) do
+ start_pleroma()
+
+ Pleroma.User.Query.build(%{
+ local: true,
+ deactivated: false,
+ is_moderator: false,
+ is_admin: false,
+ invisible: false
+ })
+ |> Pleroma.Repo.chunk_stream(500, :batches)
+ |> Stream.each(fn users ->
+ users
+ |> Enum.each(fn user -> User.need_confirmation(user, false) end)
+ end)
+ |> Stream.run()
+ end
+
+ def run(["unconfirm_all"]) do
+ start_pleroma()
+
+ Pleroma.User.Query.build(%{
+ local: true,
+ deactivated: false,
+ is_moderator: false,
+ is_admin: false,
+ invisible: false
+ })
+ |> Pleroma.Repo.chunk_stream(500, :batches)
+ |> Stream.each(fn users ->
+ users
+ |> Enum.each(fn user -> User.need_confirmation(user, true) end)
+ end)
+ |> Stream.run()
+ end
+
def run(["sign_out", nickname]) do
start_pleroma()
@@ -370,13 +413,13 @@ def run(["list"]) do
start_pleroma()
Pleroma.User.Query.build(%{local: true})
- |> Pleroma.RepoStreamer.chunk_stream(500)
+ |> Pleroma.Repo.chunk_stream(500, :batches)
|> Stream.each(fn users ->
users
|> Enum.each(fn user ->
shell_info(
"#{user.nickname} moderator: #{user.is_moderator}, admin: #{user.is_admin}, locked: #{
- user.locked
+ user.is_locked
}, deactivated: #{user.deactivated}"
)
end)
@@ -404,10 +447,17 @@ defp set_admin(user, value) do
defp set_locked(user, value) do
{:ok, user} =
user
- |> Changeset.change(%{locked: value})
+ |> Changeset.change(%{is_locked: value})
|> User.update_and_set_cache()
- shell_info("Locked status of #{user.nickname}: #{user.locked}")
+ shell_info("Locked status of #{user.nickname}: #{user.is_locked}")
+ user
+ end
+
+ defp set_confirmed(user, value) do
+ {:ok, user} = User.need_confirmation(user, !value)
+
+ shell_info("Confirmation pending status of #{user.nickname}: #{user.confirmation_pending}")
user
end
end
diff --git a/lib/transports.ex b/lib/phoenix/transports/web_socket/raw.ex
similarity index 90%
rename from lib/transports.ex
rename to lib/phoenix/transports/web_socket/raw.ex
index aab7fad99..c3665bebe 100644
--- a/lib/transports.ex
+++ b/lib/phoenix/transports/web_socket/raw.ex
@@ -31,7 +31,12 @@ def init(%Plug.Conn{method: "GET"} = conn, {endpoint, handler, transport}) do
case conn do
%{halted: false} = conn ->
- case Transport.connect(endpoint, handler, transport, __MODULE__, nil, conn.params) do
+ case handler.connect(%{
+ endpoint: endpoint,
+ transport: transport,
+ options: [serializer: nil],
+ params: conn.params
+ }) do
{:ok, socket} ->
{:ok, conn, {__MODULE__, {socket, opts}}}
diff --git a/lib/pleroma/activity.ex b/lib/pleroma/activity.ex
index 97feebeaa..553834da0 100644
--- a/lib/pleroma/activity.ex
+++ b/lib/pleroma/activity.ex
@@ -7,7 +7,6 @@ defmodule Pleroma.Activity do
alias Pleroma.Activity
alias Pleroma.Activity.Queries
- alias Pleroma.ActivityExpiration
alias Pleroma.Bookmark
alias Pleroma.Notification
alias Pleroma.Object
@@ -15,6 +14,7 @@ defmodule Pleroma.Activity do
alias Pleroma.ReportNote
alias Pleroma.ThreadMute
alias Pleroma.User
+ alias Pleroma.Web.ActivityPub.ActivityPub
import Ecto.Changeset
import Ecto.Query
@@ -60,8 +60,6 @@ defmodule Pleroma.Activity do
# typical case.
has_one(:object, Object, on_delete: :nothing, foreign_key: :id)
- has_one(:expiration, ActivityExpiration, on_delete: :delete_all)
-
timestamps()
end
@@ -156,6 +154,18 @@ def get_bookmark(%Activity{} = activity, %User{} = user) do
def get_bookmark(_, _), do: nil
+ def get_report(activity_id) do
+ opts = %{
+ type: "Flag",
+ skip_preload: true,
+ preload_report_notes: true
+ }
+
+ ActivityPub.fetch_activities_query([], opts)
+ |> where(id: ^activity_id)
+ |> Repo.one()
+ end
+
def change(struct, params \\ %{}) do
struct
|> cast(params, [:data, :recipients])
@@ -304,14 +314,14 @@ def all_by_actor_and_id(actor, status_ids) do
|> Repo.all()
end
- def follow_requests_for_actor(%Pleroma.User{ap_id: ap_id}) do
+ def follow_requests_for_actor(%User{ap_id: ap_id}) do
ap_id
|> Queries.by_object_id()
|> Queries.by_type("Follow")
|> where([a], fragment("? ->> 'state' = 'pending'", a.data))
end
- def following_requests_for_actor(%Pleroma.User{ap_id: ap_id}) do
+ def following_requests_for_actor(%User{ap_id: ap_id}) do
Queries.by_type("Follow")
|> where([a], fragment("?->>'state' = 'pending'", a.data))
|> where([a], a.actor == ^ap_id)
diff --git a/lib/pleroma/activity/ir/topics.ex b/lib/pleroma/activity/ir/topics.ex
index 9e65bedad..fe2e8cb5c 100644
--- a/lib/pleroma/activity/ir/topics.ex
+++ b/lib/pleroma/activity/ir/topics.ex
@@ -40,7 +40,8 @@ defp visibility_tags(object, activity) do
end
defp item_creation_tags(tags, object, %{data: %{"type" => "Create"}} = activity) do
- tags ++ hashtags_to_topics(object) ++ attachment_topics(object, activity)
+ tags ++
+ remote_topics(activity) ++ hashtags_to_topics(object) ++ attachment_topics(object, activity)
end
defp item_creation_tags(tags, _, _) do
@@ -55,9 +56,19 @@ defp hashtags_to_topics(%{data: %{"tag" => tags}}) do
defp hashtags_to_topics(_), do: []
+ defp remote_topics(%{local: true}), do: []
+
+ defp remote_topics(%{actor: actor}) when is_binary(actor),
+ do: ["public:remote:" <> URI.parse(actor).host]
+
+ defp remote_topics(_), do: []
+
defp attachment_topics(%{data: %{"attachment" => []}}, _act), do: []
defp attachment_topics(_object, %{local: true}), do: ["public:media", "public:local:media"]
+ defp attachment_topics(_object, %{actor: actor}) when is_binary(actor),
+ do: ["public:media", "public:remote:media:" <> URI.parse(actor).host]
+
defp attachment_topics(_object, _act), do: ["public:media"]
end
diff --git a/lib/pleroma/activity_expiration.ex b/lib/pleroma/activity_expiration.ex
deleted file mode 100644
index 955f0578e..000000000
--- a/lib/pleroma/activity_expiration.ex
+++ /dev/null
@@ -1,74 +0,0 @@
-# Pleroma: A lightweight social networking server
-# Copyright © 2017-2020 Pleroma Authors
-# SPDX-License-Identifier: AGPL-3.0-only
-
-defmodule Pleroma.ActivityExpiration do
- use Ecto.Schema
-
- alias Pleroma.Activity
- alias Pleroma.ActivityExpiration
- alias Pleroma.Repo
-
- import Ecto.Changeset
- import Ecto.Query
-
- @type t :: %__MODULE__{}
- @min_activity_lifetime :timer.hours(1)
-
- schema "activity_expirations" do
- belongs_to(:activity, Activity, type: FlakeId.Ecto.CompatType)
- field(:scheduled_at, :naive_datetime)
- end
-
- def changeset(%ActivityExpiration{} = expiration, attrs, validate_scheduled_at) do
- expiration
- |> cast(attrs, [:scheduled_at])
- |> validate_required([:scheduled_at])
- |> validate_scheduled_at(validate_scheduled_at)
- end
-
- def get_by_activity_id(activity_id) do
- ActivityExpiration
- |> where([exp], exp.activity_id == ^activity_id)
- |> Repo.one()
- end
-
- def create(%Activity{} = activity, scheduled_at, validate_scheduled_at \\ true) do
- %ActivityExpiration{activity_id: activity.id}
- |> changeset(%{scheduled_at: scheduled_at}, validate_scheduled_at)
- |> Repo.insert()
- end
-
- def due_expirations(offset \\ 0) do
- naive_datetime =
- NaiveDateTime.utc_now()
- |> NaiveDateTime.add(offset, :millisecond)
-
- ActivityExpiration
- |> where([exp], exp.scheduled_at < ^naive_datetime)
- |> limit(50)
- |> preload(:activity)
- |> Repo.all()
- |> Enum.reject(fn %{activity: activity} ->
- Activity.pinned_by_actor?(activity)
- end)
- end
-
- def validate_scheduled_at(changeset, false), do: changeset
-
- def validate_scheduled_at(changeset, true) do
- validate_change(changeset, :scheduled_at, fn _, scheduled_at ->
- if not expires_late_enough?(scheduled_at) do
- [scheduled_at: "an ephemeral activity must live for at least one hour"]
- else
- []
- end
- end)
- end
-
- def expires_late_enough?(scheduled_at) do
- now = NaiveDateTime.utc_now()
- diff = NaiveDateTime.diff(scheduled_at, now, :millisecond)
- diff > @min_activity_lifetime
- end
-end
diff --git a/lib/pleroma/application.ex b/lib/pleroma/application.ex
index c39e24919..7c4cd9626 100644
--- a/lib/pleroma/application.ex
+++ b/lib/pleroma/application.ex
@@ -52,11 +52,10 @@ def start(_type, _args) do
Pleroma.HTML.compile_scrubbers()
Pleroma.Config.Oban.warn()
Config.DeprecationWarnings.warn()
- Pleroma.Plugs.HTTPSecurityPlug.warn_if_disabled()
+ Pleroma.Web.Plugs.HTTPSecurityPlug.warn_if_disabled()
Pleroma.ApplicationRequirements.verify!()
setup_instrumenters()
load_custom_modules()
- check_system_commands()
Pleroma.Docs.JSON.compile()
adapter = Application.get_env(:tesla, :adapter)
@@ -89,18 +88,19 @@ def start(_type, _args) do
Pleroma.Repo,
Config.TransferTask,
Pleroma.Emoji,
- Pleroma.Plugs.RateLimiter.Supervisor
+ Pleroma.Web.Plugs.RateLimiter.Supervisor
] ++
cachex_children() ++
http_children(adapter, @env) ++
[
Pleroma.Stats,
Pleroma.JobQueueMonitor,
+ {Majic.Pool, [name: Pleroma.MajicPool, pool_size: Config.get([:majic_pool, :size], 2)]},
{Oban, Config.get(Oban)}
] ++
task_children(@env) ++
- streamer_child(@env) ++
- chat_child(@env, chat_enabled?()) ++
+ dont_run_in_test(@env) ++
+ chat_child(chat_enabled?()) ++
[
Pleroma.Web.Endpoint,
Pleroma.Gopher.Server
@@ -151,7 +151,10 @@ defp setup_instrumenters do
Pleroma.Web.Endpoint.MetricsExporter.setup()
Pleroma.Web.Endpoint.PipelineInstrumenter.setup()
- Pleroma.Web.Endpoint.Instrumenter.setup()
+
+ # Note: disabled until prometheus-phx is integrated into prometheus-phoenix:
+ # Pleroma.Web.Endpoint.Instrumenter.setup()
+ PrometheusPhx.setup()
end
defp cachex_children do
@@ -165,7 +168,11 @@ defp cachex_children do
build_cachex("web_resp", limit: 2500),
build_cachex("emoji_packs", expiration: emoji_packs_expiration(), limit: 10),
build_cachex("failed_proxy_url", limit: 2500),
- build_cachex("banned_urls", default_ttl: :timer.hours(24 * 30), limit: 5_000)
+ build_cachex("banned_urls", default_ttl: :timer.hours(24 * 30), limit: 5_000),
+ build_cachex("chat_message_id_idempotency_key",
+ expiration: chat_message_id_idempotency_key_expiration(),
+ limit: 500_000
+ )
]
end
@@ -175,6 +182,9 @@ defp emoji_packs_expiration,
defp idempotency_expiration,
do: expiration(default: :timer.seconds(6 * 60 * 60), interval: :timer.seconds(60))
+ defp chat_message_id_idempotency_key_expiration,
+ do: expiration(default: :timer.minutes(2), interval: :timer.seconds(60))
+
defp seconds_valid_interval,
do: :timer.seconds(Config.get!([Pleroma.Captcha, :seconds_valid]))
@@ -188,24 +198,28 @@ def build_cachex(type, opts),
defp chat_enabled?, do: Config.get([:chat, :enabled])
- defp streamer_child(env) when env in [:test, :benchmark], do: []
+ defp dont_run_in_test(env) when env in [:test, :benchmark], do: []
- defp streamer_child(_) do
+ defp dont_run_in_test(_) do
[
{Registry,
[
name: Pleroma.Web.Streamer.registry(),
keys: :duplicate,
partitions: System.schedulers_online()
- ]}
+ ]},
+ Pleroma.Web.FedSockets.Supervisor
]
end
- defp chat_child(_env, true) do
- [Pleroma.Web.ChatChannel.ChatChannelState]
+ defp chat_child(true) do
+ [
+ Pleroma.Web.ChatChannel.ChatChannelState,
+ {Phoenix.PubSub, [name: Pleroma.PubSub, adapter: Phoenix.PubSub.PG2]}
+ ]
end
- defp chat_child(_, _), do: []
+ defp chat_child(_), do: []
defp task_children(:test) do
[
@@ -259,21 +273,4 @@ defp http_children(Tesla.Adapter.Gun, _) do
end
defp http_children(_, _), do: []
-
- defp check_system_commands do
- filters = Config.get([Pleroma.Upload, :filters])
-
- check_filter = fn filter, command_required ->
- with true <- filter in filters,
- false <- Pleroma.Utils.command_available?(command_required) do
- Logger.error(
- "#{filter} is specified in list of Pleroma.Upload filters, but the #{command_required} command is not found"
- )
- end
- end
-
- check_filter.(Pleroma.Upload.Filters.Exiftool, "exiftool")
- check_filter.(Pleroma.Upload.Filters.Mogrify, "mogrify")
- check_filter.(Pleroma.Upload.Filters.Mogrifun, "mogrify")
- end
end
diff --git a/lib/pleroma/application_requirements.ex b/lib/pleroma/application_requirements.ex
index 16f62b6f5..b977257a3 100644
--- a/lib/pleroma/application_requirements.ex
+++ b/lib/pleroma/application_requirements.ex
@@ -9,6 +9,9 @@ defmodule Pleroma.ApplicationRequirements do
defmodule VerifyError, do: defexception([:message])
+ alias Pleroma.Config
+ alias Pleroma.Helpers.MediaHelper
+
import Ecto.Query
require Logger
@@ -16,7 +19,8 @@ defmodule VerifyError, do: defexception([:message])
@spec verify!() :: :ok | VerifyError.t()
def verify! do
:ok
- |> check_confirmation_accounts!
+ |> check_system_commands!()
+ |> check_confirmation_accounts!()
|> check_migrations_applied!()
|> check_welcome_message_config!()
|> check_rum!()
@@ -48,7 +52,9 @@ def check_confirmation_accounts!(:ok) do
if Pleroma.Config.get([:instance, :account_activation_required]) &&
not Pleroma.Config.get([Pleroma.Emails.Mailer, :enabled]) do
Logger.error(
- "Account activation enabled, but no Mailer settings enabled.\nPlease set config :pleroma, :instance, account_activation_required: false\nOtherwise setup and enable Mailer."
+ "Account activation enabled, but no Mailer settings enabled.\n" <>
+ "Please set config :pleroma, :instance, account_activation_required: false\n" <>
+ "Otherwise setup and enable Mailer."
)
{:error,
@@ -81,7 +87,9 @@ def check_migrations_applied!(:ok) do
Enum.map(down_migrations, fn {:down, id, name} -> "- #{name} (#{id})\n" end)
Logger.error(
- "The following migrations were not applied:\n#{down_migrations_text}If you want to start Pleroma anyway, set\nconfig :pleroma, :i_am_aware_this_may_cause_data_loss, disable_migration_check: true"
+ "The following migrations were not applied:\n#{down_migrations_text}" <>
+ "If you want to start Pleroma anyway, set\n" <>
+ "config :pleroma, :i_am_aware_this_may_cause_data_loss, disable_migration_check: true"
)
{:error, "Unapplied Migrations detected"}
@@ -124,14 +132,22 @@ defp do_check_rum!(setting, migrate) do
case {setting, migrate} do
{true, false} ->
Logger.error(
- "Use `RUM` index is enabled, but were not applied migrations for it.\nIf you want to start Pleroma anyway, set\nconfig :pleroma, :database, rum_enabled: false\nOtherwise apply the following migrations:\n`mix ecto.migrate --migrations-path priv/repo/optional_migrations/rum_indexing/`"
+ "Use `RUM` index is enabled, but were not applied migrations for it.\n" <>
+ "If you want to start Pleroma anyway, set\n" <>
+ "config :pleroma, :database, rum_enabled: false\n" <>
+ "Otherwise apply the following migrations:\n" <>
+ "`mix ecto.migrate --migrations-path priv/repo/optional_migrations/rum_indexing/`"
)
{:error, "Unapplied RUM Migrations detected"}
{false, true} ->
Logger.error(
- "Detected applied migrations to use `RUM` index, but `RUM` isn't enable in settings.\nIf you want to use `RUM`, set\nconfig :pleroma, :database, rum_enabled: true\nOtherwise roll `RUM` migrations back.\n`mix ecto.rollback --migrations-path priv/repo/optional_migrations/rum_indexing/`"
+ "Detected applied migrations to use `RUM` index, but `RUM` isn't enable in settings.\n" <>
+ "If you want to use `RUM`, set\n" <>
+ "config :pleroma, :database, rum_enabled: true\n" <>
+ "Otherwise roll `RUM` migrations back.\n" <>
+ "`mix ecto.rollback --migrations-path priv/repo/optional_migrations/rum_indexing/`"
)
{:error, "RUM Migrations detected"}
@@ -140,4 +156,50 @@ defp do_check_rum!(setting, migrate) do
:ok
end
end
+
+ defp check_system_commands!(:ok) do
+ filter_commands_statuses = [
+ check_filter(Pleroma.Upload.Filters.Exiftool, "exiftool"),
+ check_filter(Pleroma.Upload.Filters.Mogrify, "mogrify"),
+ check_filter(Pleroma.Upload.Filters.Mogrifun, "mogrify")
+ ]
+
+ preview_proxy_commands_status =
+ if !Config.get([:media_preview_proxy, :enabled]) or
+ MediaHelper.missing_dependencies() == [] do
+ true
+ else
+ Logger.error(
+ "The following dependencies required by Media preview proxy " <>
+ "(which is currently enabled) are not installed: " <>
+ inspect(MediaHelper.missing_dependencies())
+ )
+
+ false
+ end
+
+ if Enum.all?([preview_proxy_commands_status | filter_commands_statuses], & &1) do
+ :ok
+ else
+ {:error,
+ "System commands missing. Check logs and see `docs/installation` for more details."}
+ end
+ end
+
+ defp check_system_commands!(result), do: result
+
+ defp check_filter(filter, command_required) do
+ filters = Config.get([Pleroma.Upload, :filters])
+
+ if filter in filters and not Pleroma.Utils.command_available?(command_required) do
+ Logger.error(
+ "#{filter} is specified in list of Pleroma.Upload filters, but the " <>
+ "#{command_required} command is not found"
+ )
+
+ false
+ else
+ true
+ end
+ end
end
diff --git a/lib/pleroma/bbs/authenticator.ex b/lib/pleroma/bbs/authenticator.ex
index 815de7002..83ebb756d 100644
--- a/lib/pleroma/bbs/authenticator.ex
+++ b/lib/pleroma/bbs/authenticator.ex
@@ -4,8 +4,8 @@
defmodule Pleroma.BBS.Authenticator do
use Sshd.PasswordAuthenticator
- alias Pleroma.Plugs.AuthenticationPlug
alias Pleroma.User
+ alias Pleroma.Web.Plugs.AuthenticationPlug
def authenticate(username, password) do
username = to_string(username)
diff --git a/lib/pleroma/captcha/captcha.ex b/lib/pleroma/captcha.ex
similarity index 100%
rename from lib/pleroma/captcha/captcha.ex
rename to lib/pleroma/captcha.ex
diff --git a/lib/pleroma/captcha/kocaptcha.ex b/lib/pleroma/captcha/kocaptcha.ex
index 337506647..201b55ab4 100644
--- a/lib/pleroma/captcha/kocaptcha.ex
+++ b/lib/pleroma/captcha/kocaptcha.ex
@@ -10,7 +10,7 @@ defmodule Pleroma.Captcha.Kocaptcha do
def new do
endpoint = Pleroma.Config.get!([__MODULE__, :endpoint])
- case Tesla.get(endpoint <> "/new") do
+ case Pleroma.HTTP.get(endpoint <> "/new") do
{:error, _} ->
%{error: :kocaptcha_service_unavailable}
diff --git a/lib/pleroma/captcha/captcha_service.ex b/lib/pleroma/captcha/service.ex
similarity index 100%
rename from lib/pleroma/captcha/captcha_service.ex
rename to lib/pleroma/captcha/service.ex
diff --git a/lib/pleroma/chat.ex b/lib/pleroma/chat.ex
index 24a86371e..28007cd9f 100644
--- a/lib/pleroma/chat.ex
+++ b/lib/pleroma/chat.ex
@@ -6,7 +6,9 @@ defmodule Pleroma.Chat do
use Ecto.Schema
import Ecto.Changeset
+ import Ecto.Query
+ alias Pleroma.Chat
alias Pleroma.Repo
alias Pleroma.User
@@ -16,6 +18,7 @@ defmodule Pleroma.Chat do
It is a helper only, to make it easy to display a list of chats with other people, ordered by last bump. The actual messages are retrieved by querying the recipients of the ChatMessages.
"""
+ @type t :: %__MODULE__{}
@primary_key {:id, FlakeId.Ecto.CompatType, autogenerate: true}
schema "chats" do
@@ -39,16 +42,28 @@ def changeset(struct, params) do
|> unique_constraint(:user_id, name: :chats_user_id_recipient_index)
end
+ @spec get_by_user_and_id(User.t(), FlakeId.Ecto.CompatType.t()) ::
+ {:ok, t()} | {:error, :not_found}
+ def get_by_user_and_id(%User{id: user_id}, id) do
+ from(c in __MODULE__,
+ where: c.id == ^id,
+ where: c.user_id == ^user_id
+ )
+ |> Repo.find_resource()
+ end
+
+ @spec get_by_id(FlakeId.Ecto.CompatType.t()) :: t() | nil
def get_by_id(id) do
- __MODULE__
- |> Repo.get(id)
+ Repo.get(__MODULE__, id)
end
+ @spec get(FlakeId.Ecto.CompatType.t(), String.t()) :: t() | nil
def get(user_id, recipient) do
- __MODULE__
- |> Repo.get_by(user_id: user_id, recipient: recipient)
+ Repo.get_by(__MODULE__, user_id: user_id, recipient: recipient)
end
+ @spec get_or_create(FlakeId.Ecto.CompatType.t(), String.t()) ::
+ {:ok, t()} | {:error, Ecto.Changeset.t()}
def get_or_create(user_id, recipient) do
%__MODULE__{}
|> changeset(%{user_id: user_id, recipient: recipient})
@@ -60,6 +75,8 @@ def get_or_create(user_id, recipient) do
)
end
+ @spec bump_or_create(FlakeId.Ecto.CompatType.t(), String.t()) ::
+ {:ok, t()} | {:error, Ecto.Changeset.t()}
def bump_or_create(user_id, recipient) do
%__MODULE__{}
|> changeset(%{user_id: user_id, recipient: recipient})
@@ -69,4 +86,12 @@ def bump_or_create(user_id, recipient) do
conflict_target: [:user_id, :recipient]
)
end
+
+ @spec for_user_query(FlakeId.Ecto.CompatType.t()) :: Ecto.Query.t()
+ def for_user_query(user_id) do
+ from(c in Chat,
+ where: c.user_id == ^user_id,
+ order_by: [desc: c.updated_at]
+ )
+ end
end
diff --git a/lib/pleroma/config/deprecation_warnings.ex b/lib/pleroma/config/deprecation_warnings.ex
index 2bfe4ddba..59c6b0f58 100644
--- a/lib/pleroma/config/deprecation_warnings.ex
+++ b/lib/pleroma/config/deprecation_warnings.ex
@@ -8,7 +8,7 @@ defmodule Pleroma.Config.DeprecationWarnings do
require Logger
alias Pleroma.Config
- @type config_namespace() :: [atom()]
+ @type config_namespace() :: atom() | [atom()]
@type config_map() :: {config_namespace(), config_namespace(), String.t()}
@mrf_config_map [
@@ -26,37 +26,26 @@ def check_hellthread_threshold do
!!!DEPRECATION WARNING!!!
You are using the old configuration mechanism for the hellthread filter. Please check config.md.
""")
- end
- end
- def mrf_user_allowlist do
- config = Config.get(:mrf_user_allowlist)
-
- if config && Enum.any?(config, fn {k, _} -> is_atom(k) end) do
- rewritten =
- Enum.reduce(Config.get(:mrf_user_allowlist), Map.new(), fn {k, v}, acc ->
- Map.put(acc, to_string(k), v)
- end)
-
- Config.put(:mrf_user_allowlist, rewritten)
-
- Logger.error("""
- !!!DEPRECATION WARNING!!!
- As of Pleroma 2.0.7, the `mrf_user_allowlist` setting changed of format.
- Pleroma 2.1 will remove support for the old format. Please change your configuration to match this:
-
- config :pleroma, :mrf_user_allowlist, #{inspect(rewritten, pretty: true)}
- """)
+ :error
+ else
+ :ok
end
end
def warn do
- check_hellthread_threshold()
- mrf_user_allowlist()
- check_old_mrf_config()
- check_media_proxy_whitelist_config()
- check_welcome_message_config()
- check_gun_pool_options()
+ with :ok <- check_hellthread_threshold(),
+ :ok <- check_old_mrf_config(),
+ :ok <- check_media_proxy_whitelist_config(),
+ :ok <- check_welcome_message_config(),
+ :ok <- check_gun_pool_options(),
+ :ok <- check_activity_expiration_config(),
+ :ok <- check_remote_ip_plug_name() do
+ :ok
+ else
+ _ ->
+ :error
+ end
end
def check_welcome_message_config do
@@ -69,10 +58,14 @@ def check_welcome_message_config do
if use_old_config do
Logger.error("""
!!!DEPRECATION WARNING!!!
- Your config is using the old namespace for Welcome messages configuration. You need to change to the new namespace:
- \n* `config :pleroma, :instance, welcome_user_nickname` is now `config :pleroma, :welcome, :direct_message, :sender_nickname`
- \n* `config :pleroma, :instance, welcome_message` is now `config :pleroma, :welcome, :direct_message, :message`
+ Your config is using the old namespace for Welcome messages configuration. You need to convert to the new namespace. e.g.,
+ \n* `config :pleroma, :instance, welcome_user_nickname` and `config :pleroma, :instance, welcome_message` are now equal to:
+ \n* `config :pleroma, :welcome, direct_message: [enabled: true, sender_nickname: "NICKNAME", message: "Your welcome message"]`"
""")
+
+ :error
+ else
+ :ok
end
end
@@ -100,8 +93,11 @@ def move_namespace_and_warn(config_map, warning_preface) do
end
end)
- if warning != "" do
+ if warning == "" do
+ :ok
+ else
Logger.warn(warning_preface <> warning)
+ :error
end
end
@@ -114,6 +110,10 @@ def check_media_proxy_whitelist_config do
!!!DEPRECATION WARNING!!!
Your config is using old format (only domain) for MediaProxy whitelist option. Setting should work for now, but you are advised to change format to scheme with port to prevent possible issues later.
""")
+
+ :error
+ else
+ :ok
end
end
@@ -123,7 +123,7 @@ def check_gun_pool_options do
if timeout = pool_config[:await_up_timeout] do
Logger.warn("""
!!!DEPRECATION WARNING!!!
- Your config is using old setting name `await_up_timeout` instead of `connect_timeout`. Setting should work for now, but you are advised to change format to scheme with port to prevent possible issues later.
+ Your config is using old setting `config :pleroma, :connections_pool, await_up_timeout`. Please change to `config :pleroma, :connections_pool, connect_timeout` to ensure compatibility with future releases.
""")
Config.put(:connections_pool, Keyword.put_new(pool_config, :connect_timeout, timeout))
@@ -156,6 +156,41 @@ def check_gun_pool_options do
Logger.warn(Enum.join([warning_preface | pool_warnings]))
Config.put(:pools, updated_config)
+ :error
+ else
+ :ok
end
end
+
+ @spec check_activity_expiration_config() :: :ok | nil
+ def check_activity_expiration_config do
+ warning_preface = """
+ !!!DEPRECATION WARNING!!!
+ Your config is using old namespace for activity expiration configuration. Setting should work for now, but you are advised to change to new namespace to prevent possible issues later:
+ """
+
+ move_namespace_and_warn(
+ [
+ {Pleroma.ActivityExpiration, Pleroma.Workers.PurgeExpiredActivity,
+ "\n* `config :pleroma, Pleroma.ActivityExpiration` is now `config :pleroma, Pleroma.Workers.PurgeExpiredActivity`"}
+ ],
+ warning_preface
+ )
+ end
+
+ @spec check_remote_ip_plug_name() :: :ok | nil
+ def check_remote_ip_plug_name do
+ warning_preface = """
+ !!!DEPRECATION WARNING!!!
+ Your config is using old namespace for RemoteIp Plug. Setting should work for now, but you are advised to change to new namespace to prevent possible issues later:
+ """
+
+ move_namespace_and_warn(
+ [
+ {Pleroma.Plugs.RemoteIp, Pleroma.Web.Plugs.RemoteIp,
+ "\n* `config :pleroma, Pleroma.Plugs.RemoteIp` is now `config :pleroma, Pleroma.Web.Plugs.RemoteIp`"}
+ ],
+ warning_preface
+ )
+ end
end
diff --git a/lib/pleroma/config/oban.ex b/lib/pleroma/config/oban.ex
index c2d56ebab..8e0351d52 100644
--- a/lib/pleroma/config/oban.ex
+++ b/lib/pleroma/config/oban.ex
@@ -1,3 +1,7 @@
+# Pleroma: A lightweight social networking server
+# Copyright © 2017-2020 Pleroma Authors
+# SPDX-License-Identifier: AGPL-3.0-only
+
defmodule Pleroma.Config.Oban do
require Logger
@@ -5,7 +9,11 @@ def warn do
oban_config = Pleroma.Config.get(Oban)
crontab =
- [Pleroma.Workers.Cron.StatsWorker]
+ [
+ Pleroma.Workers.Cron.StatsWorker,
+ Pleroma.Workers.Cron.PurgeExpiredActivitiesWorker,
+ Pleroma.Workers.Cron.ClearOauthTokenWorker
+ ]
|> Enum.reduce(oban_config[:crontab], fn removed_worker, acc ->
with acc when is_list(acc) <- acc,
setting when is_tuple(setting) <-
diff --git a/lib/pleroma/config/config_db.ex b/lib/pleroma/config_db.ex
similarity index 100%
rename from lib/pleroma/config/config_db.ex
rename to lib/pleroma/config_db.ex
diff --git a/lib/pleroma/conversation.ex b/lib/pleroma/conversation.ex
index e76eb0087..77933f0be 100644
--- a/lib/pleroma/conversation.ex
+++ b/lib/pleroma/conversation.ex
@@ -43,7 +43,7 @@ def get_for_ap_id(ap_id) do
def maybe_create_recipientships(participation, activity) do
participation = Repo.preload(participation, :recipients)
- if participation.recipients |> Enum.empty?() do
+ if Enum.empty?(participation.recipients) do
recipients = User.get_all_by_ap_id(activity.recipients)
RecipientShip.create(recipients, participation)
end
@@ -69,10 +69,6 @@ def create_or_bump_for(activity, opts \\ []) do
Enum.map(users, fn user ->
invisible_conversation = Enum.any?(users, &User.blocks?(user, &1))
- unless invisible_conversation do
- User.increment_unread_conversation_count(conversation, user)
- end
-
opts = Keyword.put(opts, :invisible_conversation, invisible_conversation)
{:ok, participation} =
diff --git a/lib/pleroma/conversation/participation.ex b/lib/pleroma/conversation/participation.ex
index 8bc3e85d6..4c32b273a 100644
--- a/lib/pleroma/conversation/participation.ex
+++ b/lib/pleroma/conversation/participation.ex
@@ -63,21 +63,10 @@ def mark_as_read(%User{} = user, %Conversation{} = conversation) do
end
end
- def mark_as_read(participation) do
- __MODULE__
- |> where(id: ^participation.id)
- |> update(set: [read: true])
- |> select([p], p)
- |> Repo.update_all([])
- |> case do
- {1, [participation]} ->
- participation = Repo.preload(participation, :user)
- User.set_unread_conversation_count(participation.user)
- {:ok, participation}
-
- error ->
- error
- end
+ def mark_as_read(%__MODULE__{} = participation) do
+ participation
+ |> change(read: true)
+ |> Repo.update()
end
def mark_all_as_read(%User{local: true} = user, %User{} = target_user) do
@@ -93,7 +82,6 @@ def mark_all_as_read(%User{local: true} = user, %User{} = target_user) do
|> update([p], set: [read: true])
|> Repo.update_all([])
- {:ok, user} = User.set_unread_conversation_count(user)
{:ok, user, []}
end
@@ -108,7 +96,6 @@ def mark_all_as_read(%User{} = user) do
|> select([p], p)
|> Repo.update_all([])
- {:ok, user} = User.set_unread_conversation_count(user)
{:ok, user, participations}
end
@@ -220,6 +207,12 @@ def set_recipients(participation, user_ids) do
{:ok, Repo.preload(participation, :recipients, force: true)}
end
+ @spec unread_count(User.t()) :: integer()
+ def unread_count(%User{id: user_id}) do
+ from(q in __MODULE__, where: q.user_id == ^user_id and q.read == false)
+ |> Repo.aggregate(:count, :id)
+ end
+
def unread_conversation_count_for_user(user) do
from(p in __MODULE__,
where: p.user_id == ^user.id,
diff --git a/lib/pleroma/conversation/participation_recipient_ship.ex b/lib/pleroma/conversation/participation/recipient_ship.ex
similarity index 100%
rename from lib/pleroma/conversation/participation_recipient_ship.ex
rename to lib/pleroma/conversation/participation/recipient_ship.ex
diff --git a/lib/pleroma/docs/generator.ex b/lib/pleroma/docs/generator.ex
index a671a6278..a70f83b73 100644
--- a/lib/pleroma/docs/generator.ex
+++ b/lib/pleroma/docs/generator.ex
@@ -1,3 +1,7 @@
+# Pleroma: A lightweight social networking server
+# Copyright © 2017-2020 Pleroma Authors
+# SPDX-License-Identifier: AGPL-3.0-only
+
defmodule Pleroma.Docs.Generator do
@callback process(keyword()) :: {:ok, String.t()}
diff --git a/lib/pleroma/docs/json.ex b/lib/pleroma/docs/json.ex
index feeb4320e..13618b509 100644
--- a/lib/pleroma/docs/json.ex
+++ b/lib/pleroma/docs/json.ex
@@ -1,3 +1,7 @@
+# Pleroma: A lightweight social networking server
+# Copyright © 2017-2020 Pleroma Authors
+# SPDX-License-Identifier: AGPL-3.0-only
+
defmodule Pleroma.Docs.JSON do
@behaviour Pleroma.Docs.Generator
@external_resource "config/description.exs"
diff --git a/lib/pleroma/docs/markdown.ex b/lib/pleroma/docs/markdown.ex
index da3f20f43..eac0789a6 100644
--- a/lib/pleroma/docs/markdown.ex
+++ b/lib/pleroma/docs/markdown.ex
@@ -1,3 +1,7 @@
+# Pleroma: A lightweight social networking server
+# Copyright © 2017-2020 Pleroma Authors
+# SPDX-License-Identifier: AGPL-3.0-only
+
defmodule Pleroma.Docs.Markdown do
@behaviour Pleroma.Docs.Generator
diff --git a/lib/pleroma/emails/admin_email.ex b/lib/pleroma/emails/admin_email.ex
index c27ad1065..8979db2f8 100644
--- a/lib/pleroma/emails/admin_email.ex
+++ b/lib/pleroma/emails/admin_email.ex
@@ -88,7 +88,7 @@ def new_unapproved_registration(to, account) do
html_body = """
New account for review: @#{account.nickname}
#{HTML.strip_tags(account.registration_reason)}
- Visit AdminFE
+ Visit AdminFE
"""
new()
diff --git a/lib/pleroma/emails/mailer.ex b/lib/pleroma/emails/mailer.ex
index 8b1bdef75..5108c71c8 100644
--- a/lib/pleroma/emails/mailer.ex
+++ b/lib/pleroma/emails/mailer.ex
@@ -35,6 +35,11 @@ def perform(:deliver_async, email, config), do: deliver(email, config)
def deliver(email, config \\ [])
def deliver(email, config) do
+ # temporary hackney fix until hackney max_connections bug is fixed
+ # https://git.pleroma.social/pleroma/pleroma/-/issues/2101
+ email =
+ Swoosh.Email.put_private(email, :hackney_options, ssl_options: [versions: [:"tlsv1.2"]])
+
case enabled?() do
true -> Swoosh.Mailer.deliver(email, parse_config(config))
false -> {:error, :deliveries_disabled}
diff --git a/lib/pleroma/emails/user_email.ex b/lib/pleroma/emails/user_email.ex
index 1d8c72ae9..806a61fd2 100644
--- a/lib/pleroma/emails/user_email.ex
+++ b/lib/pleroma/emails/user_email.ex
@@ -189,4 +189,30 @@ def unsubscribe_url(user, notifications_type) do
Router.Helpers.subscription_url(Endpoint, :unsubscribe, token)
end
+
+ def backup_is_ready_email(backup, admin_user_id \\ nil) do
+ %{user: user} = Pleroma.Repo.preload(backup, :user)
+ download_url = Pleroma.Web.PleromaAPI.BackupView.download_url(backup)
+
+ html_body =
+ if is_nil(admin_user_id) do
+ """
+ You requested a full backup of your Pleroma account. It's ready for download:
+ #{download_url}
+ """
+ else
+ admin = Pleroma.Repo.get(User, admin_user_id)
+
+ """
+ Admin @#{admin.nickname} requested a full backup of your Pleroma account. It's ready for download:
+ #{download_url}
+ """
+ end
+
+ new()
+ |> to(recipient(user))
+ |> from(sender())
+ |> subject("Your account archive is ready")
+ |> html_body(html_body)
+ end
end
diff --git a/lib/pleroma/emoji.ex b/lib/pleroma/emoji.ex
index f6016d73f..04936155b 100644
--- a/lib/pleroma/emoji.ex
+++ b/lib/pleroma/emoji.ex
@@ -56,6 +56,9 @@ def get(name) do
end
end
+ @spec exist?(String.t()) :: boolean()
+ def exist?(name), do: not is_nil(get(name))
+
@doc "Returns all the emojos!!"
@spec get_all() :: list({String.t(), String.t(), String.t()})
def get_all do
diff --git a/lib/pleroma/emoji/pack.ex b/lib/pleroma/emoji/pack.ex
index d076ae312..ca58e5432 100644
--- a/lib/pleroma/emoji/pack.ex
+++ b/lib/pleroma/emoji/pack.ex
@@ -1,3 +1,7 @@
+# Pleroma: A lightweight social networking server
+# Copyright © 2017-2020 Pleroma Authors
+# SPDX-License-Identifier: AGPL-3.0-only
+
defmodule Pleroma.Emoji.Pack do
@derive {Jason.Encoder, only: [:files, :pack, :files_count]}
defstruct files: %{},
@@ -17,6 +21,7 @@ defmodule Pleroma.Emoji.Pack do
}
alias Pleroma.Emoji
+ alias Pleroma.Emoji.Pack
@spec create(String.t()) :: {:ok, t()} | {:error, File.posix()} | {:error, :empty_values}
def create(name) do
@@ -64,24 +69,93 @@ def delete(name) do
end
end
- @spec add_file(String.t(), String.t(), Path.t(), Plug.Upload.t() | String.t()) ::
- {:ok, t()} | {:error, File.posix() | atom()}
- def add_file(name, shortcode, filename, file) do
- with :ok <- validate_not_empty([name, shortcode, filename]),
+ @spec unpack_zip_emojies(list(tuple())) :: list(map())
+ defp unpack_zip_emojies(zip_files) do
+ Enum.reduce(zip_files, [], fn
+ {_, path, s, _, _, _}, acc when elem(s, 2) == :regular ->
+ with(
+ filename <- Path.basename(path),
+ shortcode <- Path.basename(filename, Path.extname(filename)),
+ false <- Emoji.exist?(shortcode)
+ ) do
+ [%{path: path, filename: path, shortcode: shortcode} | acc]
+ else
+ _ -> acc
+ end
+
+ _, acc ->
+ acc
+ end)
+ end
+
+ @spec add_file(t(), String.t(), Path.t(), Plug.Upload.t()) ::
+ {:ok, t()}
+ | {:error, File.posix() | atom()}
+ def add_file(%Pack{} = pack, _, _, %Plug.Upload{content_type: "application/zip"} = file) do
+ with {:ok, zip_files} <- :zip.table(to_charlist(file.path)),
+ [_ | _] = emojies <- unpack_zip_emojies(zip_files),
+ {:ok, tmp_dir} <- Pleroma.Utils.tmp_dir("emoji") do
+ try do
+ {:ok, _emoji_files} =
+ :zip.unzip(
+ to_charlist(file.path),
+ [{:file_list, Enum.map(emojies, & &1[:path])}, {:cwd, tmp_dir}]
+ )
+
+ {_, updated_pack} =
+ Enum.map_reduce(emojies, pack, fn item, emoji_pack ->
+ emoji_file = %Plug.Upload{
+ filename: item[:filename],
+ path: Path.join(tmp_dir, item[:path])
+ }
+
+ {:ok, updated_pack} =
+ do_add_file(
+ emoji_pack,
+ item[:shortcode],
+ to_string(item[:filename]),
+ emoji_file
+ )
+
+ {item, updated_pack}
+ end)
+
+ Emoji.reload()
+
+ {:ok, updated_pack}
+ after
+ File.rm_rf(tmp_dir)
+ end
+ else
+ {:error, _} = error ->
+ error
+
+ _ ->
+ {:ok, pack}
+ end
+ end
+
+ def add_file(%Pack{} = pack, shortcode, filename, %Plug.Upload{} = file) do
+ with :ok <- validate_not_empty([shortcode, filename]),
:ok <- validate_emoji_not_exists(shortcode),
- {:ok, pack} <- load_pack(name),
- :ok <- save_file(file, pack, filename),
- {:ok, updated_pack} <- pack |> put_emoji(shortcode, filename) |> save_pack() do
+ {:ok, updated_pack} <- do_add_file(pack, shortcode, filename, file) do
Emoji.reload()
{:ok, updated_pack}
end
end
- @spec delete_file(String.t(), String.t()) ::
+ defp do_add_file(pack, shortcode, filename, file) do
+ with :ok <- save_file(file, pack, filename) do
+ pack
+ |> put_emoji(shortcode, filename)
+ |> save_pack()
+ end
+ end
+
+ @spec delete_file(t(), String.t()) ::
{:ok, t()} | {:error, File.posix() | atom()}
- def delete_file(name, shortcode) do
- with :ok <- validate_not_empty([name, shortcode]),
- {:ok, pack} <- load_pack(name),
+ def delete_file(%Pack{} = pack, shortcode) do
+ with :ok <- validate_not_empty([shortcode]),
:ok <- remove_file(pack, shortcode),
{:ok, updated_pack} <- pack |> delete_emoji(shortcode) |> save_pack() do
Emoji.reload()
@@ -89,11 +163,10 @@ def delete_file(name, shortcode) do
end
end
- @spec update_file(String.t(), String.t(), String.t(), String.t(), boolean()) ::
+ @spec update_file(t(), String.t(), String.t(), String.t(), boolean()) ::
{:ok, t()} | {:error, File.posix() | atom()}
- def update_file(name, shortcode, new_shortcode, new_filename, force) do
- with :ok <- validate_not_empty([name, shortcode, new_shortcode, new_filename]),
- {:ok, pack} <- load_pack(name),
+ def update_file(%Pack{} = pack, shortcode, new_shortcode, new_filename, force) do
+ with :ok <- validate_not_empty([shortcode, new_shortcode, new_filename]),
{:ok, filename} <- get_filename(pack, shortcode),
:ok <- validate_emoji_not_exists(new_shortcode, force),
:ok <- rename_file(pack, filename, new_filename),
@@ -129,13 +202,13 @@ def import_from_filesystem do
end
end
- @spec list_remote(String.t()) :: {:ok, map()} | {:error, atom()}
- def list_remote(url) do
- uri = url |> String.trim() |> URI.parse()
+ @spec list_remote(keyword()) :: {:ok, map()} | {:error, atom()}
+ def list_remote(opts) do
+ uri = opts[:url] |> String.trim() |> URI.parse()
with :ok <- validate_shareable_packs_available(uri) do
uri
- |> URI.merge("/api/pleroma/emoji/packs")
+ |> URI.merge("/api/pleroma/emoji/packs?page=#{opts[:page]}&page_size=#{opts[:page_size]}")
|> http_get()
end
end
@@ -175,7 +248,8 @@ def download(name, url, as) do
uri = url |> String.trim() |> URI.parse()
with :ok <- validate_shareable_packs_available(uri),
- {:ok, remote_pack} <- uri |> URI.merge("/api/pleroma/emoji/packs/#{name}") |> http_get(),
+ {:ok, remote_pack} <-
+ uri |> URI.merge("/api/pleroma/emoji/pack?name=#{name}") |> http_get(),
{:ok, %{sha: sha, url: url} = pack_info} <- fetch_pack_info(remote_pack, uri, name),
{:ok, archive} <- download_archive(url, sha),
pack <- copy_as(remote_pack, as || name),
@@ -243,9 +317,10 @@ defp validate_emoji_not_exists(shortcode, force \\ false)
defp validate_emoji_not_exists(_shortcode, true), do: :ok
defp validate_emoji_not_exists(shortcode, _) do
- case Emoji.get(shortcode) do
- nil -> :ok
- _ -> {:error, :already_exists}
+ if Emoji.exist?(shortcode) do
+ {:error, :already_exists}
+ else
+ :ok
end
end
@@ -386,25 +461,18 @@ defp validate_not_empty(list) do
end
end
- defp save_file(file, pack, filename) do
+ defp save_file(%Plug.Upload{path: upload_path}, pack, filename) do
file_path = Path.join(pack.path, filename)
create_subdirs(file_path)
- case file do
- %Plug.Upload{path: upload_path} ->
- # Copy the uploaded file from the temporary directory
- with {:ok, _} <- File.copy(upload_path, file_path), do: :ok
-
- url when is_binary(url) ->
- # Download and write the file
- file_contents = Tesla.get!(url).body
- File.write(file_path, file_contents)
+ with {:ok, _} <- File.copy(upload_path, file_path) do
+ :ok
end
end
defp put_emoji(pack, shortcode, filename) do
files = Map.put(pack.files, shortcode, filename)
- %{pack | files: files}
+ %{pack | files: files, files_count: length(Map.keys(files))}
end
defp delete_emoji(pack, shortcode) do
@@ -460,7 +528,7 @@ defp get_filename(pack, shortcode) do
defp http_get(%URI{} = url), do: url |> to_string() |> http_get()
defp http_get(url) do
- with {:ok, %{body: body}} <- url |> Pleroma.HTTP.get() do
+ with {:ok, %{body: body}} <- Pleroma.HTTP.get(url, [], pool: :default) do
Jason.decode(body)
end
end
@@ -509,7 +577,7 @@ defp fetch_pack_info(remote_pack, uri, name) do
{:ok,
%{
sha: sha,
- url: URI.merge(uri, "/api/pleroma/emoji/packs/#{name}/archive") |> to_string()
+ url: URI.merge(uri, "/api/pleroma/emoji/packs/archive?name=#{name}") |> to_string()
}}
%{"fallback-src" => src, "fallback-src-sha256" => sha} when is_binary(src) ->
@@ -526,7 +594,7 @@ defp fetch_pack_info(remote_pack, uri, name) do
end
defp download_archive(url, sha) do
- with {:ok, %{body: archive}} <- Tesla.get(url) do
+ with {:ok, %{body: archive}} <- Pleroma.HTTP.get(url) do
if Base.decode16!(sha) == :crypto.hash(:sha256, archive) do
{:ok, archive}
else
@@ -549,7 +617,7 @@ defp fallback_sha_changed?(pack, data) do
end
defp update_sha_and_save_metadata(pack, data) do
- with {:ok, %{body: zip}} <- Tesla.get(data[:"fallback-src"]),
+ with {:ok, %{body: zip}} <- Pleroma.HTTP.get(data[:"fallback-src"]),
:ok <- validate_has_all_files(pack, zip) do
fallback_sha = :sha256 |> :crypto.hash(zip) |> Base.encode16()
diff --git a/lib/pleroma/gun/gun.ex b/lib/pleroma/gun.ex
similarity index 100%
rename from lib/pleroma/gun/gun.ex
rename to lib/pleroma/gun.ex
diff --git a/lib/pleroma/gun/conn.ex b/lib/pleroma/gun/conn.ex
index 75b1ffc0a..477e19c6e 100644
--- a/lib/pleroma/gun/conn.ex
+++ b/lib/pleroma/gun/conn.ex
@@ -50,10 +50,10 @@ defp do_open(uri, %{proxy: {proxy_host, proxy_port}} = opts) do
with open_opts <- Map.delete(opts, :tls_opts),
{:ok, conn} <- Gun.open(proxy_host, proxy_port, open_opts),
- {:ok, _} <- Gun.await_up(conn, opts[:connect_timeout]),
+ {:ok, protocol} <- Gun.await_up(conn, opts[:connect_timeout]),
stream <- Gun.connect(conn, connect_opts),
{:response, :fin, 200, _} <- Gun.await(conn, stream) do
- {:ok, conn}
+ {:ok, conn, protocol}
else
error ->
Logger.warn(
@@ -88,8 +88,8 @@ defp do_open(uri, %{proxy: {proxy_type, proxy_host, proxy_port}} = opts) do
|> Map.put(:socks_opts, socks_opts)
with {:ok, conn} <- Gun.open(proxy_host, proxy_port, opts),
- {:ok, _} <- Gun.await_up(conn, opts[:connect_timeout]) do
- {:ok, conn}
+ {:ok, protocol} <- Gun.await_up(conn, opts[:connect_timeout]) do
+ {:ok, conn, protocol}
else
error ->
Logger.warn(
@@ -106,8 +106,8 @@ defp do_open(%URI{host: host, port: port} = uri, opts) do
host = Pleroma.HTTP.AdapterHelper.parse_host(host)
with {:ok, conn} <- Gun.open(host, port, opts),
- {:ok, _} <- Gun.await_up(conn, opts[:connect_timeout]) do
- {:ok, conn}
+ {:ok, protocol} <- Gun.await_up(conn, opts[:connect_timeout]) do
+ {:ok, conn, protocol}
else
error ->
Logger.warn(
diff --git a/lib/pleroma/gun/connection_pool.ex b/lib/pleroma/gun/connection_pool.ex
index f34602b73..e322f192a 100644
--- a/lib/pleroma/gun/connection_pool.ex
+++ b/lib/pleroma/gun/connection_pool.ex
@@ -1,3 +1,7 @@
+# Pleroma: A lightweight social networking server
+# Copyright © 2017-2020 Pleroma Authors
+# SPDX-License-Identifier: AGPL-3.0-only
+
defmodule Pleroma.Gun.ConnectionPool do
@registry __MODULE__
diff --git a/lib/pleroma/gun/connection_pool/reclaimer.ex b/lib/pleroma/gun/connection_pool/reclaimer.ex
index cea800882..241e8b04f 100644
--- a/lib/pleroma/gun/connection_pool/reclaimer.ex
+++ b/lib/pleroma/gun/connection_pool/reclaimer.ex
@@ -1,3 +1,7 @@
+# Pleroma: A lightweight social networking server
+# Copyright © 2017-2020 Pleroma Authors
+# SPDX-License-Identifier: AGPL-3.0-only
+
defmodule Pleroma.Gun.ConnectionPool.Reclaimer do
use GenServer, restart: :temporary
diff --git a/lib/pleroma/gun/connection_pool/worker.ex b/lib/pleroma/gun/connection_pool/worker.ex
index c36332817..b71816bed 100644
--- a/lib/pleroma/gun/connection_pool/worker.ex
+++ b/lib/pleroma/gun/connection_pool/worker.ex
@@ -1,3 +1,7 @@
+# Pleroma: A lightweight social networking server
+# Copyright © 2017-2020 Pleroma Authors
+# SPDX-License-Identifier: AGPL-3.0-only
+
defmodule Pleroma.Gun.ConnectionPool.Worker do
alias Pleroma.Gun
use GenServer, restart: :temporary
@@ -15,7 +19,7 @@ def init([_key, _uri, _opts, _client_pid] = opts) do
@impl true
def handle_continue({:connect, [key, uri, opts, client_pid]}, _) do
- with {:ok, conn_pid} <- Gun.Conn.open(uri, opts),
+ with {:ok, conn_pid, protocol} <- Gun.Conn.open(uri, opts),
Process.link(conn_pid) do
time = :erlang.monotonic_time(:millisecond)
@@ -27,8 +31,12 @@ def handle_continue({:connect, [key, uri, opts, client_pid]}, _) do
send(client_pid, {:conn_pid, conn_pid})
{:noreply,
- %{key: key, timer: nil, client_monitors: %{client_pid => Process.monitor(client_pid)}},
- :hibernate}
+ %{
+ key: key,
+ timer: nil,
+ client_monitors: %{client_pid => Process.monitor(client_pid)},
+ protocol: protocol
+ }, :hibernate}
else
err ->
{:stop, {:shutdown, err}, nil}
@@ -53,14 +61,20 @@ def handle_cast({:remove_client, client_pid}, state) do
end
@impl true
- def handle_call(:add_client, {client_pid, _}, %{key: key} = state) do
+ def handle_call(:add_client, {client_pid, _}, %{key: key, protocol: protocol} = state) do
time = :erlang.monotonic_time(:millisecond)
- {{conn_pid, _, _, _}, _} =
+ {{conn_pid, used_by, _, _}, _} =
Registry.update_value(@registry, key, fn {conn_pid, used_by, crf, last_reference} ->
{conn_pid, [client_pid | used_by], crf(time - last_reference, crf), time}
end)
+ :telemetry.execute(
+ [:pleroma, :connection_pool, :client, :add],
+ %{client_pid: client_pid, clients: used_by},
+ %{key: state.key, protocol: protocol}
+ )
+
state =
if state.timer != nil do
Process.cancel_timer(state[:timer])
@@ -83,25 +97,18 @@ def handle_call(:remove_client, {client_pid, _}, %{key: key} = state) do
end)
{ref, state} = pop_in(state.client_monitors[client_pid])
- # DOWN message can receive right after `remove_client` call and cause worker to terminate
- state =
- if is_nil(ref) do
- state
+
+ Process.demonitor(ref, [:flush])
+
+ timer =
+ if used_by == [] do
+ max_idle = Pleroma.Config.get([:connections_pool, :max_idle_time], 30_000)
+ Process.send_after(self(), :idle_close, max_idle)
else
- Process.demonitor(ref)
-
- timer =
- if used_by == [] do
- max_idle = Pleroma.Config.get([:connections_pool, :max_idle_time], 30_000)
- Process.send_after(self(), :idle_close, max_idle)
- else
- nil
- end
-
- %{state | timer: timer}
+ nil
end
- {:reply, :ok, state, :hibernate}
+ {:reply, :ok, %{state | timer: timer}, :hibernate}
end
@impl true
@@ -131,7 +138,7 @@ def handle_info({:gun_down, _pid, _protocol, _reason, _killed_streams}, state) d
@impl true
def handle_info({:DOWN, _ref, :process, pid, reason}, state) do
:telemetry.execute(
- [:pleroma, :connection_pool, :client_death],
+ [:pleroma, :connection_pool, :client, :dead],
%{client_pid: pid, reason: reason},
%{key: state.key}
)
diff --git a/lib/pleroma/gun/connection_pool/worker_supervisor.ex b/lib/pleroma/gun/connection_pool/worker_supervisor.ex
index 39615c956..4c23bcbd9 100644
--- a/lib/pleroma/gun/connection_pool/worker_supervisor.ex
+++ b/lib/pleroma/gun/connection_pool/worker_supervisor.ex
@@ -1,3 +1,7 @@
+# Pleroma: A lightweight social networking server
+# Copyright © 2017-2020 Pleroma Authors
+# SPDX-License-Identifier: AGPL-3.0-only
+
defmodule Pleroma.Gun.ConnectionPool.WorkerSupervisor do
@moduledoc "Supervisor for pool workers. Does not do anything except enforce max connection limit"
diff --git a/lib/pleroma/helpers/inet_helper.ex b/lib/pleroma/helpers/inet_helper.ex
new file mode 100644
index 000000000..126f82381
--- /dev/null
+++ b/lib/pleroma/helpers/inet_helper.ex
@@ -0,0 +1,19 @@
+# Pleroma: A lightweight social networking server
+# Copyright © 2017-2020 Pleroma Authors
+# SPDX-License-Identifier: AGPL-3.0-only
+
+defmodule Pleroma.Helpers.InetHelper do
+ def parse_address(ip) when is_tuple(ip) do
+ {:ok, ip}
+ end
+
+ def parse_address(ip) when is_binary(ip) do
+ ip
+ |> String.to_charlist()
+ |> parse_address()
+ end
+
+ def parse_address(ip) do
+ :inet.parse_address(ip)
+ end
+end
diff --git a/lib/pleroma/helpers/media_helper.ex b/lib/pleroma/helpers/media_helper.ex
new file mode 100644
index 000000000..6b799173e
--- /dev/null
+++ b/lib/pleroma/helpers/media_helper.ex
@@ -0,0 +1,162 @@
+# Pleroma: A lightweight social networking server
+# Copyright © 2017-2020 Pleroma Authors
+# SPDX-License-Identifier: AGPL-3.0-only
+
+defmodule Pleroma.Helpers.MediaHelper do
+ @moduledoc """
+ Handles common media-related operations.
+ """
+
+ alias Pleroma.HTTP
+
+ require Logger
+
+ def missing_dependencies do
+ Enum.reduce([imagemagick: "convert", ffmpeg: "ffmpeg"], [], fn {sym, executable}, acc ->
+ if Pleroma.Utils.command_available?(executable) do
+ acc
+ else
+ [sym | acc]
+ end
+ end)
+ end
+
+ def image_resize(url, options) do
+ with executable when is_binary(executable) <- System.find_executable("convert"),
+ {:ok, args} <- prepare_image_resize_args(options),
+ {:ok, env} <- HTTP.get(url, [], pool: :media),
+ {:ok, fifo_path} <- mkfifo() do
+ args = List.flatten([fifo_path, args])
+ run_fifo(fifo_path, env, executable, args)
+ else
+ nil -> {:error, {:convert, :command_not_found}}
+ {:error, _} = error -> error
+ end
+ end
+
+ defp prepare_image_resize_args(
+ %{max_width: max_width, max_height: max_height, format: "png"} = options
+ ) do
+ quality = options[:quality] || 85
+ resize = Enum.join([max_width, "x", max_height, ">"])
+
+ args = [
+ "-resize",
+ resize,
+ "-quality",
+ to_string(quality),
+ "png:-"
+ ]
+
+ {:ok, args}
+ end
+
+ defp prepare_image_resize_args(%{max_width: max_width, max_height: max_height} = options) do
+ quality = options[:quality] || 85
+ resize = Enum.join([max_width, "x", max_height, ">"])
+
+ args = [
+ "-interlace",
+ "Plane",
+ "-resize",
+ resize,
+ "-quality",
+ to_string(quality),
+ "jpg:-"
+ ]
+
+ {:ok, args}
+ end
+
+ defp prepare_image_resize_args(_), do: {:error, :missing_options}
+
+ # Note: video thumbnail is intentionally not resized (always has original dimensions)
+ def video_framegrab(url) do
+ with executable when is_binary(executable) <- System.find_executable("ffmpeg"),
+ {:ok, env} <- HTTP.get(url, [], pool: :media),
+ {:ok, fifo_path} <- mkfifo(),
+ args = [
+ "-y",
+ "-i",
+ fifo_path,
+ "-vframes",
+ "1",
+ "-f",
+ "mjpeg",
+ "-loglevel",
+ "error",
+ "-"
+ ] do
+ run_fifo(fifo_path, env, executable, args)
+ else
+ nil -> {:error, {:ffmpeg, :command_not_found}}
+ {:error, _} = error -> error
+ end
+ end
+
+ defp run_fifo(fifo_path, env, executable, args) do
+ pid =
+ Port.open({:spawn_executable, executable}, [
+ :use_stdio,
+ :stream,
+ :exit_status,
+ :binary,
+ args: args
+ ])
+
+ fifo = Port.open(to_charlist(fifo_path), [:eof, :binary, :stream, :out])
+ fix = Pleroma.Helpers.QtFastStart.fix(env.body)
+ true = Port.command(fifo, fix)
+ :erlang.port_close(fifo)
+ loop_recv(pid)
+ after
+ File.rm(fifo_path)
+ end
+
+ defp mkfifo do
+ path = Path.join(System.tmp_dir!(), "pleroma-media-preview-pipe-#{Ecto.UUID.generate()}")
+
+ case System.cmd("mkfifo", [path]) do
+ {_, 0} ->
+ spawn(fifo_guard(path))
+ {:ok, path}
+
+ {_, err} ->
+ {:error, {:fifo_failed, err}}
+ end
+ end
+
+ defp fifo_guard(path) do
+ pid = self()
+
+ fn ->
+ ref = Process.monitor(pid)
+
+ receive do
+ {:DOWN, ^ref, :process, ^pid, _} ->
+ File.rm(path)
+ end
+ end
+ end
+
+ defp loop_recv(pid) do
+ loop_recv(pid, <<>>)
+ end
+
+ defp loop_recv(pid, acc) do
+ receive do
+ {^pid, {:data, data}} ->
+ loop_recv(pid, acc <> data)
+
+ {^pid, {:exit_status, 0}} ->
+ {:ok, acc}
+
+ {^pid, {:exit_status, status}} ->
+ {:error, status}
+ after
+ 5000 ->
+ :erlang.port_close(pid)
+ {:error, :timeout}
+ end
+ end
+end
diff --git a/lib/pleroma/helpers/qt_fast_start.ex b/lib/pleroma/helpers/qt_fast_start.ex
new file mode 100644
index 000000000..bb93224b5
--- /dev/null
+++ b/lib/pleroma/helpers/qt_fast_start.ex
@@ -0,0 +1,131 @@
+# Pleroma: A lightweight social networking server
+# Copyright © 2017-2020 Pleroma Authors
+# SPDX-License-Identifier: AGPL-3.0-only
+
+defmodule Pleroma.Helpers.QtFastStart do
+ @moduledoc """
+ (WIP) Converts a "slow start" (data before metadatas) mov/mp4 file to a "fast start" one (metadatas before data).
+ """
+
+ # TODO: Cleanup and optimizations
+ # Inspirations: https://www.ffmpeg.org/doxygen/3.4/qt-faststart_8c_source.html
+ # https://github.com/danielgtaylor/qtfaststart/blob/master/qtfaststart/processor.py
+ # ISO/IEC 14496-12:2015, ISO/IEC 15444-12:2015
+ # Paracetamol
+
+ def fix(<<0x00, 0x00, 0x00, _, 0x66, 0x74, 0x79, 0x70, _::bits>> = binary) do
+ index = fix(binary, 0, nil, nil, [])
+
+ case index do
+ :abort -> binary
+ [{"ftyp", _, _, _, _}, {"mdat", _, _, _, _} | _] -> faststart(index)
+ [{"ftyp", _, _, _, _}, {"free", _, _, _, _}, {"mdat", _, _, _, _} | _] -> faststart(index)
+ _ -> binary
+ end
+ end
+
+ def fix(binary) do
+ binary
+ end
+
+ # MOOV have been seen before MDAT- abort
+ defp fix(<<_::bits>>, _, true, false, _) do
+ :abort
+ end
+
+ defp fix(
+ <>,
+ pos,
+ got_moov,
+ got_mdat,
+ acc
+ ) do
+ full_size = (size - 8) * 8
+ <> = rest
+
+ acc = [
+ {fourcc, pos, pos + size, size,
+ <>}
+ | acc
+ ]
+
+ fix(rest, pos + size, got_moov || fourcc == "moov", got_mdat || fourcc == "mdat", acc)
+ end
+
+ defp fix(<<>>, _pos, _, _, acc) do
+ :lists.reverse(acc)
+ end
+
+ defp faststart(index) do
+ {{_ftyp, _, _, _, ftyp}, index} = List.keytake(index, "ftyp", 0)
+
+ # Skip re-writing the free fourcc as it's kind of useless.
+ # Why stream useless bytes when you can do without?
+ {free_size, index} =
+ case List.keytake(index, "free", 0) do
+ {{_, _, _, size, _}, index} -> {size, index}
+ _ -> {0, index}
+ end
+
+ {{_moov, _, _, moov_size, moov}, index} = List.keytake(index, "moov", 0)
+ offset = -free_size + moov_size
+ rest = for {_, _, _, _, data} <- index, do: data, into: []
+ <> = moov
+ [ftyp, moov_head, fix_moov(moov_data, offset, []), rest]
+ end
+
+ defp fix_moov(
+ <>,
+ offset,
+ acc
+ ) do
+ full_size = (size - 8) * 8
+ <> = rest
+
+ data =
+ cond do
+ fourcc in ["trak", "mdia", "minf", "stbl"] ->
+ # Theses contains sto or co64 part
+ [<>, fix_moov(data, offset, [])]
+
+ fourcc in ["stco", "co64"] ->
+ # fix the damn thing
+ <> = data
+
+ entry_size =
+ case fourcc do
+ "stco" -> 32
+ "co64" -> 64
+ end
+
+ [
+ <>,
+ rewrite_entries(entry_size, offset, rest, [])
+ ]
+
+ true ->
+ [<>, data]
+ end
+
+ acc = [acc | data]
+ fix_moov(rest, offset, acc)
+ end
+
+ defp fix_moov(<<>>, _, acc), do: acc
+
+ for size <- [32, 64] do
+ defp rewrite_entries(
+ unquote(size),
+ offset,
+ <>,
+ acc
+ ) do
+ rewrite_entries(unquote(size), offset, rest, [
+ acc | <>
+ ])
+ end
+ end
+
+ defp rewrite_entries(_, _, <<>>, acc), do: acc
+end
diff --git a/lib/pleroma/helpers/uri_helper.ex b/lib/pleroma/helpers/uri_helper.ex
index 6d205a636..f1301f055 100644
--- a/lib/pleroma/helpers/uri_helper.ex
+++ b/lib/pleroma/helpers/uri_helper.ex
@@ -3,18 +3,22 @@
# SPDX-License-Identifier: AGPL-3.0-only
defmodule Pleroma.Helpers.UriHelper do
- def append_uri_params(uri, appended_params) do
+ def modify_uri_params(uri, overridden_params, deleted_params \\ []) do
uri = URI.parse(uri)
- appended_params = for {k, v} <- appended_params, into: %{}, do: {to_string(k), v}
- existing_params = URI.query_decoder(uri.query || "") |> Enum.into(%{})
- updated_params_keys = Enum.uniq(Map.keys(existing_params) ++ Map.keys(appended_params))
+
+ existing_params = URI.query_decoder(uri.query || "") |> Map.new()
+ overridden_params = Map.new(overridden_params, fn {k, v} -> {to_string(k), v} end)
+ deleted_params = Enum.map(deleted_params, &to_string/1)
updated_params =
- for k <- updated_params_keys, do: {k, appended_params[k] || existing_params[k]}
+ existing_params
+ |> Map.merge(overridden_params)
+ |> Map.drop(deleted_params)
uri
|> Map.put(:query, URI.encode_query(updated_params))
|> URI.to_string()
+ |> String.replace_suffix("?", "")
end
def maybe_add_base("/" <> uri, base), do: Path.join([base, uri])
diff --git a/lib/pleroma/http/http.ex b/lib/pleroma/http.ex
similarity index 100%
rename from lib/pleroma/http/http.ex
rename to lib/pleroma/http.ex
diff --git a/lib/pleroma/http/adapter_helper/default.ex b/lib/pleroma/http/adapter_helper/default.ex
index e13441316..8567a616b 100644
--- a/lib/pleroma/http/adapter_helper/default.ex
+++ b/lib/pleroma/http/adapter_helper/default.ex
@@ -1,3 +1,7 @@
+# Pleroma: A lightweight social networking server
+# Copyright © 2017-2020 Pleroma Authors
+# SPDX-License-Identifier: AGPL-3.0-only
+
defmodule Pleroma.HTTP.AdapterHelper.Default do
alias Pleroma.HTTP.AdapterHelper
diff --git a/lib/pleroma/http/adapter_helper/hackney.ex b/lib/pleroma/http/adapter_helper/hackney.ex
index ef84553c1..ff60513fd 100644
--- a/lib/pleroma/http/adapter_helper/hackney.ex
+++ b/lib/pleroma/http/adapter_helper/hackney.ex
@@ -1,3 +1,7 @@
+# Pleroma: A lightweight social networking server
+# Copyright © 2017-2020 Pleroma Authors
+# SPDX-License-Identifier: AGPL-3.0-only
+
defmodule Pleroma.HTTP.AdapterHelper.Hackney do
@behaviour Pleroma.HTTP.AdapterHelper
diff --git a/lib/pleroma/http/web_push.ex b/lib/pleroma/http/web_push.ex
new file mode 100644
index 000000000..78148a12e
--- /dev/null
+++ b/lib/pleroma/http/web_push.ex
@@ -0,0 +1,12 @@
+# Pleroma: A lightweight social networking server
+# Copyright © 2017-2020 Pleroma Authors
+# SPDX-License-Identifier: AGPL-3.0-only
+
+defmodule Pleroma.HTTP.WebPush do
+ @moduledoc false
+
+ def post(url, payload, headers) do
+ list_headers = Map.to_list(headers)
+ Pleroma.HTTP.post(url, payload, list_headers)
+ end
+end
diff --git a/lib/pleroma/instances.ex b/lib/pleroma/instances.ex
index 557e8decf..7315bd7cb 100644
--- a/lib/pleroma/instances.ex
+++ b/lib/pleroma/instances.ex
@@ -11,6 +11,7 @@ defmodule Pleroma.Instances do
defdelegate reachable?(url_or_host), to: @adapter
defdelegate set_reachable(url_or_host), to: @adapter
defdelegate set_unreachable(url_or_host, unreachable_since \\ nil), to: @adapter
+ defdelegate get_consistently_unreachable(), to: @adapter
def set_consistently_unreachable(url_or_host),
do: set_unreachable(url_or_host, reachability_datetime_threshold())
diff --git a/lib/pleroma/instances/instance.ex b/lib/pleroma/instances/instance.ex
index 8bf53c090..df471a39d 100644
--- a/lib/pleroma/instances/instance.ex
+++ b/lib/pleroma/instances/instance.ex
@@ -119,6 +119,17 @@ def set_unreachable(url_or_host, unreachable_since) when is_binary(url_or_host)
def set_unreachable(_, _), do: {:error, nil}
+ def get_consistently_unreachable do
+ reachability_datetime_threshold = Instances.reachability_datetime_threshold()
+
+ from(i in Instance,
+ where: ^reachability_datetime_threshold > i.unreachable_since,
+ order_by: i.unreachable_since,
+ select: {i.host, i.unreachable_since}
+ )
+ |> Repo.all()
+ end
+
defp parse_datetime(datetime) when is_binary(datetime) do
NaiveDateTime.from_iso8601(datetime)
end
@@ -156,16 +167,12 @@ def get_or_update_favicon(%URI{host: host} = instance_uri) do
defp scrape_favicon(%URI{} = instance_uri) do
try do
with {:ok, %Tesla.Env{body: html}} <-
- Pleroma.HTTP.get(to_string(instance_uri), [{"accept", "text/html"}],
- adapter: [pool: :media]
- ),
- favicon_rel <-
- html
- |> Floki.parse_document!()
- |> Floki.attribute("link[rel=icon]", "href")
- |> List.first(),
- favicon <- URI.merge(instance_uri, favicon_rel) |> to_string(),
- true <- is_binary(favicon) do
+ Pleroma.HTTP.get(to_string(instance_uri), [{"accept", "text/html"}], pool: :media),
+ {_, [favicon_rel | _]} when is_binary(favicon_rel) <-
+ {:parse,
+ html |> Floki.parse_document!() |> Floki.attribute("link[rel=icon]", "href")},
+ {_, favicon} when is_binary(favicon) <-
+ {:merge, URI.merge(instance_uri, favicon_rel) |> to_string()} do
favicon
else
_ -> nil
diff --git a/lib/pleroma/jwt.ex b/lib/pleroma/jwt.ex
index 10102ff5d..faeb77781 100644
--- a/lib/pleroma/jwt.ex
+++ b/lib/pleroma/jwt.ex
@@ -1,3 +1,7 @@
+# Pleroma: A lightweight social networking server
+# Copyright © 2017-2020 Pleroma Authors
+# SPDX-License-Identifier: AGPL-3.0-only
+
defmodule Pleroma.JWT do
use Joken.Config
diff --git a/lib/pleroma/mfa/token.ex b/lib/pleroma/mfa/token.ex
index 0b2449971..69b64c0e8 100644
--- a/lib/pleroma/mfa/token.ex
+++ b/lib/pleroma/mfa/token.ex
@@ -10,10 +10,11 @@ defmodule Pleroma.MFA.Token do
alias Pleroma.Repo
alias Pleroma.User
alias Pleroma.Web.OAuth.Authorization
- alias Pleroma.Web.OAuth.Token, as: OAuthToken
@expires 300
+ @type t() :: %__MODULE__{}
+
schema "mfa_tokens" do
field(:token, :string)
field(:valid_until, :naive_datetime_usec)
@@ -24,6 +25,7 @@ defmodule Pleroma.MFA.Token do
timestamps()
end
+ @spec get_by_token(String.t()) :: {:ok, t()} | {:error, :not_found}
def get_by_token(token) do
from(
t in __MODULE__,
@@ -33,33 +35,40 @@ def get_by_token(token) do
|> Repo.find_resource()
end
- def validate(token) do
- with {:fetch_token, {:ok, token}} <- {:fetch_token, get_by_token(token)},
- {:expired, false} <- {:expired, is_expired?(token)} do
+ @spec validate(String.t()) :: {:ok, t()} | {:error, :not_found} | {:error, :expired_token}
+ def validate(token_str) do
+ with {:ok, token} <- get_by_token(token_str),
+ false <- expired?(token) do
{:ok, token}
- else
- {:expired, _} -> {:error, :expired_token}
- {:fetch_token, _} -> {:error, :not_found}
- error -> {:error, error}
end
end
- def create_token(%User{} = user) do
- %__MODULE__{}
- |> change
- |> assign_user(user)
- |> put_token
- |> put_valid_until
- |> Repo.insert()
+ defp expired?(%__MODULE__{valid_until: valid_until}) do
+ with true <- NaiveDateTime.diff(NaiveDateTime.utc_now(), valid_until) > 0 do
+ {:error, :expired_token}
+ end
end
- def create_token(user, authorization) do
+ @spec create(User.t(), Authorization.t() | nil) :: {:ok, t()} | {:error, Ecto.Changeset.t()}
+ def create(user, authorization \\ nil) do
+ with {:ok, token} <- do_create(user, authorization) do
+ Pleroma.Workers.PurgeExpiredToken.enqueue(%{
+ token_id: token.id,
+ valid_until: DateTime.from_naive!(token.valid_until, "Etc/UTC"),
+ mod: __MODULE__
+ })
+
+ {:ok, token}
+ end
+ end
+
+ defp do_create(user, authorization) do
%__MODULE__{}
- |> change
+ |> change()
|> assign_user(user)
- |> assign_authorization(authorization)
- |> put_token
- |> put_valid_until
+ |> maybe_assign_authorization(authorization)
+ |> put_token()
+ |> put_valid_until()
|> Repo.insert()
end
@@ -69,15 +78,19 @@ defp assign_user(changeset, user) do
|> validate_required([:user])
end
- defp assign_authorization(changeset, authorization) do
+ defp maybe_assign_authorization(changeset, %Authorization{} = authorization) do
changeset
|> put_assoc(:authorization, authorization)
|> validate_required([:authorization])
end
+ defp maybe_assign_authorization(changeset, _), do: changeset
+
defp put_token(changeset) do
+ token = Pleroma.Web.OAuth.Token.Utils.generate_token()
+
changeset
- |> change(%{token: OAuthToken.Utils.generate_token()})
+ |> change(%{token: token})
|> validate_required([:token])
|> unique_constraint(:token)
end
@@ -89,18 +102,4 @@ defp put_valid_until(changeset) do
|> change(%{valid_until: expires_in})
|> validate_required([:valid_until])
end
-
- def is_expired?(%__MODULE__{valid_until: valid_until}) do
- NaiveDateTime.diff(NaiveDateTime.utc_now(), valid_until) > 0
- end
-
- def is_expired?(_), do: false
-
- def delete_expired_tokens do
- from(
- q in __MODULE__,
- where: fragment("?", q.valid_until) < ^Timex.now()
- )
- |> Repo.delete_all()
- end
end
diff --git a/lib/pleroma/migration_helper/notification_backfill.ex b/lib/pleroma/migration_helper/notification_backfill.ex
index d260e62ca..24f4733fe 100644
--- a/lib/pleroma/migration_helper/notification_backfill.ex
+++ b/lib/pleroma/migration_helper/notification_backfill.ex
@@ -19,13 +19,13 @@ def fill_in_notification_types do
query
|> Repo.chunk_stream(100)
|> Enum.each(fn notification ->
- type =
- notification.activity
- |> type_from_activity()
+ if notification.activity do
+ type = type_from_activity(notification.activity)
- notification
- |> Ecto.Changeset.change(%{type: type})
- |> Repo.update()
+ notification
+ |> Ecto.Changeset.change(%{type: type})
+ |> Repo.update()
+ end
end)
end
@@ -72,8 +72,7 @@ defp type_from_activity(%{data: %{"type" => type}} = activity) do
"pleroma:emoji_reaction"
"Create" ->
- activity
- |> type_from_activity_object()
+ type_from_activity_object(activity)
t ->
raise "No notification type for activity type #{t}"
diff --git a/lib/pleroma/mime.ex b/lib/pleroma/mime.ex
deleted file mode 100644
index 6ee055f50..000000000
--- a/lib/pleroma/mime.ex
+++ /dev/null
@@ -1,120 +0,0 @@
-# Pleroma: A lightweight social networking server
-# Copyright © 2017-2020 Pleroma Authors
-# SPDX-License-Identifier: AGPL-3.0-only
-
-defmodule Pleroma.MIME do
- @moduledoc """
- Returns the mime-type of a binary and optionally a normalized file-name.
- """
- @default "application/octet-stream"
- @read_bytes 35
-
- @spec file_mime_type(String.t(), String.t()) ::
- {:ok, content_type :: String.t(), filename :: String.t()} | {:error, any()} | :error
- def file_mime_type(path, filename) do
- with {:ok, content_type} <- file_mime_type(path),
- filename <- fix_extension(filename, content_type) do
- {:ok, content_type, filename}
- end
- end
-
- @spec file_mime_type(String.t()) :: {:ok, String.t()} | {:error, any()} | :error
- def file_mime_type(filename) do
- File.open(filename, [:read], fn f ->
- check_mime_type(IO.binread(f, @read_bytes))
- end)
- end
-
- def bin_mime_type(binary, filename) do
- with {:ok, content_type} <- bin_mime_type(binary),
- filename <- fix_extension(filename, content_type) do
- {:ok, content_type, filename}
- end
- end
-
- @spec bin_mime_type(binary()) :: {:ok, String.t()} | :error
- def bin_mime_type(<>) do
- {:ok, check_mime_type(head)}
- end
-
- def bin_mime_type(_), do: :error
-
- def mime_type(<<_::binary>>), do: {:ok, @default}
-
- defp fix_extension(filename, content_type) do
- parts = String.split(filename, ".")
-
- new_filename =
- if length(parts) > 1 do
- Enum.drop(parts, -1) |> Enum.join(".")
- else
- Enum.join(parts)
- end
-
- cond do
- content_type == "application/octet-stream" ->
- filename
-
- ext = List.first(MIME.extensions(content_type)) ->
- new_filename <> "." <> ext
-
- true ->
- Enum.join([new_filename, String.split(content_type, "/") |> List.last()], ".")
- end
- end
-
- defp check_mime_type(<<0x89, 0x50, 0x4E, 0x47, 0x0D, 0x0A, 0x1A, 0x0A, _::binary>>) do
- "image/png"
- end
-
- defp check_mime_type(<<0x47, 0x49, 0x46, 0x38, _, 0x61, _::binary>>) do
- "image/gif"
- end
-
- defp check_mime_type(<<0xFF, 0xD8, 0xFF, _::binary>>) do
- "image/jpeg"
- end
-
- defp check_mime_type(<<0x1A, 0x45, 0xDF, 0xA3, _::binary>>) do
- "video/webm"
- end
-
- defp check_mime_type(<<0x00, 0x00, 0x00, _, 0x66, 0x74, 0x79, 0x70, _::binary>>) do
- "video/mp4"
- end
-
- defp check_mime_type(<<0x49, 0x44, 0x33, _::binary>>) do
- "audio/mpeg"
- end
-
- defp check_mime_type(<<255, 251, _, 68, 0, 0, 0, 0, _::binary>>) do
- "audio/mpeg"
- end
-
- defp check_mime_type(
- <<0x4F, 0x67, 0x67, 0x53, 0x00, 0x02, 0x00, 0x00, _::size(160), 0x80, 0x74, 0x68, 0x65,
- 0x6F, 0x72, 0x61, _::binary>>
- ) do
- "video/ogg"
- end
-
- defp check_mime_type(<<0x4F, 0x67, 0x67, 0x53, 0x00, 0x02, 0x00, 0x00, _::binary>>) do
- "audio/ogg"
- end
-
- defp check_mime_type(<<"RIFF", _::binary-size(4), "WAVE", _::binary>>) do
- "audio/wav"
- end
-
- defp check_mime_type(<<"RIFF", _::binary-size(4), "WEBP", _::binary>>) do
- "image/webp"
- end
-
- defp check_mime_type(<<"RIFF", _::binary-size(4), "AVI.", _::binary>>) do
- "video/avi"
- end
-
- defp check_mime_type(_) do
- @default
- end
-end
diff --git a/lib/pleroma/moderation_log.ex b/lib/pleroma/moderation_log.ex
index 31c9afe2a..142dd8e0a 100644
--- a/lib/pleroma/moderation_log.ex
+++ b/lib/pleroma/moderation_log.ex
@@ -1,3 +1,7 @@
+# Pleroma: A lightweight social networking server
+# Copyright © 2017-2020 Pleroma Authors
+# SPDX-License-Identifier: AGPL-3.0-only
+
defmodule Pleroma.ModerationLog do
use Ecto.Schema
@@ -320,6 +324,19 @@ def insert_log(%{
|> insert_log_entry_with_message()
end
+ @spec insert_log(%{actor: User, action: String.t(), subject_id: String.t()}) ::
+ {:ok, ModerationLog} | {:error, any}
+ def insert_log(%{actor: %User{} = actor, action: "chat_message_delete", subject_id: subject_id}) do
+ %ModerationLog{
+ data: %{
+ "actor" => %{"nickname" => actor.nickname},
+ "action" => "chat_message_delete",
+ "subject_id" => subject_id
+ }
+ }
+ |> insert_log_entry_with_message()
+ end
+
@spec insert_log_entry_with_message(ModerationLog) :: {:ok, ModerationLog} | {:error, any}
defp insert_log_entry_with_message(entry) do
entry.data["message"]
@@ -627,6 +644,27 @@ def get_log_entry_message(%ModerationLog{
"@#{actor_nickname} updated users: #{users_to_nicknames_string(subjects)}"
end
+ @spec get_log_entry_message(ModerationLog) :: String.t()
+ def get_log_entry_message(%ModerationLog{
+ data: %{
+ "actor" => %{"nickname" => actor_nickname},
+ "action" => "chat_message_delete",
+ "subject_id" => subject_id
+ }
+ }) do
+ "@#{actor_nickname} deleted chat message ##{subject_id}"
+ end
+
+ def get_log_entry_message(%ModerationLog{
+ data: %{
+ "actor" => %{"nickname" => actor_nickname},
+ "action" => "create_backup",
+ "subject" => %{"nickname" => user_nickname}
+ }
+ }) do
+ "@#{actor_nickname} requested account backup for @#{user_nickname}"
+ end
+
defp nicknames_to_string(nicknames) do
nicknames
|> Enum.map(&"@#{&1}")
diff --git a/lib/pleroma/object/containment.ex b/lib/pleroma/object/containment.ex
index bc88e8a0c..29cb3d672 100644
--- a/lib/pleroma/object/containment.ex
+++ b/lib/pleroma/object/containment.ex
@@ -44,13 +44,6 @@ def get_object(_) do
nil
end
- # TODO: We explicitly allow 'tag' URIs through, due to references to legacy OStatus
- # objects being present in the test suite environment. Once these objects are
- # removed, please also remove this.
- if Mix.env() == :test do
- defp compare_uris(_, %URI{scheme: "tag"}), do: :ok
- end
-
defp compare_uris(%URI{host: host} = _id_uri, %URI{host: host} = _other_uri), do: :ok
defp compare_uris(_id_uri, _other_uri), do: :error
diff --git a/lib/pleroma/object/fetcher.ex b/lib/pleroma/object/fetcher.ex
index 1de2ce6c3..169298b34 100644
--- a/lib/pleroma/object/fetcher.ex
+++ b/lib/pleroma/object/fetcher.ex
@@ -12,6 +12,7 @@ defmodule Pleroma.Object.Fetcher do
alias Pleroma.Web.ActivityPub.ObjectValidator
alias Pleroma.Web.ActivityPub.Transmogrifier
alias Pleroma.Web.Federator
+ alias Pleroma.Web.FedSockets
require Logger
require Pleroma.Constants
@@ -98,8 +99,8 @@ def fetch_object_from_id(id, options \\ []) do
{:containment, _} ->
{:error, "Object containment failed."}
- {:transmogrifier, {:error, {:reject, nil}}} ->
- {:reject, nil}
+ {:transmogrifier, {:error, {:reject, e}}} ->
+ {:reject, e}
{:transmogrifier, _} = e ->
{:error, e}
@@ -182,27 +183,20 @@ defp maybe_date_fetch(headers, date) do
end
end
- def fetch_and_contain_remote_object_from_id(id) when is_binary(id) do
+ def fetch_and_contain_remote_object_from_id(prm, opts \\ [])
+
+ def fetch_and_contain_remote_object_from_id(%{"id" => id}, opts),
+ do: fetch_and_contain_remote_object_from_id(id, opts)
+
+ def fetch_and_contain_remote_object_from_id(id, opts) when is_binary(id) do
Logger.debug("Fetching object #{id} via AP")
- date = Pleroma.Signature.signed_date()
-
- headers =
- [{"accept", "application/activity+json"}]
- |> maybe_date_fetch(date)
- |> sign_fetch(id, date)
-
- Logger.debug("Fetch headers: #{inspect(headers)}")
-
with {:scheme, true} <- {:scheme, String.starts_with?(id, "http")},
- {:ok, %{body: body, status: code}} when code in 200..299 <- HTTP.get(id, headers),
- {:ok, data} <- Jason.decode(body),
+ {:ok, body} <- get_object(id, opts),
+ {:ok, data} <- safe_json_decode(body),
:ok <- Containment.contain_origin_from_id(id, data) do
{:ok, data}
else
- {:ok, %{status: code}} when code in [404, 410] ->
- {:error, "Object has been deleted"}
-
{:scheme, _} ->
{:error, "Unsupported URI scheme"}
@@ -214,8 +208,44 @@ def fetch_and_contain_remote_object_from_id(id) when is_binary(id) do
end
end
- def fetch_and_contain_remote_object_from_id(%{"id" => id}),
- do: fetch_and_contain_remote_object_from_id(id)
+ def fetch_and_contain_remote_object_from_id(_id, _opts),
+ do: {:error, "id must be a string"}
- def fetch_and_contain_remote_object_from_id(_id), do: {:error, "id must be a string"}
+ defp get_object(id, opts) do
+ with false <- Keyword.get(opts, :force_http, false),
+ {:ok, fedsocket} <- FedSockets.get_or_create_fed_socket(id) do
+ Logger.debug("fetching via fedsocket - #{inspect(id)}")
+ FedSockets.fetch(fedsocket, id)
+ else
+ _other ->
+ Logger.debug("fetching via http - #{inspect(id)}")
+ get_object_http(id)
+ end
+ end
+
+ defp get_object_http(id) do
+ date = Pleroma.Signature.signed_date()
+
+ headers =
+ [{"accept", "application/activity+json"}]
+ |> maybe_date_fetch(date)
+ |> sign_fetch(id, date)
+
+ case HTTP.get(id, headers) do
+ {:ok, %{body: body, status: code}} when code in 200..299 ->
+ {:ok, body}
+
+ {:ok, %{status: code}} when code in [404, 410] ->
+ {:error, "Object has been deleted"}
+
+ {:error, e} ->
+ {:error, e}
+
+ e ->
+ {:error, e}
+ end
+ end
+
+ defp safe_json_decode(nil), do: {:ok, nil}
+ defp safe_json_decode(json), do: Jason.decode(json)
end
diff --git a/lib/pleroma/plugs/remote_ip.ex b/lib/pleroma/plugs/remote_ip.ex
deleted file mode 100644
index 0ac9050d0..000000000
--- a/lib/pleroma/plugs/remote_ip.ex
+++ /dev/null
@@ -1,54 +0,0 @@
-# Pleroma: A lightweight social networking server
-# Copyright © 2017-2020 Pleroma Authors
-# SPDX-License-Identifier: AGPL-3.0-only
-
-defmodule Pleroma.Plugs.RemoteIp do
- @moduledoc """
- This is a shim to call [`RemoteIp`](https://git.pleroma.social/pleroma/remote_ip) but with runtime configuration.
- """
-
- import Plug.Conn
-
- @behaviour Plug
-
- @headers ~w[
- x-forwarded-for
- ]
-
- # https://en.wikipedia.org/wiki/Localhost
- # https://en.wikipedia.org/wiki/Private_network
- @reserved ~w[
- 127.0.0.0/8
- ::1/128
- fc00::/7
- 10.0.0.0/8
- 172.16.0.0/12
- 192.168.0.0/16
- ]
-
- def init(_), do: nil
-
- def call(%{remote_ip: original_remote_ip} = conn, _) do
- config = Pleroma.Config.get(__MODULE__, [])
-
- if Keyword.get(config, :enabled, false) do
- %{remote_ip: new_remote_ip} = conn = RemoteIp.call(conn, remote_ip_opts(config))
- assign(conn, :remote_ip_found, original_remote_ip != new_remote_ip)
- else
- conn
- end
- end
-
- defp remote_ip_opts(config) do
- headers = config |> Keyword.get(:headers, @headers) |> MapSet.new()
- reserved = Keyword.get(config, :reserved, @reserved)
-
- proxies =
- config
- |> Keyword.get(:proxies, [])
- |> Enum.concat(reserved)
- |> Enum.map(&InetCidr.parse/1)
-
- {headers, proxies}
- end
-end
diff --git a/lib/pleroma/repo.ex b/lib/pleroma/repo.ex
index f317e4d58..4524bd5e2 100644
--- a/lib/pleroma/repo.ex
+++ b/lib/pleroma/repo.ex
@@ -49,7 +49,21 @@ def get_assoc(resource, association) do
end
end
- def chunk_stream(query, chunk_size) do
+ @doc """
+ Returns a lazy enumerable that emits all entries from the data store matching the given query.
+
+ `returns_as` use to group records. use the `batches` option to fetch records in bulk.
+
+ ## Examples
+
+ # fetch records one-by-one
+ iex> Pleroma.Repo.chunk_stream(Pleroma.Activity.Queries.by_actor(ap_id), 500)
+
+ # fetch records in bulk
+ iex> Pleroma.Repo.chunk_stream(Pleroma.Activity.Queries.by_actor(ap_id), 500, :batches)
+ """
+ @spec chunk_stream(Ecto.Query.t(), integer(), atom()) :: Enumerable.t()
+ def chunk_stream(query, chunk_size, returns_as \\ :one) do
# We don't actually need start and end funcitons of resource streaming,
# but it seems to be the only way to not fetch records one-by-one and
# have individual records be the elements of the stream, instead of
@@ -69,7 +83,12 @@ def chunk_stream(query, chunk_size) do
records ->
last_id = List.last(records).id
- {records, last_id}
+
+ if returns_as == :one do
+ {records, last_id}
+ else
+ {[records], last_id}
+ end
end
end,
fn _ -> :ok end
diff --git a/lib/pleroma/repo_streamer.ex b/lib/pleroma/repo_streamer.ex
deleted file mode 100644
index cb4d7bb7a..000000000
--- a/lib/pleroma/repo_streamer.ex
+++ /dev/null
@@ -1,34 +0,0 @@
-# Pleroma: A lightweight social networking server
-# Copyright © 2017-2020 Pleroma Authors
-# SPDX-License-Identifier: AGPL-3.0-only
-
-defmodule Pleroma.RepoStreamer do
- alias Pleroma.Repo
- import Ecto.Query
-
- def chunk_stream(query, chunk_size) do
- Stream.unfold(0, fn
- :halt ->
- {[], :halt}
-
- last_id ->
- query
- |> order_by(asc: :id)
- |> where([r], r.id > ^last_id)
- |> limit(^chunk_size)
- |> Repo.all()
- |> case do
- [] ->
- {[], :halt}
-
- records ->
- last_id = List.last(records).id
- {records, last_id}
- end
- end)
- |> Stream.take_while(fn
- [] -> false
- _ -> true
- end)
- end
-end
diff --git a/lib/pleroma/reverse_proxy/reverse_proxy.ex b/lib/pleroma/reverse_proxy.ex
similarity index 98%
rename from lib/pleroma/reverse_proxy/reverse_proxy.ex
rename to lib/pleroma/reverse_proxy.ex
index 0de4e2309..8ae1157df 100644
--- a/lib/pleroma/reverse_proxy/reverse_proxy.ex
+++ b/lib/pleroma/reverse_proxy.ex
@@ -17,6 +17,9 @@ defmodule Pleroma.ReverseProxy do
@failed_request_ttl :timer.seconds(60)
@methods ~w(GET HEAD)
+ def max_read_duration_default, do: @max_read_duration
+ def default_cache_control_header, do: @default_cache_control_header
+
@moduledoc """
A reverse proxy.
@@ -391,6 +394,8 @@ defp body_size_constraint(size, limit) when is_integer(limit) and limit > 0 and
defp body_size_constraint(_, _), do: :ok
+ defp check_read_duration(nil = _duration, max), do: check_read_duration(@max_read_duration, max)
+
defp check_read_duration(duration, max)
when is_integer(duration) and is_integer(max) and max > 0 do
if duration > max do
diff --git a/lib/pleroma/reverse_proxy/client/tesla.ex b/lib/pleroma/reverse_proxy/client/tesla.ex
index d5a339681..4b118eec2 100644
--- a/lib/pleroma/reverse_proxy/client/tesla.ex
+++ b/lib/pleroma/reverse_proxy/client/tesla.ex
@@ -28,7 +28,7 @@ def request(method, url, headers, body, opts \\ []) do
url,
body,
headers,
- Keyword.put(opts, :adapter, opts)
+ opts
) do
if is_map(response.body) and method != :head do
{:ok, response.status, response.headers, response.body}
diff --git a/lib/pleroma/signature.ex b/lib/pleroma/signature.ex
index 3aa6909d2..e388993b7 100644
--- a/lib/pleroma/signature.ex
+++ b/lib/pleroma/signature.ex
@@ -39,7 +39,7 @@ def key_id_to_actor_id(key_id) do
def fetch_public_key(conn) do
with %{"keyId" => kid} <- HTTPSignatures.signature_for_conn(conn),
{:ok, actor_id} <- key_id_to_actor_id(kid),
- {:ok, public_key} <- User.get_public_key_for_ap_id(actor_id) do
+ {:ok, public_key} <- User.get_public_key_for_ap_id(actor_id, force_http: true) do
{:ok, public_key}
else
e ->
@@ -50,8 +50,8 @@ def fetch_public_key(conn) do
def refetch_public_key(conn) do
with %{"keyId" => kid} <- HTTPSignatures.signature_for_conn(conn),
{:ok, actor_id} <- key_id_to_actor_id(kid),
- {:ok, _user} <- ActivityPub.make_user_from_ap_id(actor_id),
- {:ok, public_key} <- User.get_public_key_for_ap_id(actor_id) do
+ {:ok, _user} <- ActivityPub.make_user_from_ap_id(actor_id, force_http: true),
+ {:ok, public_key} <- User.get_public_key_for_ap_id(actor_id, force_http: true) do
{:ok, public_key}
else
e ->
diff --git a/lib/pleroma/telemetry/logger.ex b/lib/pleroma/telemetry/logger.ex
index 4cacae02f..003079cf3 100644
--- a/lib/pleroma/telemetry/logger.ex
+++ b/lib/pleroma/telemetry/logger.ex
@@ -1,3 +1,7 @@
+# Pleroma: A lightweight social networking server
+# Copyright © 2017-2020 Pleroma Authors
+# SPDX-License-Identifier: AGPL-3.0-only
+
defmodule Pleroma.Telemetry.Logger do
@moduledoc "Transforms Pleroma telemetry events to logs"
@@ -7,7 +11,8 @@ defmodule Pleroma.Telemetry.Logger do
[:pleroma, :connection_pool, :reclaim, :start],
[:pleroma, :connection_pool, :reclaim, :stop],
[:pleroma, :connection_pool, :provision_failure],
- [:pleroma, :connection_pool, :client_death]
+ [:pleroma, :connection_pool, :client, :dead],
+ [:pleroma, :connection_pool, :client, :add]
]
def attach do
:telemetry.attach_many("pleroma-logger", @events, &handle_event/4, [])
@@ -62,7 +67,7 @@ def handle_event(
end
def handle_event(
- [:pleroma, :connection_pool, :client_death],
+ [:pleroma, :connection_pool, :client, :dead],
%{client_pid: client_pid, reason: reason},
%{key: key},
_
@@ -73,4 +78,17 @@ def handle_event(
}"
end)
end
+
+ def handle_event(
+ [:pleroma, :connection_pool, :client, :add],
+ %{clients: [_, _ | _] = clients},
+ %{key: key, protocol: :http},
+ _
+ ) do
+ Logger.info(fn ->
+ "Pool worker for #{key}: #{length(clients)} clients are using an HTTP1 connection at the same time, head-of-line blocking might occur."
+ end)
+ end
+
+ def handle_event([:pleroma, :connection_pool, :client, :add], _, _, _), do: :ok
end
diff --git a/lib/pleroma/tests/auth_test_controller.ex b/lib/pleroma/tests/auth_test_controller.ex
index fb04411d9..b30d83567 100644
--- a/lib/pleroma/tests/auth_test_controller.ex
+++ b/lib/pleroma/tests/auth_test_controller.ex
@@ -8,9 +8,9 @@ defmodule Pleroma.Tests.AuthTestController do
use Pleroma.Web, :controller
- alias Pleroma.Plugs.EnsurePublicOrAuthenticatedPlug
- alias Pleroma.Plugs.OAuthScopesPlug
alias Pleroma.User
+ alias Pleroma.Web.Plugs.EnsurePublicOrAuthenticatedPlug
+ alias Pleroma.Web.Plugs.OAuthScopesPlug
# Serves only with proper OAuth token (:api and :authenticated_api)
# Skipping EnsurePublicOrAuthenticatedPlug has no effect in this case
diff --git a/lib/pleroma/upload.ex b/lib/pleroma/upload.ex
index 015c87593..db2cc1dae 100644
--- a/lib/pleroma/upload.ex
+++ b/lib/pleroma/upload.ex
@@ -66,6 +66,7 @@ defp get_description(opts, upload) do
end
@spec store(source, options :: [option()]) :: {:ok, Map.t()} | {:error, any()}
+ @doc "Store a file. If using a `Plug.Upload{}` as the source, be sure to use `Majic.Plug` to ensure its content_type and filename is correct."
def store(upload, opts \\ []) do
opts = get_opts(opts)
@@ -139,14 +140,13 @@ defp get_opts(opts) do
end
defp prepare_upload(%Plug.Upload{} = file, opts) do
- with :ok <- check_file_size(file.path, opts.size_limit),
- {:ok, content_type, name} <- Pleroma.MIME.file_mime_type(file.path, file.filename) do
+ with :ok <- check_file_size(file.path, opts.size_limit) do
{:ok,
%__MODULE__{
id: UUID.generate(),
- name: name,
+ name: file.filename,
tempfile: file.path,
- content_type: content_type
+ content_type: file.content_type
}}
end
end
@@ -154,16 +154,17 @@ defp prepare_upload(%Plug.Upload{} = file, opts) do
defp prepare_upload(%{img: "data:image/" <> image_data}, opts) do
parsed = Regex.named_captures(~r/(?jpeg|png|gif);base64,(?.*)/, image_data)
data = Base.decode64!(parsed["data"], ignore: :whitespace)
- hash = String.downcase(Base.encode16(:crypto.hash(:sha256, data)))
+ hash = Base.encode16(:crypto.hash(:sha256, data), lower: true)
with :ok <- check_binary_size(data, opts.size_limit),
tmp_path <- tempfile_for_image(data),
- {:ok, content_type, name} <-
- Pleroma.MIME.bin_mime_type(data, hash <> "." <> parsed["filetype"]) do
+ {:ok, %{mime_type: content_type}} <-
+ Majic.perform({:bytes, data}, pool: Pleroma.MajicPool),
+ [ext | _] <- MIME.extensions(content_type) do
{:ok,
%__MODULE__{
id: UUID.generate(),
- name: name,
+ name: hash <> "." <> ext,
tempfile: tmp_path,
content_type: content_type
}}
@@ -172,7 +173,7 @@ defp prepare_upload(%{img: "data:image/" <> image_data}, opts) do
# For Mix.Tasks.MigrateLocalUploads
defp prepare_upload(%__MODULE__{tempfile: path} = upload, _opts) do
- with {:ok, content_type} <- Pleroma.MIME.file_mime_type(path) do
+ with {:ok, %{mime_type: content_type}} <- Majic.perform(path, pool: Pleroma.MajicPool) do
{:ok, %__MODULE__{upload | content_type: content_type}}
end
end
diff --git a/lib/pleroma/uploaders/uploader.ex b/lib/pleroma/uploaders/uploader.ex
index 9a94534e9..6249eceb1 100644
--- a/lib/pleroma/uploaders/uploader.ex
+++ b/lib/pleroma/uploaders/uploader.ex
@@ -12,7 +12,7 @@ defmodule Pleroma.Uploaders.Uploader do
@doc """
Instructs how to get the file from the backend.
- Used by `Pleroma.Plugs.UploadedMedia`.
+ Used by `Pleroma.Web.Plugs.UploadedMedia`.
"""
@type get_method :: {:static_dir, directory :: String.t()} | {:url, url :: String.t()}
@callback get_file(file :: String.t()) :: {:ok, get_method()}
diff --git a/lib/pleroma/user.ex b/lib/pleroma/user.ex
index f323fc6ed..059d94e30 100644
--- a/lib/pleroma/user.ex
+++ b/lib/pleroma/user.ex
@@ -25,7 +25,6 @@ defmodule Pleroma.User do
alias Pleroma.Object
alias Pleroma.Registration
alias Pleroma.Repo
- alias Pleroma.RepoStreamer
alias Pleroma.User
alias Pleroma.UserRelationship
alias Pleroma.Web
@@ -108,7 +107,7 @@ defmodule Pleroma.User do
field(:note_count, :integer, default: 0)
field(:follower_count, :integer, default: 0)
field(:following_count, :integer, default: 0)
- field(:locked, :boolean, default: false)
+ field(:is_locked, :boolean, default: false)
field(:confirmation_pending, :boolean, default: false)
field(:password_reset_pending, :boolean, default: false)
field(:approval_pending, :boolean, default: false)
@@ -129,7 +128,6 @@ defmodule Pleroma.User do
field(:hide_followers, :boolean, default: false)
field(:hide_follows, :boolean, default: false)
field(:hide_favorites, :boolean, default: true)
- field(:unread_conversation_count, :integer, default: 0)
field(:pinned_activities, {:array, :string}, default: [])
field(:email_notifications, :map, default: %{"digest" => false})
field(:mascot, :map, default: nil)
@@ -137,7 +135,7 @@ defmodule Pleroma.User do
field(:pleroma_settings_store, :map, default: %{})
field(:fields, {:array, :map}, default: [])
field(:raw_fields, {:array, :map}, default: [])
- field(:discoverable, :boolean, default: false)
+ field(:is_discoverable, :boolean, default: false)
field(:invisible, :boolean, default: false)
field(:allow_following_move, :boolean, default: true)
field(:skip_thread_containment, :boolean, default: false)
@@ -276,9 +274,9 @@ def binary_id(%User{} = user), do: binary_id(user.id)
@spec account_status(User.t()) :: account_status()
def account_status(%User{deactivated: true}), do: :deactivated
def account_status(%User{password_reset_pending: true}), do: :password_reset_pending
- def account_status(%User{approval_pending: true}), do: :approval_pending
+ def account_status(%User{local: true, approval_pending: true}), do: :approval_pending
- def account_status(%User{confirmation_pending: true}) do
+ def account_status(%User{local: true, confirmation_pending: true}) do
if Config.get([:instance, :account_activation_required]) do
:confirmation_pending
else
@@ -427,7 +425,6 @@ def remote_user_changeset(struct \\ %User{local: false}, params) do
params,
[
:bio,
- :name,
:emoji,
:ap_id,
:inbox,
@@ -437,7 +434,7 @@ def remote_user_changeset(struct \\ %User{local: false}, params) do
:avatar,
:ap_enabled,
:banner,
- :locked,
+ :is_locked,
:last_refreshed_at,
:uri,
:follower_address,
@@ -449,14 +446,16 @@ def remote_user_changeset(struct \\ %User{local: false}, params) do
:follower_count,
:fields,
:following_count,
- :discoverable,
+ :is_discoverable,
:invisible,
:actor_type,
:also_known_as,
:accepts_chat_messages
]
)
- |> validate_required([:name, :ap_id])
+ |> cast(params, [:name], empty_values: [])
+ |> validate_required([:ap_id])
+ |> validate_required([:name], trim: false)
|> unique_constraint(:nickname)
|> validate_format(:nickname, @email_regex)
|> validate_length(:bio, max: bio_limit)
@@ -480,7 +479,7 @@ def update_changeset(struct, params \\ %{}) do
:public_key,
:inbox,
:shared_inbox,
- :locked,
+ :is_locked,
:no_rich_text,
:default_scope,
:banner,
@@ -496,7 +495,7 @@ def update_changeset(struct, params \\ %{}) do
:fields,
:raw_fields,
:pleroma_settings_store,
- :discoverable,
+ :is_discoverable,
:actor_type,
:also_known_as,
:accepts_chat_messages
@@ -766,6 +765,16 @@ defp autofollow_users(user) do
follow_all(user, autofollowed_users)
end
+ defp autofollowing_users(user) do
+ candidates = Config.get([:instance, :autofollowing_nicknames])
+
+ User.Query.build(%{nickname: candidates, local: true, deactivated: false})
+ |> Repo.all()
+ |> Enum.each(&follow(&1, user, :follow_accept))
+
+ {:ok, :success}
+ end
+
@doc "Inserts provided changeset, performs post-registration actions (confirmation email sending etc.)"
def register(%Ecto.Changeset{} = changeset) do
with {:ok, user} <- Repo.insert(changeset) do
@@ -775,6 +784,7 @@ def register(%Ecto.Changeset{} = changeset) do
def post_register_action(%User{} = user) do
with {:ok, user} <- autofollow_users(user),
+ {:ok, _} <- autofollowing_users(user),
{:ok, user} <- set_cache(user),
{:ok, _} <- send_welcome_email(user),
{:ok, _} <- send_welcome_message(user),
@@ -814,7 +824,8 @@ def send_welcome_email(%User{email: email} = user) when is_binary(email) do
def send_welcome_email(_), do: {:ok, :noop}
@spec try_send_confirmation_email(User.t()) :: {:ok, :enqueued | :noop}
- def try_send_confirmation_email(%User{confirmation_pending: true} = user) do
+ def try_send_confirmation_email(%User{confirmation_pending: true, email: email} = user)
+ when is_binary(email) do
if Config.get([:instance, :account_activation_required]) do
send_confirmation_email(user)
{:ok, :enqueued}
@@ -847,7 +858,7 @@ def needs_update?(_), do: true
@spec maybe_direct_follow(User.t(), User.t()) :: {:ok, User.t()} | {:error, String.t()}
# "Locked" (self-locked) users demand explicit authorization of follow requests
- def maybe_direct_follow(%User{} = follower, %User{local: true, locked: true} = followed) do
+ def maybe_direct_follow(%User{} = follower, %User{local: true, is_locked: true} = followed) do
follow(follower, followed, :follow_pending)
end
@@ -915,9 +926,7 @@ defp do_unfollow(%User{} = follower, %User{} = followed) do
FollowingRelationship.unfollow(follower, followed)
{:ok, followed} = update_follower_count(followed)
- {:ok, follower} =
- follower
- |> update_following_count()
+ {:ok, follower} = update_following_count(follower)
{:ok, follower, followed}
@@ -956,7 +965,7 @@ def get_follow_state(
end
def locked?(%User{} = user) do
- user.locked || false
+ user.is_locked || false
end
def get_by_id(id) do
@@ -1295,47 +1304,6 @@ def update_following_count(%User{local: true} = user) do
|> update_and_set_cache()
end
- def set_unread_conversation_count(%User{local: true} = user) do
- unread_query = Participation.unread_conversation_count_for_user(user)
-
- User
- |> join(:inner, [u], p in subquery(unread_query))
- |> update([u, p],
- set: [unread_conversation_count: p.count]
- )
- |> where([u], u.id == ^user.id)
- |> select([u], u)
- |> Repo.update_all([])
- |> case do
- {1, [user]} -> set_cache(user)
- _ -> {:error, user}
- end
- end
-
- def set_unread_conversation_count(user), do: {:ok, user}
-
- def increment_unread_conversation_count(conversation, %User{local: true} = user) do
- unread_query =
- Participation.unread_conversation_count_for_user(user)
- |> where([p], p.conversation_id == ^conversation.id)
-
- User
- |> join(:inner, [u], p in subquery(unread_query))
- |> update([u, p],
- inc: [unread_conversation_count: 1]
- )
- |> where([u], u.id == ^user.id)
- |> where([u, p], p.count == 0)
- |> select([u], u)
- |> Repo.update_all([])
- |> case do
- {1, [user]} -> set_cache(user)
- _ -> {:error, user}
- end
- end
-
- def increment_unread_conversation_count(_, user), do: {:ok, user}
-
@spec get_users_from_set([String.t()], keyword()) :: [User.t()]
def get_users_from_set(ap_ids, opts \\ []) do
local_only = Keyword.get(opts, :local_only, true)
@@ -1603,7 +1571,7 @@ def purge_user_changeset(user) do
note_count: 0,
follower_count: 0,
following_count: 0,
- locked: false,
+ is_locked: false,
confirmation_pending: false,
password_reset_pending: false,
approval_pending: false,
@@ -1620,7 +1588,7 @@ def purge_user_changeset(user) do
pleroma_settings_store: %{},
fields: [],
raw_fields: [],
- discoverable: false,
+ is_discoverable: false,
also_known_as: []
})
end
@@ -1686,42 +1654,6 @@ def perform(:delete, %User{} = user) do
def perform(:deactivate_async, user, status), do: deactivate(user, status)
- @spec perform(atom(), User.t(), list()) :: list() | {:error, any()}
- def perform(:blocks_import, %User{} = blocker, blocked_identifiers)
- when is_list(blocked_identifiers) do
- Enum.map(
- blocked_identifiers,
- fn blocked_identifier ->
- with {:ok, %User{} = blocked} <- get_or_fetch(blocked_identifier),
- {:ok, _block} <- CommonAPI.block(blocker, blocked) do
- blocked
- else
- err ->
- Logger.debug("blocks_import failed for #{blocked_identifier} with: #{inspect(err)}")
- err
- end
- end
- )
- end
-
- def perform(:follow_import, %User{} = follower, followed_identifiers)
- when is_list(followed_identifiers) do
- Enum.map(
- followed_identifiers,
- fn followed_identifier ->
- with {:ok, %User{} = followed} <- get_or_fetch(followed_identifier),
- {:ok, follower} <- maybe_direct_follow(follower, followed),
- {:ok, _, _, _} <- CommonAPI.follow(follower, followed) do
- followed
- else
- err ->
- Logger.debug("follow_import failed for #{followed_identifier} with: #{inspect(err)}")
- err
- end
- end
- )
- end
-
@spec external_users_query() :: Ecto.Query.t()
def external_users_query do
User.Query.build(%{
@@ -1750,21 +1682,6 @@ def external_users(opts \\ []) do
Repo.all(query)
end
- def blocks_import(%User{} = blocker, blocked_identifiers) when is_list(blocked_identifiers) do
- BackgroundWorker.enqueue("blocks_import", %{
- "blocker_id" => blocker.id,
- "blocked_identifiers" => blocked_identifiers
- })
- end
-
- def follow_import(%User{} = follower, followed_identifiers)
- when is_list(followed_identifiers) do
- BackgroundWorker.enqueue("follow_import", %{
- "follower_id" => follower.id,
- "followed_identifiers" => followed_identifiers
- })
- end
-
def delete_notifications_from_user_activities(%User{ap_id: ap_id}) do
Notification
|> join(:inner, [n], activity in assoc(n, :activity))
@@ -1775,7 +1692,7 @@ def delete_notifications_from_user_activities(%User{ap_id: ap_id}) do
def delete_user_activities(%User{ap_id: ap_id} = user) do
ap_id
|> Activity.Queries.by_actor()
- |> RepoStreamer.chunk_stream(50)
+ |> Repo.chunk_stream(50, :batches)
|> Stream.each(fn activities ->
Enum.each(activities, fn activity -> delete_activity(activity, user) end)
end)
@@ -1821,12 +1738,12 @@ def html_filter_policy(%User{no_rich_text: true}) do
def html_filter_policy(_), do: Config.get([:markup, :scrub_policy])
- def fetch_by_ap_id(ap_id), do: ActivityPub.make_user_from_ap_id(ap_id)
+ def fetch_by_ap_id(ap_id, opts \\ []), do: ActivityPub.make_user_from_ap_id(ap_id, opts)
- def get_or_fetch_by_ap_id(ap_id) do
+ def get_or_fetch_by_ap_id(ap_id, opts \\ []) do
cached_user = get_cached_by_ap_id(ap_id)
- maybe_fetched_user = needs_update?(cached_user) && fetch_by_ap_id(ap_id)
+ maybe_fetched_user = needs_update?(cached_user) && fetch_by_ap_id(ap_id, opts)
case {cached_user, maybe_fetched_user} do
{_, {:ok, %User{} = user}} ->
@@ -1899,8 +1816,8 @@ def public_key(%{public_key: public_key_pem}) when is_binary(public_key_pem) do
def public_key(_), do: {:error, "key not found"}
- def get_public_key_for_ap_id(ap_id) do
- with {:ok, %User{} = user} <- get_or_fetch_by_ap_id(ap_id),
+ def get_public_key_for_ap_id(ap_id, opts \\ []) do
+ with {:ok, %User{} = user} <- get_or_fetch_by_ap_id(ap_id, opts),
{:ok, public_key} <- public_key(user) do
{:ok, public_key}
else
@@ -2123,6 +2040,13 @@ def toggle_confirmation(users) do
Enum.map(users, &toggle_confirmation/1)
end
+ @spec need_confirmation(User.t(), boolean()) :: {:ok, User.t()} | {:error, Changeset.t()}
+ def need_confirmation(%User{} = user, bool) do
+ user
+ |> confirmation_changeset(need_confirmation: bool)
+ |> update_and_set_cache()
+ end
+
def get_mascot(%{mascot: %{} = mascot}) when not is_nil(mascot) do
mascot
end
@@ -2315,6 +2239,11 @@ def add_pinnned_activity(user, %Pleroma.Activity{id: id}) do
max_pinned_statuses = Config.get([:instance, :max_pinned_statuses], 0)
params = %{pinned_activities: user.pinned_activities ++ [id]}
+ # if pinned activity was scheduled for deletion, we remove job
+ if expiration = Pleroma.Workers.PurgeExpiredActivity.get_expiration(id) do
+ Oban.cancel_job(expiration.id)
+ end
+
user
|> cast(params, [:pinned_activities])
|> validate_length(:pinned_activities,
@@ -2327,9 +2256,21 @@ def add_pinnned_activity(user, %Pleroma.Activity{id: id}) do
|> update_and_set_cache()
end
- def remove_pinnned_activity(user, %Pleroma.Activity{id: id}) do
+ def remove_pinnned_activity(user, %Pleroma.Activity{id: id, data: data}) do
params = %{pinned_activities: List.delete(user.pinned_activities, id)}
+ # if pinned activity was scheduled for deletion, we reschedule it for deletion
+ if data["expires_at"] do
+ # MRF.ActivityExpirationPolicy used UTC timestamps for expires_at in original implementation
+ {:ok, expires_at} =
+ data["expires_at"] |> Pleroma.EctoType.ActivityPub.ObjectValidators.DateTime.cast()
+
+ Pleroma.Workers.PurgeExpiredActivity.enqueue(%{
+ activity_id: id,
+ expires_at: expires_at
+ })
+ end
+
user
|> cast(params, [:pinned_activities])
|> update_and_set_cache()
diff --git a/lib/pleroma/user/backup.ex b/lib/pleroma/user/backup.ex
new file mode 100644
index 000000000..a9041fd94
--- /dev/null
+++ b/lib/pleroma/user/backup.ex
@@ -0,0 +1,258 @@
+# Pleroma: A lightweight social networking server
+# Copyright © 2017-2020 Pleroma Authors
+# SPDX-License-Identifier: AGPL-3.0-only
+
+defmodule Pleroma.User.Backup do
+ use Ecto.Schema
+
+ import Ecto.Changeset
+ import Ecto.Query
+ import Pleroma.Web.Gettext
+
+ require Pleroma.Constants
+
+ alias Pleroma.Activity
+ alias Pleroma.Bookmark
+ alias Pleroma.Repo
+ alias Pleroma.User
+ alias Pleroma.Web.ActivityPub.ActivityPub
+ alias Pleroma.Web.ActivityPub.Transmogrifier
+ alias Pleroma.Web.ActivityPub.UserView
+ alias Pleroma.Workers.BackupWorker
+
+ schema "backups" do
+ field(:content_type, :string)
+ field(:file_name, :string)
+ field(:file_size, :integer, default: 0)
+ field(:processed, :boolean, default: false)
+
+ belongs_to(:user, User, type: FlakeId.Ecto.CompatType)
+
+ timestamps()
+ end
+
+ def create(user, admin_id \\ nil) do
+ with :ok <- validate_email_enabled(),
+ :ok <- validate_user_email(user),
+ :ok <- validate_limit(user, admin_id),
+ {:ok, backup} <- user |> new() |> Repo.insert() do
+ BackupWorker.process(backup, admin_id)
+ end
+ end
+
+ def new(user) do
+ rand_str = :crypto.strong_rand_bytes(32) |> Base.url_encode64(padding: false)
+ datetime = Calendar.NaiveDateTime.Format.iso8601_basic(NaiveDateTime.utc_now())
+ name = "archive-#{user.nickname}-#{datetime}-#{rand_str}.zip"
+
+ %__MODULE__{
+ user_id: user.id,
+ content_type: "application/zip",
+ file_name: name
+ }
+ end
+
+ def delete(backup) do
+ uploader = Pleroma.Config.get([Pleroma.Upload, :uploader])
+
+ with :ok <- uploader.delete_file(Path.join("backups", backup.file_name)) do
+ Repo.delete(backup)
+ end
+ end
+
+ defp validate_limit(_user, admin_id) when is_binary(admin_id), do: :ok
+
+ defp validate_limit(user, nil) do
+ case get_last(user.id) do
+ %__MODULE__{inserted_at: inserted_at} ->
+ days = Pleroma.Config.get([__MODULE__, :limit_days])
+ diff = Timex.diff(NaiveDateTime.utc_now(), inserted_at, :days)
+
+ if diff > days do
+ :ok
+ else
+ {:error,
+ dngettext(
+ "errors",
+ "Last export was less than a day ago",
+ "Last export was less than %{days} days ago",
+ days,
+ days: days
+ )}
+ end
+
+ nil ->
+ :ok
+ end
+ end
+
+ defp validate_email_enabled do
+ if Pleroma.Config.get([Pleroma.Emails.Mailer, :enabled]) do
+ :ok
+ else
+ {:error, dgettext("errors", "Backups require enabled email")}
+ end
+ end
+
+ defp validate_user_email(%User{email: nil}) do
+ {:error, dgettext("errors", "Email is required")}
+ end
+
+ defp validate_user_email(%User{email: email}) when is_binary(email), do: :ok
+
+ def get_last(user_id) do
+ __MODULE__
+ |> where(user_id: ^user_id)
+ |> order_by(desc: :id)
+ |> limit(1)
+ |> Repo.one()
+ end
+
+ def list(%User{id: user_id}) do
+ __MODULE__
+ |> where(user_id: ^user_id)
+ |> order_by(desc: :id)
+ |> Repo.all()
+ end
+
+ def remove_outdated(%__MODULE__{id: latest_id, user_id: user_id}) do
+ __MODULE__
+ |> where(user_id: ^user_id)
+ |> where([b], b.id != ^latest_id)
+ |> Repo.all()
+ |> Enum.each(&BackupWorker.delete/1)
+ end
+
+ def get(id), do: Repo.get(__MODULE__, id)
+
+ def process(%__MODULE__{} = backup) do
+ with {:ok, zip_file} <- export(backup),
+ {:ok, %{size: size}} <- File.stat(zip_file),
+ {:ok, _upload} <- upload(backup, zip_file) do
+ backup
+ |> cast(%{file_size: size, processed: true}, [:file_size, :processed])
+ |> Repo.update()
+ end
+ end
+
+ @files ['actor.json', 'outbox.json', 'likes.json', 'bookmarks.json']
+ def export(%__MODULE__{} = backup) do
+ backup = Repo.preload(backup, :user)
+ name = String.trim_trailing(backup.file_name, ".zip")
+ dir = dir(name)
+
+ with :ok <- File.mkdir(dir),
+ :ok <- actor(dir, backup.user),
+ :ok <- statuses(dir, backup.user),
+ :ok <- likes(dir, backup.user),
+ :ok <- bookmarks(dir, backup.user),
+ {:ok, zip_path} <- :zip.create(String.to_charlist(dir <> ".zip"), @files, cwd: dir),
+ {:ok, _} <- File.rm_rf(dir) do
+ {:ok, to_string(zip_path)}
+ end
+ end
+
+ def dir(name) do
+ dir = Pleroma.Config.get([__MODULE__, :dir]) || System.tmp_dir!()
+ Path.join(dir, name)
+ end
+
+ def upload(%__MODULE__{} = backup, zip_path) do
+ uploader = Pleroma.Config.get([Pleroma.Upload, :uploader])
+
+ upload = %Pleroma.Upload{
+ name: backup.file_name,
+ tempfile: zip_path,
+ content_type: backup.content_type,
+ path: Path.join("backups", backup.file_name)
+ }
+
+ with {:ok, _} <- Pleroma.Uploaders.Uploader.put_file(uploader, upload),
+ :ok <- File.rm(zip_path) do
+ {:ok, upload}
+ end
+ end
+
+ defp actor(dir, user) do
+ with {:ok, json} <-
+ UserView.render("user.json", %{user: user})
+ |> Map.merge(%{"likes" => "likes.json", "bookmarks" => "bookmarks.json"})
+ |> Jason.encode() do
+ File.write(Path.join(dir, "actor.json"), json)
+ end
+ end
+
+ defp write_header(file, name) do
+ IO.write(
+ file,
+ """
+ {
+ "@context": "https://www.w3.org/ns/activitystreams",
+ "id": "#{name}.json",
+ "type": "OrderedCollection",
+ "orderedItems": [
+
+ """
+ )
+ end
+
+ defp write(query, dir, name, fun) do
+ path = Path.join(dir, "#{name}.json")
+
+ with {:ok, file} <- File.open(path, [:write, :utf8]),
+ :ok <- write_header(file, name) do
+ total =
+ query
+ |> Pleroma.Repo.chunk_stream(100)
+ |> Enum.reduce(0, fn i, acc ->
+ with {:ok, data} <- fun.(i),
+ {:ok, str} <- Jason.encode(data),
+ :ok <- IO.write(file, str <> ",\n") do
+ acc + 1
+ else
+ _ -> acc
+ end
+ end)
+
+ with :ok <- :file.pwrite(file, {:eof, -2}, "\n],\n \"totalItems\": #{total}}") do
+ File.close(file)
+ end
+ end
+ end
+
+ defp bookmarks(dir, %{id: user_id} = _user) do
+ Bookmark
+ |> where(user_id: ^user_id)
+ |> join(:inner, [b], activity in assoc(b, :activity))
+ |> select([b, a], %{id: b.id, object: fragment("(?)->>'object'", a.data)})
+ |> write(dir, "bookmarks", fn a -> {:ok, a.object} end)
+ end
+
+ defp likes(dir, user) do
+ user.ap_id
+ |> Activity.Queries.by_actor()
+ |> Activity.Queries.by_type("Like")
+ |> select([like], %{id: like.id, object: fragment("(?)->>'object'", like.data)})
+ |> write(dir, "likes", fn a -> {:ok, a.object} end)
+ end
+
+ defp statuses(dir, user) do
+ opts =
+ %{}
+ |> Map.put(:type, ["Create", "Announce"])
+ |> Map.put(:actor_id, user.ap_id)
+
+ [
+ [Pleroma.Constants.as_public(), user.ap_id],
+ User.following(user),
+ Pleroma.List.memberships(user)
+ ]
+ |> Enum.concat()
+ |> ActivityPub.fetch_activities_query(opts)
+ |> write(dir, "outbox", fn a ->
+ with {:ok, activity} <- Transmogrifier.prepare_outgoing(a.data) do
+ {:ok, Map.delete(activity, "@context")}
+ end
+ end)
+ end
+end
diff --git a/lib/pleroma/user/import.ex b/lib/pleroma/user/import.ex
new file mode 100644
index 000000000..e458021c8
--- /dev/null
+++ b/lib/pleroma/user/import.ex
@@ -0,0 +1,85 @@
+# Pleroma: A lightweight social networking server
+# Copyright © 2017-2020 Pleroma Authors
+# SPDX-License-Identifier: AGPL-3.0-only
+
+defmodule Pleroma.User.Import do
+ use Ecto.Schema
+
+ alias Pleroma.User
+ alias Pleroma.Web.CommonAPI
+ alias Pleroma.Workers.BackgroundWorker
+
+ require Logger
+
+ @spec perform(atom(), User.t(), list()) :: :ok | list() | {:error, any()}
+ def perform(:mutes_import, %User{} = user, [_ | _] = identifiers) do
+ Enum.map(
+ identifiers,
+ fn identifier ->
+ with {:ok, %User{} = muted_user} <- User.get_or_fetch(identifier),
+ {:ok, _} <- User.mute(user, muted_user) do
+ muted_user
+ else
+ error -> handle_error(:mutes_import, identifier, error)
+ end
+ end
+ )
+ end
+
+ def perform(:blocks_import, %User{} = blocker, [_ | _] = identifiers) do
+ Enum.map(
+ identifiers,
+ fn identifier ->
+ with {:ok, %User{} = blocked} <- User.get_or_fetch(identifier),
+ {:ok, _block} <- CommonAPI.block(blocker, blocked) do
+ blocked
+ else
+ error -> handle_error(:blocks_import, identifier, error)
+ end
+ end
+ )
+ end
+
+ def perform(:follow_import, %User{} = follower, [_ | _] = identifiers) do
+ Enum.map(
+ identifiers,
+ fn identifier ->
+ with {:ok, %User{} = followed} <- User.get_or_fetch(identifier),
+ {:ok, follower} <- User.maybe_direct_follow(follower, followed),
+ {:ok, _, _, _} <- CommonAPI.follow(follower, followed) do
+ followed
+ else
+ error -> handle_error(:follow_import, identifier, error)
+ end
+ end
+ )
+ end
+
+ def perform(_, _, _), do: :ok
+
+ defp handle_error(op, user_id, error) do
+ Logger.debug("#{op} failed for #{user_id} with: #{inspect(error)}")
+ error
+ end
+
+ def blocks_import(%User{} = blocker, [_ | _] = identifiers) do
+ BackgroundWorker.enqueue(
+ "blocks_import",
+ %{"user_id" => blocker.id, "identifiers" => identifiers}
+ )
+ end
+
+ def follow_import(%User{} = follower, [_ | _] = identifiers) do
+ BackgroundWorker.enqueue(
+ "follow_import",
+ %{"user_id" => follower.id, "identifiers" => identifiers}
+ )
+ end
+
+ def mutes_import(%User{} = user, [_ | _] = identifiers) do
+ BackgroundWorker.enqueue(
+ "mutes_import",
+ %{"user_id" => user.id, "identifiers" => identifiers}
+ )
+ end
+end
diff --git a/lib/pleroma/user/query.ex b/lib/pleroma/user/query.ex
index d618432ff..7ef2a1455 100644
--- a/lib/pleroma/user/query.ex
+++ b/lib/pleroma/user/query.ex
@@ -43,10 +43,12 @@ defmodule Pleroma.User.Query do
active: boolean(),
deactivated: boolean(),
need_approval: boolean(),
+ unconfirmed: boolean(),
is_admin: boolean(),
is_moderator: boolean(),
super_users: boolean(),
invisible: boolean(),
+ internal: boolean(),
followers: User.t(),
friends: User.t(),
recipients_from_activity: [String.t()],
@@ -54,7 +56,8 @@ defmodule Pleroma.User.Query do
ap_id: [String.t()],
order_by: term(),
select: term(),
- limit: pos_integer()
+ limit: pos_integer(),
+ actor_types: [String.t()]
}
| map()
@@ -80,7 +83,9 @@ defp base_query do
end
defp prepare_query(query, criteria) do
- Enum.reduce(criteria, query, &compose_query/2)
+ criteria
+ |> Map.put_new(:internal, false)
+ |> Enum.reduce(query, &compose_query/2)
end
defp compose_query({key, value}, query)
@@ -107,12 +112,16 @@ defp compose_query({:tags, tags}, query) when is_list(tags) and length(tags) > 0
where(query, [u], fragment("? && ?", u.tags, ^tags))
end
- defp compose_query({:is_admin, _}, query) do
- where(query, [u], u.is_admin)
+ defp compose_query({:is_admin, bool}, query) do
+ where(query, [u], u.is_admin == ^bool)
end
- defp compose_query({:is_moderator, _}, query) do
- where(query, [u], u.is_moderator)
+ defp compose_query({:actor_types, actor_types}, query) when is_list(actor_types) do
+ where(query, [u], u.actor_type in ^actor_types)
+ end
+
+ defp compose_query({:is_moderator, bool}, query) do
+ where(query, [u], u.is_moderator == ^bool)
end
defp compose_query({:super_users, _}, query) do
@@ -129,14 +138,12 @@ defp compose_query({:external, _}, query), do: location_query(query, false)
defp compose_query({:active, _}, query) do
User.restrict_deactivated(query)
- |> where([u], not is_nil(u.nickname))
|> where([u], u.approval_pending == false)
end
defp compose_query({:legacy_active, _}, query) do
query
|> where([u], fragment("not (?->'deactivated' @> 'true')", u.info))
- |> where([u], not is_nil(u.nickname))
end
defp compose_query({:deactivated, false}, query) do
@@ -145,13 +152,20 @@ defp compose_query({:deactivated, false}, query) do
defp compose_query({:deactivated, true}, query) do
where(query, [u], u.deactivated == ^true)
- |> where([u], not is_nil(u.nickname))
+ end
+
+ defp compose_query({:confirmation_pending, bool}, query) do
+ where(query, [u], u.confirmation_pending == ^bool)
end
defp compose_query({:need_approval, _}, query) do
where(query, [u], u.approval_pending)
end
+ defp compose_query({:unconfirmed, _}, query) do
+ where(query, [u], u.confirmation_pending)
+ end
+
defp compose_query({:followers, %User{id: id}}, query) do
query
|> where([u], u.id != ^id)
@@ -199,10 +213,15 @@ defp compose_query({:limit, limit}, query) do
limit(query, ^limit)
end
+ defp compose_query({:internal, false}, query) do
+ query
+ |> where([u], not is_nil(u.nickname))
+ |> where([u], not like(u.nickname, "internal.%"))
+ end
+
defp compose_query(_unsupported_param, query), do: query
defp location_query(query, local) do
where(query, [u], u.local == ^local)
- |> where([u], not is_nil(u.nickname))
end
end
diff --git a/lib/pleroma/user/search.ex b/lib/pleroma/user/search.ex
index 7babd47ea..2dab67211 100644
--- a/lib/pleroma/user/search.ex
+++ b/lib/pleroma/user/search.ex
@@ -3,8 +3,10 @@
# SPDX-License-Identifier: AGPL-3.0-only
defmodule Pleroma.User.Search do
+ alias Pleroma.EctoType.ActivityPub.ObjectValidators.Uri, as: UriType
alias Pleroma.Pagination
alias Pleroma.User
+
import Ecto.Query
@limit 20
@@ -19,16 +21,47 @@ def search(query_string, opts \\ []) do
query_string = format_query(query_string)
- maybe_resolve(resolve, for_user, query_string)
+ # If this returns anything, it should bounce to the top
+ maybe_resolved = maybe_resolve(resolve, for_user, query_string)
+
+ top_user_ids =
+ []
+ |> maybe_add_resolved(maybe_resolved)
+ |> maybe_add_ap_id_match(query_string)
+ |> maybe_add_uri_match(query_string)
results =
query_string
- |> search_query(for_user, following)
+ |> search_query(for_user, following, top_user_ids)
|> Pagination.fetch_paginated(%{"offset" => offset, "limit" => result_limit}, :offset)
results
end
+ defp maybe_add_resolved(list, {:ok, %User{} = user}) do
+ [user.id | list]
+ end
+
+ defp maybe_add_resolved(list, _), do: list
+
+ defp maybe_add_ap_id_match(list, query) do
+ if user = User.get_cached_by_ap_id(query) do
+ [user.id | list]
+ else
+ list
+ end
+ end
+
+ defp maybe_add_uri_match(list, query) do
+ with {:ok, query} <- UriType.cast(query),
+ q = from(u in User, where: u.uri == ^query, select: u.id),
+ users = Pleroma.Repo.all(q) do
+ users ++ list
+ else
+ _ -> list
+ end
+ end
+
defp format_query(query_string) do
# Strip the beginning @ off if there is a query
query_string = String.trim_leading(query_string, "@")
@@ -47,21 +80,29 @@ defp format_query(query_string) do
end
end
- defp search_query(query_string, for_user, following) do
+ defp search_query(query_string, for_user, following, top_user_ids) do
for_user
|> base_query(following)
|> filter_blocked_user(for_user)
|> filter_invisible_users()
+ |> filter_discoverable_users()
|> filter_internal_users()
|> filter_blocked_domains(for_user)
|> fts_search(query_string)
+ |> select_top_users(top_user_ids)
|> trigram_rank(query_string)
- |> boost_search_rank(for_user)
+ |> boost_search_rank(for_user, top_user_ids)
|> subquery()
|> order_by(desc: :search_rank)
|> maybe_restrict_local(for_user)
end
+ defp select_top_users(query, top_user_ids) do
+ from(u in query,
+ or_where: u.id in ^top_user_ids
+ )
+ end
+
defp fts_search(query, query_string) do
query_string = to_tsquery(query_string)
@@ -122,6 +163,10 @@ defp filter_invisible_users(query) do
from(q in query, where: q.invisible == false)
end
+ defp filter_discoverable_users(query) do
+ from(q in query, where: q.is_discoverable == true)
+ end
+
defp filter_internal_users(query) do
from(q in query, where: q.actor_type != "Application")
end
@@ -175,7 +220,7 @@ defp restrict_local(q), do: where(q, [u], u.local == true)
defp local_domain, do: Pleroma.Config.get([Pleroma.Web.Endpoint, :url, :host])
- defp boost_search_rank(query, %User{} = for_user) do
+ defp boost_search_rank(query, %User{} = for_user, top_user_ids) do
friends_ids = User.get_friends_ids(for_user)
followers_ids = User.get_followers_ids(for_user)
@@ -187,6 +232,7 @@ defp boost_search_rank(query, %User{} = for_user) do
CASE WHEN (?) THEN (?) * 1.5
WHEN (?) THEN (?) * 1.3
WHEN (?) THEN (?) * 1.1
+ WHEN (?) THEN 9001
ELSE (?) END
""",
u.id in ^friends_ids and u.id in ^followers_ids,
@@ -195,11 +241,26 @@ defp boost_search_rank(query, %User{} = for_user) do
u.search_rank,
u.id in ^followers_ids,
u.search_rank,
+ u.id in ^top_user_ids,
u.search_rank
)
}
)
end
- defp boost_search_rank(query, _for_user), do: query
+ defp boost_search_rank(query, _for_user, top_user_ids) do
+ from(u in subquery(query),
+ select_merge: %{
+ search_rank:
+ fragment(
+ """
+ CASE WHEN (?) THEN 9001
+ ELSE (?) END
+ """,
+ u.id in ^top_user_ids,
+ u.search_rank
+ )
+ }
+ )
+ end
end
diff --git a/lib/pleroma/utils.ex b/lib/pleroma/utils.ex
index 21d1159be..e95766223 100644
--- a/lib/pleroma/utils.ex
+++ b/lib/pleroma/utils.ex
@@ -24,4 +24,24 @@ def compile_dir(dir) when is_binary(dir) do
def command_available?(command) do
match?({_output, 0}, System.cmd("sh", ["-c", "command -v #{command}"]))
end
+
+ @doc "creates the uniq temporary directory"
+ @spec tmp_dir(String.t()) :: {:ok, String.t()} | {:error, :file.posix()}
+ def tmp_dir(prefix \\ "") do
+ sub_dir =
+ [
+ prefix,
+ Timex.to_unix(Timex.now()),
+ :os.getpid(),
+ String.downcase(Integer.to_string(:rand.uniform(0x100000000), 36))
+ ]
+ |> Enum.join("-")
+
+ tmp_dir = Path.join(System.tmp_dir!(), sub_dir)
+
+ case File.mkdir(tmp_dir) do
+ :ok -> {:ok, tmp_dir}
+ error -> error
+ end
+ end
end
diff --git a/lib/pleroma/web/web.ex b/lib/pleroma/web.ex
similarity index 93%
rename from lib/pleroma/web/web.ex
rename to lib/pleroma/web.ex
index 4f9281851..6ed19d3dd 100644
--- a/lib/pleroma/web/web.ex
+++ b/lib/pleroma/web.ex
@@ -2,11 +2,6 @@
# Copyright © 2017-2020 Pleroma Authors
# SPDX-License-Identifier: AGPL-3.0-only
-defmodule Pleroma.Web.Plug do
- # Substitute for `call/2` which is defined with `use Pleroma.Web, :plug`
- @callback perform(Plug.Conn.t(), Plug.opts()) :: Plug.Conn.t()
-end
-
defmodule Pleroma.Web do
@moduledoc """
A module that keeps using definitions for controllers,
@@ -25,12 +20,12 @@ defmodule Pleroma.Web do
below.
"""
- alias Pleroma.Plugs.EnsureAuthenticatedPlug
- alias Pleroma.Plugs.EnsurePublicOrAuthenticatedPlug
- alias Pleroma.Plugs.ExpectAuthenticatedCheckPlug
- alias Pleroma.Plugs.ExpectPublicOrAuthenticatedCheckPlug
- alias Pleroma.Plugs.OAuthScopesPlug
- alias Pleroma.Plugs.PlugHelper
+ alias Pleroma.Web.Plugs.EnsureAuthenticatedPlug
+ alias Pleroma.Web.Plugs.EnsurePublicOrAuthenticatedPlug
+ alias Pleroma.Web.Plugs.ExpectAuthenticatedCheckPlug
+ alias Pleroma.Web.Plugs.ExpectPublicOrAuthenticatedCheckPlug
+ alias Pleroma.Web.Plugs.OAuthScopesPlug
+ alias Pleroma.Web.Plugs.PlugHelper
def controller do
quote do
@@ -177,7 +172,7 @@ def router do
def channel do
quote do
# credo:disable-for-next-line Credo.Check.Consistency.MultiAliasImportRequireUse
- use Phoenix.Channel
+ import Phoenix.Channel
import Pleroma.Web.Gettext
end
end
diff --git a/lib/pleroma/web/activity_pub/activity_pub.ex b/lib/pleroma/web/activity_pub/activity_pub.ex
index 333621413..d8f685d38 100644
--- a/lib/pleroma/web/activity_pub/activity_pub.ex
+++ b/lib/pleroma/web/activity_pub/activity_pub.ex
@@ -5,7 +5,6 @@
defmodule Pleroma.Web.ActivityPub.ActivityPub do
alias Pleroma.Activity
alias Pleroma.Activity.Ir.Topics
- alias Pleroma.ActivityExpiration
alias Pleroma.Config
alias Pleroma.Constants
alias Pleroma.Conversation
@@ -85,7 +84,7 @@ defp increase_replies_count_if_reply(%{
defp increase_replies_count_if_reply(_create_data), do: :noop
- @object_types ~w[ChatMessage Question Answer Audio Event]
+ @object_types ~w[ChatMessage Question Answer Audio Video Event Article]
@spec persist(map(), keyword()) :: {:ok, Activity.t() | Object.t()}
def persist(%{"type" => type} = object, meta) when type in @object_types do
with {:ok, object} <- Object.create(object) do
@@ -102,7 +101,9 @@ def persist(object, meta) do
local: local,
recipients: recipients,
actor: object["actor"]
- }) do
+ }),
+ # TODO: add tests for expired activities, when Note type will be supported in new pipeline
+ {:ok, _} <- maybe_create_activity_expiration(activity) do
{:ok, activity, meta}
end
end
@@ -111,23 +112,14 @@ def persist(object, meta) do
def insert(map, local \\ true, fake \\ false, bypass_actor_check \\ false) when is_map(map) do
with nil <- Activity.normalize(map),
map <- lazy_put_activity_defaults(map, fake),
- true <- bypass_actor_check || check_actor_is_active(map["actor"]),
- {_, true} <- {:remote_limit_error, check_remote_limit(map)},
+ {_, true} <- {:actor_check, bypass_actor_check || check_actor_is_active(map["actor"])},
+ {_, true} <- {:remote_limit_pass, check_remote_limit(map)},
{:ok, map} <- MRF.filter(map),
{recipients, _, _} = get_recipients(map),
{:fake, false, map, recipients} <- {:fake, fake, map, recipients},
{:containment, :ok} <- {:containment, Containment.contain_child(map)},
- {:ok, map, object} <- insert_full_object(map) do
- {:ok, activity} =
- %Activity{
- data: map,
- local: local,
- actor: map["actor"],
- recipients: recipients
- }
- |> Repo.insert()
- |> maybe_create_activity_expiration()
-
+ {:ok, map, object} <- insert_full_object(map),
+ {:ok, activity} <- insert_activity_with_expiration(map, local, recipients) do
# Splice in the child object if we have one.
activity = Maps.put_if_present(activity, :object, object)
@@ -138,6 +130,15 @@ def insert(map, local \\ true, fake \\ false, bypass_actor_check \\ false) when
%Activity{} = activity ->
{:ok, activity}
+ {:actor_check, _} ->
+ {:error, false}
+
+ {:containment, _} = error ->
+ error
+
+ {:error, _} = error ->
+ error
+
{:fake, true, map, recipients} ->
activity = %Activity{
data: map,
@@ -150,8 +151,24 @@ def insert(map, local \\ true, fake \\ false, bypass_actor_check \\ false) when
Pleroma.Web.RichMedia.Helpers.fetch_data_for_activity(activity)
{:ok, activity}
- error ->
- {:error, error}
+ {:remote_limit_pass, _} ->
+ {:error, :remote_limit}
+
+ {:reject, _} = e ->
+ {:error, e}
+ end
+ end
+
+ defp insert_activity_with_expiration(data, local, recipients) do
+ struct = %Activity{
+ data: data,
+ local: local,
+ actor: data["actor"],
+ recipients: recipients
+ }
+
+ with {:ok, activity} <- Repo.insert(struct) do
+ maybe_create_activity_expiration(activity)
end
end
@@ -164,13 +181,19 @@ def notify_and_stream(activity) do
stream_out_participations(participations)
end
- defp maybe_create_activity_expiration({:ok, %{data: %{"expires_at" => expires_at}} = activity}) do
- with {:ok, _} <- ActivityExpiration.create(activity, expires_at) do
+ defp maybe_create_activity_expiration(
+ %{data: %{"expires_at" => %DateTime{} = expires_at}} = activity
+ ) do
+ with {:ok, _job} <-
+ Pleroma.Workers.PurgeExpiredActivity.enqueue(%{
+ activity_id: activity.id,
+ expires_at: expires_at
+ }) do
{:ok, activity}
end
end
- defp maybe_create_activity_expiration(result), do: result
+ defp maybe_create_activity_expiration(activity), do: {:ok, activity}
defp create_or_bump_conversation(activity, actor) do
with {:ok, conversation} <- Conversation.create_or_bump_for(activity),
@@ -744,7 +767,7 @@ defp restrict_replies(query, %{exclude_replies: true}) do
end
defp restrict_replies(query, %{
- reply_filtering_user: user,
+ reply_filtering_user: %User{} = user,
reply_visibility: "self"
}) do
from(
@@ -760,14 +783,24 @@ defp restrict_replies(query, %{
end
defp restrict_replies(query, %{
- reply_filtering_user: user,
+ reply_filtering_user: %User{} = user,
reply_visibility: "following"
}) do
from(
[activity, object] in query,
where:
fragment(
- "?->>'inReplyTo' is null OR ? && array_remove(?, ?) OR ? = ?",
+ """
+ ?->>'type' != 'Create' -- This isn't a Create
+ OR ?->>'inReplyTo' is null -- this isn't a reply
+ OR ? && array_remove(?, ?) -- The recipient is us or one of our friends,
+ -- unless they are the author (because authors
+ -- are also part of the recipients). This leads
+ -- to a bug that self-replies by friends won't
+ -- show up.
+ OR ? = ? -- The actor is us
+ """,
+ activity.data,
object.data,
^[user.ap_id | User.get_cached_user_friends_ap_ids(user)],
activity.recipients,
@@ -794,7 +827,14 @@ defp restrict_muted(query, %{muting_user: %User{} = user} = opts) do
query =
from([activity] in query,
where: fragment("not (? = ANY(?))", activity.actor, ^mutes),
- where: fragment("not (?->'to' \\?| ?)", activity.data, ^mutes)
+ where:
+ fragment(
+ "not (?->'to' \\?| ?) or ? = ?",
+ activity.data,
+ ^mutes,
+ activity.actor,
+ ^user.ap_id
+ )
)
unless opts[:skip_preload] do
@@ -818,7 +858,14 @@ defp restrict_blocked(query, %{blocking_user: %User{} = user} = opts) do
from(
[activity, object: o] in query,
where: fragment("not (? = ANY(?))", activity.actor, ^blocked_ap_ids),
- where: fragment("not (? && ?)", activity.recipients, ^blocked_ap_ids),
+ where:
+ fragment(
+ "((not (? && ?)) or ? = ?)",
+ activity.recipients,
+ ^blocked_ap_ids,
+ activity.actor,
+ ^user.ap_id
+ ),
where:
fragment(
"recipients_contain_blocked_domains(?, ?) = false",
@@ -890,16 +937,11 @@ defp restrict_muted_reblogs(query, %{muting_user: %User{} = user} = opts) do
defp restrict_muted_reblogs(query, _), do: query
- defp restrict_instance(query, %{instance: instance}) do
- users =
- from(
- u in User,
- select: u.ap_id,
- where: fragment("? LIKE ?", u.nickname, ^"%@#{instance}")
- )
- |> Repo.all()
-
- from(activity in query, where: activity.actor in ^users)
+ defp restrict_instance(query, %{instance: instance}) when is_binary(instance) do
+ from(
+ activity in query,
+ where: fragment("split_part(actor::text, '/'::text, 3) = ?", ^instance)
+ )
end
defp restrict_instance(query, _), do: query
@@ -1188,11 +1230,11 @@ defp object_to_user_data(data) do
{String.trim(name, ":"), url}
end)
- locked = data["manuallyApprovesFollowers"] || false
+ is_locked = data["manuallyApprovesFollowers"] || false
capabilities = data["capabilities"] || %{}
accepts_chat_messages = capabilities["acceptsChatMessages"]
data = Transmogrifier.maybe_fix_user_object(data)
- discoverable = data["discoverable"] || false
+ is_discoverable = data["discoverable"] || false
invisible = data["invisible"] || false
actor_type = data["type"] || "Person"
@@ -1217,8 +1259,8 @@ defp object_to_user_data(data) do
banner: banner,
fields: fields,
emoji: emojis,
- locked: locked,
- discoverable: discoverable,
+ is_locked: is_locked,
+ is_discoverable: is_discoverable,
invisible: invisible,
avatar: avatar,
name: data["name"],
@@ -1247,10 +1289,12 @@ defp object_to_user_data(data) do
def fetch_follow_information_for_user(user) do
with {:ok, following_data} <-
- Fetcher.fetch_and_contain_remote_object_from_id(user.following_address),
+ Fetcher.fetch_and_contain_remote_object_from_id(user.following_address,
+ force_http: true
+ ),
{:ok, hide_follows} <- collection_private(following_data),
{:ok, followers_data} <-
- Fetcher.fetch_and_contain_remote_object_from_id(user.follower_address),
+ Fetcher.fetch_and_contain_remote_object_from_id(user.follower_address, force_http: true),
{:ok, hide_followers} <- collection_private(followers_data) do
{:ok,
%{
@@ -1324,11 +1368,12 @@ def user_data_from_user_object(data) do
end
end
- def fetch_and_prepare_user_from_ap_id(ap_id) do
- with {:ok, data} <- Fetcher.fetch_and_contain_remote_object_from_id(ap_id),
+ def fetch_and_prepare_user_from_ap_id(ap_id, opts \\ []) do
+ with {:ok, data} <- Fetcher.fetch_and_contain_remote_object_from_id(ap_id, opts),
{:ok, data} <- user_data_from_user_object(data) do
{:ok, maybe_update_follow_information(data)}
else
+ # If this has been deleted, only log a debug and not an error
{:error, "Object has been deleted" = e} ->
Logger.debug("Could not decode user at fetch #{ap_id}, #{inspect(e)}")
{:error, e}
@@ -1367,13 +1412,13 @@ def maybe_handle_clashing_nickname(data) do
end
end
- def make_user_from_ap_id(ap_id) do
+ def make_user_from_ap_id(ap_id, opts \\ []) do
user = User.get_cached_by_ap_id(ap_id)
if user && !User.ap_enabled?(user) do
Transmogrifier.upgrade_user_from_ap_id(ap_id)
else
- with {:ok, data} <- fetch_and_prepare_user_from_ap_id(ap_id) do
+ with {:ok, data} <- fetch_and_prepare_user_from_ap_id(ap_id, opts) do
if user do
user
|> User.remote_user_changeset(data)
diff --git a/lib/pleroma/web/activity_pub/activity_pub_controller.ex b/lib/pleroma/web/activity_pub/activity_pub_controller.ex
index 732c44271..31df80adb 100644
--- a/lib/pleroma/web/activity_pub/activity_pub_controller.ex
+++ b/lib/pleroma/web/activity_pub/activity_pub_controller.ex
@@ -9,7 +9,6 @@ defmodule Pleroma.Web.ActivityPub.ActivityPubController do
alias Pleroma.Delivery
alias Pleroma.Object
alias Pleroma.Object.Fetcher
- alias Pleroma.Plugs.EnsureAuthenticatedPlug
alias Pleroma.User
alias Pleroma.Web.ActivityPub.ActivityPub
alias Pleroma.Web.ActivityPub.Builder
@@ -23,8 +22,9 @@ defmodule Pleroma.Web.ActivityPub.ActivityPubController do
alias Pleroma.Web.ActivityPub.Visibility
alias Pleroma.Web.ControllerHelper
alias Pleroma.Web.Endpoint
- alias Pleroma.Web.FederatingPlug
alias Pleroma.Web.Federator
+ alias Pleroma.Web.Plugs.EnsureAuthenticatedPlug
+ alias Pleroma.Web.Plugs.FederatingPlug
require Logger
@@ -45,8 +45,10 @@ defmodule Pleroma.Web.ActivityPub.ActivityPubController do
when action in [:read_inbox, :update_outbox, :whoami, :upload_media]
)
+ plug(Majic.Plug, [pool: Pleroma.MajicPool] when action in [:upload_media])
+
plug(
- Pleroma.Plugs.Cache,
+ Pleroma.Web.Plugs.Cache,
[query_params: false, tracking_fun: &__MODULE__.track_object_fetch/2]
when action in [:activity, :object]
)
@@ -412,7 +414,7 @@ defp handle_user_activity(
object =
object
|> Map.merge(Map.take(params, ["to", "cc"]))
- |> Map.put("attributedTo", user.ap_id())
+ |> Map.put("attributedTo", user.ap_id)
|> Transmogrifier.fix_object()
ActivityPub.create(%{
@@ -456,7 +458,7 @@ def update_outbox(
%{assigns: %{user: %User{nickname: nickname} = user}} = conn,
%{"nickname" => nickname} = params
) do
- actor = user.ap_id()
+ actor = user.ap_id
params =
params
@@ -523,19 +525,6 @@ defp ensure_user_keys_present_and_maybe_refresh_for_user(user, for_user) do
{new_user, for_user}
end
- @doc """
- Endpoint based on
-
- Parameters:
- - (required) `file`: data of the media
- - (optionnal) `description`: description of the media, intended for accessibility
-
- Response:
- - HTTP Code: 201 Created
- - HTTP Body: ActivityPub object to be inserted into another's `attachment` field
-
- Note: Will not point to a URL with a `Location` header because no standalone Activity has been created.
- """
def upload_media(%{assigns: %{user: %User{} = user}} = conn, %{"file" => file} = data) do
with {:ok, object} <-
ActivityPub.upload(
diff --git a/lib/pleroma/web/activity_pub/builder.ex b/lib/pleroma/web/activity_pub/builder.ex
index 9a7b7d9de..298aff6b7 100644
--- a/lib/pleroma/web/activity_pub/builder.ex
+++ b/lib/pleroma/web/activity_pub/builder.ex
@@ -1,3 +1,7 @@
+# Pleroma: A lightweight social networking server
+# Copyright © 2017-2020 Pleroma Authors
+# SPDX-License-Identifier: AGPL-3.0-only
+
defmodule Pleroma.Web.ActivityPub.Builder do
@moduledoc """
This module builds the objects. Meant to be used for creating local objects.
diff --git a/lib/pleroma/web/activity_pub/mrf.ex b/lib/pleroma/web/activity_pub/mrf.ex
index 206d6af52..5e5361082 100644
--- a/lib/pleroma/web/activity_pub/mrf.ex
+++ b/lib/pleroma/web/activity_pub/mrf.ex
@@ -5,16 +5,34 @@
defmodule Pleroma.Web.ActivityPub.MRF do
@callback filter(Map.t()) :: {:ok | :reject, Map.t()}
- def filter(policies, %{} = object) do
+ def filter(policies, %{} = message) do
policies
- |> Enum.reduce({:ok, object}, fn
- policy, {:ok, object} -> policy.filter(object)
+ |> Enum.reduce({:ok, message}, fn
+ policy, {:ok, message} -> policy.filter(message)
_, error -> error
end)
end
def filter(%{} = object), do: get_policies() |> filter(object)
+ def pipeline_filter(%{} = message, meta) do
+ object = meta[:object_data]
+ ap_id = message["object"]
+
+ if object && ap_id do
+ with {:ok, message} <- filter(Map.put(message, "object", object)) do
+ meta = Keyword.put(meta, :object_data, message["object"])
+ {:ok, Map.put(message, "object", ap_id), meta}
+ else
+ {err, message} -> {err, message, meta}
+ end
+ else
+ {err, message} = filter(message)
+
+ {err, message, meta}
+ end
+ end
+
def get_policies do
Pleroma.Config.get([:mrf, :policies], []) |> get_policies()
end
diff --git a/lib/pleroma/web/activity_pub/mrf/activity_expiration_policy.ex b/lib/pleroma/web/activity_pub/mrf/activity_expiration_policy.ex
index 7b4c78e0f..bee47b4ed 100644
--- a/lib/pleroma/web/activity_pub/mrf/activity_expiration_policy.ex
+++ b/lib/pleroma/web/activity_pub/mrf/activity_expiration_policy.ex
@@ -31,10 +31,10 @@ defp note?(activity) do
defp maybe_add_expiration(activity) do
days = Pleroma.Config.get([:mrf_activity_expiration, :days], 365)
- expires_at = NaiveDateTime.utc_now() |> Timex.shift(days: days)
+ expires_at = DateTime.utc_now() |> Timex.shift(days: days)
with %{"expires_at" => existing_expires_at} <- activity,
- :lt <- NaiveDateTime.compare(existing_expires_at, expires_at) do
+ :lt <- DateTime.compare(existing_expires_at, expires_at) do
activity
else
_ -> Map.put(activity, "expires_at", expires_at)
diff --git a/lib/pleroma/web/activity_pub/mrf/keyword_policy.ex b/lib/pleroma/web/activity_pub/mrf/keyword_policy.ex
index 15e09dcf0..db66cfa3e 100644
--- a/lib/pleroma/web/activity_pub/mrf/keyword_policy.ex
+++ b/lib/pleroma/web/activity_pub/mrf/keyword_policy.ex
@@ -20,9 +20,17 @@ defp string_matches?(string, pattern) do
String.match?(string, pattern)
end
- defp check_reject(%{"object" => %{"content" => content, "summary" => summary}} = message) do
+ defp object_payload(%{} = object) do
+ [object["content"], object["summary"], object["name"]]
+ |> Enum.filter(& &1)
+ |> Enum.join("\n")
+ end
+
+ defp check_reject(%{"object" => %{} = object} = message) do
+ payload = object_payload(object)
+
if Enum.any?(Pleroma.Config.get([:mrf_keyword, :reject]), fn pattern ->
- string_matches?(content, pattern) or string_matches?(summary, pattern)
+ string_matches?(payload, pattern)
end) do
{:reject, "[KeywordPolicy] Matches with rejected keyword"}
else
@@ -30,12 +38,12 @@ defp check_reject(%{"object" => %{"content" => content, "summary" => summary}} =
end
end
- defp check_ftl_removal(
- %{"to" => to, "object" => %{"content" => content, "summary" => summary}} = message
- ) do
+ defp check_ftl_removal(%{"to" => to, "object" => %{} = object} = message) do
+ payload = object_payload(object)
+
if Pleroma.Constants.as_public() in to and
Enum.any?(Pleroma.Config.get([:mrf_keyword, :federated_timeline_removal]), fn pattern ->
- string_matches?(content, pattern) or string_matches?(summary, pattern)
+ string_matches?(payload, pattern)
end) do
to = List.delete(to, Pleroma.Constants.as_public())
cc = [Pleroma.Constants.as_public() | message["cc"] || []]
@@ -51,35 +59,24 @@ defp check_ftl_removal(
end
end
- defp check_replace(%{"object" => %{"content" => content, "summary" => summary}} = message) do
- content =
- if is_binary(content) do
- content
- else
- ""
- end
+ defp check_replace(%{"object" => %{} = object} = message) do
+ object =
+ ["content", "name", "summary"]
+ |> Enum.filter(fn field -> Map.has_key?(object, field) && object[field] end)
+ |> Enum.reduce(object, fn field, object ->
+ data =
+ Enum.reduce(
+ Pleroma.Config.get([:mrf_keyword, :replace]),
+ object[field],
+ fn {pat, repl}, acc -> String.replace(acc, pat, repl) end
+ )
- summary =
- if is_binary(summary) do
- summary
- else
- ""
- end
+ Map.put(object, field, data)
+ end)
- {content, summary} =
- Enum.reduce(
- Pleroma.Config.get([:mrf_keyword, :replace]),
- {content, summary},
- fn {pattern, replacement}, {content_acc, summary_acc} ->
- {String.replace(content_acc, pattern, replacement),
- String.replace(summary_acc, pattern, replacement)}
- end
- )
+ message = Map.put(message, "object", object)
- {:ok,
- message
- |> put_in(["object", "content"], content)
- |> put_in(["object", "summary"], summary)}
+ {:ok, message}
end
@impl true
diff --git a/lib/pleroma/web/activity_pub/mrf/media_proxy_warming_policy.ex b/lib/pleroma/web/activity_pub/mrf/media_proxy_warming_policy.ex
index 98d595469..0fb05d3c4 100644
--- a/lib/pleroma/web/activity_pub/mrf/media_proxy_warming_policy.ex
+++ b/lib/pleroma/web/activity_pub/mrf/media_proxy_warming_policy.ex
@@ -12,17 +12,21 @@ defmodule Pleroma.Web.ActivityPub.MRF.MediaProxyWarmingPolicy do
require Logger
- @options [
+ @adapter_options [
pool: :media,
recv_timeout: 10_000
]
def perform(:prefetch, url) do
- Logger.debug("Prefetching #{inspect(url)}")
+ # Fetching only proxiable resources
+ if MediaProxy.enabled?() and MediaProxy.url_proxiable?(url) do
+ # If preview proxy is enabled, it'll also hit media proxy (so we're caching both requests)
+ prefetch_url = MediaProxy.preview_url(url)
- url
- |> MediaProxy.url()
- |> HTTP.get([], @options)
+ Logger.debug("Prefetching #{inspect(url)} as #{inspect(prefetch_url)}")
+
+ HTTP.get(prefetch_url, [], @adapter_options)
+ end
end
def perform(:preload, %{"object" => %{"attachment" => attachments}} = _message) do
diff --git a/lib/pleroma/web/activity_pub/mrf/simple_policy.ex b/lib/pleroma/web/activity_pub/mrf/simple_policy.ex
index bb193475a..161177727 100644
--- a/lib/pleroma/web/activity_pub/mrf/simple_policy.ex
+++ b/lib/pleroma/web/activity_pub/mrf/simple_policy.ex
@@ -66,7 +66,8 @@ defp check_media_nsfw(
"type" => "Create",
"object" => child_object
} = object
- ) do
+ )
+ when is_map(child_object) do
media_nsfw =
Config.get([:mrf_simple, :media_nsfw])
|> MRF.subdomains_regex()
diff --git a/lib/pleroma/web/activity_pub/mrf/subchain_policy.ex b/lib/pleroma/web/activity_pub/mrf/subchain_policy.ex
index c9f20571f..048052da6 100644
--- a/lib/pleroma/web/activity_pub/mrf/subchain_policy.ex
+++ b/lib/pleroma/web/activity_pub/mrf/subchain_policy.ex
@@ -28,8 +28,7 @@ def filter(%{"actor" => actor} = message) do
}"
)
- subchain
- |> MRF.filter(message)
+ MRF.filter(subchain, message)
else
_e -> {:ok, message}
end
diff --git a/lib/pleroma/web/activity_pub/object_validator.ex b/lib/pleroma/web/activity_pub/object_validator.ex
index b77c06395..bd0a2a8dc 100644
--- a/lib/pleroma/web/activity_pub/object_validator.ex
+++ b/lib/pleroma/web/activity_pub/object_validator.ex
@@ -12,11 +12,13 @@ defmodule Pleroma.Web.ActivityPub.ObjectValidator do
alias Pleroma.Activity
alias Pleroma.EctoType.ActivityPub.ObjectValidators
alias Pleroma.Object
+ alias Pleroma.Object.Containment
alias Pleroma.User
alias Pleroma.Web.ActivityPub.ObjectValidators.AcceptRejectValidator
alias Pleroma.Web.ActivityPub.ObjectValidators.AnnounceValidator
alias Pleroma.Web.ActivityPub.ObjectValidators.AnswerValidator
- alias Pleroma.Web.ActivityPub.ObjectValidators.AudioValidator
+ alias Pleroma.Web.ActivityPub.ObjectValidators.ArticleNoteValidator
+ alias Pleroma.Web.ActivityPub.ObjectValidators.AudioVideoValidator
alias Pleroma.Web.ActivityPub.ObjectValidators.BlockValidator
alias Pleroma.Web.ActivityPub.ObjectValidators.ChatMessageValidator
alias Pleroma.Web.ActivityPub.ObjectValidators.CreateChatMessageValidator
@@ -149,10 +151,20 @@ def validate(%{"type" => "Question"} = object, meta) do
end
end
- def validate(%{"type" => "Audio"} = object, meta) do
+ def validate(%{"type" => type} = object, meta) when type in ~w[Audio Video] do
with {:ok, object} <-
object
- |> AudioValidator.cast_and_validate()
+ |> AudioVideoValidator.cast_and_validate()
+ |> Ecto.Changeset.apply_action(:insert) do
+ object = stringify_keys(object)
+ {:ok, object, meta}
+ end
+ end
+
+ def validate(%{"type" => "Article"} = object, meta) do
+ with {:ok, object} <-
+ object
+ |> ArticleNoteValidator.cast_and_validate()
|> Ecto.Changeset.apply_action(:insert) do
object = stringify_keys(object)
{:ok, object, meta}
@@ -198,7 +210,7 @@ def validate(
%{"type" => "Create", "object" => %{"type" => objtype} = object} = create_activity,
meta
)
- when objtype in ~w[Question Answer Audio Event] do
+ when objtype in ~w[Question Answer Audio Video Event Article] do
with {:ok, object_data} <- cast_and_apply(object),
meta = Keyword.put(meta, :object_data, object_data |> stringify_keys),
{:ok, create_activity} <-
@@ -232,14 +244,18 @@ def cast_and_apply(%{"type" => "Answer"} = object) do
AnswerValidator.cast_and_apply(object)
end
- def cast_and_apply(%{"type" => "Audio"} = object) do
- AudioValidator.cast_and_apply(object)
+ def cast_and_apply(%{"type" => type} = object) when type in ~w[Audio Video] do
+ AudioVideoValidator.cast_and_apply(object)
end
def cast_and_apply(%{"type" => "Event"} = object) do
EventValidator.cast_and_apply(object)
end
+ def cast_and_apply(%{"type" => "Article"} = object) do
+ ArticleNoteValidator.cast_and_apply(object)
+ end
+
def cast_and_apply(o), do: {:error, {:validator_not_set, o}}
# is_struct/1 isn't present in Elixir 1.8.x
@@ -262,7 +278,8 @@ def stringify_keys(object) when is_list(object) do
def stringify_keys(object), do: object
def fetch_actor(object) do
- with {:ok, actor} <- ObjectValidators.ObjectID.cast(object["actor"]) do
+ with actor <- Containment.get_actor(object),
+ {:ok, actor} <- ObjectValidators.ObjectID.cast(actor) do
User.get_or_fetch_by_ap_id(actor)
end
end
diff --git a/lib/pleroma/web/activity_pub/object_validators/audio_validator.ex b/lib/pleroma/web/activity_pub/object_validators/article_note_validator.ex
similarity index 80%
rename from lib/pleroma/web/activity_pub/object_validators/audio_validator.ex
rename to lib/pleroma/web/activity_pub/object_validators/article_note_validator.ex
index 1a97c504a..5b7dad517 100644
--- a/lib/pleroma/web/activity_pub/object_validators/audio_validator.ex
+++ b/lib/pleroma/web/activity_pub/object_validators/article_note_validator.ex
@@ -2,7 +2,7 @@
# Copyright © 2017-2020 Pleroma Authors
# SPDX-License-Identifier: AGPL-3.0-only
-defmodule Pleroma.Web.ActivityPub.ObjectValidators.AudioValidator do
+defmodule Pleroma.Web.ActivityPub.ObjectValidators.ArticleNoteValidator do
use Ecto.Schema
alias Pleroma.EctoType.ActivityPub.ObjectValidators
@@ -25,14 +25,19 @@ defmodule Pleroma.Web.ActivityPub.ObjectValidators.AudioValidator do
# TODO: Write type
field(:tag, {:array, :map}, default: [])
field(:type, :string)
+
+ field(:name, :string)
+ field(:summary, :string)
field(:content, :string)
+
field(:context, :string)
+ # short identifier for PleromaFE to group statuses by context
+ field(:context_id, :integer)
# TODO: Remove actor on objects
field(:actor, ObjectValidators.ObjectID)
field(:attributedTo, ObjectValidators.ObjectID)
- field(:summary, :string)
field(:published, ObjectValidators.DateTime)
field(:emoji, ObjectValidators.Emoji, default: %{})
field(:sensitive, :boolean, default: false)
@@ -40,13 +45,11 @@ defmodule Pleroma.Web.ActivityPub.ObjectValidators.AudioValidator do
field(:replies_count, :integer, default: 0)
field(:like_count, :integer, default: 0)
field(:announcement_count, :integer, default: 0)
- field(:inReplyTo, :string)
+ field(:inReplyTo, ObjectValidators.ObjectID)
field(:url, ObjectValidators.Uri)
- # short identifier for PleromaFE to group statuses by context
- field(:context_id, :integer)
- field(:likes, {:array, :string}, default: [])
- field(:announcements, {:array, :string}, default: [])
+ field(:likes, {:array, ObjectValidators.ObjectID}, default: [])
+ field(:announcements, {:array, ObjectValidators.ObjectID}, default: [])
end
def cast_and_apply(data) do
@@ -62,19 +65,14 @@ def cast_and_validate(data) do
end
def cast_data(data) do
+ data = fix(data)
+
%__MODULE__{}
|> changeset(data)
end
- defp fix_url(%{"url" => url} = data) when is_list(url) do
- attachment =
- Enum.find(url, fn x -> is_map(x) and String.starts_with?(x["mimeType"], "audio/") end)
-
- link_element = Enum.find(url, fn x -> is_map(x) and x["mimeType"] == "text/html" end)
-
- data
- |> Map.put("attachment", [attachment])
- |> Map.put("url", link_element["href"])
+ defp fix_url(%{"url" => url} = data) when is_map(url) do
+ Map.put(data, "url", url["href"])
end
defp fix_url(data), do: data
@@ -83,8 +81,9 @@ defp fix(data) do
data
|> CommonFixes.fix_defaults()
|> CommonFixes.fix_attribution()
- |> Transmogrifier.fix_emoji()
+ |> CommonFixes.fix_actor()
|> fix_url()
+ |> Transmogrifier.fix_emoji()
end
def changeset(struct, data) do
@@ -97,8 +96,8 @@ def changeset(struct, data) do
def validate_data(data_cng) do
data_cng
- |> validate_inclusion(:type, ["Audio"])
- |> validate_required([:id, :actor, :attributedTo, :type, :context, :attachment])
+ |> validate_inclusion(:type, ["Article", "Note"])
+ |> validate_required([:id, :actor, :attributedTo, :type, :context, :context_id])
|> CommonValidations.validate_any_presence([:cc, :to])
|> CommonValidations.validate_fields_match([:actor, :attributedTo])
|> CommonValidations.validate_actor_presence()
diff --git a/lib/pleroma/web/activity_pub/object_validators/attachment_validator.ex b/lib/pleroma/web/activity_pub/object_validators/attachment_validator.ex
index c8b148280..df102a134 100644
--- a/lib/pleroma/web/activity_pub/object_validators/attachment_validator.ex
+++ b/lib/pleroma/web/activity_pub/object_validators/attachment_validator.ex
@@ -5,6 +5,7 @@
defmodule Pleroma.Web.ActivityPub.ObjectValidators.AttachmentValidator do
use Ecto.Schema
+ alias Pleroma.EctoType.ActivityPub.ObjectValidators
alias Pleroma.Web.ActivityPub.ObjectValidators.UrlObjectValidator
import Ecto.Changeset
@@ -15,7 +16,11 @@ defmodule Pleroma.Web.ActivityPub.ObjectValidators.AttachmentValidator do
field(:mediaType, :string, default: "application/octet-stream")
field(:name, :string)
- embeds_many(:url, UrlObjectValidator)
+ embeds_many :url, UrlObjectValidator, primary_key: false do
+ field(:type, :string)
+ field(:href, ObjectValidators.Uri)
+ field(:mediaType, :string, default: "application/octet-stream")
+ end
end
def cast_and_validate(data) do
@@ -37,7 +42,18 @@ def changeset(struct, data) do
struct
|> cast(data, [:type, :mediaType, :name])
- |> cast_embed(:url, required: true)
+ |> cast_embed(:url, with: &url_changeset/2)
+ |> validate_inclusion(:type, ~w[Link Document Audio Image Video])
+ |> validate_required([:type, :mediaType, :url])
+ end
+
+ def url_changeset(struct, data) do
+ data = fix_media_type(data)
+
+ struct
+ |> cast(data, [:type, :href, :mediaType])
+ |> validate_inclusion(:type, ["Link"])
+ |> validate_required([:type, :href, :mediaType])
end
def fix_media_type(data) do
@@ -75,6 +91,7 @@ defp fix_url(data) do
def validate_data(cng) do
cng
+ |> validate_inclusion(:type, ~w[Document Audio Image Video])
|> validate_required([:mediaType, :url, :type])
end
end
diff --git a/lib/pleroma/web/activity_pub/object_validators/audio_video_validator.ex b/lib/pleroma/web/activity_pub/object_validators/audio_video_validator.ex
new file mode 100644
index 000000000..16973e5db
--- /dev/null
+++ b/lib/pleroma/web/activity_pub/object_validators/audio_video_validator.ex
@@ -0,0 +1,134 @@
+# Pleroma: A lightweight social networking server
+# Copyright © 2017-2020 Pleroma Authors
+# SPDX-License-Identifier: AGPL-3.0-only
+
+defmodule Pleroma.Web.ActivityPub.ObjectValidators.AudioVideoValidator do
+ use Ecto.Schema
+
+ alias Pleroma.EarmarkRenderer
+ alias Pleroma.EctoType.ActivityPub.ObjectValidators
+ alias Pleroma.Web.ActivityPub.ObjectValidators.AttachmentValidator
+ alias Pleroma.Web.ActivityPub.ObjectValidators.CommonFixes
+ alias Pleroma.Web.ActivityPub.ObjectValidators.CommonValidations
+ alias Pleroma.Web.ActivityPub.Transmogrifier
+
+ import Ecto.Changeset
+
+ @primary_key false
+ @derive Jason.Encoder
+
+ embedded_schema do
+ field(:id, ObjectValidators.ObjectID, primary_key: true)
+ field(:to, ObjectValidators.Recipients, default: [])
+ field(:cc, ObjectValidators.Recipients, default: [])
+ field(:bto, ObjectValidators.Recipients, default: [])
+ field(:bcc, ObjectValidators.Recipients, default: [])
+ # TODO: Write type
+ field(:tag, {:array, :map}, default: [])
+ field(:type, :string)
+
+ field(:name, :string)
+ field(:summary, :string)
+ field(:content, :string)
+
+ field(:context, :string)
+ # short identifier for PleromaFE to group statuses by context
+ field(:context_id, :integer)
+
+ # TODO: Remove actor on objects
+ field(:actor, ObjectValidators.ObjectID)
+
+ field(:attributedTo, ObjectValidators.ObjectID)
+ field(:published, ObjectValidators.DateTime)
+ field(:emoji, ObjectValidators.Emoji, default: %{})
+ field(:sensitive, :boolean, default: false)
+ embeds_many(:attachment, AttachmentValidator)
+ field(:replies_count, :integer, default: 0)
+ field(:like_count, :integer, default: 0)
+ field(:announcement_count, :integer, default: 0)
+ field(:inReplyTo, ObjectValidators.ObjectID)
+ field(:url, ObjectValidators.Uri)
+
+ field(:likes, {:array, ObjectValidators.ObjectID}, default: [])
+ field(:announcements, {:array, ObjectValidators.ObjectID}, default: [])
+ end
+
+ def cast_and_apply(data) do
+ data
+ |> cast_data
+ |> apply_action(:insert)
+ end
+
+ def cast_and_validate(data) do
+ data
+ |> cast_data()
+ |> validate_data()
+ end
+
+ def cast_data(data) do
+ %__MODULE__{}
+ |> changeset(data)
+ end
+
+ defp fix_url(%{"url" => url} = data) when is_list(url) do
+ attachment =
+ Enum.find(url, fn x ->
+ mime_type = x["mimeType"] || x["mediaType"] || ""
+
+ is_map(x) and String.starts_with?(mime_type, ["video/", "audio/"])
+ end)
+
+ link_element =
+ Enum.find(url, fn x ->
+ mime_type = x["mimeType"] || x["mediaType"] || ""
+
+ is_map(x) and mime_type == "text/html"
+ end)
+
+ data
+ |> Map.put("attachment", [attachment])
+ |> Map.put("url", link_element["href"])
+ end
+
+ defp fix_url(data), do: data
+
+ defp fix_content(%{"mediaType" => "text/markdown", "content" => content} = data)
+ when is_binary(content) do
+ content =
+ content
+ |> Earmark.as_html!(%Earmark.Options{renderer: EarmarkRenderer})
+ |> Pleroma.HTML.filter_tags()
+
+ Map.put(data, "content", content)
+ end
+
+ defp fix_content(data), do: data
+
+ defp fix(data) do
+ data
+ |> CommonFixes.fix_defaults()
+ |> CommonFixes.fix_attribution()
+ |> CommonFixes.fix_actor()
+ |> Transmogrifier.fix_emoji()
+ |> fix_url()
+ |> fix_content()
+ end
+
+ def changeset(struct, data) do
+ data = fix(data)
+
+ struct
+ |> cast(data, __schema__(:fields) -- [:attachment])
+ |> cast_embed(:attachment)
+ end
+
+ def validate_data(data_cng) do
+ data_cng
+ |> validate_inclusion(:type, ["Audio", "Video"])
+ |> validate_required([:id, :actor, :attributedTo, :type, :context, :attachment])
+ |> CommonValidations.validate_any_presence([:cc, :to])
+ |> CommonValidations.validate_fields_match([:actor, :attributedTo])
+ |> CommonValidations.validate_actor_presence()
+ |> CommonValidations.validate_host_match()
+ end
+end
diff --git a/lib/pleroma/web/activity_pub/object_validators/common_fixes.ex b/lib/pleroma/web/activity_pub/object_validators/common_fixes.ex
index 720213d73..b3638cfc7 100644
--- a/lib/pleroma/web/activity_pub/object_validators/common_fixes.ex
+++ b/lib/pleroma/web/activity_pub/object_validators/common_fixes.ex
@@ -3,6 +3,7 @@
# SPDX-License-Identifier: AGPL-3.0-only
defmodule Pleroma.Web.ActivityPub.ObjectValidators.CommonFixes do
+ alias Pleroma.Object.Containment
alias Pleroma.Web.ActivityPub.Utils
# based on Pleroma.Web.ActivityPub.Utils.lazy_put_objects_defaults
@@ -19,4 +20,12 @@ def fix_attribution(data) do
data
|> Map.put_new("actor", data["attributedTo"])
end
+
+ def fix_actor(data) do
+ actor = Containment.get_actor(data)
+
+ data
+ |> Map.put("actor", actor)
+ |> Map.put("attributedTo", actor)
+ end
end
diff --git a/lib/pleroma/web/activity_pub/object_validators/create_generic_validator.ex b/lib/pleroma/web/activity_pub/object_validators/create_generic_validator.ex
index b3dbeea57..422ee07be 100644
--- a/lib/pleroma/web/activity_pub/object_validators/create_generic_validator.ex
+++ b/lib/pleroma/web/activity_pub/object_validators/create_generic_validator.ex
@@ -10,9 +10,10 @@ defmodule Pleroma.Web.ActivityPub.ObjectValidators.CreateGenericValidator do
alias Pleroma.EctoType.ActivityPub.ObjectValidators
alias Pleroma.Object
+ alias Pleroma.Web.ActivityPub.ObjectValidators.CommonFixes
+ alias Pleroma.Web.ActivityPub.ObjectValidators.CommonValidations
import Ecto.Changeset
- import Pleroma.Web.ActivityPub.ObjectValidators.CommonValidations
@primary_key false
@@ -75,14 +76,15 @@ defp fix(data, meta) do
data
|> fix_context(meta)
|> fix_addressing(meta)
+ |> CommonFixes.fix_actor()
end
def validate_data(cng, meta \\ []) do
cng
|> validate_required([:actor, :type, :object])
|> validate_inclusion(:type, ["Create"])
- |> validate_actor_presence()
- |> validate_any_presence([:to, :cc])
+ |> CommonValidations.validate_actor_presence()
+ |> CommonValidations.validate_any_presence([:to, :cc])
|> validate_actors_match(meta)
|> validate_context_match(meta)
|> validate_object_nonexistence()
diff --git a/lib/pleroma/web/activity_pub/object_validators/note_validator.ex b/lib/pleroma/web/activity_pub/object_validators/note_validator.ex
deleted file mode 100644
index ab4469a59..000000000
--- a/lib/pleroma/web/activity_pub/object_validators/note_validator.ex
+++ /dev/null
@@ -1,73 +0,0 @@
-# Pleroma: A lightweight social networking server
-# Copyright © 2017-2020 Pleroma Authors
-# SPDX-License-Identifier: AGPL-3.0-only
-
-defmodule Pleroma.Web.ActivityPub.ObjectValidators.NoteValidator do
- use Ecto.Schema
-
- alias Pleroma.EctoType.ActivityPub.ObjectValidators
- alias Pleroma.Web.ActivityPub.Transmogrifier
-
- import Ecto.Changeset
-
- @primary_key false
-
- embedded_schema do
- field(:id, ObjectValidators.ObjectID, primary_key: true)
- field(:to, ObjectValidators.Recipients, default: [])
- field(:cc, ObjectValidators.Recipients, default: [])
- field(:bto, ObjectValidators.Recipients, default: [])
- field(:bcc, ObjectValidators.Recipients, default: [])
- # TODO: Write type
- field(:tag, {:array, :map}, default: [])
- field(:type, :string)
-
- field(:name, :string)
- field(:summary, :string)
- field(:content, :string)
-
- field(:context, :string)
- # short identifier for PleromaFE to group statuses by context
- field(:context_id, :integer)
-
- field(:actor, ObjectValidators.ObjectID)
- field(:attributedTo, ObjectValidators.ObjectID)
- field(:published, ObjectValidators.DateTime)
- field(:emoji, ObjectValidators.Emoji, default: %{})
- field(:sensitive, :boolean, default: false)
- # TODO: Write type
- field(:attachment, {:array, :map}, default: [])
- field(:replies_count, :integer, default: 0)
- field(:like_count, :integer, default: 0)
- field(:announcement_count, :integer, default: 0)
- field(:inReplyTo, ObjectValidators.ObjectID)
- field(:url, ObjectValidators.Uri)
-
- field(:likes, {:array, :string}, default: [])
- field(:announcements, {:array, :string}, default: [])
- end
-
- def cast_and_validate(data) do
- data
- |> cast_data()
- |> validate_data()
- end
-
- defp fix(data) do
- data
- |> Transmogrifier.fix_emoji()
- end
-
- def cast_data(data) do
- data = fix(data)
-
- %__MODULE__{}
- |> cast(data, __schema__(:fields))
- end
-
- def validate_data(data_cng) do
- data_cng
- |> validate_inclusion(:type, ["Note"])
- |> validate_required([:id, :actor, :to, :cc, :type, :content, :context])
- end
-end
diff --git a/lib/pleroma/web/activity_pub/object_validators/question_validator.ex b/lib/pleroma/web/activity_pub/object_validators/question_validator.ex
index 934d3c1ea..9310485dc 100644
--- a/lib/pleroma/web/activity_pub/object_validators/question_validator.ex
+++ b/lib/pleroma/web/activity_pub/object_validators/question_validator.ex
@@ -47,8 +47,8 @@ defmodule Pleroma.Web.ActivityPub.ObjectValidators.QuestionValidator do
# short identifier for PleromaFE to group statuses by context
field(:context_id, :integer)
- field(:likes, {:array, :string}, default: [])
- field(:announcements, {:array, :string}, default: [])
+ field(:likes, {:array, ObjectValidators.ObjectID}, default: [])
+ field(:announcements, {:array, ObjectValidators.ObjectID}, default: [])
field(:closed, ObjectValidators.DateTime)
field(:voters, {:array, ObjectValidators.ObjectID}, default: [])
diff --git a/lib/pleroma/web/activity_pub/object_validators/url_object_validator.ex b/lib/pleroma/web/activity_pub/object_validators/url_object_validator.ex
deleted file mode 100644
index 881030f38..000000000
--- a/lib/pleroma/web/activity_pub/object_validators/url_object_validator.ex
+++ /dev/null
@@ -1,24 +0,0 @@
-# Pleroma: A lightweight social networking server
-# Copyright © 2017-2020 Pleroma Authors
-# SPDX-License-Identifier: AGPL-3.0-only
-
-defmodule Pleroma.Web.ActivityPub.ObjectValidators.UrlObjectValidator do
- use Ecto.Schema
-
- alias Pleroma.EctoType.ActivityPub.ObjectValidators
-
- import Ecto.Changeset
- @primary_key false
-
- embedded_schema do
- field(:type, :string)
- field(:href, ObjectValidators.Uri)
- field(:mediaType, :string, default: "application/octet-stream")
- end
-
- def changeset(struct, data) do
- struct
- |> cast(data, __schema__(:fields))
- |> validate_required([:type, :href, :mediaType])
- end
-end
diff --git a/lib/pleroma/web/activity_pub/pipeline.ex b/lib/pleroma/web/activity_pub/pipeline.ex
index 36e325c37..2db86f116 100644
--- a/lib/pleroma/web/activity_pub/pipeline.ex
+++ b/lib/pleroma/web/activity_pub/pipeline.ex
@@ -26,13 +26,17 @@ def common_pipeline(object, meta) do
{:error, e} ->
{:error, e}
+
+ {:reject, e} ->
+ {:reject, e}
end
end
def do_common_pipeline(object, meta) do
with {_, {:ok, validated_object, meta}} <-
{:validate_object, ObjectValidator.validate(object, meta)},
- {_, {:ok, mrfd_object}} <- {:mrf_object, MRF.filter(validated_object)},
+ {_, {:ok, mrfd_object, meta}} <-
+ {:mrf_object, MRF.pipeline_filter(validated_object, meta)},
{_, {:ok, activity, meta}} <-
{:persist_object, ActivityPub.persist(mrfd_object, meta)},
{_, {:ok, activity, meta}} <-
@@ -40,7 +44,7 @@ def do_common_pipeline(object, meta) do
{_, {:ok, _}} <- {:federation, maybe_federate(activity, meta)} do
{:ok, activity, meta}
else
- {:mrf_object, {:reject, _}} -> {:ok, nil, meta}
+ {:mrf_object, {:reject, message, _}} -> {:reject, message}
e -> {:error, e}
end
end
diff --git a/lib/pleroma/web/activity_pub/publisher.ex b/lib/pleroma/web/activity_pub/publisher.ex
index d88f7f3ee..a2930c1cd 100644
--- a/lib/pleroma/web/activity_pub/publisher.ex
+++ b/lib/pleroma/web/activity_pub/publisher.ex
@@ -13,6 +13,7 @@ defmodule Pleroma.Web.ActivityPub.Publisher do
alias Pleroma.User
alias Pleroma.Web.ActivityPub.Relay
alias Pleroma.Web.ActivityPub.Transmogrifier
+ alias Pleroma.Web.FedSockets
require Pleroma.Constants
@@ -50,15 +51,35 @@ def is_representable?(%Activity{} = activity) do
def publish_one(%{inbox: inbox, json: json, actor: %User{} = actor, id: id} = params) do
Logger.debug("Federating #{id} to #{inbox}")
- uri = URI.parse(inbox)
+ case FedSockets.get_or_create_fed_socket(inbox) do
+ {:ok, fedsocket} ->
+ Logger.debug("publishing via fedsockets - #{inspect(inbox)}")
+ FedSockets.publish(fedsocket, json)
+ _ ->
+ Logger.debug("publishing via http - #{inspect(inbox)}")
+ http_publish(inbox, actor, json, params)
+ end
+ end
+
+ def publish_one(%{actor_id: actor_id} = params) do
+ actor = User.get_cached_by_id(actor_id)
+
+ params
+ |> Map.delete(:actor_id)
+ |> Map.put(:actor, actor)
+ |> publish_one()
+ end
+
+ defp http_publish(inbox, actor, json, params) do
+ uri = %{path: path} = URI.parse(inbox)
digest = "SHA-256=" <> (:crypto.hash(:sha256, json) |> Base.encode64())
date = Pleroma.Signature.signed_date()
signature =
Pleroma.Signature.sign(actor, %{
- "(request-target)": "post #{uri.path}",
+ "(request-target)": "post #{path}",
host: signature_host(uri),
"content-length": byte_size(json),
digest: digest,
@@ -89,15 +110,6 @@ def publish_one(%{inbox: inbox, json: json, actor: %User{} = actor, id: id} = pa
end
end
- def publish_one(%{actor_id: actor_id} = params) do
- actor = User.get_cached_by_id(actor_id)
-
- params
- |> Map.delete(:actor_id)
- |> Map.put(:actor, actor)
- |> publish_one()
- end
-
defp signature_host(%URI{port: port, scheme: scheme, host: host}) do
if port == URI.default_port(scheme) do
host
@@ -230,9 +242,7 @@ def publish(%User{} = actor, %{data: %{"bcc" => bcc}} = activity)
end)
end
- @doc """
- Publishes an activity to all relevant peers.
- """
+ # Publishes an activity to all relevant peers.
def publish(%User{} = actor, %Activity{} = activity) do
public = is_public?(activity)
diff --git a/lib/pleroma/web/activity_pub/relay.ex b/lib/pleroma/web/activity_pub/relay.ex
index b65710a94..6606e1780 100644
--- a/lib/pleroma/web/activity_pub/relay.ex
+++ b/lib/pleroma/web/activity_pub/relay.ex
@@ -30,12 +30,16 @@ def follow(target_instance) do
end
end
- @spec unfollow(String.t()) :: {:ok, Activity.t()} | {:error, any()}
- def unfollow(target_instance) do
+ @spec unfollow(String.t(), map()) :: {:ok, Activity.t()} | {:error, any()}
+ def unfollow(target_instance, opts \\ %{}) do
with %User{} = local_user <- get_actor(),
- {:ok, %User{} = target_user} <- User.get_or_fetch_by_ap_id(target_instance),
+ {:ok, target_user} <- fetch_target_user(target_instance, opts),
{:ok, activity} <- ActivityPub.unfollow(local_user, target_user) do
- User.unfollow(local_user, target_user)
+ case target_user.id do
+ nil -> User.update_following_count(local_user)
+ _ -> User.unfollow(local_user, target_user)
+ end
+
Logger.info("relay: unfollowed instance: #{target_instance}: id=#{activity.data["id"]}")
{:ok, activity}
else
@@ -43,6 +47,14 @@ def unfollow(target_instance) do
end
end
+ defp fetch_target_user(ap_id, opts) do
+ case {opts[:force], User.get_or_fetch_by_ap_id(ap_id)} do
+ {_, {:ok, %User{} = user}} -> {:ok, user}
+ {true, _} -> {:ok, %User{ap_id: ap_id}}
+ {_, error} -> error
+ end
+ end
+
@spec publish(any()) :: {:ok, Activity.t()} | {:error, any()}
def publish(%Activity{data: %{"type" => "Create"}} = activity) do
with %User{} = user <- get_actor(),
diff --git a/lib/pleroma/web/activity_pub/side_effects.ex b/lib/pleroma/web/activity_pub/side_effects.ex
index a5e2323bd..bbff35c36 100644
--- a/lib/pleroma/web/activity_pub/side_effects.ex
+++ b/lib/pleroma/web/activity_pub/side_effects.ex
@@ -1,3 +1,7 @@
+# Pleroma: A lightweight social networking server
+# Copyright © 2017-2020 Pleroma Authors
+# SPDX-License-Identifier: AGPL-3.0-only
+
defmodule Pleroma.Web.ActivityPub.SideEffects do
@moduledoc """
This module looks at an inserted object and executes the side effects that it
@@ -7,7 +11,6 @@ defmodule Pleroma.Web.ActivityPub.SideEffects do
"""
alias Pleroma.Activity
alias Pleroma.Activity.Ir.Topics
- alias Pleroma.ActivityExpiration
alias Pleroma.Chat
alias Pleroma.Chat.MessageReference
alias Pleroma.FollowingRelationship
@@ -99,7 +102,7 @@ def handle(
%User{} = followed <- User.get_cached_by_ap_id(followed_user),
{_, {:ok, _}, _, _} <-
{:following, User.follow(follower, followed, :follow_pending), follower, followed} do
- if followed.local && !followed.locked do
+ if followed.local && !followed.is_locked do
{:ok, accept_data, _} = Builder.accept(followed, object)
{:ok, _activity, _} = Pipeline.common_pipeline(accept_data, local: true)
end
@@ -184,14 +187,10 @@ def handle(%{data: %{"type" => "Create"}} = activity, meta) do
{:ok, notifications} = Notification.create_notifications(activity, do_send: false)
{:ok, _user} = ActivityPub.increase_note_count_if_public(user, object)
- if in_reply_to = object.data["inReplyTo"] do
+ if in_reply_to = object.data["inReplyTo"] && object.data["type"] != "Answer" do
Object.increase_replies_count(in_reply_to)
end
- if expires_at = activity.data["expires_at"] do
- ActivityExpiration.create(activity, expires_at)
- end
-
BackgroundWorker.enqueue("fetch_data_for_activity", %{"activity_id" => activity.id})
meta =
@@ -307,11 +306,18 @@ def handle_object_creation(%{"type" => "ChatMessage"} = object, meta) do
streamables =
[[actor, recipient], [recipient, actor]]
+ |> Enum.uniq()
|> Enum.map(fn [user, other_user] ->
if user.local do
{:ok, chat} = Chat.bump_or_create(user.id, other_user.ap_id)
{:ok, cm_ref} = MessageReference.create(chat, object, user.ap_id != actor.ap_id)
+ Cachex.put(
+ :chat_message_id_idempotency_key_cache,
+ cm_ref.id,
+ meta[:idempotency_key]
+ )
+
{
["user", "user:pleroma_chat"],
{user, %{cm_ref | chat: chat, object: object}}
@@ -341,7 +347,7 @@ def handle_object_creation(%{"type" => "Answer"} = object_map, meta) do
end
def handle_object_creation(%{"type" => objtype} = object, meta)
- when objtype in ~w[Audio Question Event] do
+ when objtype in ~w[Audio Video Question Event Article] do
with {:ok, object, meta} <- Pipeline.common_pipeline(object, meta) do
{:ok, object, meta}
end
diff --git a/lib/pleroma/web/activity_pub/transmogrifier.ex b/lib/pleroma/web/activity_pub/transmogrifier.ex
index 0831efadc..39c8f7e39 100644
--- a/lib/pleroma/web/activity_pub/transmogrifier.ex
+++ b/lib/pleroma/web/activity_pub/transmogrifier.ex
@@ -7,7 +7,6 @@ defmodule Pleroma.Web.ActivityPub.Transmogrifier do
A module to handle coding from internal to wire ActivityPub and back.
"""
alias Pleroma.Activity
- alias Pleroma.EarmarkRenderer
alias Pleroma.EctoType.ActivityPub.ObjectValidators
alias Pleroma.Maps
alias Pleroma.Object
@@ -41,11 +40,11 @@ def fix_object(object, options \\ []) do
|> fix_in_reply_to(options)
|> fix_emoji
|> fix_tag
+ |> set_sensitive
|> fix_content_map
|> fix_addressing
|> fix_summary
|> fix_type(options)
- |> fix_content
end
def fix_summary(%{"summary" => nil} = object) do
@@ -168,7 +167,6 @@ def fix_in_reply_to(object, options \\ [])
def fix_in_reply_to(%{"inReplyTo" => in_reply_to} = object, options)
when not is_nil(in_reply_to) do
in_reply_to_id = prepare_in_reply_to(in_reply_to)
- object = Map.put(object, "inReplyToAtomUri", in_reply_to_id)
depth = (options[:depth] || 0) + 1
if Federator.allowed_thread_distance?(depth) do
@@ -176,9 +174,8 @@ def fix_in_reply_to(%{"inReplyTo" => in_reply_to} = object, options)
%Activity{} <- Activity.get_create_by_object_ap_id(replied_object.data["id"]) do
object
|> Map.put("inReplyTo", replied_object.data["id"])
- |> Map.put("inReplyToAtomUri", object["inReplyToAtomUri"] || in_reply_to_id)
|> Map.put("context", replied_object.data["context"] || object["conversation"])
- |> Map.drop(["conversation"])
+ |> Map.drop(["conversation", "inReplyToAtomUri"])
else
e ->
Logger.warn("Couldn't fetch #{inspect(in_reply_to_id)}, error: #{inspect(e)}")
@@ -276,24 +273,7 @@ def fix_url(%{"url" => url} = object) when is_map(url) do
Map.put(object, "url", url["href"])
end
- def fix_url(%{"type" => "Video", "url" => url} = object) when is_list(url) do
- attachment =
- Enum.find(url, fn x ->
- media_type = x["mediaType"] || x["mimeType"] || ""
-
- is_map(x) and String.starts_with?(media_type, "video/")
- end)
-
- link_element =
- Enum.find(url, fn x -> is_map(x) and (x["mediaType"] || x["mimeType"]) == "text/html" end)
-
- object
- |> Map.put("attachment", [attachment])
- |> Map.put("url", link_element["href"])
- end
-
- def fix_url(%{"type" => object_type, "url" => url} = object)
- when object_type != "Video" and is_list(url) do
+ def fix_url(%{"url" => url} = object) when is_list(url) do
first_element = Enum.at(url, 0)
url_string =
@@ -311,7 +291,7 @@ def fix_url(object), do: object
def fix_emoji(%{"tag" => tags} = object) when is_list(tags) do
emoji =
tags
- |> Enum.filter(fn data -> data["type"] == "Emoji" and data["icon"] end)
+ |> Enum.filter(fn data -> is_map(data) and data["type"] == "Emoji" and data["icon"] end)
|> Enum.reduce(%{}, fn data, mapping ->
name = String.trim(data["name"], ":")
@@ -334,19 +314,21 @@ def fix_tag(%{"tag" => tag} = object) when is_list(tag) do
tags =
tag
|> Enum.filter(fn data -> data["type"] == "Hashtag" and data["name"] end)
- |> Enum.map(fn data -> String.slice(data["name"], 1..-1) end)
+ |> Enum.map(fn %{"name" => name} ->
+ name
+ |> String.slice(1..-1)
+ |> String.downcase()
+ end)
Map.put(object, "tag", tag ++ tags)
end
- def fix_tag(%{"tag" => %{"type" => "Hashtag", "name" => hashtag} = tag} = object) do
- combined = [tag, String.slice(hashtag, 1..-1)]
-
- Map.put(object, "tag", combined)
+ def fix_tag(%{"tag" => %{} = tag} = object) do
+ object
+ |> Map.put("tag", [tag])
+ |> fix_tag
end
- def fix_tag(%{"tag" => %{} = tag} = object), do: Map.put(object, "tag", [tag])
-
def fix_tag(object), do: object
# content map usually only has one language so this will do for now.
@@ -373,18 +355,6 @@ def fix_type(%{"inReplyTo" => reply_id, "name" => _} = object, options)
def fix_type(object, _), do: object
- defp fix_content(%{"mediaType" => "text/markdown", "content" => content} = object)
- when is_binary(content) do
- html_content =
- content
- |> Earmark.as_html!(%Earmark.Options{renderer: EarmarkRenderer})
- |> Pleroma.HTML.filter_tags()
-
- Map.merge(object, %{"content" => html_content, "mediaType" => "text/html"})
- end
-
- defp fix_content(object), do: object
-
# Reduce the object list to find the reported user.
defp get_reported(objects) do
Enum.reduce_while(objects, nil, fn ap_id, _ ->
@@ -457,7 +427,7 @@ def handle_incoming(
%{"type" => "Create", "object" => %{"type" => objtype} = object} = data,
options
)
- when objtype in ~w{Article Note Video Page} do
+ when objtype in ~w{Note Page} do
actor = Containment.get_actor(data)
with nil <- Activity.get_create_by_object_ap_id(object["id"]),
@@ -548,13 +518,19 @@ def handle_incoming(
end
def handle_incoming(
- %{"type" => "Create", "object" => %{"type" => objtype}} = data,
+ %{"type" => "Create", "object" => %{"type" => objtype, "id" => obj_id}} = data,
_options
)
- when objtype in ~w{Question Answer ChatMessage Audio Event} do
+ when objtype in ~w{Question Answer ChatMessage Audio Video Event Article} do
+ data = Map.put(data, "object", strip_internal_fields(data["object"]))
+
with {:ok, %User{}} <- ObjectValidator.fetch_actor(data),
+ nil <- Activity.get_create_by_object_ap_id(obj_id),
{:ok, activity, _} <- Pipeline.common_pipeline(data, local: false) do
{:ok, activity}
+ else
+ %Activity{} = activity -> {:ok, activity}
+ e -> e
end
end
@@ -954,7 +930,7 @@ def set_conversation(object) do
Map.put(object, "conversation", object["context"])
end
- def set_sensitive(%{"sensitive" => true} = object) do
+ def set_sensitive(%{"sensitive" => _} = object) do
object
end
@@ -1031,7 +1007,7 @@ def perform(:user_upgrade, user) do
def upgrade_user_from_ap_id(ap_id) do
with %User{local: false} = user <- User.get_cached_by_ap_id(ap_id),
- {:ok, data} <- ActivityPub.fetch_and_prepare_user_from_ap_id(ap_id),
+ {:ok, data} <- ActivityPub.fetch_and_prepare_user_from_ap_id(ap_id, force_http: true),
{:ok, user} <- update_user(user, data) do
TransmogrifierWorker.enqueue("user_upgrade", %{"user_id" => user.id})
{:ok, user}
diff --git a/lib/pleroma/web/activity_pub/views/user_view.ex b/lib/pleroma/web/activity_pub/views/user_view.ex
index 3a4564912..4dc45cde3 100644
--- a/lib/pleroma/web/activity_pub/views/user_view.ex
+++ b/lib/pleroma/web/activity_pub/views/user_view.ex
@@ -101,7 +101,7 @@ def render("user.json", %{user: user}) do
"name" => user.name,
"summary" => user.bio,
"url" => user.ap_id,
- "manuallyApprovesFollowers" => user.locked,
+ "manuallyApprovesFollowers" => user.is_locked,
"publicKey" => %{
"id" => "#{user.ap_id}#main-key",
"owner" => user.ap_id,
@@ -110,7 +110,7 @@ def render("user.json", %{user: user}) do
"endpoints" => endpoints,
"attachment" => fields,
"tag" => emoji_tags,
- "discoverable" => user.discoverable,
+ "discoverable" => user.is_discoverable,
"capabilities" => capabilities
}
|> Map.merge(maybe_make_image(&User.avatar_url/2, "icon", user))
diff --git a/lib/pleroma/web/activity_pub/visibility.ex b/lib/pleroma/web/activity_pub/visibility.ex
index 5c349bb7a..76bd54a42 100644
--- a/lib/pleroma/web/activity_pub/visibility.ex
+++ b/lib/pleroma/web/activity_pub/visibility.ex
@@ -44,29 +44,30 @@ def is_direct?(activity) do
def is_list?(%{data: %{"listMessage" => _}}), do: true
def is_list?(_), do: false
- @spec visible_for_user?(Activity.t(), User.t() | nil) :: boolean()
- def visible_for_user?(%{actor: ap_id}, %User{ap_id: ap_id}), do: true
+ @spec visible_for_user?(Activity.t() | nil, User.t() | nil) :: boolean()
+ def visible_for_user?(%Activity{actor: ap_id}, %User{ap_id: ap_id}), do: true
def visible_for_user?(nil, _), do: false
- def visible_for_user?(%{data: %{"listMessage" => _}}, nil), do: false
+ def visible_for_user?(%Activity{data: %{"listMessage" => _}}, nil), do: false
- def visible_for_user?(%{data: %{"listMessage" => list_ap_id}} = activity, %User{} = user) do
+ def visible_for_user?(
+ %Activity{data: %{"listMessage" => list_ap_id}} = activity,
+ %User{} = user
+ ) do
user.ap_id in activity.data["to"] ||
list_ap_id
|> Pleroma.List.get_by_ap_id()
|> Pleroma.List.member?(user)
end
- def visible_for_user?(%{local: local} = activity, nil) do
- cfg_key = if local, do: :local, else: :remote
-
- if Pleroma.Config.restrict_unauthenticated_access?(:activities, cfg_key),
+ def visible_for_user?(%Activity{} = activity, nil) do
+ if restrict_unauthenticated_access?(activity),
do: false,
else: is_public?(activity)
end
- def visible_for_user?(activity, user) do
+ def visible_for_user?(%Activity{} = activity, user) do
x = [user.ap_id | User.following(user)]
y = [activity.actor] ++ activity.data["to"] ++ (activity.data["cc"] || [])
is_public?(activity) || Enum.any?(x, &(&1 in y))
@@ -82,6 +83,26 @@ def entire_thread_visible_for_user?(%Activity{} = activity, %User{} = user) do
result
end
+ def restrict_unauthenticated_access?(%Activity{local: local}) do
+ restrict_unauthenticated_access_to_activity?(local)
+ end
+
+ def restrict_unauthenticated_access?(%Object{} = object) do
+ object
+ |> Object.local?()
+ |> restrict_unauthenticated_access_to_activity?()
+ end
+
+ def restrict_unauthenticated_access?(%User{} = user) do
+ User.visible_for(user, _reading_user = nil)
+ end
+
+ defp restrict_unauthenticated_access_to_activity?(local?) when is_boolean(local?) do
+ cfg_key = if local?, do: :local, else: :remote
+
+ Pleroma.Config.restrict_unauthenticated_access?(:activities, cfg_key)
+ end
+
def get_visibility(object) do
to = object.data["to"] || []
cc = object.data["cc"] || []
diff --git a/lib/pleroma/web/admin_api/controllers/admin_api_controller.ex b/lib/pleroma/web/admin_api/controllers/admin_api_controller.ex
index f5e4d49f9..5c2c282b3 100644
--- a/lib/pleroma/web/admin_api/controllers/admin_api_controller.ex
+++ b/lib/pleroma/web/admin_api/controllers/admin_api_controller.ex
@@ -5,32 +5,28 @@
defmodule Pleroma.Web.AdminAPI.AdminAPIController do
use Pleroma.Web, :controller
- import Pleroma.Web.ControllerHelper, only: [json_response: 3]
+ import Pleroma.Web.ControllerHelper,
+ only: [json_response: 3, fetch_integer_param: 3]
alias Pleroma.Config
alias Pleroma.MFA
alias Pleroma.ModerationLog
- alias Pleroma.Plugs.OAuthScopesPlug
alias Pleroma.Stats
alias Pleroma.User
alias Pleroma.Web.ActivityPub.ActivityPub
- alias Pleroma.Web.ActivityPub.Builder
- alias Pleroma.Web.ActivityPub.Pipeline
alias Pleroma.Web.AdminAPI
alias Pleroma.Web.AdminAPI.AccountView
alias Pleroma.Web.AdminAPI.ModerationLogView
- alias Pleroma.Web.AdminAPI.Search
alias Pleroma.Web.Endpoint
+ alias Pleroma.Web.Plugs.OAuthScopesPlug
alias Pleroma.Web.Router
- require Logger
-
@users_page_size 50
plug(
OAuthScopesPlug,
%{scopes: ["read:accounts"], admin: true}
- when action in [:list_users, :user_show, :right_get, :show_user_credentials]
+ when action in [:right_get, :show_user_credentials, :create_backup]
)
plug(
@@ -39,12 +35,6 @@ defmodule Pleroma.Web.AdminAPI.AdminAPIController do
when action in [
:get_password_reset,
:force_password_reset,
- :user_delete,
- :users_create,
- :user_toggle_activation,
- :user_activate,
- :user_deactivate,
- :user_approve,
:tag_users,
:untag_users,
:right_add,
@@ -58,14 +48,14 @@ defmodule Pleroma.Web.AdminAPI.AdminAPIController do
plug(
OAuthScopesPlug,
- %{scopes: ["write:follows"], admin: true}
- when action in [:user_follow, :user_unfollow]
+ %{scopes: ["read:statuses"], admin: true}
+ when action in [:list_user_statuses, :list_instance_statuses]
)
plug(
OAuthScopesPlug,
- %{scopes: ["read:statuses"], admin: true}
- when action in [:list_user_statuses, :list_instance_statuses]
+ %{scopes: ["read:chats"], admin: true}
+ when action in [:list_user_chats]
)
plug(
@@ -91,132 +81,6 @@ defmodule Pleroma.Web.AdminAPI.AdminAPIController do
action_fallback(AdminAPI.FallbackController)
- def user_delete(conn, %{"nickname" => nickname}) do
- user_delete(conn, %{"nicknames" => [nickname]})
- end
-
- def user_delete(%{assigns: %{user: admin}} = conn, %{"nicknames" => nicknames}) do
- users =
- nicknames
- |> Enum.map(&User.get_cached_by_nickname/1)
-
- users
- |> Enum.each(fn user ->
- {:ok, delete_data, _} = Builder.delete(admin, user.ap_id)
- Pipeline.common_pipeline(delete_data, local: true)
- end)
-
- ModerationLog.insert_log(%{
- actor: admin,
- subject: users,
- action: "delete"
- })
-
- json(conn, nicknames)
- end
-
- def user_follow(%{assigns: %{user: admin}} = conn, %{
- "follower" => follower_nick,
- "followed" => followed_nick
- }) do
- with %User{} = follower <- User.get_cached_by_nickname(follower_nick),
- %User{} = followed <- User.get_cached_by_nickname(followed_nick) do
- User.follow(follower, followed)
-
- ModerationLog.insert_log(%{
- actor: admin,
- followed: followed,
- follower: follower,
- action: "follow"
- })
- end
-
- json(conn, "ok")
- end
-
- def user_unfollow(%{assigns: %{user: admin}} = conn, %{
- "follower" => follower_nick,
- "followed" => followed_nick
- }) do
- with %User{} = follower <- User.get_cached_by_nickname(follower_nick),
- %User{} = followed <- User.get_cached_by_nickname(followed_nick) do
- User.unfollow(follower, followed)
-
- ModerationLog.insert_log(%{
- actor: admin,
- followed: followed,
- follower: follower,
- action: "unfollow"
- })
- end
-
- json(conn, "ok")
- end
-
- def users_create(%{assigns: %{user: admin}} = conn, %{"users" => users}) do
- changesets =
- Enum.map(users, fn %{"nickname" => nickname, "email" => email, "password" => password} ->
- user_data = %{
- nickname: nickname,
- name: nickname,
- email: email,
- password: password,
- password_confirmation: password,
- bio: "."
- }
-
- User.register_changeset(%User{}, user_data, need_confirmation: false)
- end)
- |> Enum.reduce(Ecto.Multi.new(), fn changeset, multi ->
- Ecto.Multi.insert(multi, Ecto.UUID.generate(), changeset)
- end)
-
- case Pleroma.Repo.transaction(changesets) do
- {:ok, users} ->
- res =
- users
- |> Map.values()
- |> Enum.map(fn user ->
- {:ok, user} = User.post_register_action(user)
-
- user
- end)
- |> Enum.map(&AccountView.render("created.json", %{user: &1}))
-
- ModerationLog.insert_log(%{
- actor: admin,
- subjects: Map.values(users),
- action: "create"
- })
-
- json(conn, res)
-
- {:error, id, changeset, _} ->
- res =
- Enum.map(changesets.operations, fn
- {current_id, {:changeset, _current_changeset, _}} when current_id == id ->
- AccountView.render("create-error.json", %{changeset: changeset})
-
- {_, {:changeset, current_changeset, _}} ->
- AccountView.render("create-error.json", %{changeset: current_changeset})
- end)
-
- conn
- |> put_status(:conflict)
- |> json(res)
- end
- end
-
- def user_show(%{assigns: %{user: admin}} = conn, %{"nickname" => nickname}) do
- with %User{} = user <- User.get_cached_by_nickname_or_id(nickname, for: admin) do
- conn
- |> put_view(AccountView)
- |> render("show.json", %{user: user})
- else
- _ -> {:error, :not_found}
- end
- end
-
def list_instance_statuses(conn, %{"instance" => instance} = params) do
with_reblogs = params["with_reblogs"] == "true" || params["with_reblogs"] == true
{page, page_size} = page_params(params)
@@ -256,67 +120,18 @@ def list_user_statuses(%{assigns: %{user: admin}} = conn, %{"nickname" => nickna
end
end
- def user_toggle_activation(%{assigns: %{user: admin}} = conn, %{"nickname" => nickname}) do
- user = User.get_cached_by_nickname(nickname)
+ def list_user_chats(%{assigns: %{user: admin}} = conn, %{"nickname" => nickname} = _params) do
+ with %User{id: user_id} <- User.get_cached_by_nickname_or_id(nickname, for: admin) do
+ chats =
+ Pleroma.Chat.for_user_query(user_id)
+ |> Pleroma.Repo.all()
- {:ok, updated_user} = User.deactivate(user, !user.deactivated)
-
- action = if user.deactivated, do: "activate", else: "deactivate"
-
- ModerationLog.insert_log(%{
- actor: admin,
- subject: [user],
- action: action
- })
-
- conn
- |> put_view(AccountView)
- |> render("show.json", %{user: updated_user})
- end
-
- def user_activate(%{assigns: %{user: admin}} = conn, %{"nicknames" => nicknames}) do
- users = Enum.map(nicknames, &User.get_cached_by_nickname/1)
- {:ok, updated_users} = User.deactivate(users, false)
-
- ModerationLog.insert_log(%{
- actor: admin,
- subject: users,
- action: "activate"
- })
-
- conn
- |> put_view(AccountView)
- |> render("index.json", %{users: Keyword.values(updated_users)})
- end
-
- def user_deactivate(%{assigns: %{user: admin}} = conn, %{"nicknames" => nicknames}) do
- users = Enum.map(nicknames, &User.get_cached_by_nickname/1)
- {:ok, updated_users} = User.deactivate(users, true)
-
- ModerationLog.insert_log(%{
- actor: admin,
- subject: users,
- action: "deactivate"
- })
-
- conn
- |> put_view(AccountView)
- |> render("index.json", %{users: Keyword.values(updated_users)})
- end
-
- def user_approve(%{assigns: %{user: admin}} = conn, %{"nicknames" => nicknames}) do
- users = Enum.map(nicknames, &User.get_cached_by_nickname/1)
- {:ok, updated_users} = User.approve(users)
-
- ModerationLog.insert_log(%{
- actor: admin,
- subject: users,
- action: "approve"
- })
-
- conn
- |> put_view(AccountView)
- |> render("index.json", %{users: updated_users})
+ conn
+ |> put_view(AdminAPI.ChatView)
+ |> render("index.json", chats: chats)
+ else
+ _ -> {:error, :not_found}
+ end
end
def tag_users(%{assigns: %{user: admin}} = conn, %{"nicknames" => nicknames, "tags" => tags}) do
@@ -345,43 +160,6 @@ def untag_users(%{assigns: %{user: admin}} = conn, %{"nicknames" => nicknames, "
end
end
- def list_users(conn, params) do
- {page, page_size} = page_params(params)
- filters = maybe_parse_filters(params["filters"])
-
- search_params = %{
- query: params["query"],
- page: page,
- page_size: page_size,
- tags: params["tags"],
- name: params["name"],
- email: params["email"]
- }
-
- with {:ok, users, count} <- Search.user(Map.merge(search_params, filters)) do
- json(
- conn,
- AccountView.render("index.json",
- users: users,
- count: count,
- page_size: page_size
- )
- )
- end
- end
-
- @filters ~w(local external active deactivated need_approval is_admin is_moderator)
-
- @spec maybe_parse_filters(String.t()) :: %{required(String.t()) => true} | %{}
- defp maybe_parse_filters(filters) when is_nil(filters) or filters == "", do: %{}
-
- defp maybe_parse_filters(filters) do
- filters
- |> String.split(",")
- |> Enum.filter(&Enum.member?(@filters, &1))
- |> Map.new(&{String.to_existing_atom(&1), true})
- end
-
def right_add_multiple(%{assigns: %{user: admin}} = conn, %{
"permission_group" => permission_group,
"nicknames" => nicknames
@@ -663,25 +441,19 @@ def stats(conn, params) do
json(conn, %{"status_visibility" => counters})
end
+ def create_backup(%{assigns: %{user: admin}} = conn, %{"nickname" => nickname}) do
+ with %User{} = user <- User.get_by_nickname(nickname),
+ {:ok, _} <- Pleroma.User.Backup.create(user, admin.id) do
+ ModerationLog.insert_log(%{actor: admin, subject: user, action: "create_backup"})
+
+ json(conn, "")
+ end
+ end
+
defp page_params(params) do
- {get_page(params["page"]), get_page_size(params["page_size"])}
- end
-
- defp get_page(page_string) when is_nil(page_string), do: 1
-
- defp get_page(page_string) do
- case Integer.parse(page_string) do
- {page, _} -> page
- :error -> 1
- end
- end
-
- defp get_page_size(page_size_string) when is_nil(page_size_string), do: @users_page_size
-
- defp get_page_size(page_size_string) do
- case Integer.parse(page_size_string) do
- {page_size, _} -> page_size
- :error -> @users_page_size
- end
+ {
+ fetch_integer_param(params, "page", 1),
+ fetch_integer_param(params, "page_size", @users_page_size)
+ }
end
end
diff --git a/lib/pleroma/web/admin_api/controllers/chat_controller.ex b/lib/pleroma/web/admin_api/controllers/chat_controller.ex
new file mode 100644
index 000000000..af8ff8292
--- /dev/null
+++ b/lib/pleroma/web/admin_api/controllers/chat_controller.ex
@@ -0,0 +1,85 @@
+# Pleroma: A lightweight social networking server
+# Copyright © 2017-2020 Pleroma Authors
+# SPDX-License-Identifier: AGPL-3.0-only
+
+defmodule Pleroma.Web.AdminAPI.ChatController do
+ use Pleroma.Web, :controller
+
+ alias Pleroma.Activity
+ alias Pleroma.Chat
+ alias Pleroma.Chat.MessageReference
+ alias Pleroma.ModerationLog
+ alias Pleroma.Pagination
+ alias Pleroma.Web.AdminAPI
+ alias Pleroma.Web.CommonAPI
+ alias Pleroma.Web.PleromaAPI.Chat.MessageReferenceView
+ alias Pleroma.Web.Plugs.OAuthScopesPlug
+
+ require Logger
+
+ plug(Pleroma.Web.ApiSpec.CastAndValidate)
+
+ plug(
+ OAuthScopesPlug,
+ %{scopes: ["read:chats"], admin: true} when action in [:show, :messages]
+ )
+
+ plug(
+ OAuthScopesPlug,
+ %{scopes: ["write:chats"], admin: true} when action in [:delete_message]
+ )
+
+ action_fallback(Pleroma.Web.AdminAPI.FallbackController)
+
+ defdelegate open_api_operation(action), to: Pleroma.Web.ApiSpec.Admin.ChatOperation
+
+ def delete_message(%{assigns: %{user: user}} = conn, %{
+ message_id: message_id,
+ id: chat_id
+ }) do
+ with %MessageReference{object: %{data: %{"id" => object_ap_id}}} = cm_ref <-
+ MessageReference.get_by_id(message_id),
+ ^chat_id <- to_string(cm_ref.chat_id),
+ %Activity{id: activity_id} <- Activity.get_create_by_object_ap_id(object_ap_id),
+ {:ok, _} <- CommonAPI.delete(activity_id, user) do
+ ModerationLog.insert_log(%{
+ action: "chat_message_delete",
+ actor: user,
+ subject_id: message_id
+ })
+
+ conn
+ |> put_view(MessageReferenceView)
+ |> render("show.json", chat_message_reference: cm_ref)
+ else
+ _e ->
+ {:error, :could_not_delete}
+ end
+ end
+
+ def messages(conn, %{id: id} = params) do
+ with %Chat{} = chat <- Chat.get_by_id(id) do
+ cm_refs =
+ chat
+ |> MessageReference.for_chat_query()
+ |> Pagination.fetch_paginated(params)
+
+ conn
+ |> put_view(MessageReferenceView)
+ |> render("index.json", chat_message_references: cm_refs)
+ else
+ _ ->
+ conn
+ |> put_status(:not_found)
+ |> json(%{error: "not found"})
+ end
+ end
+
+ def show(conn, %{id: id}) do
+ with %Chat{} = chat <- Chat.get_by_id(id) do
+ conn
+ |> put_view(AdminAPI.ChatView)
+ |> render("show.json", chat: chat)
+ end
+ end
+end
diff --git a/lib/pleroma/web/admin_api/controllers/config_controller.ex b/lib/pleroma/web/admin_api/controllers/config_controller.ex
index 0df13007f..5d155af3d 100644
--- a/lib/pleroma/web/admin_api/controllers/config_controller.ex
+++ b/lib/pleroma/web/admin_api/controllers/config_controller.ex
@@ -7,7 +7,7 @@ defmodule Pleroma.Web.AdminAPI.ConfigController do
alias Pleroma.Config
alias Pleroma.ConfigDB
- alias Pleroma.Plugs.OAuthScopesPlug
+ alias Pleroma.Web.Plugs.OAuthScopesPlug
plug(Pleroma.Web.ApiSpec.CastAndValidate)
plug(OAuthScopesPlug, %{scopes: ["write"], admin: true} when action == :update)
diff --git a/lib/pleroma/web/admin_api/controllers/instance_document_controller.ex b/lib/pleroma/web/admin_api/controllers/instance_document_controller.ex
new file mode 100644
index 000000000..37dbfeb72
--- /dev/null
+++ b/lib/pleroma/web/admin_api/controllers/instance_document_controller.ex
@@ -0,0 +1,41 @@
+# Pleroma: A lightweight social networking server
+# Copyright © 2017-2020 Pleroma Authors
+# SPDX-License-Identifier: AGPL-3.0-only
+
+defmodule Pleroma.Web.AdminAPI.InstanceDocumentController do
+ use Pleroma.Web, :controller
+
+ alias Pleroma.Web.InstanceDocument
+ alias Pleroma.Web.Plugs.InstanceStatic
+ alias Pleroma.Web.Plugs.OAuthScopesPlug
+
+ plug(Pleroma.Web.ApiSpec.CastAndValidate)
+
+ action_fallback(Pleroma.Web.AdminAPI.FallbackController)
+
+ defdelegate open_api_operation(action), to: Pleroma.Web.ApiSpec.Admin.InstanceDocumentOperation
+
+ plug(OAuthScopesPlug, %{scopes: ["read"], admin: true} when action == :show)
+ plug(OAuthScopesPlug, %{scopes: ["write"], admin: true} when action in [:update, :delete])
+
+ def show(conn, %{name: document_name}) do
+ with {:ok, url} <- InstanceDocument.get(document_name),
+ {:ok, content} <- File.read(InstanceStatic.file_path(url)) do
+ conn
+ |> put_resp_content_type("text/html")
+ |> send_resp(200, content)
+ end
+ end
+
+ def update(%{body_params: %{file: file}} = conn, %{name: document_name}) do
+ with {:ok, url} <- InstanceDocument.put(document_name, file.path) do
+ json(conn, %{"url" => url})
+ end
+ end
+
+ def delete(conn, %{name: document_name}) do
+ with :ok <- InstanceDocument.delete(document_name) do
+ json(conn, %{})
+ end
+ end
+end
diff --git a/lib/pleroma/web/admin_api/controllers/invite_controller.ex b/lib/pleroma/web/admin_api/controllers/invite_controller.ex
index 7d169b8d2..6a9b4038a 100644
--- a/lib/pleroma/web/admin_api/controllers/invite_controller.ex
+++ b/lib/pleroma/web/admin_api/controllers/invite_controller.ex
@@ -8,8 +8,8 @@ defmodule Pleroma.Web.AdminAPI.InviteController do
import Pleroma.Web.ControllerHelper, only: [json_response: 3]
alias Pleroma.Config
- alias Pleroma.Plugs.OAuthScopesPlug
alias Pleroma.UserInviteToken
+ alias Pleroma.Web.Plugs.OAuthScopesPlug
require Logger
diff --git a/lib/pleroma/web/admin_api/controllers/media_proxy_cache_controller.ex b/lib/pleroma/web/admin_api/controllers/media_proxy_cache_controller.ex
index 131e22d78..6d92e9f7f 100644
--- a/lib/pleroma/web/admin_api/controllers/media_proxy_cache_controller.ex
+++ b/lib/pleroma/web/admin_api/controllers/media_proxy_cache_controller.ex
@@ -5,9 +5,9 @@
defmodule Pleroma.Web.AdminAPI.MediaProxyCacheController do
use Pleroma.Web, :controller
- alias Pleroma.Plugs.OAuthScopesPlug
alias Pleroma.Web.ApiSpec.Admin, as: Spec
alias Pleroma.Web.MediaProxy
+ alias Pleroma.Web.Plugs.OAuthScopesPlug
plug(Pleroma.Web.ApiSpec.CastAndValidate)
diff --git a/lib/pleroma/web/admin_api/controllers/oauth_app_controller.ex b/lib/pleroma/web/admin_api/controllers/o_auth_app_controller.ex
similarity index 97%
rename from lib/pleroma/web/admin_api/controllers/oauth_app_controller.ex
rename to lib/pleroma/web/admin_api/controllers/o_auth_app_controller.ex
index dca23ea73..116a05a4d 100644
--- a/lib/pleroma/web/admin_api/controllers/oauth_app_controller.ex
+++ b/lib/pleroma/web/admin_api/controllers/o_auth_app_controller.ex
@@ -7,8 +7,8 @@ defmodule Pleroma.Web.AdminAPI.OAuthAppController do
import Pleroma.Web.ControllerHelper, only: [json_response: 3]
- alias Pleroma.Plugs.OAuthScopesPlug
alias Pleroma.Web.OAuth.App
+ alias Pleroma.Web.Plugs.OAuthScopesPlug
require Logger
diff --git a/lib/pleroma/web/admin_api/controllers/relay_controller.ex b/lib/pleroma/web/admin_api/controllers/relay_controller.ex
index 95d06dde7..611388447 100644
--- a/lib/pleroma/web/admin_api/controllers/relay_controller.ex
+++ b/lib/pleroma/web/admin_api/controllers/relay_controller.ex
@@ -6,8 +6,8 @@ defmodule Pleroma.Web.AdminAPI.RelayController do
use Pleroma.Web, :controller
alias Pleroma.ModerationLog
- alias Pleroma.Plugs.OAuthScopesPlug
alias Pleroma.Web.ActivityPub.Relay
+ alias Pleroma.Web.Plugs.OAuthScopesPlug
require Logger
@@ -33,11 +33,7 @@ def index(conn, _params) do
def follow(%{assigns: %{user: admin}, body_params: %{relay_url: target}} = conn, _) do
with {:ok, _message} <- Relay.follow(target) do
- ModerationLog.insert_log(%{
- action: "relay_follow",
- actor: admin,
- target: target
- })
+ ModerationLog.insert_log(%{action: "relay_follow", actor: admin, target: target})
json(conn, %{actor: target, followed_back: target in Relay.following()})
else
@@ -48,13 +44,9 @@ def follow(%{assigns: %{user: admin}, body_params: %{relay_url: target}} = conn,
end
end
- def unfollow(%{assigns: %{user: admin}, body_params: %{relay_url: target}} = conn, _) do
- with {:ok, _message} <- Relay.unfollow(target) do
- ModerationLog.insert_log(%{
- action: "relay_unfollow",
- actor: admin,
- target: target
- })
+ def unfollow(%{assigns: %{user: admin}, body_params: %{relay_url: target} = params} = conn, _) do
+ with {:ok, _message} <- Relay.unfollow(target, %{force: params[:force]}) do
+ ModerationLog.insert_log(%{action: "relay_unfollow", actor: admin, target: target})
json(conn, target)
else
diff --git a/lib/pleroma/web/admin_api/controllers/report_controller.ex b/lib/pleroma/web/admin_api/controllers/report_controller.ex
index 4c011e174..6a0e56f5f 100644
--- a/lib/pleroma/web/admin_api/controllers/report_controller.ex
+++ b/lib/pleroma/web/admin_api/controllers/report_controller.ex
@@ -9,12 +9,12 @@ defmodule Pleroma.Web.AdminAPI.ReportController do
alias Pleroma.Activity
alias Pleroma.ModerationLog
- alias Pleroma.Plugs.OAuthScopesPlug
alias Pleroma.ReportNote
alias Pleroma.Web.ActivityPub.Utils
alias Pleroma.Web.AdminAPI
alias Pleroma.Web.AdminAPI.Report
alias Pleroma.Web.CommonAPI
+ alias Pleroma.Web.Plugs.OAuthScopesPlug
require Logger
@@ -38,7 +38,7 @@ def index(conn, params) do
end
def show(conn, %{id: id}) do
- with %Activity{} = report <- Activity.get_by_id(id) do
+ with %Activity{} = report <- Activity.get_report(id) do
render(conn, "show.json", Report.extract_report_info(report))
else
_ -> {:error, :not_found}
diff --git a/lib/pleroma/web/admin_api/controllers/status_controller.ex b/lib/pleroma/web/admin_api/controllers/status_controller.ex
index bc48cc527..2bb437cfe 100644
--- a/lib/pleroma/web/admin_api/controllers/status_controller.ex
+++ b/lib/pleroma/web/admin_api/controllers/status_controller.ex
@@ -7,10 +7,10 @@ defmodule Pleroma.Web.AdminAPI.StatusController do
alias Pleroma.Activity
alias Pleroma.ModerationLog
- alias Pleroma.Plugs.OAuthScopesPlug
alias Pleroma.Web.ActivityPub.ActivityPub
alias Pleroma.Web.CommonAPI
alias Pleroma.Web.MastodonAPI
+ alias Pleroma.Web.Plugs.OAuthScopesPlug
require Logger
diff --git a/lib/pleroma/web/admin_api/controllers/user_controller.ex b/lib/pleroma/web/admin_api/controllers/user_controller.ex
new file mode 100644
index 000000000..a2a1c875d
--- /dev/null
+++ b/lib/pleroma/web/admin_api/controllers/user_controller.ex
@@ -0,0 +1,281 @@
+# Pleroma: A lightweight social networking server
+# Copyright © 2017-2020 Pleroma Authors
+# SPDX-License-Identifier: AGPL-3.0-only
+
+defmodule Pleroma.Web.AdminAPI.UserController do
+ use Pleroma.Web, :controller
+
+ import Pleroma.Web.ControllerHelper,
+ only: [fetch_integer_param: 3]
+
+ alias Pleroma.ModerationLog
+ alias Pleroma.User
+ alias Pleroma.Web.ActivityPub.Builder
+ alias Pleroma.Web.ActivityPub.Pipeline
+ alias Pleroma.Web.AdminAPI
+ alias Pleroma.Web.AdminAPI.AccountView
+ alias Pleroma.Web.AdminAPI.Search
+ alias Pleroma.Web.Plugs.OAuthScopesPlug
+
+ @users_page_size 50
+
+ plug(
+ OAuthScopesPlug,
+ %{scopes: ["read:accounts"], admin: true}
+ when action in [:list, :show]
+ )
+
+ plug(
+ OAuthScopesPlug,
+ %{scopes: ["write:accounts"], admin: true}
+ when action in [
+ :delete,
+ :create,
+ :toggle_activation,
+ :activate,
+ :deactivate,
+ :approve
+ ]
+ )
+
+ plug(
+ OAuthScopesPlug,
+ %{scopes: ["write:follows"], admin: true}
+ when action in [:follow, :unfollow]
+ )
+
+ action_fallback(AdminAPI.FallbackController)
+
+ def delete(conn, %{"nickname" => nickname}) do
+ delete(conn, %{"nicknames" => [nickname]})
+ end
+
+ def delete(%{assigns: %{user: admin}} = conn, %{"nicknames" => nicknames}) do
+ users = Enum.map(nicknames, &User.get_cached_by_nickname/1)
+
+ Enum.each(users, fn user ->
+ {:ok, delete_data, _} = Builder.delete(admin, user.ap_id)
+ Pipeline.common_pipeline(delete_data, local: true)
+ end)
+
+ ModerationLog.insert_log(%{
+ actor: admin,
+ subject: users,
+ action: "delete"
+ })
+
+ json(conn, nicknames)
+ end
+
+ def follow(%{assigns: %{user: admin}} = conn, %{
+ "follower" => follower_nick,
+ "followed" => followed_nick
+ }) do
+ with %User{} = follower <- User.get_cached_by_nickname(follower_nick),
+ %User{} = followed <- User.get_cached_by_nickname(followed_nick) do
+ User.follow(follower, followed)
+
+ ModerationLog.insert_log(%{
+ actor: admin,
+ followed: followed,
+ follower: follower,
+ action: "follow"
+ })
+ end
+
+ json(conn, "ok")
+ end
+
+ def unfollow(%{assigns: %{user: admin}} = conn, %{
+ "follower" => follower_nick,
+ "followed" => followed_nick
+ }) do
+ with %User{} = follower <- User.get_cached_by_nickname(follower_nick),
+ %User{} = followed <- User.get_cached_by_nickname(followed_nick) do
+ User.unfollow(follower, followed)
+
+ ModerationLog.insert_log(%{
+ actor: admin,
+ followed: followed,
+ follower: follower,
+ action: "unfollow"
+ })
+ end
+
+ json(conn, "ok")
+ end
+
+ def create(%{assigns: %{user: admin}} = conn, %{"users" => users}) do
+ changesets =
+ Enum.map(users, fn %{"nickname" => nickname, "email" => email, "password" => password} ->
+ user_data = %{
+ nickname: nickname,
+ name: nickname,
+ email: email,
+ password: password,
+ password_confirmation: password,
+ bio: "."
+ }
+
+ User.register_changeset(%User{}, user_data, need_confirmation: false)
+ end)
+ |> Enum.reduce(Ecto.Multi.new(), fn changeset, multi ->
+ Ecto.Multi.insert(multi, Ecto.UUID.generate(), changeset)
+ end)
+
+ case Pleroma.Repo.transaction(changesets) do
+ {:ok, users} ->
+ res =
+ users
+ |> Map.values()
+ |> Enum.map(fn user ->
+ {:ok, user} = User.post_register_action(user)
+
+ user
+ end)
+ |> Enum.map(&AccountView.render("created.json", %{user: &1}))
+
+ ModerationLog.insert_log(%{
+ actor: admin,
+ subjects: Map.values(users),
+ action: "create"
+ })
+
+ json(conn, res)
+
+ {:error, id, changeset, _} ->
+ res =
+ Enum.map(changesets.operations, fn
+ {current_id, {:changeset, _current_changeset, _}} when current_id == id ->
+ AccountView.render("create-error.json", %{changeset: changeset})
+
+ {_, {:changeset, current_changeset, _}} ->
+ AccountView.render("create-error.json", %{changeset: current_changeset})
+ end)
+
+ conn
+ |> put_status(:conflict)
+ |> json(res)
+ end
+ end
+
+ def show(%{assigns: %{user: admin}} = conn, %{"nickname" => nickname}) do
+ with %User{} = user <- User.get_cached_by_nickname_or_id(nickname, for: admin) do
+ conn
+ |> put_view(AccountView)
+ |> render("show.json", %{user: user})
+ else
+ _ -> {:error, :not_found}
+ end
+ end
+
+ def toggle_activation(%{assigns: %{user: admin}} = conn, %{"nickname" => nickname}) do
+ user = User.get_cached_by_nickname(nickname)
+
+ {:ok, updated_user} = User.deactivate(user, !user.deactivated)
+
+ action = if user.deactivated, do: "activate", else: "deactivate"
+
+ ModerationLog.insert_log(%{
+ actor: admin,
+ subject: [user],
+ action: action
+ })
+
+ conn
+ |> put_view(AccountView)
+ |> render("show.json", %{user: updated_user})
+ end
+
+ def activate(%{assigns: %{user: admin}} = conn, %{"nicknames" => nicknames}) do
+ users = Enum.map(nicknames, &User.get_cached_by_nickname/1)
+ {:ok, updated_users} = User.deactivate(users, false)
+
+ ModerationLog.insert_log(%{
+ actor: admin,
+ subject: users,
+ action: "activate"
+ })
+
+ conn
+ |> put_view(AccountView)
+ |> render("index.json", %{users: Keyword.values(updated_users)})
+ end
+
+ def deactivate(%{assigns: %{user: admin}} = conn, %{"nicknames" => nicknames}) do
+ users = Enum.map(nicknames, &User.get_cached_by_nickname/1)
+ {:ok, updated_users} = User.deactivate(users, true)
+
+ ModerationLog.insert_log(%{
+ actor: admin,
+ subject: users,
+ action: "deactivate"
+ })
+
+ conn
+ |> put_view(AccountView)
+ |> render("index.json", %{users: Keyword.values(updated_users)})
+ end
+
+ def approve(%{assigns: %{user: admin}} = conn, %{"nicknames" => nicknames}) do
+ users = Enum.map(nicknames, &User.get_cached_by_nickname/1)
+ {:ok, updated_users} = User.approve(users)
+
+ ModerationLog.insert_log(%{
+ actor: admin,
+ subject: users,
+ action: "approve"
+ })
+
+ conn
+ |> put_view(AccountView)
+ |> render("index.json", %{users: updated_users})
+ end
+
+ def list(conn, params) do
+ {page, page_size} = page_params(params)
+ filters = maybe_parse_filters(params["filters"])
+
+ search_params =
+ %{
+ query: params["query"],
+ page: page,
+ page_size: page_size,
+ tags: params["tags"],
+ name: params["name"],
+ email: params["email"],
+ actor_types: params["actor_types"]
+ }
+ |> Map.merge(filters)
+
+ with {:ok, users, count} <- Search.user(search_params) do
+ json(
+ conn,
+ AccountView.render("index.json",
+ users: users,
+ count: count,
+ page_size: page_size
+ )
+ )
+ end
+ end
+
+ @filters ~w(local external active deactivated need_approval unconfirmed is_admin is_moderator)
+
+ @spec maybe_parse_filters(String.t()) :: %{required(String.t()) => true} | %{}
+ defp maybe_parse_filters(filters) when is_nil(filters) or filters == "", do: %{}
+
+ defp maybe_parse_filters(filters) do
+ filters
+ |> String.split(",")
+ |> Enum.filter(&Enum.member?(@filters, &1))
+ |> Map.new(&{String.to_existing_atom(&1), true})
+ end
+
+ defp page_params(params) do
+ {
+ fetch_integer_param(params, "page", 1),
+ fetch_integer_param(params, "page_size", @users_page_size)
+ }
+ end
+end
diff --git a/lib/pleroma/web/admin_api/views/account_view.ex b/lib/pleroma/web/admin_api/views/account_view.ex
index 9c477feab..8bac24d3e 100644
--- a/lib/pleroma/web/admin_api/views/account_view.ex
+++ b/lib/pleroma/web/admin_api/views/account_view.ex
@@ -39,7 +39,7 @@ def render("credentials.json", %{user: user, for: for_user}) do
:fields,
:name,
:nickname,
- :locked,
+ :is_locked,
:no_rich_text,
:default_scope,
:hide_follows,
@@ -52,7 +52,7 @@ def render("credentials.json", %{user: user, for: for_user}) do
:skip_thread_containment,
:pleroma_settings_store,
:raw_fields,
- :discoverable,
+ :is_discoverable,
:actor_type
])
|> Map.merge(%{
diff --git a/lib/pleroma/web/admin_api/views/chat_view.ex b/lib/pleroma/web/admin_api/views/chat_view.ex
new file mode 100644
index 000000000..847df1423
--- /dev/null
+++ b/lib/pleroma/web/admin_api/views/chat_view.ex
@@ -0,0 +1,30 @@
+# Pleroma: A lightweight social networking server
+# Copyright © 2017-2020 Pleroma Authors
+# SPDX-License-Identifier: AGPL-3.0-only
+
+defmodule Pleroma.Web.AdminAPI.ChatView do
+ use Pleroma.Web, :view
+
+ alias Pleroma.Chat
+ alias Pleroma.User
+ alias Pleroma.Web.MastodonAPI
+ alias Pleroma.Web.PleromaAPI
+
+ def render("index.json", %{chats: chats} = opts) do
+ render_many(chats, __MODULE__, "show.json", Map.delete(opts, :chats))
+ end
+
+ def render("show.json", %{chat: %Chat{user_id: user_id}} = opts) do
+ user = User.get_by_id(user_id)
+ sender = MastodonAPI.AccountView.render("show.json", user: user, skip_visibility_check: true)
+
+ serialized_chat = PleromaAPI.ChatView.render("show.json", opts)
+
+ serialized_chat
+ |> Map.put(:sender, sender)
+ |> Map.put(:receiver, serialized_chat[:account])
+ |> Map.delete(:account)
+ end
+
+ def render(view, opts), do: PleromaAPI.ChatView.render(view, opts)
+end
diff --git a/lib/pleroma/web/admin_api/views/report_view.ex b/lib/pleroma/web/admin_api/views/report_view.ex
index 773f798fe..535556370 100644
--- a/lib/pleroma/web/admin_api/views/report_view.ex
+++ b/lib/pleroma/web/admin_api/views/report_view.ex
@@ -52,7 +52,7 @@ def render("show.json", %{report: report, user: user, account: account, statuses
end
def render("index_notes.json", %{notes: notes}) when is_list(notes) do
- Enum.map(notes, &render(__MODULE__, "show_note.json", &1))
+ Enum.map(notes, &render(__MODULE__, "show_note.json", Map.from_struct(&1)))
end
def render("index_notes.json", _), do: []
diff --git a/lib/pleroma/web/admin_api/views/status_view.ex b/lib/pleroma/web/admin_api/views/status_view.ex
index 500800be2..6042a22b6 100644
--- a/lib/pleroma/web/admin_api/views/status_view.ex
+++ b/lib/pleroma/web/admin_api/views/status_view.ex
@@ -8,6 +8,7 @@ defmodule Pleroma.Web.AdminAPI.StatusView do
require Pleroma.Constants
alias Pleroma.Web.AdminAPI
+ alias Pleroma.Web.CommonAPI
alias Pleroma.Web.MastodonAPI
defdelegate merge_account_views(user), to: AdminAPI.AccountView
@@ -17,7 +18,7 @@ def render("index.json", opts) do
end
def render("show.json", %{activity: %{data: %{"object" => _object}} = activity} = opts) do
- user = MastodonAPI.StatusView.get_user(activity.data["actor"])
+ user = CommonAPI.get_user(activity.data["actor"])
MastodonAPI.StatusView.render("show.json", opts)
|> Map.merge(%{account: merge_account_views(user)})
diff --git a/lib/pleroma/web/api_spec.ex b/lib/pleroma/web/api_spec.ex
index 79fd5f871..93a5273e3 100644
--- a/lib/pleroma/web/api_spec.ex
+++ b/lib/pleroma/web/api_spec.ex
@@ -13,10 +13,15 @@ defmodule Pleroma.Web.ApiSpec do
@impl OpenApi
def spec do
%OpenApi{
- servers: [
- # Populate the Server info from a phoenix endpoint
- OpenApiSpex.Server.from_endpoint(Endpoint)
- ],
+ servers:
+ if Phoenix.Endpoint.server?(:pleroma, Endpoint) do
+ [
+ # Populate the Server info from a phoenix endpoint
+ OpenApiSpex.Server.from_endpoint(Endpoint)
+ ]
+ else
+ []
+ end,
info: %OpenApiSpex.Info{
title: "Pleroma",
description: Application.spec(:pleroma, :description) |> to_string(),
diff --git a/lib/pleroma/web/api_spec/cast_and_validate.ex b/lib/pleroma/web/api_spec/cast_and_validate.ex
index fbfc27d6f..6d1a7ebbc 100644
--- a/lib/pleroma/web/api_spec/cast_and_validate.ex
+++ b/lib/pleroma/web/api_spec/cast_and_validate.ex
@@ -115,6 +115,10 @@ defp cast_and_validate(spec, operation, conn, content_type, false = _strict) do
%{reason: :unexpected_field, name: name, path: [name]}, params ->
Map.delete(params, name)
+ # Filter out empty params
+ %{reason: :invalid_type, path: [name_atom], value: ""}, params ->
+ Map.delete(params, to_string(name_atom))
+
%{reason: :invalid_enum, name: nil, path: path, value: value}, params ->
path = path |> Enum.reverse() |> tl() |> Enum.reverse() |> list_items_to_string()
update_in(params, path, &List.delete(&1, value))
diff --git a/lib/pleroma/web/api_spec/helpers.ex b/lib/pleroma/web/api_spec/helpers.ex
index 2a7f1a706..34de2ed57 100644
--- a/lib/pleroma/web/api_spec/helpers.ex
+++ b/lib/pleroma/web/api_spec/helpers.ex
@@ -72,7 +72,11 @@ def empty_object_response do
end
def empty_array_response do
- Operation.response("Empty array", "application/json", %Schema{type: :array, example: []})
+ Operation.response("Empty array", "application/json", %Schema{
+ type: :array,
+ items: %Schema{type: :object, example: %{}},
+ example: []
+ })
end
def no_content_response do
diff --git a/lib/pleroma/web/api_spec/operations/account_operation.ex b/lib/pleroma/web/api_spec/operations/account_operation.ex
index aaebc9b5c..4934b7788 100644
--- a/lib/pleroma/web/api_spec/operations/account_operation.ex
+++ b/lib/pleroma/web/api_spec/operations/account_operation.ex
@@ -335,6 +335,7 @@ def mutes_operation do
operationId: "AccountController.mutes",
description: "Accounts the user has muted.",
security: [%{"oAuth" => ["follow", "read:mutes"]}],
+ parameters: pagination_params(),
responses: %{
200 => Operation.response("Accounts", "application/json", array_of_accounts())
}
@@ -348,6 +349,7 @@ def blocks_operation do
operationId: "AccountController.blocks",
description: "View your blocks. See also accounts/:id/{block,unblock}",
security: [%{"oAuth" => ["read:blocks"]}],
+ parameters: pagination_params(),
responses: %{
200 => Operation.response("Accounts", "application/json", array_of_accounts())
}
@@ -372,6 +374,10 @@ def identity_proofs_operation do
tags: ["accounts"],
summary: "Identity proofs",
operationId: "AccountController.identity_proofs",
+ # Validators complains about unused path params otherwise
+ parameters: [
+ %Reference{"$ref": "#/components/parameters/accountIdOrNickname"}
+ ],
description: "Not implemented",
responses: %{
200 => empty_array_response()
@@ -469,7 +475,6 @@ defp create_response do
identifier: %Schema{type: :string},
message: %Schema{type: :string}
},
- required: [],
# Note: example of successful registration with failed login response:
# example: %{
# "identifier" => "missing_confirmed_email",
@@ -530,7 +535,7 @@ defp update_credentials_request do
nullable: true,
oneOf: [
%Schema{type: :array, items: attribute_field()},
- %Schema{type: :object, additionalProperties: %Schema{type: attribute_field()}}
+ %Schema{type: :object, additionalProperties: attribute_field()}
]
},
# NOTE: `source` field is not supported
diff --git a/lib/pleroma/web/api_spec/operations/admin/chat_operation.ex b/lib/pleroma/web/api_spec/operations/admin/chat_operation.ex
new file mode 100644
index 000000000..d3e5dfc1c
--- /dev/null
+++ b/lib/pleroma/web/api_spec/operations/admin/chat_operation.ex
@@ -0,0 +1,96 @@
+# Pleroma: A lightweight social networking server
+# Copyright © 2017-2020 Pleroma Authors
+# SPDX-License-Identifier: AGPL-3.0-only
+
+defmodule Pleroma.Web.ApiSpec.Admin.ChatOperation do
+ alias OpenApiSpex.Operation
+ alias Pleroma.Web.ApiSpec.Schemas.Chat
+ alias Pleroma.Web.ApiSpec.Schemas.ChatMessage
+
+ import Pleroma.Web.ApiSpec.Helpers
+
+ def open_api_operation(action) do
+ operation = String.to_existing_atom("#{action}_operation")
+ apply(__MODULE__, operation, [])
+ end
+
+ def delete_message_operation do
+ %Operation{
+ tags: ["admin", "chat"],
+ summary: "Delete an individual chat message",
+ operationId: "AdminAPI.ChatController.delete_message",
+ parameters: [
+ Operation.parameter(:id, :path, :string, "The ID of the Chat"),
+ Operation.parameter(:message_id, :path, :string, "The ID of the message")
+ ],
+ responses: %{
+ 200 =>
+ Operation.response(
+ "The deleted ChatMessage",
+ "application/json",
+ ChatMessage
+ )
+ },
+ security: [
+ %{
+ "oAuth" => ["write:chats"]
+ }
+ ]
+ }
+ end
+
+ def messages_operation do
+ %Operation{
+ tags: ["admin", "chat"],
+ summary: "Get the most recent messages of the chat",
+ operationId: "AdminAPI.ChatController.messages",
+ parameters:
+ [Operation.parameter(:id, :path, :string, "The ID of the Chat")] ++
+ pagination_params(),
+ responses: %{
+ 200 =>
+ Operation.response(
+ "The messages in the chat",
+ "application/json",
+ Pleroma.Web.ApiSpec.ChatOperation.chat_messages_response()
+ )
+ },
+ security: [
+ %{
+ "oAuth" => ["read:chats"]
+ }
+ ]
+ }
+ end
+
+ def show_operation do
+ %Operation{
+ tags: ["chat"],
+ summary: "Create a chat",
+ operationId: "AdminAPI.ChatController.show",
+ parameters: [
+ Operation.parameter(
+ :id,
+ :path,
+ :string,
+ "The id of the chat",
+ required: true,
+ example: "1234"
+ )
+ ],
+ responses: %{
+ 200 =>
+ Operation.response(
+ "The existing chat",
+ "application/json",
+ Chat
+ )
+ },
+ security: [
+ %{
+ "oAuth" => ["read"]
+ }
+ ]
+ }
+ end
+end
diff --git a/lib/pleroma/web/api_spec/operations/admin/instance_document_operation.ex b/lib/pleroma/web/api_spec/operations/admin/instance_document_operation.ex
new file mode 100644
index 000000000..a120ff4e8
--- /dev/null
+++ b/lib/pleroma/web/api_spec/operations/admin/instance_document_operation.ex
@@ -0,0 +1,115 @@
+# Pleroma: A lightweight social networking server
+# Copyright © 2017-2020 Pleroma Authors
+# SPDX-License-Identifier: AGPL-3.0-only
+
+defmodule Pleroma.Web.ApiSpec.Admin.InstanceDocumentOperation do
+ alias OpenApiSpex.Operation
+ alias OpenApiSpex.Schema
+ alias Pleroma.Web.ApiSpec.Helpers
+ alias Pleroma.Web.ApiSpec.Schemas.ApiError
+
+ def open_api_operation(action) do
+ operation = String.to_existing_atom("#{action}_operation")
+ apply(__MODULE__, operation, [])
+ end
+
+ def show_operation do
+ %Operation{
+ tags: ["Admin", "InstanceDocument"],
+ summary: "Get the instance document",
+ operationId: "AdminAPI.InstanceDocumentController.show",
+ security: [%{"oAuth" => ["read"]}],
+ parameters: [
+ Operation.parameter(:name, :path, %Schema{type: :string}, "The document name",
+ required: true
+ )
+ | Helpers.admin_api_params()
+ ],
+ responses: %{
+ 200 => document_content(),
+ 400 => Operation.response("Bad Request", "application/json", ApiError),
+ 403 => Operation.response("Forbidden", "application/json", ApiError),
+ 404 => Operation.response("Not Found", "application/json", ApiError)
+ }
+ }
+ end
+
+ def update_operation do
+ %Operation{
+ tags: ["Admin", "InstanceDocument"],
+ summary: "Update the instance document",
+ operationId: "AdminAPI.InstanceDocumentController.update",
+ security: [%{"oAuth" => ["write"]}],
+ requestBody: Helpers.request_body("Parameters", update_request()),
+ parameters: [
+ Operation.parameter(:name, :path, %Schema{type: :string}, "The document name",
+ required: true
+ )
+ | Helpers.admin_api_params()
+ ],
+ responses: %{
+ 200 => Operation.response("InstanceDocument", "application/json", instance_document()),
+ 400 => Operation.response("Bad Request", "application/json", ApiError),
+ 403 => Operation.response("Forbidden", "application/json", ApiError),
+ 404 => Operation.response("Not Found", "application/json", ApiError)
+ }
+ }
+ end
+
+ defp update_request do
+ %Schema{
+ title: "UpdateRequest",
+ description: "POST body for uploading the file",
+ type: :object,
+ required: [:file],
+ properties: %{
+ file: %Schema{
+ type: :string,
+ format: :binary,
+ description: "The file to be uploaded, using multipart form data."
+ }
+ }
+ }
+ end
+
+ def delete_operation do
+ %Operation{
+ tags: ["Admin", "InstanceDocument"],
+ summary: "Get the instance document",
+ operationId: "AdminAPI.InstanceDocumentController.delete",
+ security: [%{"oAuth" => ["write"]}],
+ parameters: [
+ Operation.parameter(:name, :path, %Schema{type: :string}, "The document name",
+ required: true
+ )
+ | Helpers.admin_api_params()
+ ],
+ responses: %{
+ 200 => Operation.response("InstanceDocument", "application/json", instance_document()),
+ 400 => Operation.response("Bad Request", "application/json", ApiError),
+ 403 => Operation.response("Forbidden", "application/json", ApiError),
+ 404 => Operation.response("Not Found", "application/json", ApiError)
+ }
+ }
+ end
+
+ defp instance_document do
+ %Schema{
+ title: "InstanceDocument",
+ type: :object,
+ properties: %{
+ url: %Schema{type: :string}
+ },
+ example: %{
+ "url" => "https://example.com/static/terms-of-service.html"
+ }
+ }
+ end
+
+ defp document_content do
+ Operation.response("InstanceDocumentContent", "text/html", %Schema{
+ type: :string,
+ example: "Instance panel
"
+ })
+ end
+end
diff --git a/lib/pleroma/web/api_spec/operations/admin/oauth_app_operation.ex b/lib/pleroma/web/api_spec/operations/admin/o_auth_app_operation.ex
similarity index 100%
rename from lib/pleroma/web/api_spec/operations/admin/oauth_app_operation.ex
rename to lib/pleroma/web/api_spec/operations/admin/o_auth_app_operation.ex
diff --git a/lib/pleroma/web/api_spec/operations/admin/relay_operation.ex b/lib/pleroma/web/api_spec/operations/admin/relay_operation.ex
index e06b2d164..f754bb9f5 100644
--- a/lib/pleroma/web/api_spec/operations/admin/relay_operation.ex
+++ b/lib/pleroma/web/api_spec/operations/admin/relay_operation.ex
@@ -56,7 +56,7 @@ def unfollow_operation do
operationId: "AdminAPI.RelayController.unfollow",
security: [%{"oAuth" => ["write:follows"]}],
parameters: admin_api_params(),
- requestBody: request_body("Parameters", relay_url()),
+ requestBody: request_body("Parameters", relay_unfollow()),
responses: %{
200 =>
Operation.response("Status", "application/json", %Schema{
@@ -91,4 +91,14 @@ defp relay_url do
}
}
end
+
+ defp relay_unfollow do
+ %Schema{
+ type: :object,
+ properties: %{
+ relay_url: %Schema{type: :string, format: :uri},
+ force: %Schema{type: :boolean, default: false}
+ }
+ }
+ end
end
diff --git a/lib/pleroma/web/api_spec/operations/chat_operation.ex b/lib/pleroma/web/api_spec/operations/chat_operation.ex
index b1a0d26ab..560b81f17 100644
--- a/lib/pleroma/web/api_spec/operations/chat_operation.ex
+++ b/lib/pleroma/web/api_spec/operations/chat_operation.ex
@@ -6,6 +6,7 @@ defmodule Pleroma.Web.ApiSpec.ChatOperation do
alias OpenApiSpex.Operation
alias OpenApiSpex.Schema
alias Pleroma.Web.ApiSpec.Schemas.ApiError
+ alias Pleroma.Web.ApiSpec.Schemas.BooleanLike
alias Pleroma.Web.ApiSpec.Schemas.Chat
alias Pleroma.Web.ApiSpec.Schemas.ChatMessage
@@ -132,7 +133,10 @@ def index_operation do
tags: ["chat"],
summary: "Get a list of chats that you participated in",
operationId: "ChatController.index",
- parameters: pagination_params(),
+ parameters: [
+ Operation.parameter(:with_muted, :query, BooleanLike, "Include chats from muted users")
+ | pagination_params()
+ ],
responses: %{
200 => Operation.response("The chats of the user", "application/json", chats_response())
},
@@ -158,7 +162,8 @@ def messages_operation do
"The messages in the chat",
"application/json",
chat_messages_response()
- )
+ ),
+ 404 => Operation.response("Not Found", "application/json", ApiError)
},
security: [
%{
@@ -184,7 +189,8 @@ def post_chat_message_operation do
"application/json",
ChatMessage
),
- 400 => Operation.response("Bad Request", "application/json", ApiError)
+ 400 => Operation.response("Bad Request", "application/json", ApiError),
+ 422 => Operation.response("MRF Rejection", "application/json", ApiError)
},
security: [
%{
diff --git a/lib/pleroma/web/api_spec/operations/custom_emoji_operation.ex b/lib/pleroma/web/api_spec/operations/custom_emoji_operation.ex
index 2f812ac77..5ff263ceb 100644
--- a/lib/pleroma/web/api_spec/operations/custom_emoji_operation.ex
+++ b/lib/pleroma/web/api_spec/operations/custom_emoji_operation.ex
@@ -69,7 +69,7 @@ defp custom_emoji do
type: :object,
properties: %{
category: %Schema{type: :string},
- tags: %Schema{type: :array}
+ tags: %Schema{type: :array, items: %Schema{type: :string}}
}
}
],
diff --git a/lib/pleroma/web/api_spec/operations/emoji_reaction_operation.ex b/lib/pleroma/web/api_spec/operations/emoji_reaction_operation.ex
index 1a49fece0..745d41f88 100644
--- a/lib/pleroma/web/api_spec/operations/emoji_reaction_operation.ex
+++ b/lib/pleroma/web/api_spec/operations/emoji_reaction_operation.ex
@@ -23,7 +23,7 @@ def index_operation do
parameters: [
Operation.parameter(:id, :path, FlakeID, "Status ID", required: true),
Operation.parameter(:emoji, :path, :string, "Filter by a single unicode emoji",
- required: false
+ required: nil
)
],
security: [%{"oAuth" => ["read:statuses"]}],
diff --git a/lib/pleroma/web/api_spec/operations/list_operation.ex b/lib/pleroma/web/api_spec/operations/list_operation.ex
index 15039052e..f6e73968a 100644
--- a/lib/pleroma/web/api_spec/operations/list_operation.ex
+++ b/lib/pleroma/web/api_spec/operations/list_operation.ex
@@ -187,8 +187,7 @@ defp add_remove_accounts_request(required) when is_boolean(required) do
type: :object,
properties: %{
account_ids: %Schema{type: :array, description: "Array of account IDs", items: FlakeID}
- },
- required: required && [:account_ids]
+ }
},
required: required
)
diff --git a/lib/pleroma/web/api_spec/operations/pleroma_backup_operation.ex b/lib/pleroma/web/api_spec/operations/pleroma_backup_operation.ex
new file mode 100644
index 000000000..6993794db
--- /dev/null
+++ b/lib/pleroma/web/api_spec/operations/pleroma_backup_operation.ex
@@ -0,0 +1,79 @@
+# Pleroma: A lightweight social networking server
+# Copyright © 2017-2020 Pleroma Authors
+# SPDX-License-Identifier: AGPL-3.0-only
+
+defmodule Pleroma.Web.ApiSpec.PleromaBackupOperation do
+ alias OpenApiSpex.Operation
+ alias OpenApiSpex.Schema
+ alias Pleroma.Web.ApiSpec.Schemas.ApiError
+
+ def open_api_operation(action) do
+ operation = String.to_existing_atom("#{action}_operation")
+ apply(__MODULE__, operation, [])
+ end
+
+ def index_operation do
+ %Operation{
+ tags: ["Backups"],
+ summary: "List backups",
+ security: [%{"oAuth" => ["read:account"]}],
+ operationId: "PleromaAPI.BackupController.index",
+ responses: %{
+ 200 =>
+ Operation.response(
+ "An array of backups",
+ "application/json",
+ %Schema{
+ type: :array,
+ items: backup()
+ }
+ ),
+ 400 => Operation.response("Bad Request", "application/json", ApiError)
+ }
+ }
+ end
+
+ def create_operation do
+ %Operation{
+ tags: ["Backups"],
+ summary: "Create a backup",
+ security: [%{"oAuth" => ["read:account"]}],
+ operationId: "PleromaAPI.BackupController.create",
+ responses: %{
+ 200 =>
+ Operation.response(
+ "An array of backups",
+ "application/json",
+ %Schema{
+ type: :array,
+ items: backup()
+ }
+ ),
+ 400 => Operation.response("Bad Request", "application/json", ApiError)
+ }
+ }
+ end
+
+ defp backup do
+ %Schema{
+ title: "Backup",
+ description: "Response schema for a backup",
+ type: :object,
+ properties: %{
+ inserted_at: %Schema{type: :string, format: :"date-time"},
+ content_type: %Schema{type: :string},
+ file_name: %Schema{type: :string},
+ file_size: %Schema{type: :integer},
+ processed: %Schema{type: :boolean}
+ },
+ example: %{
+ "content_type" => "application/zip",
+ "file_name" =>
+ "https://cofe.fe:4000/media/backups/archive-foobar-20200908T164207-Yr7vuT5Wycv-sN3kSN2iJ0k-9pMo60j9qmvRCdDqIew.zip",
+ "file_size" => 4105,
+ "inserted_at" => "2020-09-08T16:42:07.000Z",
+ "processed" => true
+ }
+ }
+ end
+end
diff --git a/lib/pleroma/web/api_spec/operations/pleroma_emoji_file_operation.ex b/lib/pleroma/web/api_spec/operations/pleroma_emoji_file_operation.ex
new file mode 100644
index 000000000..a56641426
--- /dev/null
+++ b/lib/pleroma/web/api_spec/operations/pleroma_emoji_file_operation.ex
@@ -0,0 +1,139 @@
+# Pleroma: A lightweight social networking server
+# Copyright © 2017-2020 Pleroma Authors
+# SPDX-License-Identifier: AGPL-3.0-only
+
+defmodule Pleroma.Web.ApiSpec.PleromaEmojiFileOperation do
+ alias OpenApiSpex.Operation
+ alias OpenApiSpex.Schema
+ alias Pleroma.Web.ApiSpec.Schemas.ApiError
+
+ import Pleroma.Web.ApiSpec.Helpers
+
+ def open_api_operation(action) do
+ operation = String.to_existing_atom("#{action}_operation")
+ apply(__MODULE__, operation, [])
+ end
+
+ def create_operation do
+ %Operation{
+ tags: ["Emoji Packs"],
+ summary: "Add new file to the pack",
+ operationId: "PleromaAPI.EmojiPackController.add_file",
+ security: [%{"oAuth" => ["write"]}],
+ requestBody: request_body("Parameters", create_request(), required: true),
+ parameters: [name_param()],
+ responses: %{
+ 200 => Operation.response("Files Object", "application/json", files_object()),
+ 422 => Operation.response("Unprocessable Entity", "application/json", ApiError),
+ 404 => Operation.response("Not Found", "application/json", ApiError),
+ 400 => Operation.response("Bad Request", "application/json", ApiError),
+ 409 => Operation.response("Conflict", "application/json", ApiError)
+ }
+ }
+ end
+
+ defp create_request do
+ %Schema{
+ type: :object,
+ required: [:file],
+ properties: %{
+ file: %Schema{
+ description:
+ "File needs to be uploaded with the multipart request or link to remote file",
+ anyOf: [
+ %Schema{type: :string, format: :binary},
+ %Schema{type: :string, format: :uri}
+ ]
+ },
+ shortcode: %Schema{
+ type: :string,
+ description:
+ "Shortcode for new emoji, must be unique for all emoji. If not sended, shortcode will be taken from original filename."
+ },
+ filename: %Schema{
+ type: :string,
+ description:
+ "New emoji file name. If not specified will be taken from original filename."
+ }
+ }
+ }
+ end
+
+ def update_operation do
+ %Operation{
+ tags: ["Emoji Packs"],
+ summary: "Add new file to the pack",
+ operationId: "PleromaAPI.EmojiPackController.update_file",
+ security: [%{"oAuth" => ["write"]}],
+ requestBody: request_body("Parameters", update_request(), required: true),
+ parameters: [name_param()],
+ responses: %{
+ 200 => Operation.response("Files Object", "application/json", files_object()),
+ 404 => Operation.response("Not Found", "application/json", ApiError),
+ 400 => Operation.response("Bad Request", "application/json", ApiError),
+ 409 => Operation.response("Conflict", "application/json", ApiError),
+ 422 => Operation.response("Unprocessable Entity", "application/json", ApiError)
+ }
+ }
+ end
+
+ defp update_request do
+ %Schema{
+ type: :object,
+ required: [:shortcode, :new_shortcode, :new_filename],
+ properties: %{
+ shortcode: %Schema{
+ type: :string,
+ description: "Emoji file shortcode"
+ },
+ new_shortcode: %Schema{
+ type: :string,
+ description: "New emoji file shortcode"
+ },
+ new_filename: %Schema{
+ type: :string,
+ description: "New filename for emoji file"
+ },
+ force: %Schema{
+ type: :boolean,
+ description: "With true value to overwrite existing emoji with new shortcode",
+ default: false
+ }
+ }
+ }
+ end
+
+ def delete_operation do
+ %Operation{
+ tags: ["Emoji Packs"],
+ summary: "Delete emoji file from pack",
+ operationId: "PleromaAPI.EmojiPackController.delete_file",
+ security: [%{"oAuth" => ["write"]}],
+ parameters: [
+ name_param(),
+ Operation.parameter(:shortcode, :query, :string, "File shortcode",
+ example: "cofe",
+ required: true
+ )
+ ],
+ responses: %{
+ 200 => Operation.response("Files Object", "application/json", files_object()),
+ 400 => Operation.response("Bad Request", "application/json", ApiError),
+ 404 => Operation.response("Not Found", "application/json", ApiError),
+ 422 => Operation.response("Unprocessable Entity", "application/json", ApiError)
+ }
+ }
+ end
+
+ defp name_param do
+ Operation.parameter(:name, :query, :string, "Pack Name", example: "cofe", required: true)
+ end
+
+ defp files_object do
+ %Schema{
+ type: :object,
+ additionalProperties: %Schema{type: :string},
+ description: "Object with emoji names as keys and filenames as values"
+ }
+ end
+end
diff --git a/lib/pleroma/web/api_spec/operations/pleroma_emoji_pack_operation.ex b/lib/pleroma/web/api_spec/operations/pleroma_emoji_pack_operation.ex
index b2b4f8713..79f52dcb3 100644
--- a/lib/pleroma/web/api_spec/operations/pleroma_emoji_pack_operation.ex
+++ b/lib/pleroma/web/api_spec/operations/pleroma_emoji_pack_operation.ex
@@ -19,7 +19,21 @@ def remote_operation do
tags: ["Emoji Packs"],
summary: "Make request to another instance for emoji packs list",
security: [%{"oAuth" => ["write"]}],
- parameters: [url_param()],
+ parameters: [
+ url_param(),
+ Operation.parameter(
+ :page,
+ :query,
+ %Schema{type: :integer, default: 1},
+ "Page"
+ ),
+ Operation.parameter(
+ :page_size,
+ :query,
+ %Schema{type: :integer, default: 30},
+ "Number of emoji to return"
+ )
+ ],
operationId: "PleromaAPI.EmojiPackController.remote",
responses: %{
200 => emoji_packs_response(),
@@ -175,111 +189,6 @@ def update_operation do
}
end
- def add_file_operation do
- %Operation{
- tags: ["Emoji Packs"],
- summary: "Add new file to the pack",
- operationId: "PleromaAPI.EmojiPackController.add_file",
- security: [%{"oAuth" => ["write"]}],
- requestBody: request_body("Parameters", add_file_request(), required: true),
- parameters: [name_param()],
- responses: %{
- 200 => Operation.response("Files Object", "application/json", files_object()),
- 400 => Operation.response("Bad Request", "application/json", ApiError),
- 409 => Operation.response("Conflict", "application/json", ApiError)
- }
- }
- end
-
- defp add_file_request do
- %Schema{
- type: :object,
- required: [:file],
- properties: %{
- file: %Schema{
- description:
- "File needs to be uploaded with the multipart request or link to remote file",
- anyOf: [
- %Schema{type: :string, format: :binary},
- %Schema{type: :string, format: :uri}
- ]
- },
- shortcode: %Schema{
- type: :string,
- description:
- "Shortcode for new emoji, must be unique for all emoji. If not sended, shortcode will be taken from original filename."
- },
- filename: %Schema{
- type: :string,
- description:
- "New emoji file name. If not specified will be taken from original filename."
- }
- }
- }
- end
-
- def update_file_operation do
- %Operation{
- tags: ["Emoji Packs"],
- summary: "Add new file to the pack",
- operationId: "PleromaAPI.EmojiPackController.update_file",
- security: [%{"oAuth" => ["write"]}],
- requestBody: request_body("Parameters", update_file_request(), required: true),
- parameters: [name_param()],
- responses: %{
- 200 => Operation.response("Files Object", "application/json", files_object()),
- 400 => Operation.response("Bad Request", "application/json", ApiError),
- 409 => Operation.response("Conflict", "application/json", ApiError)
- }
- }
- end
-
- defp update_file_request do
- %Schema{
- type: :object,
- required: [:shortcode, :new_shortcode, :new_filename],
- properties: %{
- shortcode: %Schema{
- type: :string,
- description: "Emoji file shortcode"
- },
- new_shortcode: %Schema{
- type: :string,
- description: "New emoji file shortcode"
- },
- new_filename: %Schema{
- type: :string,
- description: "New filename for emoji file"
- },
- force: %Schema{
- type: :boolean,
- description: "With true value to overwrite existing emoji with new shortcode",
- default: false
- }
- }
- }
- end
-
- def delete_file_operation do
- %Operation{
- tags: ["Emoji Packs"],
- summary: "Delete emoji file from pack",
- operationId: "PleromaAPI.EmojiPackController.delete_file",
- security: [%{"oAuth" => ["write"]}],
- parameters: [
- name_param(),
- Operation.parameter(:shortcode, :query, :string, "File shortcode",
- example: "cofe",
- required: true
- )
- ],
- responses: %{
- 200 => Operation.response("Files Object", "application/json", files_object()),
- 400 => Operation.response("Bad Request", "application/json", ApiError)
- }
- }
- end
-
def import_from_filesystem_operation do
%Operation{
tags: ["Emoji Packs"],
@@ -297,7 +206,7 @@ def import_from_filesystem_operation do
end
defp name_param do
- Operation.parameter(:name, :path, :string, "Pack Name", example: "cofe", required: true)
+ Operation.parameter(:name, :query, :string, "Pack Name", example: "cofe", required: true)
end
defp url_param do
diff --git a/lib/pleroma/web/api_spec/operations/pleroma_instances_operation.ex b/lib/pleroma/web/api_spec/operations/pleroma_instances_operation.ex
new file mode 100644
index 000000000..2c455b0df
--- /dev/null
+++ b/lib/pleroma/web/api_spec/operations/pleroma_instances_operation.ex
@@ -0,0 +1,40 @@
+# Pleroma: A lightweight social networking server
+# Copyright © 2017-2020 Pleroma Authors
+# SPDX-License-Identifier: AGPL-3.0-only
+
+defmodule Pleroma.Web.ApiSpec.PleromaInstancesOperation do
+ alias OpenApiSpex.Operation
+ alias OpenApiSpex.Schema
+
+ def open_api_operation(action) do
+ operation = String.to_existing_atom("#{action}_operation")
+ apply(__MODULE__, operation, [])
+ end
+
+ def show_operation do
+ %Operation{
+ tags: ["PleromaInstances"],
+ summary: "Instances federation status",
+ description: "Information about instances deemed unreachable by the server",
+ operationId: "PleromaInstances.show",
+ responses: %{
+ 200 => Operation.response("PleromaInstances", "application/json", pleroma_instances())
+ }
+ }
+ end
+
+ def pleroma_instances do
+ %Schema{
+ type: :object,
+ properties: %{
+ unreachable: %Schema{
+ type: :object,
+ properties: %{hostname: %Schema{type: :string, format: :"date-time"}}
+ }
+ },
+ example: %{
+ "unreachable" => %{"consistently-unreachable.name" => "2020-10-14 22:07:58.216473"}
+ }
+ }
+ end
+end
diff --git a/lib/pleroma/web/api_spec/operations/status_operation.ex b/lib/pleroma/web/api_spec/operations/status_operation.ex
index 5bd4619d5..d7ebde6f6 100644
--- a/lib/pleroma/web/api_spec/operations/status_operation.ex
+++ b/lib/pleroma/web/api_spec/operations/status_operation.ex
@@ -55,7 +55,7 @@ def create_operation do
"application/json",
%Schema{oneOf: [Status, ScheduledStatus]}
),
- 422 => Operation.response("Bad Request", "application/json", ApiError)
+ 422 => Operation.response("Bad Request / MRF Rejection", "application/json", ApiError)
}
}
end
diff --git a/lib/pleroma/web/api_spec/operations/timeline_operation.ex b/lib/pleroma/web/api_spec/operations/timeline_operation.ex
index 8e19bace7..95720df9f 100644
--- a/lib/pleroma/web/api_spec/operations/timeline_operation.ex
+++ b/lib/pleroma/web/api_spec/operations/timeline_operation.ex
@@ -59,6 +59,7 @@ def public_operation do
security: [%{"oAuth" => ["read:statuses"]}],
parameters: [
local_param(),
+ instance_param(),
only_media_param(),
with_muted_param(),
exclude_visibilities_param(),
@@ -158,8 +159,17 @@ defp local_param do
)
end
+ defp instance_param do
+ Operation.parameter(
+ :instance,
+ :query,
+ %Schema{type: :string},
+ "Show only statuses from the given domain"
+ )
+ end
+
defp with_muted_param do
- Operation.parameter(:with_muted, :query, BooleanLike, "Includeactivities by muted users")
+ Operation.parameter(:with_muted, :query, BooleanLike, "Include activities by muted users")
end
defp exclude_visibilities_param do
diff --git a/lib/pleroma/web/api_spec/operations/user_import_operation.ex b/lib/pleroma/web/api_spec/operations/user_import_operation.ex
new file mode 100644
index 000000000..a50314fb7
--- /dev/null
+++ b/lib/pleroma/web/api_spec/operations/user_import_operation.ex
@@ -0,0 +1,80 @@
+# Pleroma: A lightweight social networking server
+# Copyright © 2017-2020 Pleroma Authors
+# SPDX-License-Identifier: AGPL-3.0-only
+
+defmodule Pleroma.Web.ApiSpec.UserImportOperation do
+ alias OpenApiSpex.Operation
+ alias OpenApiSpex.Schema
+ alias Pleroma.Web.ApiSpec.Schemas.ApiError
+
+ import Pleroma.Web.ApiSpec.Helpers
+
+ @spec open_api_operation(atom) :: Operation.t()
+ def open_api_operation(action) do
+ operation = String.to_existing_atom("#{action}_operation")
+ apply(__MODULE__, operation, [])
+ end
+
+ def follow_operation do
+ %Operation{
+ tags: ["follow_import"],
+ summary: "Imports your follows.",
+ operationId: "UserImportController.follow",
+ requestBody: request_body("Parameters", import_request(), required: true),
+ responses: %{
+ 200 => ok_response(),
+ 500 => Operation.response("Error", "application/json", ApiError)
+ },
+ security: [%{"oAuth" => ["write:follow"]}]
+ }
+ end
+
+ def blocks_operation do
+ %Operation{
+ tags: ["blocks_import"],
+ summary: "Imports your blocks.",
+ operationId: "UserImportController.blocks",
+ requestBody: request_body("Parameters", import_request(), required: true),
+ responses: %{
+ 200 => ok_response(),
+ 500 => Operation.response("Error", "application/json", ApiError)
+ },
+ security: [%{"oAuth" => ["write:blocks"]}]
+ }
+ end
+
+ def mutes_operation do
+ %Operation{
+ tags: ["mutes_import"],
+ summary: "Imports your mutes.",
+ operationId: "UserImportController.mutes",
+ requestBody: request_body("Parameters", import_request(), required: true),
+ responses: %{
+ 200 => ok_response(),
+ 500 => Operation.response("Error", "application/json", ApiError)
+ },
+ security: [%{"oAuth" => ["write:mutes"]}]
+ }
+ end
+
+ defp import_request do
+ %Schema{
+ type: :object,
+ required: [:list],
+ properties: %{
+ list: %Schema{
+ description:
+ "STRING or FILE containing a whitespace-separated list of accounts to import.",
+ anyOf: [
+ %Schema{type: :string, format: :binary},
+ %Schema{type: :string}
+ ]
+ }
+ }
+ }
+ end
+
+ defp ok_response do
+ Operation.response("Ok", "application/json", %Schema{type: :string, example: "ok"})
+ end
+end
diff --git a/lib/pleroma/web/api_spec/schemas/chat.ex b/lib/pleroma/web/api_spec/schemas/chat.ex
index b4986b734..65f908e33 100644
--- a/lib/pleroma/web/api_spec/schemas/chat.ex
+++ b/lib/pleroma/web/api_spec/schemas/chat.ex
@@ -50,7 +50,7 @@ defmodule Pleroma.Web.ApiSpec.Schemas.Chat do
"fields" => []
},
"statuses_count" => 1,
- "locked" => false,
+ "is_locked" => false,
"created_at" => "2020-04-16T13:40:15.000Z",
"display_name" => "lain",
"fields" => [],
diff --git a/lib/pleroma/web/api_spec/schemas/chat_message.ex b/lib/pleroma/web/api_spec/schemas/chat_message.ex
index bbf2a4427..9d2799618 100644
--- a/lib/pleroma/web/api_spec/schemas/chat_message.ex
+++ b/lib/pleroma/web/api_spec/schemas/chat_message.ex
@@ -4,6 +4,7 @@
defmodule Pleroma.Web.ApiSpec.Schemas.ChatMessage do
alias OpenApiSpex.Schema
+ alias Pleroma.Web.ApiSpec.Schemas.Emoji
require OpenApiSpex
@@ -18,7 +19,7 @@ defmodule Pleroma.Web.ApiSpec.Schemas.ChatMessage do
chat_id: %Schema{type: :string},
content: %Schema{type: :string, nullable: true},
created_at: %Schema{type: :string, format: :"date-time"},
- emojis: %Schema{type: :array},
+ emojis: %Schema{type: :array, items: Emoji},
attachment: %Schema{type: :object, nullable: true},
card: %Schema{
type: :object,
diff --git a/lib/pleroma/web/api_spec/schemas/poll.ex b/lib/pleroma/web/api_spec/schemas/poll.ex
index c62096db0..0dfa60b97 100644
--- a/lib/pleroma/web/api_spec/schemas/poll.ex
+++ b/lib/pleroma/web/api_spec/schemas/poll.ex
@@ -28,8 +28,11 @@ defmodule Pleroma.Web.ApiSpec.Schemas.Poll do
},
votes_count: %Schema{
type: :integer,
- nullable: true,
- description: "How many votes have been received. Number, or null if `multiple` is false."
+ description: "How many votes have been received. Number."
+ },
+ voters_count: %Schema{
+ type: :integer,
+ description: "How many unique accounts have voted. Number."
},
voted: %Schema{
type: :boolean,
@@ -61,7 +64,7 @@ defmodule Pleroma.Web.ApiSpec.Schemas.Poll do
expired: true,
multiple: false,
votes_count: 10,
- voters_count: nil,
+ voters_count: 10,
voted: true,
own_votes: [
1
diff --git a/lib/pleroma/web/api_spec/schemas/scheduled_status.ex b/lib/pleroma/web/api_spec/schemas/scheduled_status.ex
index 0520d0848..addefa9d3 100644
--- a/lib/pleroma/web/api_spec/schemas/scheduled_status.ex
+++ b/lib/pleroma/web/api_spec/schemas/scheduled_status.ex
@@ -27,9 +27,9 @@ defmodule Pleroma.Web.ApiSpec.Schemas.ScheduledStatus do
media_ids: %Schema{type: :array, nullable: true, items: %Schema{type: :string}},
sensitive: %Schema{type: :boolean, nullable: true},
spoiler_text: %Schema{type: :string, nullable: true},
- visibility: %Schema{type: VisibilityScope, nullable: true},
+ visibility: %Schema{allOf: [VisibilityScope], nullable: true},
scheduled_at: %Schema{type: :string, format: :"date-time", nullable: true},
- poll: %Schema{type: Poll, nullable: true},
+ poll: %Schema{allOf: [Poll], nullable: true},
in_reply_to_id: %Schema{type: :string, nullable: true}
}
}
diff --git a/lib/pleroma/web/api_spec/schemas/status.ex b/lib/pleroma/web/api_spec/schemas/status.ex
index 947e42890..e6890df2d 100644
--- a/lib/pleroma/web/api_spec/schemas/status.ex
+++ b/lib/pleroma/web/api_spec/schemas/status.ex
@@ -252,7 +252,7 @@ defmodule Pleroma.Web.ApiSpec.Schemas.Status do
"header" => "http://localhost:4001/images/banner.png",
"header_static" => "http://localhost:4001/images/banner.png",
"id" => "9toJCsKN7SmSf3aj5c",
- "locked" => false,
+ "is_locked" => false,
"note" => "Tester Number 6",
"pleroma" => %{
"background_image" => nil,
diff --git a/lib/pleroma/web/auth/pleroma_authenticator.ex b/lib/pleroma/web/auth/pleroma_authenticator.ex
index c611b3e09..d6d2a8d06 100644
--- a/lib/pleroma/web/auth/pleroma_authenticator.ex
+++ b/lib/pleroma/web/auth/pleroma_authenticator.ex
@@ -3,10 +3,10 @@
# SPDX-License-Identifier: AGPL-3.0-only
defmodule Pleroma.Web.Auth.PleromaAuthenticator do
- alias Pleroma.Plugs.AuthenticationPlug
alias Pleroma.Registration
alias Pleroma.Repo
alias Pleroma.User
+ alias Pleroma.Web.Plugs.AuthenticationPlug
import Pleroma.Web.Auth.Authenticator,
only: [fetch_credentials: 1, fetch_user: 1]
diff --git a/lib/pleroma/web/auth/totp_authenticator.ex b/lib/pleroma/web/auth/totp_authenticator.ex
index 1794e407c..edc9871ea 100644
--- a/lib/pleroma/web/auth/totp_authenticator.ex
+++ b/lib/pleroma/web/auth/totp_authenticator.ex
@@ -5,8 +5,8 @@
defmodule Pleroma.Web.Auth.TOTPAuthenticator do
alias Pleroma.MFA
alias Pleroma.MFA.TOTP
- alias Pleroma.Plugs.AuthenticationPlug
alias Pleroma.User
+ alias Pleroma.Web.Plugs.AuthenticationPlug
@doc "Verify code or check backup code."
@spec verify(String.t(), User.t()) ::
diff --git a/lib/pleroma/web/common_api/common_api.ex b/lib/pleroma/web/common_api.ex
similarity index 96%
rename from lib/pleroma/web/common_api/common_api.ex
rename to lib/pleroma/web/common_api.ex
index 4ab533658..318ffc5d0 100644
--- a/lib/pleroma/web/common_api/common_api.ex
+++ b/lib/pleroma/web/common_api.ex
@@ -4,7 +4,6 @@
defmodule Pleroma.Web.CommonAPI do
alias Pleroma.Activity
- alias Pleroma.ActivityExpiration
alias Pleroma.Conversation.Participation
alias Pleroma.Formatter
alias Pleroma.Object
@@ -46,9 +45,13 @@ def post_chat_message(%User{} = user, %User{} = recipient, content, opts \\ [])
{_, {:ok, %Activity{} = activity, _meta}} <-
{:common_pipeline,
Pipeline.common_pipeline(create_activity_data,
- local: true
+ local: true,
+ idempotency_key: opts[:idempotency_key]
)} do
{:ok, activity}
+ else
+ {:common_pipeline, {:reject, _} = e} -> e
+ e -> e
end
end
@@ -381,9 +384,9 @@ def get_replied_to_visibility(activity) do
def check_expiry_date({:ok, nil} = res), do: res
def check_expiry_date({:ok, in_seconds}) do
- expiry = NaiveDateTime.utc_now() |> NaiveDateTime.add(in_seconds)
+ expiry = DateTime.add(DateTime.utc_now(), in_seconds)
- if ActivityExpiration.expires_late_enough?(expiry) do
+ if Pleroma.Workers.PurgeExpiredActivity.expires_late_enough?(expiry) do
{:ok, expiry}
else
{:error, "Expiry date is too soon"}
@@ -551,4 +554,21 @@ def hide_reblogs(%User{} = user, %User{} = target) do
def show_reblogs(%User{} = user, %User{} = target) do
UserRelationship.delete_reblog_mute(user, target)
end
+
+ def get_user(ap_id, fake_record_fallback \\ true) do
+ cond do
+ user = User.get_cached_by_ap_id(ap_id) ->
+ user
+
+ user = User.get_by_guessed_nickname(ap_id) ->
+ user
+
+ fake_record_fallback ->
+ # TODO: refactor (fake records is never a good idea)
+ User.error_user(ap_id)
+
+ true ->
+ nil
+ end
+ end
end
diff --git a/lib/pleroma/web/common_api/activity_draft.ex b/lib/pleroma/web/common_api/activity_draft.ex
index f849b2e01..548f76609 100644
--- a/lib/pleroma/web/common_api/activity_draft.ex
+++ b/lib/pleroma/web/common_api/activity_draft.ex
@@ -202,7 +202,7 @@ defp changes(draft) do
additional =
case draft.expires_at do
- %NaiveDateTime{} = expires_at -> Map.put(additional, "expires_at", expires_at)
+ %DateTime{} = expires_at -> Map.put(additional, "expires_at", expires_at)
_ -> additional
end
diff --git a/lib/pleroma/web/common_api/utils.ex b/lib/pleroma/web/common_api/utils.ex
index 9d7b24eb2..3b71adf0e 100644
--- a/lib/pleroma/web/common_api/utils.ex
+++ b/lib/pleroma/web/common_api/utils.ex
@@ -12,12 +12,12 @@ defmodule Pleroma.Web.CommonAPI.Utils do
alias Pleroma.Conversation.Participation
alias Pleroma.Formatter
alias Pleroma.Object
- alias Pleroma.Plugs.AuthenticationPlug
alias Pleroma.Repo
alias Pleroma.User
alias Pleroma.Web.ActivityPub.Utils
alias Pleroma.Web.ActivityPub.Visibility
alias Pleroma.Web.MediaProxy
+ alias Pleroma.Web.Plugs.AuthenticationPlug
require Logger
require Pleroma.Constants
@@ -274,7 +274,7 @@ defp build_attachment_link(_), do: ""
def format_input(text, format, options \\ [])
@doc """
- Formatting text to plain text.
+ Formatting text to plain text, BBCode, HTML, or Markdown
"""
def format_input(text, "text/plain", options) do
text
@@ -285,9 +285,6 @@ def format_input(text, "text/plain", options) do
end).()
end
- @doc """
- Formatting text as BBCode.
- """
def format_input(text, "text/bbcode", options) do
text
|> String.replace(~r/\r/, "")
@@ -297,18 +294,12 @@ def format_input(text, "text/bbcode", options) do
|> Formatter.linkify(options)
end
- @doc """
- Formatting text to html.
- """
def format_input(text, "text/html", options) do
text
|> Formatter.html_escape("text/html")
|> Formatter.linkify(options)
end
- @doc """
- Formatting text to markdown.
- """
def format_input(text, "text/markdown", options) do
text
|> Formatter.mentions_escape(options)
diff --git a/lib/pleroma/web/controller_helper.ex b/lib/pleroma/web/controller_helper.ex
index 6445966e0..69188a882 100644
--- a/lib/pleroma/web/controller_helper.ex
+++ b/lib/pleroma/web/controller_helper.ex
@@ -48,13 +48,13 @@ defp param_to_integer(val, default) when is_binary(val) do
defp param_to_integer(_, default), do: default
- def add_link_headers(conn, activities, extra_params \\ %{})
+ def add_link_headers(conn, entries, extra_params \\ %{})
- def add_link_headers(%{assigns: %{skip_link_headers: true}} = conn, _activities, _extra_params),
+ def add_link_headers(%{assigns: %{skip_link_headers: true}} = conn, _entries, _extra_params),
do: conn
- def add_link_headers(conn, activities, extra_params) do
- case get_pagination_fields(conn, activities, extra_params) do
+ def add_link_headers(conn, entries, extra_params) do
+ case get_pagination_fields(conn, entries, extra_params) do
%{"next" => next_url, "prev" => prev_url} ->
put_resp_header(conn, "link", "<#{next_url}>; rel=\"next\", <#{prev_url}>; rel=\"prev\"")
@@ -78,19 +78,15 @@ defp build_pagination_fields(conn, min_id, max_id, extra_params) do
}
end
- def get_pagination_fields(conn, activities, extra_params \\ %{}) do
- case List.last(activities) do
+ def get_pagination_fields(conn, entries, extra_params \\ %{}) do
+ case List.last(entries) do
%{pagination_id: max_id} when not is_nil(max_id) ->
- %{pagination_id: min_id} =
- activities
- |> List.first()
+ %{pagination_id: min_id} = List.first(entries)
build_pagination_fields(conn, min_id, max_id, extra_params)
%{id: max_id} ->
- %{id: min_id} =
- activities
- |> List.first()
+ %{id: min_id} = List.first(entries)
build_pagination_fields(conn, min_id, max_id, extra_params)
diff --git a/lib/pleroma/web/endpoint.ex b/lib/pleroma/web/endpoint.ex
index 8b153763d..f26542e88 100644
--- a/lib/pleroma/web/endpoint.ex
+++ b/lib/pleroma/web/endpoint.ex
@@ -7,19 +7,23 @@ defmodule Pleroma.Web.Endpoint do
require Pleroma.Constants
+ alias Pleroma.Config
+
socket("/socket", Pleroma.Web.UserSocket)
- plug(Pleroma.Plugs.SetLocalePlug)
+ plug(Plug.Telemetry, event_prefix: [:phoenix, :endpoint])
+
+ plug(Pleroma.Web.Plugs.SetLocalePlug)
plug(CORSPlug)
- plug(Pleroma.Plugs.HTTPSecurityPlug)
- plug(Pleroma.Plugs.UploadedMedia)
+ plug(Pleroma.Web.Plugs.HTTPSecurityPlug)
+ plug(Pleroma.Web.Plugs.UploadedMedia)
@static_cache_control "public, no-cache"
# InstanceStatic needs to be before Plug.Static to be able to override shipped-static files
# If you're adding new paths to `only:` you'll need to configure them in InstanceStatic as well
# Cache-control headers are duplicated in case we turn off etags in the future
- plug(Pleroma.Plugs.InstanceStatic,
+ plug(Pleroma.Web.Plugs.InstanceStatic,
at: "/",
gzip: true,
cache_control_for_etags: @static_cache_control,
@@ -29,7 +33,7 @@ defmodule Pleroma.Web.Endpoint do
)
# Careful! No `only` restriction here, as we don't know what frontends contain.
- plug(Pleroma.Plugs.FrontendStatic,
+ plug(Pleroma.Web.Plugs.FrontendStatic,
at: "/",
frontend_type: :primary,
gzip: true,
@@ -41,7 +45,7 @@ defmodule Pleroma.Web.Endpoint do
plug(Plug.Static.IndexHtml, at: "/pleroma/admin/")
- plug(Pleroma.Plugs.FrontendStatic,
+ plug(Pleroma.Web.Plugs.FrontendStatic,
at: "/pleroma/admin",
frontend_type: :admin,
gzip: true,
@@ -79,26 +83,26 @@ defmodule Pleroma.Web.Endpoint do
plug(Phoenix.CodeReloader)
end
- plug(Pleroma.Plugs.TrailingFormatPlug)
+ plug(Pleroma.Web.Plugs.TrailingFormatPlug)
plug(Plug.RequestId)
plug(Plug.Logger, log: :debug)
plug(Plug.Parsers,
parsers: [
:urlencoded,
- {:multipart, length: {Pleroma.Config, :get, [[:instance, :upload_limit]]}},
+ {:multipart, length: {Config, :get, [[:instance, :upload_limit]]}},
:json
],
pass: ["*/*"],
json_decoder: Jason,
- length: Pleroma.Config.get([:instance, :upload_limit]),
+ length: Config.get([:instance, :upload_limit]),
body_reader: {Pleroma.Web.Plugs.DigestPlug, :read_body, []}
)
plug(Plug.MethodOverride)
plug(Plug.Head)
- secure_cookies = Pleroma.Config.get([__MODULE__, :secure_cookie_flag])
+ secure_cookies = Config.get([__MODULE__, :secure_cookie_flag])
cookie_name =
if secure_cookies,
@@ -106,7 +110,7 @@ defmodule Pleroma.Web.Endpoint do
else: "pleroma_key"
extra =
- Pleroma.Config.get([__MODULE__, :extra_cookie_attrs])
+ Config.get([__MODULE__, :extra_cookie_attrs])
|> Enum.join(";")
# The session will be stored in the cookie and signed,
@@ -116,13 +120,13 @@ defmodule Pleroma.Web.Endpoint do
Plug.Session,
store: :cookie,
key: cookie_name,
- signing_salt: Pleroma.Config.get([__MODULE__, :signing_salt], "CqaoopA2"),
+ signing_salt: Config.get([__MODULE__, :signing_salt], "CqaoopA2"),
http_only: true,
secure: secure_cookies,
extra: extra
)
- plug(Pleroma.Plugs.RemoteIp)
+ plug(Pleroma.Web.Plugs.RemoteIp)
defmodule Instrumenter do
use Prometheus.PhoenixInstrumenter
@@ -136,8 +140,34 @@ defmodule MetricsExporter do
use Prometheus.PlugExporter
end
+ defmodule MetricsExporterCaller do
+ @behaviour Plug
+
+ def init(opts), do: opts
+
+ def call(conn, opts) do
+ prometheus_config = Application.get_env(:prometheus, MetricsExporter, [])
+ ip_whitelist = List.wrap(prometheus_config[:ip_whitelist])
+
+ cond do
+ !prometheus_config[:enabled] ->
+ conn
+
+ ip_whitelist != [] and
+ !Enum.find(ip_whitelist, fn ip ->
+ Pleroma.Helpers.InetHelper.parse_address(ip) == {:ok, conn.remote_ip}
+ end) ->
+ conn
+
+ true ->
+ MetricsExporter.call(conn, opts)
+ end
+ end
+ end
+
plug(PipelineInstrumenter)
- plug(MetricsExporter)
+
+ plug(MetricsExporterCaller)
plug(Pleroma.Web.Router)
diff --git a/lib/pleroma/web/fallback_redirect_controller.ex b/lib/pleroma/web/fallback/redirect_controller.ex
similarity index 95%
rename from lib/pleroma/web/fallback_redirect_controller.ex
rename to lib/pleroma/web/fallback/redirect_controller.ex
index 431ad5485..6f759d559 100644
--- a/lib/pleroma/web/fallback_redirect_controller.ex
+++ b/lib/pleroma/web/fallback/redirect_controller.ex
@@ -2,7 +2,7 @@
# Copyright © 2017-2020 Pleroma Authors
# SPDX-License-Identifier: AGPL-3.0-only
-defmodule Fallback.RedirectController do
+defmodule Pleroma.Web.Fallback.RedirectController do
use Pleroma.Web, :controller
require Logger
@@ -75,7 +75,7 @@ def empty(conn, _params) do
end
defp index_file_path do
- Pleroma.Plugs.InstanceStatic.file_path("index.html")
+ Pleroma.Web.Plugs.InstanceStatic.file_path("index.html")
end
defp build_tags(conn, params) do
diff --git a/lib/pleroma/web/fed_sockets.ex b/lib/pleroma/web/fed_sockets.ex
new file mode 100644
index 000000000..1fd5899c8
--- /dev/null
+++ b/lib/pleroma/web/fed_sockets.ex
@@ -0,0 +1,185 @@
+# Pleroma: A lightweight social networking server
+# Copyright © 2017-2020 Pleroma Authors
+# SPDX-License-Identifier: AGPL-3.0-only
+
+defmodule Pleroma.Web.FedSockets do
+ @moduledoc """
+ This documents the FedSockets framework. A framework for federating
+ ActivityPub objects between servers via persistant WebSocket connections.
+
+ FedSockets allow servers to authenticate on first contact and maintain that
+ connection, eliminating the need to authenticate every time data needs to be shared.
+
+ ## Protocol
+ FedSockets currently support 2 types of data transfer:
+ * `publish` method which doesn't require a response
+ * `fetch` method requires a response be sent
+
+ ### Publish
+ The publish operation sends a json encoded map of the shape:
+ %{action: :publish, data: json}
+ and accepts (but does not require) a reply of form:
+ %{"action" => "publish_reply"}
+
+ The outgoing params represent
+ * data: ActivityPub object encoded into json
+
+
+ ### Fetch
+ The fetch operation sends a json encoded map of the shape:
+ %{action: :fetch, data: id, uuid: fetch_uuid}
+ and requires a reply of form:
+ %{"action" => "fetch_reply", "uuid" => uuid, "data" => data}
+
+ The outgoing params represent
+ * id: an ActivityPub object URI
+ * uuid: a unique uuid generated by the sender
+
+ The reply params represent
+ * data: an ActivityPub object encoded into json
+ * uuid: the uuid sent along with the fetch request
+
+ ## Examples
+ Clients of FedSocket transfers shouldn't need to use any of the functions outside of this module.
+
+ A typical publish operation can be performed through the following code, and a fetch operation in a similar manner.
+
+ case FedSockets.get_or_create_fed_socket(inbox) do
+ {:ok, fedsocket} ->
+ FedSockets.publish(fedsocket, json)
+
+ _ ->
+ alternative_publish(inbox, actor, json, params)
+ end
+
+ ## Configuration
+ FedSockets have the following config settings
+
+ config :pleroma, :fed_sockets,
+ enabled: true,
+ ping_interval: :timer.seconds(15),
+ connection_duration: :timer.hours(1),
+ rejection_duration: :timer.hours(1),
+ fed_socket_fetches: [
+ default: 12_000,
+ interval: 3_000,
+ lazy: false
+ ]
+ * enabled - turn FedSockets on or off with this flag. Can be toggled at runtime.
+ * connection_duration - How long a FedSocket can sit idle before it's culled.
+ * rejection_duration - After failing to make a FedSocket connection a host will be excluded
+ from further connections for this amount of time
+ * fed_socket_fetches - Use these parameters to pass options to the Cachex queue backing the FetchRegistry
+ * fed_socket_rejections - Use these parameters to pass options to the Cachex queue backing the FedRegistry
+
+ Cachex options are
+ * default: the minimum amount of time a fetch can wait before it times out.
+ * interval: the interval between checks for timed out entries. This plus the default represent the maximum time allowed
+ * lazy: leave at false for consistant and fast lookups, set to true for stricter timeout enforcement
+
+ """
+ require Logger
+
+ alias Pleroma.Web.FedSockets.FedRegistry
+ alias Pleroma.Web.FedSockets.FedSocket
+ alias Pleroma.Web.FedSockets.SocketInfo
+
+ @doc """
+ returns a FedSocket for the given origin. Will reuse an existing one or create a new one.
+
+ address is expected to be a fully formed URL such as:
+ "http://www.example.com" or "http://www.example.com:8080"
+
+ It can and usually does include additional path parameters,
+ but these are ignored as the FedSockets are organized by host and port info alone.
+ """
+ def get_or_create_fed_socket(address) do
+ with {:cache, {:error, :missing}} <- {:cache, get_fed_socket(address)},
+ {:connect, {:ok, _pid}} <- {:connect, FedSocket.connect_to_host(address)},
+ {:cache, {:ok, fed_socket}} <- {:cache, get_fed_socket(address)} do
+ Logger.debug("fedsocket created for - #{inspect(address)}")
+ {:ok, fed_socket}
+ else
+ {:cache, {:ok, socket}} ->
+ Logger.debug("fedsocket found in cache - #{inspect(address)}")
+ {:ok, socket}
+
+ {:cache, {:error, :rejected} = e} ->
+ e
+
+ {:connect, {:error, _host}} ->
+ Logger.debug("set host rejected for - #{inspect(address)}")
+ FedRegistry.set_host_rejected(address)
+ {:error, :rejected}
+
+ {_, {:error, :disabled}} ->
+ {:error, :disabled}
+
+ {_, {:error, reason}} ->
+ Logger.warn("get_or_create_fed_socket error - #{inspect(reason)}")
+ {:error, reason}
+ end
+ end
+
+ @doc """
+ returns a FedSocket for the given origin. Will not create a new FedSocket if one does not exist.
+
+ address is expected to be a fully formed URL such as:
+ "http://www.example.com" or "http://www.example.com:8080"
+ """
+ def get_fed_socket(address) do
+ origin = SocketInfo.origin(address)
+
+ with {:config, true} <- {:config, Pleroma.Config.get([:fed_sockets, :enabled], false)},
+ {:ok, socket} <- FedRegistry.get_fed_socket(origin) do
+ {:ok, socket}
+ else
+ {:config, _} ->
+ {:error, :disabled}
+
+ {:error, :rejected} ->
+ Logger.debug("FedSocket previously rejected - #{inspect(origin)}")
+ {:error, :rejected}
+
+ {:error, reason} ->
+ {:error, reason}
+ end
+ end
+
+ @doc """
+ Sends the supplied data via the publish protocol.
+ It will not block waiting for a reply.
+ Returns :ok but this is not an indication of a successful transfer.
+
+ the data is expected to be JSON encoded binary data.
+ """
+ def publish(%SocketInfo{} = fed_socket, json) do
+ FedSocket.publish(fed_socket, json)
+ end
+
+ @doc """
+ Sends the supplied data via the fetch protocol.
+ It will block waiting for a reply or timeout.
+
+ Returns {:ok, object} where object is the requested object (or nil)
+ {:error, :timeout} in the event the message was not responded to
+
+ the id is expected to be the URI of an ActivityPub object.
+ """
+ def fetch(%SocketInfo{} = fed_socket, id) do
+ FedSocket.fetch(fed_socket, id)
+ end
+
+ @doc """
+ Disconnect all and restart FedSockets.
+ This is mainly used in development and testing but could be useful in production.
+ """
+ def reset do
+ FedRegistry
+ |> Process.whereis()
+ |> Process.exit(:testing)
+ end
+
+ def uri_for_origin(origin),
+ do: "ws://#{origin}/api/fedsocket/v1"
+end
diff --git a/lib/pleroma/web/fed_sockets/fed_registry.ex b/lib/pleroma/web/fed_sockets/fed_registry.ex
new file mode 100644
index 000000000..e00ea69c0
--- /dev/null
+++ b/lib/pleroma/web/fed_sockets/fed_registry.ex
@@ -0,0 +1,185 @@
+# Pleroma: A lightweight social networking server
+# Copyright © 2017-2020 Pleroma Authors
+# SPDX-License-Identifier: AGPL-3.0-only
+
+defmodule Pleroma.Web.FedSockets.FedRegistry do
+ @moduledoc """
+ The FedRegistry stores the active FedSockets for quick retrieval.
+
+ The storage and retrieval portion of the FedRegistry is done in process through
+ elixir's `Registry` module for speed and its ability to monitor for terminated processes.
+
+ Dropped connections will be caught by `Registry` and deleted. Since the next
+ message will initiate a new connection there is no reason to try and reconnect at that point.
+
+ Normally outside modules should have no need to call or use the FedRegistry themselves.
+ """
+
+ alias Pleroma.Web.FedSockets.FedSocket
+ alias Pleroma.Web.FedSockets.SocketInfo
+
+ require Logger
+
+ @default_rejection_duration 15 * 60 * 1000
+ @rejections :fed_socket_rejections
+
+ @doc """
+ Retrieves a FedSocket from the Registry given it's origin.
+
+ The origin is expected to be a string identifying the endpoint "example.com" or "example2.com:8080"
+
+ Will return:
+ * {:ok, fed_socket} for working FedSockets
+ * {:error, :rejected} for origins that have been tried and refused within the rejection duration interval
+ * {:error, some_reason} usually :missing for unknown origins
+ """
+ def get_fed_socket(origin) do
+ case get_registry_data(origin) do
+ {:error, reason} ->
+ {:error, reason}
+
+ {:ok, %{state: :connected} = socket_info} ->
+ {:ok, socket_info}
+ end
+ end
+
+ @doc """
+ Adds a connected FedSocket to the Registry.
+
+ Always returns {:ok, fed_socket}
+ """
+ def add_fed_socket(origin, pid \\ nil) do
+ origin
+ |> SocketInfo.build(pid)
+ |> SocketInfo.connect()
+ |> add_socket_info
+ end
+
+ defp add_socket_info(%{origin: origin, state: :connected} = socket_info) do
+ case Registry.register(FedSockets.Registry, origin, socket_info) do
+ {:ok, _owner} ->
+ clear_prior_rejection(origin)
+ Logger.debug("fedsocket added: #{inspect(origin)}")
+
+ {:ok, socket_info}
+
+ {:error, {:already_registered, _pid}} ->
+ FedSocket.close(socket_info)
+ existing_socket_info = Registry.lookup(FedSockets.Registry, origin)
+
+ {:ok, existing_socket_info}
+
+ _ ->
+ {:error, :error_adding_socket}
+ end
+ end
+
+ @doc """
+ Mark this origin as having rejected a connection attempt.
+ This will keep it from getting additional connection attempts
+ for a period of time specified in the config.
+
+ Always returns {:ok, new_reg_data}
+ """
+ def set_host_rejected(uri) do
+ new_reg_data =
+ uri
+ |> SocketInfo.origin()
+ |> get_or_create_registry_data()
+ |> set_to_rejected()
+ |> save_registry_data()
+
+ {:ok, new_reg_data}
+ end
+
+ @doc """
+ Retrieves the FedRegistryData from the Registry given it's origin.
+
+ The origin is expected to be a string identifying the endpoint "example.com" or "example2.com:8080"
+
+ Will return:
+ * {:ok, fed_registry_data} for known origins
+ * {:error, :missing} for uniknown origins
+ * {:error, :cache_error} indicating some low level runtime issues
+ """
+ def get_registry_data(origin) do
+ case Registry.lookup(FedSockets.Registry, origin) do
+ [] ->
+ if is_rejected?(origin) do
+ Logger.debug("previously rejected fedsocket requested")
+ {:error, :rejected}
+ else
+ {:error, :missing}
+ end
+
+ [{_pid, %{state: :connected} = socket_info}] ->
+ {:ok, socket_info}
+
+ _ ->
+ {:error, :cache_error}
+ end
+ end
+
+ @doc """
+ Retrieves a map of all sockets from the Registry. The keys are the origins and the values are the corresponding SocketInfo
+ """
+ def list_all do
+ (list_all_connected() ++ list_all_rejected())
+ |> Enum.into(%{})
+ end
+
+ defp list_all_connected do
+ FedSockets.Registry
+ |> Registry.select([{{:"$1", :_, :"$3"}, [], [{{:"$1", :"$3"}}]}])
+ end
+
+ defp list_all_rejected do
+ {:ok, keys} = Cachex.keys(@rejections)
+
+ {:ok, registry_data} =
+ Cachex.execute(@rejections, fn worker ->
+ Enum.map(keys, fn k -> {k, Cachex.get!(worker, k)} end)
+ end)
+
+ registry_data
+ end
+
+ defp clear_prior_rejection(origin),
+ do: Cachex.del(@rejections, origin)
+
+ defp is_rejected?(origin) do
+ case Cachex.get(@rejections, origin) do
+ {:ok, nil} ->
+ false
+
+ {:ok, _} ->
+ true
+ end
+ end
+
+ defp get_or_create_registry_data(origin) do
+ case get_registry_data(origin) do
+ {:error, :missing} ->
+ %SocketInfo{origin: origin}
+
+ {:ok, socket_info} ->
+ socket_info
+ end
+ end
+
+ defp save_registry_data(%SocketInfo{origin: origin, state: :connected} = socket_info) do
+ {:ok, true} = Registry.update_value(FedSockets.Registry, origin, fn _ -> socket_info end)
+ socket_info
+ end
+
+ defp save_registry_data(%SocketInfo{origin: origin, state: :rejected} = socket_info) do
+ rejection_expiration =
+ Pleroma.Config.get([:fed_sockets, :rejection_duration], @default_rejection_duration)
+
+ {:ok, true} = Cachex.put(@rejections, origin, socket_info, ttl: rejection_expiration)
+ socket_info
+ end
+
+ defp set_to_rejected(%SocketInfo{} = socket_info),
+ do: %SocketInfo{socket_info | state: :rejected}
+end
diff --git a/lib/pleroma/web/fed_sockets/fed_socket.ex b/lib/pleroma/web/fed_sockets/fed_socket.ex
new file mode 100644
index 000000000..98d64e65a
--- /dev/null
+++ b/lib/pleroma/web/fed_sockets/fed_socket.ex
@@ -0,0 +1,137 @@
+# Pleroma: A lightweight social networking server
+# Copyright © 2017-2020 Pleroma Authors
+# SPDX-License-Identifier: AGPL-3.0-only
+
+defmodule Pleroma.Web.FedSockets.FedSocket do
+ @moduledoc """
+ The FedSocket module abstracts the actions to be taken taken on connections regardless of
+ whether the connection started as inbound or outbound.
+
+
+ Normally outside modules will have no need to call the FedSocket module directly.
+ """
+
+ alias Pleroma.Object
+ alias Pleroma.Object.Containment
+ alias Pleroma.User
+ alias Pleroma.Web.ActivityPub.ObjectView
+ alias Pleroma.Web.ActivityPub.UserView
+ alias Pleroma.Web.ActivityPub.Visibility
+ alias Pleroma.Web.FedSockets.FetchRegistry
+ alias Pleroma.Web.FedSockets.IngesterWorker
+ alias Pleroma.Web.FedSockets.OutgoingHandler
+ alias Pleroma.Web.FedSockets.SocketInfo
+
+ require Logger
+
+ @shake "61dd18f7-f1e6-49a4-939a-a749fcdc1103"
+
+ def connect_to_host(uri) do
+ case OutgoingHandler.start_link(uri) do
+ {:ok, pid} ->
+ {:ok, pid}
+
+ error ->
+ {:error, error}
+ end
+ end
+
+ def close(%SocketInfo{pid: socket_pid}),
+ do: Process.send(socket_pid, :close, [])
+
+ def publish(%SocketInfo{pid: socket_pid}, json) do
+ %{action: :publish, data: json}
+ |> Jason.encode!()
+ |> send_packet(socket_pid)
+ end
+
+ def fetch(%SocketInfo{pid: socket_pid}, id) do
+ fetch_uuid = FetchRegistry.register_fetch(id)
+
+ %{action: :fetch, data: id, uuid: fetch_uuid}
+ |> Jason.encode!()
+ |> send_packet(socket_pid)
+
+ wait_for_fetch_to_return(fetch_uuid, 0)
+ end
+
+ def receive_package(%SocketInfo{} = fed_socket, json) do
+ json
+ |> Jason.decode!()
+ |> process_package(fed_socket)
+ end
+
+ defp wait_for_fetch_to_return(uuid, cntr) do
+ case FetchRegistry.check_fetch(uuid) do
+ {:error, :waiting} ->
+ Process.sleep(:math.pow(cntr, 3) |> Kernel.trunc())
+ wait_for_fetch_to_return(uuid, cntr + 1)
+
+ {:error, :missing} ->
+ Logger.error("FedSocket fetch timed out - #{inspect(uuid)}")
+ {:error, :timeout}
+
+ {:ok, _fr} ->
+ FetchRegistry.pop_fetch(uuid)
+ end
+ end
+
+ defp process_package(%{"action" => "publish", "data" => data}, %{origin: origin} = _fed_socket) do
+ if Containment.contain_origin(origin, data) do
+ IngesterWorker.enqueue("ingest", %{"object" => data})
+ end
+
+ {:reply, %{"action" => "publish_reply", "status" => "processed"}}
+ end
+
+ defp process_package(%{"action" => "fetch_reply", "uuid" => uuid, "data" => data}, _fed_socket) do
+ FetchRegistry.register_fetch_received(uuid, data)
+ {:noreply, nil}
+ end
+
+ defp process_package(%{"action" => "fetch", "uuid" => uuid, "data" => ap_id}, _fed_socket) do
+ {:ok, data} = render_fetched_data(ap_id, uuid)
+ {:reply, data}
+ end
+
+ defp process_package(%{"action" => "publish_reply"}, _fed_socket) do
+ {:noreply, nil}
+ end
+
+ defp process_package(other, _fed_socket) do
+ Logger.warn("unknown json packages received #{inspect(other)}")
+ {:noreply, nil}
+ end
+
+ defp render_fetched_data(ap_id, uuid) do
+ {:ok,
+ %{
+ "action" => "fetch_reply",
+ "status" => "processed",
+ "uuid" => uuid,
+ "data" => represent_item(ap_id)
+ }}
+ end
+
+ defp represent_item(ap_id) do
+ case User.get_by_ap_id(ap_id) do
+ nil ->
+ object = Object.get_cached_by_ap_id(ap_id)
+
+ if Visibility.is_public?(object) do
+ Phoenix.View.render_to_string(ObjectView, "object.json", object: object)
+ else
+ nil
+ end
+
+ user ->
+ Phoenix.View.render_to_string(UserView, "user.json", user: user)
+ end
+ end
+
+ defp send_packet(data, socket_pid) do
+ Process.send(socket_pid, {:send, data}, [])
+ end
+
+ def shake, do: @shake
+end
diff --git a/lib/pleroma/web/fed_sockets/fetch_registry.ex b/lib/pleroma/web/fed_sockets/fetch_registry.ex
new file mode 100644
index 000000000..7897f0fc6
--- /dev/null
+++ b/lib/pleroma/web/fed_sockets/fetch_registry.ex
@@ -0,0 +1,151 @@
+# Pleroma: A lightweight social networking server
+# Copyright © 2017-2020 Pleroma Authors
+# SPDX-License-Identifier: AGPL-3.0-only
+
+defmodule Pleroma.Web.FedSockets.FetchRegistry do
+ @moduledoc """
+ The FetchRegistry acts as a broker for fetch requests and return values.
+ This allows calling processes to block while waiting for a reply.
+ It doesn't impose it's own process instead using `Cachex` to handle fetches in process, allowing
+ multi threaded processes to avoid bottlenecking.
+
+ Normally outside modules will have no need to call or use the FetchRegistry themselves.
+
+ The `Cachex` parameters can be controlled from the config. Since exact timeout intervals
+ aren't necessary the following settings are used by default:
+
+ config :pleroma, :fed_sockets,
+ fed_socket_fetches: [
+ default: 12_000,
+ interval: 3_000,
+ lazy: false
+ ]
+
+ """
+
+ defmodule FetchRegistryData do
+ defstruct uuid: nil,
+ sent_json: nil,
+ received_json: nil,
+ sent_at: nil,
+ received_at: nil
+ end
+
+ alias Ecto.UUID
+
+ require Logger
+
+ @fetches :fed_socket_fetches
+
+ @doc """
+ Registers a json request wth the FetchRegistry and returns the identifying UUID.
+ """
+ def register_fetch(json) do
+ %FetchRegistryData{uuid: uuid} =
+ json
+ |> new_registry_data
+ |> save_registry_data
+
+ uuid
+ end
+
+ @doc """
+ Reports on the status of a Fetch given the identifying UUID.
+
+ Will return
+ * {:ok, fetched_object} if a fetch has completed
+ * {:error, :waiting} if a fetch is still pending
+ * {:error, other_error} usually :missing to indicate a fetch that has timed out
+ """
+ def check_fetch(uuid) do
+ case get_registry_data(uuid) do
+ {:ok, %FetchRegistryData{received_at: nil}} ->
+ {:error, :waiting}
+
+ {:ok, %FetchRegistryData{} = reg_data} ->
+ {:ok, reg_data}
+
+ e ->
+ e
+ end
+ end
+
+ @doc """
+ Retrieves the response to a fetch given the identifying UUID.
+ The completed fetch will be deleted from the FetchRegistry
+
+ Will return
+ * {:ok, fetched_object} if a fetch has completed
+ * {:error, :waiting} if a fetch is still pending
+ * {:error, other_error} usually :missing to indicate a fetch that has timed out
+ """
+ def pop_fetch(uuid) do
+ case check_fetch(uuid) do
+ {:ok, %FetchRegistryData{received_json: received_json}} ->
+ delete_registry_data(uuid)
+ {:ok, received_json}
+
+ e ->
+ e
+ end
+ end
+
+ @doc """
+ This is called to register a fetch has returned.
+ It expects the result data along with the UUID that was sent in the request
+
+ Will return the fetched object or :error
+ """
+ def register_fetch_received(uuid, data) do
+ case get_registry_data(uuid) do
+ {:ok, %FetchRegistryData{received_at: nil} = reg_data} ->
+ reg_data
+ |> set_fetch_received(data)
+ |> save_registry_data()
+
+ {:ok, %FetchRegistryData{} = reg_data} ->
+ Logger.warn("tried to add fetched data twice - #{uuid}")
+ reg_data
+
+ {:error, _} ->
+ Logger.warn("Error adding fetch to registry - #{uuid}")
+ :error
+ end
+ end
+
+ defp new_registry_data(json) do
+ %FetchRegistryData{
+ uuid: UUID.generate(),
+ sent_json: json,
+ sent_at: :erlang.monotonic_time(:millisecond)
+ }
+ end
+
+ defp get_registry_data(origin) do
+ case Cachex.get(@fetches, origin) do
+ {:ok, nil} ->
+ {:error, :missing}
+
+ {:ok, reg_data} ->
+ {:ok, reg_data}
+
+ _ ->
+ {:error, :cache_error}
+ end
+ end
+
+ defp set_fetch_received(%FetchRegistryData{} = reg_data, data),
+ do: %FetchRegistryData{
+ reg_data
+ | received_at: :erlang.monotonic_time(:millisecond),
+ received_json: data
+ }
+
+ defp save_registry_data(%FetchRegistryData{uuid: uuid} = reg_data) do
+ {:ok, true} = Cachex.put(@fetches, uuid, reg_data)
+ reg_data
+ end
+
+ defp delete_registry_data(origin),
+ do: {:ok, true} = Cachex.del(@fetches, origin)
+end
diff --git a/lib/pleroma/web/fed_sockets/incoming_handler.ex b/lib/pleroma/web/fed_sockets/incoming_handler.ex
new file mode 100644
index 000000000..49d0d9d84
--- /dev/null
+++ b/lib/pleroma/web/fed_sockets/incoming_handler.ex
@@ -0,0 +1,88 @@
+# Pleroma: A lightweight social networking server
+# Copyright © 2017-2020 Pleroma Authors
+# SPDX-License-Identifier: AGPL-3.0-only
+
+defmodule Pleroma.Web.FedSockets.IncomingHandler do
+ require Logger
+
+ alias Pleroma.Web.FedSockets.FedRegistry
+ alias Pleroma.Web.FedSockets.FedSocket
+ alias Pleroma.Web.FedSockets.SocketInfo
+
+ import HTTPSignatures, only: [validate_conn: 1, split_signature: 1]
+
+ @behaviour :cowboy_websocket
+
+ def init(req, state) do
+ shake = FedSocket.shake()
+
+ with true <- Pleroma.Config.get([:fed_sockets, :enabled]),
+ sec_protocol <- :cowboy_req.header("sec-websocket-protocol", req, nil),
+ headers = %{"(request-target)" => ^shake} <- :cowboy_req.headers(req),
+ true <- validate_conn(%{req_headers: headers}),
+ %{"keyId" => origin} <- split_signature(headers["signature"]) do
+ req =
+ if is_nil(sec_protocol) do
+ req
+ else
+ :cowboy_req.set_resp_header("sec-websocket-protocol", sec_protocol, req)
+ end
+
+ {:cowboy_websocket, req, %{origin: origin}, %{}}
+ else
+ _ ->
+ {:ok, req, state}
+ end
+ end
+
+ def websocket_init(%{origin: origin}) do
+ case FedRegistry.add_fed_socket(origin) do
+ {:ok, socket_info} ->
+ {:ok, socket_info}
+
+ e ->
+ Logger.error("FedSocket websocket_init failed - #{inspect(e)}")
+ {:error, inspect(e)}
+ end
+ end
+
+ # Use the ping to check if the connection should be expired
+ def websocket_handle(:ping, socket_info) do
+ if SocketInfo.expired?(socket_info) do
+ {:stop, socket_info}
+ else
+ {:ok, socket_info, :hibernate}
+ end
+ end
+
+ def websocket_handle({:text, data}, socket_info) do
+ socket_info = SocketInfo.touch(socket_info)
+
+ case FedSocket.receive_package(socket_info, data) do
+ {:noreply, _} ->
+ {:ok, socket_info}
+
+ {:reply, reply} ->
+ {:reply, {:text, Jason.encode!(reply)}, socket_info}
+
+ {:error, reason} ->
+ Logger.error("incoming error - receive_package: #{inspect(reason)}")
+ {:ok, socket_info}
+ end
+ end
+
+ def websocket_info({:send, message}, socket_info) do
+ socket_info = SocketInfo.touch(socket_info)
+
+ {:reply, {:text, message}, socket_info}
+ end
+
+ def websocket_info(:close, state) do
+ {:stop, state}
+ end
+
+ def websocket_info(message, state) do
+ Logger.debug("#{__MODULE__} unknown message #{inspect(message)}")
+ {:ok, state}
+ end
+end
diff --git a/lib/pleroma/web/fed_sockets/ingester_worker.ex b/lib/pleroma/web/fed_sockets/ingester_worker.ex
new file mode 100644
index 000000000..325f2a4ab
--- /dev/null
+++ b/lib/pleroma/web/fed_sockets/ingester_worker.ex
@@ -0,0 +1,33 @@
+# Pleroma: A lightweight social networking server
+# Copyright © 2017-2020 Pleroma Authors
+# SPDX-License-Identifier: AGPL-3.0-only
+
+defmodule Pleroma.Web.FedSockets.IngesterWorker do
+ use Pleroma.Workers.WorkerHelper, queue: "ingestion_queue"
+ require Logger
+
+ alias Pleroma.Web.Federator
+
+ @impl Oban.Worker
+ def perform(%Job{args: %{"op" => "ingest", "object" => ingestee}}) do
+ try do
+ ingestee
+ |> Jason.decode!()
+ |> do_ingestion()
+ rescue
+ e ->
+ Logger.error("IngesterWorker error - #{inspect(e)}")
+ e
+ end
+ end
+
+ defp do_ingestion(params) do
+ case Federator.incoming_ap_doc(params) do
+ {:error, reason} ->
+ {:error, reason}
+
+ {:ok, object} ->
+ {:ok, object}
+ end
+ end
+end
diff --git a/lib/pleroma/web/fed_sockets/outgoing_handler.ex b/lib/pleroma/web/fed_sockets/outgoing_handler.ex
new file mode 100644
index 000000000..e235a7c43
--- /dev/null
+++ b/lib/pleroma/web/fed_sockets/outgoing_handler.ex
@@ -0,0 +1,151 @@
+# Pleroma: A lightweight social networking server
+# Copyright © 2017-2020 Pleroma Authors
+# SPDX-License-Identifier: AGPL-3.0-only
+
+defmodule Pleroma.Web.FedSockets.OutgoingHandler do
+ use GenServer
+
+ require Logger
+
+ alias Pleroma.Application
+ alias Pleroma.Web.ActivityPub.InternalFetchActor
+ alias Pleroma.Web.FedSockets
+ alias Pleroma.Web.FedSockets.FedRegistry
+ alias Pleroma.Web.FedSockets.FedSocket
+ alias Pleroma.Web.FedSockets.SocketInfo
+
+ def start_link(uri) do
+ GenServer.start_link(__MODULE__, %{uri: uri})
+ end
+
+ def init(%{uri: uri}) do
+ case initiate_connection(uri) do
+ {:ok, ws_origin, conn_pid} ->
+ FedRegistry.add_fed_socket(ws_origin, conn_pid)
+
+ {:error, reason} ->
+ Logger.debug("Outgoing connection failed - #{inspect(reason)}")
+ :ignore
+ end
+ end
+
+ def handle_info({:gun_ws, conn_pid, _ref, {:text, data}}, socket_info) do
+ socket_info = SocketInfo.touch(socket_info)
+
+ case FedSocket.receive_package(socket_info, data) do
+ {:noreply, _} ->
+ {:noreply, socket_info}
+
+ {:reply, reply} ->
+ :gun.ws_send(conn_pid, {:text, Jason.encode!(reply)})
+ {:noreply, socket_info}
+
+ {:error, reason} ->
+ Logger.error("incoming error - receive_package: #{inspect(reason)}")
+ {:noreply, socket_info}
+ end
+ end
+
+ def handle_info(:close, state) do
+ Logger.debug("Sending close frame !!!!!!!")
+ {:close, state}
+ end
+
+ def handle_info({:gun_down, _pid, _prot, :closed, _}, state) do
+ {:stop, :normal, state}
+ end
+
+ def handle_info({:send, data}, %{conn_pid: conn_pid} = socket_info) do
+ socket_info = SocketInfo.touch(socket_info)
+ :gun.ws_send(conn_pid, {:text, data})
+ {:noreply, socket_info}
+ end
+
+ def handle_info({:gun_ws, _, _, :pong}, state) do
+ {:noreply, state, :hibernate}
+ end
+
+ def handle_info(msg, state) do
+ Logger.debug("#{__MODULE__} unhandled event #{inspect(msg)}")
+ {:noreply, state}
+ end
+
+ def terminate(reason, state) do
+ Logger.debug(
+ "#{__MODULE__} terminating outgoing connection for #{inspect(state)} for #{inspect(reason)}"
+ )
+
+ {:ok, state}
+ end
+
+ def initiate_connection(uri) do
+ ws_uri =
+ uri
+ |> SocketInfo.origin()
+ |> FedSockets.uri_for_origin()
+
+ %{host: host, port: port, path: path} = URI.parse(ws_uri)
+
+ with {:ok, conn_pid} <- :gun.open(to_charlist(host), port, %{protocols: [:http]}),
+ {:ok, _} <- :gun.await_up(conn_pid),
+ reference <-
+ :gun.get(conn_pid, to_charlist(path), [
+ {'user-agent', to_charlist(Application.user_agent())}
+ ]),
+ {:response, :fin, 204, _} <- :gun.await(conn_pid, reference),
+ headers <- build_headers(uri),
+ ref <- :gun.ws_upgrade(conn_pid, to_charlist(path), headers, %{silence_pings: false}) do
+ receive do
+ {:gun_upgrade, ^conn_pid, ^ref, [<<"websocket">>], _} ->
+ {:ok, ws_uri, conn_pid}
+ after
+ 15_000 ->
+ Logger.debug("Fedsocket timeout connecting to #{inspect(uri)}")
+ {:error, :timeout}
+ end
+ else
+ {:response, :nofin, 404, _} ->
+ {:error, :fedsockets_not_supported}
+
+ e ->
+ Logger.debug("Fedsocket error connecting to #{inspect(uri)}")
+ {:error, e}
+ end
+ end
+
+ defp build_headers(uri) do
+ host_for_sig = uri |> URI.parse() |> host_signature()
+
+ shake = FedSocket.shake()
+ digest = "SHA-256=" <> (:crypto.hash(:sha256, shake) |> Base.encode64())
+ date = Pleroma.Signature.signed_date()
+ shake_size = byte_size(shake)
+
+ signature_opts = %{
+ "(request-target)": shake,
+ "content-length": to_charlist("#{shake_size}"),
+ date: date,
+ digest: digest,
+ host: host_for_sig
+ }
+
+ signature = Pleroma.Signature.sign(InternalFetchActor.get_actor(), signature_opts)
+
+ [
+ {'signature', to_charlist(signature)},
+ {'date', date},
+ {'digest', to_charlist(digest)},
+ {'content-length', to_charlist("#{shake_size}")},
+ {to_charlist("(request-target)"), to_charlist(shake)},
+ {'user-agent', to_charlist(Application.user_agent())}
+ ]
+ end
+
+ defp host_signature(%{host: host, scheme: scheme, port: port}) do
+ if port == URI.default_port(scheme) do
+ host
+ else
+ "#{host}:#{port}"
+ end
+ end
+end
diff --git a/lib/pleroma/web/fed_sockets/socket_info.ex b/lib/pleroma/web/fed_sockets/socket_info.ex
new file mode 100644
index 000000000..d6fdffe1a
--- /dev/null
+++ b/lib/pleroma/web/fed_sockets/socket_info.ex
@@ -0,0 +1,52 @@
+# Pleroma: A lightweight social networking server
+# Copyright © 2017-2020 Pleroma Authors
+# SPDX-License-Identifier: AGPL-3.0-only
+
+defmodule Pleroma.Web.FedSockets.SocketInfo do
+ defstruct origin: nil,
+ pid: nil,
+ conn_pid: nil,
+ state: :default,
+ connected_until: nil
+
+ alias Pleroma.Web.FedSockets.SocketInfo
+ @default_connection_duration 15 * 60 * 1000
+
+ def build(uri, conn_pid \\ nil) do
+ uri
+ |> build_origin()
+ |> build_pids(conn_pid)
+ |> touch()
+ end
+
+ def touch(%SocketInfo{} = socket_info),
+ do: %{socket_info | connected_until: new_ttl()}
+
+ def connect(%SocketInfo{} = socket_info),
+ do: %{socket_info | state: :connected}
+
+ def expired?(%{connected_until: connected_until}),
+ do: connected_until < :erlang.monotonic_time(:millisecond)
+
+ def origin(uri),
+ do: build_origin(uri).origin
+
+ defp build_pids(socket_info, conn_pid),
+ do: struct(socket_info, pid: self(), conn_pid: conn_pid)
+
+ defp build_origin(uri) when is_binary(uri),
+ do: uri |> URI.parse() |> build_origin
+
+ defp build_origin(%{host: host, port: nil, scheme: scheme}),
+ do: build_origin(%{host: host, port: URI.default_port(scheme)})
+
+ defp build_origin(%{host: host, port: port}),
+ do: %SocketInfo{origin: "#{host}:#{port}"}
+
+ defp new_ttl do
+ connection_duration =
+ Pleroma.Config.get([:fed_sockets, :connection_duration], @default_connection_duration)
+
+ :erlang.monotonic_time(:millisecond) + connection_duration
+ end
+end
diff --git a/lib/pleroma/web/fed_sockets/supervisor.ex b/lib/pleroma/web/fed_sockets/supervisor.ex
new file mode 100644
index 000000000..a5f4bebfb
--- /dev/null
+++ b/lib/pleroma/web/fed_sockets/supervisor.ex
@@ -0,0 +1,59 @@
+# Pleroma: A lightweight social networking server
+# Copyright © 2017-2020 Pleroma Authors
+# SPDX-License-Identifier: AGPL-3.0-only
+
+defmodule Pleroma.Web.FedSockets.Supervisor do
+ use Supervisor
+ import Cachex.Spec
+
+ def start_link(opts) do
+ Supervisor.start_link(__MODULE__, opts, name: __MODULE__)
+ end
+
+ def init(args) do
+ children = [
+ build_cache(:fed_socket_fetches, args),
+ build_cache(:fed_socket_rejections, args),
+ {Registry, keys: :unique, name: FedSockets.Registry, meta: [rejected: %{}]}
+ ]
+
+ opts = [strategy: :one_for_all, name: Pleroma.Web.Streamer.Supervisor]
+ Supervisor.init(children, opts)
+ end
+
+ defp build_cache(name, args) do
+ opts = get_opts(name, args)
+
+ %{
+ id: String.to_atom("#{name}_cache"),
+ start: {Cachex, :start_link, [name, opts]},
+ type: :worker
+ }
+ end
+
+ defp get_opts(cache_name, args)
+ when cache_name in [:fed_socket_fetches, :fed_socket_rejections] do
+ default = get_opts_or_config(args, cache_name, :default, 15_000)
+ interval = get_opts_or_config(args, cache_name, :interval, 3_000)
+ lazy = get_opts_or_config(args, cache_name, :lazy, false)
+
+ [expiration: expiration(default: default, interval: interval, lazy: lazy)]
+ end
+
+ defp get_opts(name, args) do
+ Keyword.get(args, name, [])
+ end
+
+ defp get_opts_or_config(args, name, key, default) do
+ args
+ |> Keyword.get(name, [])
+ |> Keyword.get(key)
+ |> case do
+ nil ->
+ Pleroma.Config.get([:fed_sockets, name, key], default)
+
+ value ->
+ value
+ end
+ end
+end
diff --git a/lib/pleroma/web/federator/federator.ex b/lib/pleroma/web/federator.ex
similarity index 89%
rename from lib/pleroma/web/federator/federator.ex
rename to lib/pleroma/web/federator.ex
index f5803578d..130654145 100644
--- a/lib/pleroma/web/federator/federator.ex
+++ b/lib/pleroma/web/federator.ex
@@ -66,14 +66,17 @@ def perform(:publish, activity) do
def perform(:incoming_ap_doc, params) do
Logger.debug("Handling incoming AP activity")
- params = Utils.normalize_params(params)
+ actor =
+ params
+ |> Map.get("actor")
+ |> Utils.get_ap_id()
# NOTE: we use the actor ID to do the containment, this is fine because an
# actor shouldn't be acting on objects outside their own AP server.
- with {:ok, _user} <- ap_enabled_actor(params["actor"]),
+ with {_, {:ok, _user}} <- {:actor, ap_enabled_actor(actor)},
nil <- Activity.normalize(params["id"]),
{_, :ok} <-
- {:correct_origin?, Containment.contain_origin_from_id(params["actor"], params)},
+ {:correct_origin?, Containment.contain_origin_from_id(actor, params)},
{:ok, activity} <- Transmogrifier.handle_incoming(params) do
{:ok, activity}
else
@@ -85,10 +88,13 @@ def perform(:incoming_ap_doc, params) do
Logger.debug("Already had #{params["id"]}")
{:error, :already_present}
+ {:actor, e} ->
+ Logger.debug("Unhandled actor #{actor}, #{inspect(e)}")
+ {:error, e}
+
e ->
# Just drop those for now
- Logger.debug("Unhandled activity")
- Logger.debug(Jason.encode!(params, pretty: true))
+ Logger.debug(fn -> "Unhandled activity\n" <> Jason.encode!(params, pretty: true) end)
{:error, e}
end
end
diff --git a/lib/pleroma/web/feed/tag_controller.ex b/lib/pleroma/web/feed/tag_controller.ex
index 93a8294b7..218cdbdf3 100644
--- a/lib/pleroma/web/feed/tag_controller.ex
+++ b/lib/pleroma/web/feed/tag_controller.ex
@@ -10,14 +10,14 @@ defmodule Pleroma.Web.Feed.TagController do
alias Pleroma.Web.Feed.FeedView
def feed(conn, params) do
- unless Pleroma.Config.restrict_unauthenticated_access?(:activities, :local) do
+ if Config.get!([:instance, :public]) do
render_feed(conn, params)
else
render_error(conn, :not_found, "Not found")
end
end
- def render_feed(conn, %{"tag" => raw_tag} = params) do
+ defp render_feed(conn, %{"tag" => raw_tag} = params) do
{format, tag} = parse_tag(raw_tag)
activities =
@@ -36,12 +36,13 @@ def render_feed(conn, %{"tag" => raw_tag} = params) do
end
@spec parse_tag(binary() | any()) :: {format :: String.t(), tag :: String.t()}
- defp parse_tag(raw_tag) when is_binary(raw_tag) do
- case Enum.reverse(String.split(raw_tag, ".")) do
- [format | tag] when format in ["atom", "rss"] -> {format, Enum.join(tag, ".")}
- _ -> {"rss", raw_tag}
+ defp parse_tag(raw_tag) do
+ case is_binary(raw_tag) && Enum.reverse(String.split(raw_tag, ".")) do
+ [format | tag] when format in ["rss", "atom"] ->
+ {format, Enum.join(tag, ".")}
+
+ _ ->
+ {"atom", raw_tag}
end
end
-
- defp parse_tag(raw_tag), do: {"rss", raw_tag}
end
diff --git a/lib/pleroma/web/feed/user_controller.ex b/lib/pleroma/web/feed/user_controller.ex
index 71eb1ea7e..a5013d2c0 100644
--- a/lib/pleroma/web/feed/user_controller.ex
+++ b/lib/pleroma/web/feed/user_controller.ex
@@ -5,30 +5,25 @@
defmodule Pleroma.Web.Feed.UserController do
use Pleroma.Web, :controller
- alias Fallback.RedirectController
+ alias Pleroma.Config
alias Pleroma.User
alias Pleroma.Web.ActivityPub.ActivityPub
alias Pleroma.Web.ActivityPub.ActivityPubController
alias Pleroma.Web.Feed.FeedView
- plug(Pleroma.Plugs.SetFormatPlug when action in [:feed_redirect])
+ plug(Pleroma.Web.Plugs.SetFormatPlug when action in [:feed_redirect])
action_fallback(:errors)
def feed_redirect(%{assigns: %{format: "html"}} = conn, %{"nickname" => nickname}) do
with {_, %User{} = user} <- {:fetch_user, User.get_cached_by_nickname_or_id(nickname)} do
- RedirectController.redirector_with_meta(conn, %{user: user})
+ Pleroma.Web.Fallback.RedirectController.redirector_with_meta(conn, %{user: user})
end
end
def feed_redirect(%{assigns: %{format: format}} = conn, _params)
when format in ["json", "activity+json"] do
- with %{halted: false} = conn <-
- Pleroma.Plugs.EnsureAuthenticatedPlug.call(conn,
- unless_func: &Pleroma.Web.FederatingPlug.federating?/1
- ) do
- ActivityPubController.call(conn, :user)
- end
+ ActivityPubController.call(conn, :user)
end
def feed_redirect(conn, %{"nickname" => nickname}) do
@@ -37,25 +32,18 @@ def feed_redirect(conn, %{"nickname" => nickname}) do
end
end
- def feed(conn, params) do
- unless Pleroma.Config.restrict_unauthenticated_access?(:profiles, :local) do
- render_feed(conn, params)
- else
- errors(conn, {:error, :not_found})
- end
- end
-
- def render_feed(conn, %{"nickname" => nickname} = params) do
+ def feed(conn, %{"nickname" => nickname} = params) do
format = get_format(conn)
format =
- if format in ["rss", "atom"] do
+ if format in ["atom", "rss"] do
format
else
"atom"
end
- with {_, %User{local: true} = user} <- {:fetch_user, User.get_cached_by_nickname(nickname)} do
+ with {_, %User{local: true} = user} <- {:fetch_user, User.get_cached_by_nickname(nickname)},
+ {_, :visible} <- {:visibility, User.visible_for(user, _reading_user = nil)} do
activities =
%{
type: ["Create"],
@@ -70,7 +58,7 @@ def render_feed(conn, %{"nickname" => nickname} = params) do
|> render("user.#{format}",
user: user,
activities: activities,
- feed_config: Pleroma.Config.get([:feed])
+ feed_config: Config.get([:feed])
)
end
end
@@ -82,6 +70,8 @@ def errors(conn, {:error, :not_found}) do
def errors(conn, {:fetch_user, %User{local: false}}), do: errors(conn, {:error, :not_found})
def errors(conn, {:fetch_user, nil}), do: errors(conn, {:error, :not_found})
+ def errors(conn, {:visibility, _}), do: errors(conn, {:error, :not_found})
+
def errors(conn, _) do
render_error(conn, :internal_server_error, "Something went wrong")
end
diff --git a/lib/pleroma/web/instance_document.ex b/lib/pleroma/web/instance_document.ex
new file mode 100644
index 000000000..df5caebf0
--- /dev/null
+++ b/lib/pleroma/web/instance_document.ex
@@ -0,0 +1,62 @@
+# Pleroma: A lightweight social networking server
+# Copyright © 2017-2020 Pleroma Authors
+# SPDX-License-Identifier: AGPL-3.0-only
+
+defmodule Pleroma.Web.InstanceDocument do
+ alias Pleroma.Config
+ alias Pleroma.Web.Endpoint
+
+ @instance_documents %{
+ "terms-of-service" => "/static/terms-of-service.html",
+ "instance-panel" => "/instance/panel.html"
+ }
+
+ @spec get(String.t()) :: {:ok, String.t()} | {:error, atom()}
+ def get(document_name) do
+ case Map.fetch(@instance_documents, document_name) do
+ {:ok, path} -> {:ok, path}
+ _ -> {:error, :not_found}
+ end
+ end
+
+ @spec put(String.t(), String.t()) :: {:ok, String.t()} | {:error, atom()}
+ def put(document_name, origin_path) do
+ with {_, {:ok, destination_path}} <-
+ {:instance_document, Map.fetch(@instance_documents, document_name)},
+ :ok <- put_file(origin_path, destination_path) do
+ {:ok, Path.join(Endpoint.url(), destination_path)}
+ else
+ {:instance_document, :error} -> {:error, :not_found}
+ error -> error
+ end
+ end
+
+ @spec delete(String.t()) :: :ok | {:error, atom()}
+ def delete(document_name) do
+ with {_, {:ok, path}} <- {:instance_document, Map.fetch(@instance_documents, document_name)},
+ instance_static_dir_path <- instance_static_dir(path),
+ :ok <- File.rm(instance_static_dir_path) do
+ :ok
+ else
+ {:instance_document, :error} -> {:error, :not_found}
+ {:error, :enoent} -> {:error, :not_found}
+ error -> error
+ end
+ end
+
+ defp put_file(origin_path, destination_path) do
+ with destination <- instance_static_dir(destination_path),
+ {_, :ok} <- {:mkdir_p, File.mkdir_p(Path.dirname(destination))},
+ {_, {:ok, _}} <- {:copy, File.copy(origin_path, destination)} do
+ :ok
+ else
+ {error, _} -> {:error, error}
+ end
+ end
+
+ defp instance_static_dir(filename) do
+ [:instance, :static_dir]
+ |> Config.get!()
+ |> Path.join(filename)
+ end
+end
diff --git a/lib/pleroma/web/mailer/subscription_controller.ex b/lib/pleroma/web/mailer/subscription_controller.ex
index 478a83518..ace44afd1 100644
--- a/lib/pleroma/web/mailer/subscription_controller.ex
+++ b/lib/pleroma/web/mailer/subscription_controller.ex
@@ -1,3 +1,7 @@
+# Pleroma: A lightweight social networking server
+# Copyright © 2017-2020 Pleroma Authors
+# SPDX-License-Identifier: AGPL-3.0-only
+
defmodule Pleroma.Web.Mailer.SubscriptionController do
use Pleroma.Web, :controller
diff --git a/lib/pleroma/web/masto_fe_controller.ex b/lib/pleroma/web/masto_fe_controller.ex
index 43ec70021..08f92d55f 100644
--- a/lib/pleroma/web/masto_fe_controller.ex
+++ b/lib/pleroma/web/masto_fe_controller.ex
@@ -5,9 +5,9 @@
defmodule Pleroma.Web.MastoFEController do
use Pleroma.Web, :controller
- alias Pleroma.Plugs.EnsurePublicOrAuthenticatedPlug
- alias Pleroma.Plugs.OAuthScopesPlug
alias Pleroma.User
+ alias Pleroma.Web.Plugs.EnsurePublicOrAuthenticatedPlug
+ alias Pleroma.Web.Plugs.OAuthScopesPlug
plug(OAuthScopesPlug, %{scopes: ["write:accounts"]} when action == :put_settings)
diff --git a/lib/pleroma/web/mastodon_api/controllers/account_controller.ex b/lib/pleroma/web/mastodon_api/controllers/account_controller.ex
index 95d8452df..a2715cf28 100644
--- a/lib/pleroma/web/mastodon_api/controllers/account_controller.ex
+++ b/lib/pleroma/web/mastodon_api/controllers/account_controller.ex
@@ -15,9 +15,6 @@ defmodule Pleroma.Web.MastodonAPI.AccountController do
]
alias Pleroma.Maps
- alias Pleroma.Plugs.EnsurePublicOrAuthenticatedPlug
- alias Pleroma.Plugs.OAuthScopesPlug
- alias Pleroma.Plugs.RateLimiter
alias Pleroma.User
alias Pleroma.Web.ActivityPub.ActivityPub
alias Pleroma.Web.ActivityPub.Builder
@@ -29,6 +26,9 @@ defmodule Pleroma.Web.MastodonAPI.AccountController do
alias Pleroma.Web.MastodonAPI.StatusView
alias Pleroma.Web.OAuth.OAuthController
alias Pleroma.Web.OAuth.OAuthView
+ alias Pleroma.Web.Plugs.EnsurePublicOrAuthenticatedPlug
+ alias Pleroma.Web.Plugs.OAuthScopesPlug
+ alias Pleroma.Web.Plugs.RateLimiter
alias Pleroma.Web.TwitterAPI.TwitterAPI
plug(Pleroma.Web.ApiSpec.CastAndValidate)
@@ -177,7 +177,6 @@ def update_credentials(%{assigns: %{user: user}, body_params: params} = conn, _p
user_params =
[
:no_rich_text,
- :locked,
:hide_followers_count,
:hide_follows_count,
:hide_followers,
@@ -186,7 +185,6 @@ def update_credentials(%{assigns: %{user: user}, body_params: params} = conn, _p
:show_role,
:skip_thread_containment,
:allow_following_move,
- :discoverable,
:accepts_chat_messages
]
|> Enum.reduce(%{}, fn key, acc ->
@@ -210,6 +208,8 @@ def update_credentials(%{assigns: %{user: user}, body_params: params} = conn, _p
if bot, do: {:ok, "Service"}, else: {:ok, "Person"}
end)
|> Maps.put_if_present(:actor_type, params[:actor_type])
+ |> Maps.put_if_present(:is_locked, params[:locked])
+ |> Maps.put_if_present(:is_discoverable, params[:discoverable])
# What happens here:
#
@@ -442,15 +442,27 @@ def follow_by_uri(%{body_params: %{uri: uri}} = conn, _) do
end
@doc "GET /api/v1/mutes"
- def mutes(%{assigns: %{user: user}} = conn, _) do
- users = User.muted_users(user, _restrict_deactivated = true)
- render(conn, "index.json", users: users, for: user, as: :user)
+ def mutes(%{assigns: %{user: user}} = conn, params) do
+ users =
+ user
+ |> User.muted_users_relation(_restrict_deactivated = true)
+ |> Pleroma.Pagination.fetch_paginated(Map.put(params, :skip_order, true))
+
+ conn
+ |> add_link_headers(users)
+ |> render("index.json", users: users, for: user, as: :user)
end
@doc "GET /api/v1/blocks"
- def blocks(%{assigns: %{user: user}} = conn, _) do
- users = User.blocked_users(user, _restrict_deactivated = true)
- render(conn, "index.json", users: users, for: user, as: :user)
+ def blocks(%{assigns: %{user: user}} = conn, params) do
+ users =
+ user
+ |> User.blocked_users_relation(_restrict_deactivated = true)
+ |> Pleroma.Pagination.fetch_paginated(Map.put(params, :skip_order, true))
+
+ conn
+ |> add_link_headers(users)
+ |> render("index.json", users: users, for: user, as: :user)
end
@doc "GET /api/v1/endorsements"
diff --git a/lib/pleroma/web/mastodon_api/controllers/app_controller.ex b/lib/pleroma/web/mastodon_api/controllers/app_controller.ex
index a516b6c20..143dcf80c 100644
--- a/lib/pleroma/web/mastodon_api/controllers/app_controller.ex
+++ b/lib/pleroma/web/mastodon_api/controllers/app_controller.ex
@@ -5,12 +5,12 @@
defmodule Pleroma.Web.MastodonAPI.AppController do
use Pleroma.Web, :controller
- alias Pleroma.Plugs.EnsurePublicOrAuthenticatedPlug
- alias Pleroma.Plugs.OAuthScopesPlug
alias Pleroma.Repo
alias Pleroma.Web.OAuth.App
alias Pleroma.Web.OAuth.Scopes
alias Pleroma.Web.OAuth.Token
+ alias Pleroma.Web.Plugs.EnsurePublicOrAuthenticatedPlug
+ alias Pleroma.Web.Plugs.OAuthScopesPlug
action_fallback(Pleroma.Web.MastodonAPI.FallbackController)
diff --git a/lib/pleroma/web/mastodon_api/controllers/auth_controller.ex b/lib/pleroma/web/mastodon_api/controllers/auth_controller.ex
index 9f09550e1..9cc3984d0 100644
--- a/lib/pleroma/web/mastodon_api/controllers/auth_controller.ex
+++ b/lib/pleroma/web/mastodon_api/controllers/auth_controller.ex
@@ -5,6 +5,8 @@
defmodule Pleroma.Web.MastodonAPI.AuthController do
use Pleroma.Web, :controller
+ import Pleroma.Web.ControllerHelper, only: [json_response: 3]
+
alias Pleroma.User
alias Pleroma.Web.OAuth.App
alias Pleroma.Web.OAuth.Authorization
@@ -13,7 +15,7 @@ defmodule Pleroma.Web.MastodonAPI.AuthController do
action_fallback(Pleroma.Web.MastodonAPI.FallbackController)
- plug(Pleroma.Plugs.RateLimiter, [name: :password_reset] when action == :password_reset)
+ plug(Pleroma.Web.Plugs.RateLimiter, [name: :password_reset] when action == :password_reset)
@local_mastodon_name "Mastodon-Local"
@@ -22,7 +24,7 @@ def login(%{assigns: %{user: %User{}}} = conn, _params) do
redirect(conn, to: local_mastodon_root_path(conn))
end
- @doc "Local Mastodon FE login init action"
+ # Local Mastodon FE login init action
def login(conn, %{"code" => auth_token}) do
with {:ok, app} <- get_or_make_app(),
{:ok, auth} <- Authorization.get_by_token(app, auth_token),
@@ -33,7 +35,7 @@ def login(conn, %{"code" => auth_token}) do
end
end
- @doc "Local Mastodon FE callback action"
+ # Local Mastodon FE callback action
def login(conn, _) do
with {:ok, app} <- get_or_make_app() do
path =
@@ -61,9 +63,7 @@ def password_reset(conn, params) do
TwitterAPI.password_reset(nickname_or_email)
- conn
- |> put_status(:no_content)
- |> json("")
+ json_response(conn, :no_content, "")
end
defp local_mastodon_root_path(conn) do
diff --git a/lib/pleroma/web/mastodon_api/controllers/conversation_controller.ex b/lib/pleroma/web/mastodon_api/controllers/conversation_controller.ex
index f35ec3596..61347d8db 100644
--- a/lib/pleroma/web/mastodon_api/controllers/conversation_controller.ex
+++ b/lib/pleroma/web/mastodon_api/controllers/conversation_controller.ex
@@ -8,8 +8,8 @@ defmodule Pleroma.Web.MastodonAPI.ConversationController do
import Pleroma.Web.ControllerHelper, only: [add_link_headers: 2]
alias Pleroma.Conversation.Participation
- alias Pleroma.Plugs.OAuthScopesPlug
alias Pleroma.Repo
+ alias Pleroma.Web.Plugs.OAuthScopesPlug
action_fallback(Pleroma.Web.MastodonAPI.FallbackController)
diff --git a/lib/pleroma/web/mastodon_api/controllers/custom_emoji_controller.ex b/lib/pleroma/web/mastodon_api/controllers/custom_emoji_controller.ex
index c5f47c5df..872cb1f4d 100644
--- a/lib/pleroma/web/mastodon_api/controllers/custom_emoji_controller.ex
+++ b/lib/pleroma/web/mastodon_api/controllers/custom_emoji_controller.ex
@@ -9,7 +9,7 @@ defmodule Pleroma.Web.MastodonAPI.CustomEmojiController do
plug(
:skip_plug,
- [Pleroma.Plugs.OAuthScopesPlug, Pleroma.Plugs.EnsurePublicOrAuthenticatedPlug]
+ [Pleroma.Web.Plugs.OAuthScopesPlug, Pleroma.Web.Plugs.EnsurePublicOrAuthenticatedPlug]
when action == :index
)
diff --git a/lib/pleroma/web/mastodon_api/controllers/domain_block_controller.ex b/lib/pleroma/web/mastodon_api/controllers/domain_block_controller.ex
index 9c2d093cd..503bd7d5f 100644
--- a/lib/pleroma/web/mastodon_api/controllers/domain_block_controller.ex
+++ b/lib/pleroma/web/mastodon_api/controllers/domain_block_controller.ex
@@ -5,8 +5,8 @@
defmodule Pleroma.Web.MastodonAPI.DomainBlockController do
use Pleroma.Web, :controller
- alias Pleroma.Plugs.OAuthScopesPlug
alias Pleroma.User
+ alias Pleroma.Web.Plugs.OAuthScopesPlug
plug(Pleroma.Web.ApiSpec.CastAndValidate)
defdelegate open_api_operation(action), to: Pleroma.Web.ApiSpec.DomainBlockOperation
diff --git a/lib/pleroma/web/mastodon_api/controllers/filter_controller.ex b/lib/pleroma/web/mastodon_api/controllers/filter_controller.ex
index abbf0ce02..c71a34b15 100644
--- a/lib/pleroma/web/mastodon_api/controllers/filter_controller.ex
+++ b/lib/pleroma/web/mastodon_api/controllers/filter_controller.ex
@@ -6,7 +6,7 @@ defmodule Pleroma.Web.MastodonAPI.FilterController do
use Pleroma.Web, :controller
alias Pleroma.Filter
- alias Pleroma.Plugs.OAuthScopesPlug
+ alias Pleroma.Web.Plugs.OAuthScopesPlug
@oauth_read_actions [:show, :index]
diff --git a/lib/pleroma/web/mastodon_api/controllers/follow_request_controller.ex b/lib/pleroma/web/mastodon_api/controllers/follow_request_controller.ex
index 748b6b475..f8cd7fa9f 100644
--- a/lib/pleroma/web/mastodon_api/controllers/follow_request_controller.ex
+++ b/lib/pleroma/web/mastodon_api/controllers/follow_request_controller.ex
@@ -5,9 +5,9 @@
defmodule Pleroma.Web.MastodonAPI.FollowRequestController do
use Pleroma.Web, :controller
- alias Pleroma.Plugs.OAuthScopesPlug
alias Pleroma.User
alias Pleroma.Web.CommonAPI
+ alias Pleroma.Web.Plugs.OAuthScopesPlug
plug(:put_view, Pleroma.Web.MastodonAPI.AccountView)
plug(Pleroma.Web.ApiSpec.CastAndValidate)
diff --git a/lib/pleroma/web/mastodon_api/controllers/instance_controller.ex b/lib/pleroma/web/mastodon_api/controllers/instance_controller.ex
index d8859731d..07a32491a 100644
--- a/lib/pleroma/web/mastodon_api/controllers/instance_controller.ex
+++ b/lib/pleroma/web/mastodon_api/controllers/instance_controller.ex
@@ -9,7 +9,7 @@ defmodule Pleroma.Web.MastodonAPI.InstanceController do
plug(
:skip_plug,
- [Pleroma.Plugs.OAuthScopesPlug, Pleroma.Plugs.EnsurePublicOrAuthenticatedPlug]
+ [Pleroma.Web.Plugs.OAuthScopesPlug, Pleroma.Web.Plugs.EnsurePublicOrAuthenticatedPlug]
when action in [:show, :peers]
)
diff --git a/lib/pleroma/web/mastodon_api/controllers/list_controller.ex b/lib/pleroma/web/mastodon_api/controllers/list_controller.ex
index 5daeaa780..f6b51bf02 100644
--- a/lib/pleroma/web/mastodon_api/controllers/list_controller.ex
+++ b/lib/pleroma/web/mastodon_api/controllers/list_controller.ex
@@ -5,9 +5,9 @@
defmodule Pleroma.Web.MastodonAPI.ListController do
use Pleroma.Web, :controller
- alias Pleroma.Plugs.OAuthScopesPlug
alias Pleroma.User
alias Pleroma.Web.MastodonAPI.AccountView
+ alias Pleroma.Web.Plugs.OAuthScopesPlug
@oauth_read_actions [:index, :show, :list_accounts]
diff --git a/lib/pleroma/web/mastodon_api/controllers/marker_controller.ex b/lib/pleroma/web/mastodon_api/controllers/marker_controller.ex
index 85310edfa..0628b2b49 100644
--- a/lib/pleroma/web/mastodon_api/controllers/marker_controller.ex
+++ b/lib/pleroma/web/mastodon_api/controllers/marker_controller.ex
@@ -4,7 +4,7 @@
defmodule Pleroma.Web.MastodonAPI.MarkerController do
use Pleroma.Web, :controller
- alias Pleroma.Plugs.OAuthScopesPlug
+ alias Pleroma.Web.Plugs.OAuthScopesPlug
plug(Pleroma.Web.ApiSpec.CastAndValidate)
diff --git a/lib/pleroma/web/mastodon_api/controllers/mastodon_api_controller.ex b/lib/pleroma/web/mastodon_api/controllers/mastodon_api_controller.ex
index e7767de4e..9cf682c7b 100644
--- a/lib/pleroma/web/mastodon_api/controllers/mastodon_api_controller.ex
+++ b/lib/pleroma/web/mastodon_api/controllers/mastodon_api_controller.ex
@@ -17,7 +17,7 @@ defmodule Pleroma.Web.MastodonAPI.MastodonAPIController do
plug(
:skip_plug,
- [Pleroma.Plugs.OAuthScopesPlug, Pleroma.Plugs.EnsurePublicOrAuthenticatedPlug]
+ [Pleroma.Web.Plugs.OAuthScopesPlug, Pleroma.Web.Plugs.EnsurePublicOrAuthenticatedPlug]
when action in [:empty_array, :empty_object]
)
diff --git a/lib/pleroma/web/mastodon_api/controllers/media_controller.ex b/lib/pleroma/web/mastodon_api/controllers/media_controller.ex
index 513de279f..161193134 100644
--- a/lib/pleroma/web/mastodon_api/controllers/media_controller.ex
+++ b/lib/pleroma/web/mastodon_api/controllers/media_controller.ex
@@ -6,11 +6,12 @@ defmodule Pleroma.Web.MastodonAPI.MediaController do
use Pleroma.Web, :controller
alias Pleroma.Object
- alias Pleroma.Plugs.OAuthScopesPlug
alias Pleroma.User
alias Pleroma.Web.ActivityPub.ActivityPub
+ alias Pleroma.Web.Plugs.OAuthScopesPlug
action_fallback(Pleroma.Web.MastodonAPI.FallbackController)
+ plug(Majic.Plug, [pool: Pleroma.MajicPool] when action in [:create, :create2])
plug(Pleroma.Web.ApiSpec.CastAndValidate)
plug(:put_view, Pleroma.Web.MastodonAPI.StatusView)
diff --git a/lib/pleroma/web/mastodon_api/controllers/notification_controller.ex b/lib/pleroma/web/mastodon_api/controllers/notification_controller.ex
index e25cef30b..c3c8606f2 100644
--- a/lib/pleroma/web/mastodon_api/controllers/notification_controller.ex
+++ b/lib/pleroma/web/mastodon_api/controllers/notification_controller.ex
@@ -8,8 +8,8 @@ defmodule Pleroma.Web.MastodonAPI.NotificationController do
import Pleroma.Web.ControllerHelper, only: [add_link_headers: 2]
alias Pleroma.Notification
- alias Pleroma.Plugs.OAuthScopesPlug
alias Pleroma.Web.MastodonAPI.MastodonAPI
+ alias Pleroma.Web.Plugs.OAuthScopesPlug
@oauth_read_actions [:show, :index]
diff --git a/lib/pleroma/web/mastodon_api/controllers/poll_controller.ex b/lib/pleroma/web/mastodon_api/controllers/poll_controller.ex
index db46ffcfc..3dcd1c44f 100644
--- a/lib/pleroma/web/mastodon_api/controllers/poll_controller.ex
+++ b/lib/pleroma/web/mastodon_api/controllers/poll_controller.ex
@@ -9,9 +9,9 @@ defmodule Pleroma.Web.MastodonAPI.PollController do
alias Pleroma.Activity
alias Pleroma.Object
- alias Pleroma.Plugs.OAuthScopesPlug
alias Pleroma.Web.ActivityPub.Visibility
alias Pleroma.Web.CommonAPI
+ alias Pleroma.Web.Plugs.OAuthScopesPlug
action_fallback(Pleroma.Web.MastodonAPI.FallbackController)
diff --git a/lib/pleroma/web/mastodon_api/controllers/report_controller.ex b/lib/pleroma/web/mastodon_api/controllers/report_controller.ex
index 405167108..156544f40 100644
--- a/lib/pleroma/web/mastodon_api/controllers/report_controller.ex
+++ b/lib/pleroma/web/mastodon_api/controllers/report_controller.ex
@@ -3,14 +3,12 @@
# SPDX-License-Identifier: AGPL-3.0-only
defmodule Pleroma.Web.MastodonAPI.ReportController do
- alias Pleroma.Plugs.OAuthScopesPlug
-
use Pleroma.Web, :controller
action_fallback(Pleroma.Web.MastodonAPI.FallbackController)
plug(Pleroma.Web.ApiSpec.CastAndValidate)
- plug(OAuthScopesPlug, %{scopes: ["write:reports"]} when action == :create)
+ plug(Pleroma.Web.Plugs.OAuthScopesPlug, %{scopes: ["write:reports"]} when action == :create)
defdelegate open_api_operation(action), to: Pleroma.Web.ApiSpec.ReportOperation
diff --git a/lib/pleroma/web/mastodon_api/controllers/scheduled_activity_controller.ex b/lib/pleroma/web/mastodon_api/controllers/scheduled_activity_controller.ex
index 1719c67ea..322a46497 100644
--- a/lib/pleroma/web/mastodon_api/controllers/scheduled_activity_controller.ex
+++ b/lib/pleroma/web/mastodon_api/controllers/scheduled_activity_controller.ex
@@ -7,9 +7,9 @@ defmodule Pleroma.Web.MastodonAPI.ScheduledActivityController do
import Pleroma.Web.ControllerHelper, only: [add_link_headers: 2]
- alias Pleroma.Plugs.OAuthScopesPlug
alias Pleroma.ScheduledActivity
alias Pleroma.Web.MastodonAPI.MastodonAPI
+ alias Pleroma.Web.Plugs.OAuthScopesPlug
@oauth_read_actions [:show, :index]
diff --git a/lib/pleroma/web/mastodon_api/controllers/search_controller.ex b/lib/pleroma/web/mastodon_api/controllers/search_controller.ex
index 5a983db39..0043c3a56 100644
--- a/lib/pleroma/web/mastodon_api/controllers/search_controller.ex
+++ b/lib/pleroma/web/mastodon_api/controllers/search_controller.ex
@@ -6,14 +6,14 @@ defmodule Pleroma.Web.MastodonAPI.SearchController do
use Pleroma.Web, :controller
alias Pleroma.Activity
- alias Pleroma.Plugs.OAuthScopesPlug
- alias Pleroma.Plugs.RateLimiter
alias Pleroma.Repo
alias Pleroma.User
alias Pleroma.Web
alias Pleroma.Web.ControllerHelper
alias Pleroma.Web.MastodonAPI.AccountView
alias Pleroma.Web.MastodonAPI.StatusView
+ alias Pleroma.Web.Plugs.OAuthScopesPlug
+ alias Pleroma.Web.Plugs.RateLimiter
require Logger
diff --git a/lib/pleroma/web/mastodon_api/controllers/status_controller.ex b/lib/pleroma/web/mastodon_api/controllers/status_controller.ex
index ecfa38489..6848adace 100644
--- a/lib/pleroma/web/mastodon_api/controllers/status_controller.ex
+++ b/lib/pleroma/web/mastodon_api/controllers/status_controller.ex
@@ -13,8 +13,6 @@ defmodule Pleroma.Web.MastodonAPI.StatusController do
alias Pleroma.Activity
alias Pleroma.Bookmark
alias Pleroma.Object
- alias Pleroma.Plugs.OAuthScopesPlug
- alias Pleroma.Plugs.RateLimiter
alias Pleroma.Repo
alias Pleroma.ScheduledActivity
alias Pleroma.User
@@ -23,9 +21,15 @@ defmodule Pleroma.Web.MastodonAPI.StatusController do
alias Pleroma.Web.CommonAPI
alias Pleroma.Web.MastodonAPI.AccountView
alias Pleroma.Web.MastodonAPI.ScheduledActivityView
+ alias Pleroma.Web.Plugs.OAuthScopesPlug
+ alias Pleroma.Web.Plugs.RateLimiter
plug(Pleroma.Web.ApiSpec.CastAndValidate)
- plug(:skip_plug, Pleroma.Plugs.EnsurePublicOrAuthenticatedPlug when action in [:index, :show])
+
+ plug(
+ :skip_plug,
+ Pleroma.Web.Plugs.EnsurePublicOrAuthenticatedPlug when action in [:index, :show]
+ )
@unauthenticated_access %{fallback: :proceed_unauthenticated, scopes: []}
@@ -123,9 +127,8 @@ def index(%{assigns: %{user: user}} = conn, %{ids: ids} = _params) do
@doc """
POST /api/v1/statuses
-
- Creates a scheduled status when `scheduled_at` param is present and it's far enough
"""
+ # Creates a scheduled status when `scheduled_at` param is present and it's far enough
def create(
%{
assigns: %{user: user},
@@ -156,11 +159,7 @@ def create(
end
end
- @doc """
- POST /api/v1/statuses
-
- Creates a regular status
- """
+ # Creates a regular status
def create(%{assigns: %{user: user}, body_params: %{status: _} = params} = conn, _) do
params = Map.put(params, :in_reply_to_status_id, params[:in_reply_to_id])
diff --git a/lib/pleroma/web/mastodon_api/controllers/subscription_controller.ex b/lib/pleroma/web/mastodon_api/controllers/subscription_controller.ex
index 34eac97c5..20138908c 100644
--- a/lib/pleroma/web/mastodon_api/controllers/subscription_controller.ex
+++ b/lib/pleroma/web/mastodon_api/controllers/subscription_controller.ex
@@ -13,7 +13,7 @@ defmodule Pleroma.Web.MastodonAPI.SubscriptionController do
plug(Pleroma.Web.ApiSpec.CastAndValidate)
plug(:restrict_push_enabled)
- plug(Pleroma.Plugs.OAuthScopesPlug, %{scopes: ["push"]})
+ plug(Pleroma.Web.Plugs.OAuthScopesPlug, %{scopes: ["push"]})
defdelegate open_api_operation(action), to: Pleroma.Web.ApiSpec.SubscriptionOperation
diff --git a/lib/pleroma/web/mastodon_api/controllers/suggestion_controller.ex b/lib/pleroma/web/mastodon_api/controllers/suggestion_controller.ex
index f91df9ab7..5765271cf 100644
--- a/lib/pleroma/web/mastodon_api/controllers/suggestion_controller.ex
+++ b/lib/pleroma/web/mastodon_api/controllers/suggestion_controller.ex
@@ -8,7 +8,7 @@ defmodule Pleroma.Web.MastodonAPI.SuggestionController do
require Logger
plug(Pleroma.Web.ApiSpec.CastAndValidate)
- plug(Pleroma.Plugs.OAuthScopesPlug, %{scopes: ["read"]} when action == :index)
+ plug(Pleroma.Web.Plugs.OAuthScopesPlug, %{scopes: ["read"]} when action == :index)
def open_api_operation(action) do
operation = String.to_existing_atom("#{action}_operation")
diff --git a/lib/pleroma/web/mastodon_api/controllers/timeline_controller.ex b/lib/pleroma/web/mastodon_api/controllers/timeline_controller.ex
index 5272790d3..ac96520a3 100644
--- a/lib/pleroma/web/mastodon_api/controllers/timeline_controller.ex
+++ b/lib/pleroma/web/mastodon_api/controllers/timeline_controller.ex
@@ -10,11 +10,11 @@ defmodule Pleroma.Web.MastodonAPI.TimelineController do
alias Pleroma.Config
alias Pleroma.Pagination
- alias Pleroma.Plugs.EnsurePublicOrAuthenticatedPlug
- alias Pleroma.Plugs.OAuthScopesPlug
- alias Pleroma.Plugs.RateLimiter
alias Pleroma.User
alias Pleroma.Web.ActivityPub.ActivityPub
+ alias Pleroma.Web.Plugs.EnsurePublicOrAuthenticatedPlug
+ alias Pleroma.Web.Plugs.OAuthScopesPlug
+ alias Pleroma.Web.Plugs.RateLimiter
plug(Pleroma.Web.ApiSpec.CastAndValidate)
plug(:skip_plug, EnsurePublicOrAuthenticatedPlug when action in [:public, :hashtag])
@@ -111,6 +111,7 @@ def public(%{assigns: %{user: user}} = conn, params) do
|> Map.put(:blocking_user, user)
|> Map.put(:muting_user, user)
|> Map.put(:reply_filtering_user, user)
+ |> Map.put(:instance, params[:instance])
|> ActivityPub.fetch_public_activities()
conn
diff --git a/lib/pleroma/web/mastodon_api/views/account_view.ex b/lib/pleroma/web/mastodon_api/views/account_view.ex
index d2a30a548..3158d09ed 100644
--- a/lib/pleroma/web/mastodon_api/views/account_view.ex
+++ b/lib/pleroma/web/mastodon_api/views/account_view.ex
@@ -181,8 +181,10 @@ defp do_render("show.json", %{user: user} = opts) do
user = User.sanitize_html(user, User.html_filter_policy(opts[:for]))
display_name = user.name || user.nickname
- image = User.avatar_url(user) |> MediaProxy.url()
+ avatar = User.avatar_url(user) |> MediaProxy.url()
+ avatar_static = User.avatar_url(user) |> MediaProxy.preview_url(static: true)
header = User.banner_url(user) |> MediaProxy.url()
+ header_static = User.banner_url(user) |> MediaProxy.preview_url(static: true)
following_count =
if !user.hide_follows_count or !user.hide_follows or opts[:for] == user do
@@ -240,17 +242,17 @@ defp do_render("show.json", %{user: user} = opts) do
username: username_from_nickname(user.nickname),
acct: user.nickname,
display_name: display_name,
- locked: user.locked,
+ locked: user.is_locked,
created_at: Utils.to_masto_date(user.inserted_at),
followers_count: followers_count,
following_count: following_count,
statuses_count: user.note_count,
note: user.bio,
url: user.uri || user.ap_id,
- avatar: image,
- avatar_static: image,
+ avatar: avatar,
+ avatar_static: avatar_static,
header: header,
- header_static: header,
+ header_static: header_static,
emojis: emojis,
fields: user.fields,
bot: bot,
@@ -259,7 +261,7 @@ defp do_render("show.json", %{user: user} = opts) do
sensitive: false,
fields: user.raw_fields,
pleroma: %{
- discoverable: user.discoverable,
+ discoverable: user.is_discoverable,
actor_type: user.actor_type
}
},
@@ -386,7 +388,7 @@ defp maybe_put_unread_conversation_count(data, %User{id: user_id} = user, %User{
data
|> Kernel.put_in(
[:pleroma, :unread_conversation_count],
- user.unread_conversation_count
+ Pleroma.Conversation.Participation.unread_count(user)
)
end
diff --git a/lib/pleroma/web/mastodon_api/views/conversation_view.ex b/lib/pleroma/web/mastodon_api/views/conversation_view.ex
index a91994915..82fcff062 100644
--- a/lib/pleroma/web/mastodon_api/views/conversation_view.ex
+++ b/lib/pleroma/web/mastodon_api/views/conversation_view.ex
@@ -33,8 +33,15 @@ def render("participation.json", %{participation: participation, for: user}) do
end
activity = Activity.get_by_id_with_object(last_activity_id)
- # Conversations return all users except the current user.
- users = Enum.reject(participation.recipients, &(&1.id == user.id))
+
+ # Conversations return all users except the current user,
+ # except when the current user is the only participant
+ users =
+ if length(participation.recipients) > 1 do
+ Enum.reject(participation.recipients, &(&1.id == user.id))
+ else
+ participation.recipients
+ end
%{
id: participation.id |> to_string(),
@@ -43,7 +50,8 @@ def render("participation.json", %{participation: participation, for: user}) do
last_status:
render(StatusView, "show.json",
activity: activity,
- direct_conversation_id: participation.id
+ direct_conversation_id: participation.id,
+ for: user
)
}
end
diff --git a/lib/pleroma/web/mastodon_api/views/poll_view.ex b/lib/pleroma/web/mastodon_api/views/poll_view.ex
index 1208dc9a0..4101f21d0 100644
--- a/lib/pleroma/web/mastodon_api/views/poll_view.ex
+++ b/lib/pleroma/web/mastodon_api/views/poll_view.ex
@@ -19,7 +19,7 @@ def render("show.json", %{object: object, multiple: multiple, options: options}
expired: expired,
multiple: multiple,
votes_count: votes_count,
- voters_count: (multiple || nil) && voters_count(object),
+ voters_count: voters_count(object),
options: options,
voted: voted?(params),
emojis: Pleroma.Web.MastodonAPI.StatusView.build_emojis(object.data["emoji"])
diff --git a/lib/pleroma/web/mastodon_api/views/status_view.ex b/lib/pleroma/web/mastodon_api/views/status_view.ex
index 3fe1967be..435bcde15 100644
--- a/lib/pleroma/web/mastodon_api/views/status_view.ex
+++ b/lib/pleroma/web/mastodon_api/views/status_view.ex
@@ -8,7 +8,6 @@ defmodule Pleroma.Web.MastodonAPI.StatusView do
require Pleroma.Constants
alias Pleroma.Activity
- alias Pleroma.ActivityExpiration
alias Pleroma.HTML
alias Pleroma.Object
alias Pleroma.Repo
@@ -56,23 +55,6 @@ defp get_replied_to_activities(activities) do
end)
end
- def get_user(ap_id, fake_record_fallback \\ true) do
- cond do
- user = User.get_cached_by_ap_id(ap_id) ->
- user
-
- user = User.get_by_guessed_nickname(ap_id) ->
- user
-
- fake_record_fallback ->
- # TODO: refactor (fake records is never a good idea)
- User.error_user(ap_id)
-
- true ->
- nil
- end
- end
-
defp get_context_id(%{data: %{"context_id" => context_id}}) when not is_nil(context_id),
do: context_id
@@ -120,7 +102,7 @@ def render("index.json", opts) do
# Note: unresolved users are filtered out
actors =
(activities ++ parent_activities)
- |> Enum.map(&get_user(&1.data["actor"], false))
+ |> Enum.map(&CommonAPI.get_user(&1.data["actor"], false))
|> Enum.filter(& &1)
UserRelationship.view_relationships_option(reading_user, actors, subset: :source_mutes)
@@ -139,7 +121,7 @@ def render(
"show.json",
%{activity: %{data: %{"type" => "Announce", "object" => _object}} = activity} = opts
) do
- user = get_user(activity.data["actor"])
+ user = CommonAPI.get_user(activity.data["actor"])
created_at = Utils.to_masto_date(activity.data["published"])
activity_object = Object.normalize(activity)
@@ -212,7 +194,7 @@ def render(
def render("show.json", %{activity: %{data: %{"object" => _object}} = activity} = opts) do
object = Object.normalize(activity)
- user = get_user(activity.data["actor"])
+ user = CommonAPI.get_user(activity.data["actor"])
user_follower_address = user.follower_address
like_count = object.data["like_count"] || 0
@@ -245,8 +227,8 @@ def render("show.json", %{activity: %{data: %{"object" => _object}} = activity}
expires_at =
with true <- client_posted_this_activity,
- %ActivityExpiration{scheduled_at: scheduled_at} <-
- ActivityExpiration.get_by_activity_id(activity.id) do
+ %Oban.Job{scheduled_at: scheduled_at} <-
+ Pleroma.Workers.PurgeExpiredActivity.get_expiration(activity.id) do
scheduled_at
else
_ -> nil
@@ -266,7 +248,7 @@ def render("show.json", %{activity: %{data: %{"object" => _object}} = activity}
reply_to = get_reply_to(activity, opts)
- reply_to_user = reply_to && get_user(reply_to.data["actor"])
+ reply_to_user = reply_to && CommonAPI.get_user(reply_to.data["actor"])
content =
object
@@ -433,6 +415,7 @@ def render("attachment.json", %{attachment: attachment}) do
[attachment_url | _] = attachment["url"]
media_type = attachment_url["mediaType"] || attachment_url["mimeType"] || "image"
href = attachment_url["href"] |> MediaProxy.url()
+ href_preview = attachment_url["href"] |> MediaProxy.preview_url()
type =
cond do
@@ -448,7 +431,7 @@ def render("attachment.json", %{attachment: attachment}) do
id: to_string(attachment["id"] || hash_id),
url: href,
remote_url: href,
- preview_url: href,
+ preview_url: href_preview,
text_url: href,
type: type,
description: attachment["name"],
diff --git a/lib/pleroma/web/mastodon_api/websocket_handler.ex b/lib/pleroma/web/mastodon_api/websocket_handler.ex
index 94e4595d8..439cdd716 100644
--- a/lib/pleroma/web/mastodon_api/websocket_handler.ex
+++ b/lib/pleroma/web/mastodon_api/websocket_handler.ex
@@ -23,8 +23,8 @@ def init(%{qs: qs} = req, state) do
with params <- Enum.into(:cow_qs.parse_qs(qs), %{}),
sec_websocket <- :cowboy_req.header("sec-websocket-protocol", req, nil),
access_token <- Map.get(params, "access_token"),
- {:ok, user} <- authenticate_request(access_token, sec_websocket),
- {:ok, topic} <- Streamer.get_topic(Map.get(params, "stream"), user, params) do
+ {:ok, user, oauth_token} <- authenticate_request(access_token, sec_websocket),
+ {:ok, topic} <- Streamer.get_topic(params["stream"], user, oauth_token, params) do
req =
if sec_websocket do
:cowboy_req.set_resp_header("sec-websocket-protocol", sec_websocket, req)
@@ -37,12 +37,12 @@ def init(%{qs: qs} = req, state) do
else
{:error, :bad_topic} ->
Logger.debug("#{__MODULE__} bad topic #{inspect(req)}")
- {:ok, req} = :cowboy_req.reply(404, req)
+ req = :cowboy_req.reply(404, req)
{:ok, req, state}
{:error, :unauthorized} ->
Logger.debug("#{__MODULE__} authentication error: #{inspect(req)}")
- {:ok, req} = :cowboy_req.reply(401, req)
+ req = :cowboy_req.reply(401, req)
{:ok, req, state}
end
end
@@ -64,7 +64,9 @@ def websocket_handle(:pong, state) do
{:ok, %{state | timer: timer()}}
end
- # We never receive messages.
+ # We only receive pings for now
+ def websocket_handle(:ping, state), do: {:ok, state}
+
def websocket_handle(frame, state) do
Logger.error("#{__MODULE__} received frame: #{inspect(frame)}")
{:ok, state}
@@ -98,6 +100,10 @@ def websocket_info(:tick, state) do
{:reply, :ping, %{state | timer: nil, count: 0}, :hibernate}
end
+ # State can be `[]` only in case we terminate before switching to websocket,
+ # we already log errors for these cases in `init/1`, so just do nothing here
+ def terminate(_reason, _req, []), do: :ok
+
def terminate(reason, _req, state) do
Logger.debug(
"#{__MODULE__} terminating websocket connection for user #{
@@ -111,7 +117,7 @@ def terminate(reason, _req, state) do
# Public streams without authentication.
defp authenticate_request(nil, nil) do
- {:ok, nil}
+ {:ok, nil, nil}
end
# Authenticated streams.
@@ -119,9 +125,9 @@ defp authenticate_request(access_token, sec_websocket) do
token = access_token || sec_websocket
with true <- is_bitstring(token),
- %Token{user_id: user_id} <- Repo.get_by(Token, token: token),
+ oauth_token = %Token{user_id: user_id} <- Repo.get_by(Token, token: token),
user = %User{} <- User.get_cached_by_id(user_id) do
- {:ok, user}
+ {:ok, user, oauth_token}
else
_ -> {:error, :unauthorized}
end
diff --git a/lib/pleroma/web/media_proxy/media_proxy.ex b/lib/pleroma/web/media_proxy.ex
similarity index 57%
rename from lib/pleroma/web/media_proxy/media_proxy.ex
rename to lib/pleroma/web/media_proxy.ex
index e18dd8224..8656b8cad 100644
--- a/lib/pleroma/web/media_proxy/media_proxy.ex
+++ b/lib/pleroma/web/media_proxy.ex
@@ -4,6 +4,7 @@
defmodule Pleroma.Web.MediaProxy do
alias Pleroma.Config
+ alias Pleroma.Helpers.UriHelper
alias Pleroma.Upload
alias Pleroma.Web
alias Pleroma.Web.MediaProxy.Invalidation
@@ -40,27 +41,35 @@ def url(url) when is_nil(url) or url == "", do: nil
def url("/" <> _ = url), do: url
def url(url) do
- if disabled?() or not url_proxiable?(url) do
- url
- else
+ if enabled?() and url_proxiable?(url) do
encode_url(url)
+ else
+ url
end
end
@spec url_proxiable?(String.t()) :: boolean()
def url_proxiable?(url) do
- if local?(url) or whitelisted?(url) do
- false
+ not local?(url) and not whitelisted?(url)
+ end
+
+ def preview_url(url, preview_params \\ []) do
+ if preview_enabled?() do
+ encode_preview_url(url, preview_params)
else
- true
+ url(url)
end
end
- defp disabled?, do: !Config.get([:media_proxy, :enabled], false)
+ def enabled?, do: Config.get([:media_proxy, :enabled], false)
- defp local?(url), do: String.starts_with?(url, Pleroma.Web.base_url())
+ # Note: media proxy must be enabled for media preview proxy in order to load all
+ # non-local non-whitelisted URLs through it and be sure that body size constraint is preserved.
+ def preview_enabled?, do: enabled?() and !!Config.get([:media_preview_proxy, :enabled])
- defp whitelisted?(url) do
+ def local?(url), do: String.starts_with?(url, Pleroma.Web.base_url())
+
+ def whitelisted?(url) do
%{host: domain} = URI.parse(url)
mediaproxy_whitelist_domains =
@@ -85,17 +94,29 @@ defp maybe_get_domain_from_url("http" <> _ = url) do
defp maybe_get_domain_from_url(domain), do: domain
- def encode_url(url) do
+ defp base64_sig64(url) do
base64 = Base.url_encode64(url, @base64_opts)
sig64 =
base64
- |> signed_url
+ |> signed_url()
|> Base.url_encode64(@base64_opts)
+ {base64, sig64}
+ end
+
+ def encode_url(url) do
+ {base64, sig64} = base64_sig64(url)
+
build_url(sig64, base64, filename(url))
end
+ def encode_preview_url(url, preview_params \\ []) do
+ {base64, sig64} = base64_sig64(url)
+
+ build_preview_url(sig64, base64, filename(url), preview_params)
+ end
+
def decode_url(sig, url) do
with {:ok, sig} <- Base.url_decode64(sig, @base64_opts),
signature when signature == sig <- signed_url(url) do
@@ -113,10 +134,14 @@ def filename(url_or_path) do
if path = URI.parse(url_or_path).path, do: Path.basename(path)
end
- def build_url(sig_base64, url_base64, filename \\ nil) do
+ def base_url do
+ Config.get([:media_proxy, :base_url], Web.base_url())
+ end
+
+ defp proxy_url(path, sig_base64, url_base64, filename) do
[
- Config.get([:media_proxy, :base_url], Web.base_url()),
- "proxy",
+ base_url(),
+ path,
sig_base64,
url_base64,
filename
@@ -124,4 +149,38 @@ def build_url(sig_base64, url_base64, filename \\ nil) do
|> Enum.filter(& &1)
|> Path.join()
end
+
+ def build_url(sig_base64, url_base64, filename \\ nil) do
+ proxy_url("proxy", sig_base64, url_base64, filename)
+ end
+
+ def build_preview_url(sig_base64, url_base64, filename \\ nil, preview_params \\ []) do
+ uri = proxy_url("proxy/preview", sig_base64, url_base64, filename)
+
+ UriHelper.modify_uri_params(uri, preview_params)
+ end
+
+ def verify_request_path_and_url(
+ %Plug.Conn{params: %{"filename" => _}, request_path: request_path},
+ url
+ ) do
+ verify_request_path_and_url(request_path, url)
+ end
+
+ def verify_request_path_and_url(request_path, url) when is_binary(request_path) do
+ filename = filename(url)
+
+ if filename && not basename_matches?(request_path, filename) do
+ {:wrong_filename, filename}
+ else
+ :ok
+ end
+ end
+
+ def verify_request_path_and_url(_, _), do: :ok
+
+ defp basename_matches?(path, filename) do
+ basename = Path.basename(path)
+ basename == filename or URI.decode(basename) == filename or URI.encode(basename) == filename
+ end
end
diff --git a/lib/pleroma/web/media_proxy/invalidation.ex b/lib/pleroma/web/media_proxy/invalidation.ex
index 5808861e6..4f4340478 100644
--- a/lib/pleroma/web/media_proxy/invalidation.ex
+++ b/lib/pleroma/web/media_proxy/invalidation.ex
@@ -33,6 +33,8 @@ defp do_purge(urls) do
def prepare_urls(urls) do
urls
|> List.wrap()
- |> Enum.map(&MediaProxy.url/1)
+ |> Enum.map(fn url -> [MediaProxy.url(url), MediaProxy.preview_url(url)] end)
+ |> List.flatten()
+ |> Enum.uniq()
end
end
diff --git a/lib/pleroma/web/media_proxy/invalidations/http.ex b/lib/pleroma/web/media_proxy/invalidation/http.ex
similarity index 97%
rename from lib/pleroma/web/media_proxy/invalidations/http.ex
rename to lib/pleroma/web/media_proxy/invalidation/http.ex
index bb81d8888..0b0cde68c 100644
--- a/lib/pleroma/web/media_proxy/invalidations/http.ex
+++ b/lib/pleroma/web/media_proxy/invalidation/http.ex
@@ -30,7 +30,7 @@ defp do_purge(method, url, headers, options) do
{:ok, %{status: status} = env} when 400 <= status and status < 500 ->
{:error, env}
- {:error, error} = error ->
+ {:error, _} = error ->
error
_ ->
diff --git a/lib/pleroma/web/media_proxy/invalidations/script.ex b/lib/pleroma/web/media_proxy/invalidation/script.ex
similarity index 100%
rename from lib/pleroma/web/media_proxy/invalidations/script.ex
rename to lib/pleroma/web/media_proxy/invalidation/script.ex
diff --git a/lib/pleroma/web/media_proxy/media_proxy_controller.ex b/lib/pleroma/web/media_proxy/media_proxy_controller.ex
index 9a64b0ef3..90651ed9b 100644
--- a/lib/pleroma/web/media_proxy/media_proxy_controller.ex
+++ b/lib/pleroma/web/media_proxy/media_proxy_controller.ex
@@ -5,44 +5,201 @@
defmodule Pleroma.Web.MediaProxy.MediaProxyController do
use Pleroma.Web, :controller
+ alias Pleroma.Config
+ alias Pleroma.Helpers.MediaHelper
+ alias Pleroma.Helpers.UriHelper
alias Pleroma.ReverseProxy
alias Pleroma.Web.MediaProxy
+ alias Plug.Conn
- @default_proxy_opts [max_body_length: 25 * 1_048_576, http: [follow_redirect: true]]
-
- def remote(conn, %{"sig" => sig64, "url" => url64} = params) do
- with config <- Pleroma.Config.get([:media_proxy], []),
- true <- Keyword.get(config, :enabled, false),
+ def remote(conn, %{"sig" => sig64, "url" => url64}) do
+ with {_, true} <- {:enabled, MediaProxy.enabled?()},
{:ok, url} <- MediaProxy.decode_url(sig64, url64),
{_, false} <- {:in_banned_urls, MediaProxy.in_banned_urls(url)},
- :ok <- filename_matches(params, conn.request_path, url) do
- ReverseProxy.call(conn, url, Keyword.get(config, :proxy_opts, @default_proxy_opts))
+ :ok <- MediaProxy.verify_request_path_and_url(conn, url) do
+ ReverseProxy.call(conn, url, media_proxy_opts())
else
- error when error in [false, {:in_banned_urls, true}] ->
- send_resp(conn, 404, Plug.Conn.Status.reason_phrase(404))
+ {:enabled, false} ->
+ send_resp(conn, 404, Conn.Status.reason_phrase(404))
+
+ {:in_banned_urls, true} ->
+ send_resp(conn, 404, Conn.Status.reason_phrase(404))
{:error, :invalid_signature} ->
- send_resp(conn, 403, Plug.Conn.Status.reason_phrase(403))
+ send_resp(conn, 403, Conn.Status.reason_phrase(403))
{:wrong_filename, filename} ->
redirect(conn, external: MediaProxy.build_url(sig64, url64, filename))
end
end
- def filename_matches(%{"filename" => _} = _, path, url) do
- filename = MediaProxy.filename(url)
-
- if filename && does_not_match(path, filename) do
- {:wrong_filename, filename}
+ def preview(%Conn{} = conn, %{"sig" => sig64, "url" => url64}) do
+ with {_, true} <- {:enabled, MediaProxy.preview_enabled?()},
+ {:ok, url} <- MediaProxy.decode_url(sig64, url64),
+ :ok <- MediaProxy.verify_request_path_and_url(conn, url) do
+ handle_preview(conn, url)
else
- :ok
+ {:enabled, false} ->
+ send_resp(conn, 404, Conn.Status.reason_phrase(404))
+
+ {:error, :invalid_signature} ->
+ send_resp(conn, 403, Conn.Status.reason_phrase(403))
+
+ {:wrong_filename, filename} ->
+ redirect(conn, external: MediaProxy.build_preview_url(sig64, url64, filename))
end
end
- def filename_matches(_, _, _), do: :ok
+ defp handle_preview(conn, url) do
+ media_proxy_url = MediaProxy.url(url)
- defp does_not_match(path, filename) do
- basename = Path.basename(path)
- basename != filename and URI.decode(basename) != filename and URI.encode(basename) != filename
+ with {:ok, %{status: status} = head_response} when status in 200..299 <-
+ Pleroma.HTTP.request("head", media_proxy_url, [], [], pool: :media) do
+ content_type = Tesla.get_header(head_response, "content-type")
+ content_length = Tesla.get_header(head_response, "content-length")
+ content_length = content_length && String.to_integer(content_length)
+ static = conn.params["static"] in ["true", true]
+
+ cond do
+ static and content_type == "image/gif" ->
+ handle_jpeg_preview(conn, media_proxy_url)
+
+ static ->
+ drop_static_param_and_redirect(conn)
+
+ content_type == "image/gif" ->
+ redirect(conn, external: media_proxy_url)
+
+ min_content_length_for_preview() > 0 and content_length > 0 and
+ content_length < min_content_length_for_preview() ->
+ redirect(conn, external: media_proxy_url)
+
+ true ->
+ handle_preview(content_type, conn, media_proxy_url)
+ end
+ else
+ # If HEAD failed, redirecting to media proxy URI doesn't make much sense; returning an error
+ {_, %{status: status}} ->
+ send_resp(conn, :failed_dependency, "Can't fetch HTTP headers (HTTP #{status}).")
+
+ {:error, :recv_response_timeout} ->
+ send_resp(conn, :failed_dependency, "HEAD request timeout.")
+
+ _ ->
+ send_resp(conn, :failed_dependency, "Can't fetch HTTP headers.")
+ end
+ end
+
+ defp handle_preview("image/png" <> _ = _content_type, conn, media_proxy_url) do
+ handle_png_preview(conn, media_proxy_url)
+ end
+
+ defp handle_preview("image/" <> _ = _content_type, conn, media_proxy_url) do
+ handle_jpeg_preview(conn, media_proxy_url)
+ end
+
+ defp handle_preview("video/" <> _ = _content_type, conn, media_proxy_url) do
+ handle_video_preview(conn, media_proxy_url)
+ end
+
+ defp handle_preview(_unsupported_content_type, conn, media_proxy_url) do
+ fallback_on_preview_error(conn, media_proxy_url)
+ end
+
+ defp handle_png_preview(conn, media_proxy_url) do
+ quality = Config.get!([:media_preview_proxy, :image_quality])
+ {thumbnail_max_width, thumbnail_max_height} = thumbnail_max_dimensions()
+
+ with {:ok, thumbnail_binary} <-
+ MediaHelper.image_resize(
+ media_proxy_url,
+ %{
+ max_width: thumbnail_max_width,
+ max_height: thumbnail_max_height,
+ quality: quality,
+ format: "png"
+ }
+ ) do
+ conn
+ |> put_preview_response_headers(["image/png", "preview.png"])
+ |> send_resp(200, thumbnail_binary)
+ else
+ _ ->
+ fallback_on_preview_error(conn, media_proxy_url)
+ end
+ end
+
+ defp handle_jpeg_preview(conn, media_proxy_url) do
+ quality = Config.get!([:media_preview_proxy, :image_quality])
+ {thumbnail_max_width, thumbnail_max_height} = thumbnail_max_dimensions()
+
+ with {:ok, thumbnail_binary} <-
+ MediaHelper.image_resize(
+ media_proxy_url,
+ %{max_width: thumbnail_max_width, max_height: thumbnail_max_height, quality: quality}
+ ) do
+ conn
+ |> put_preview_response_headers()
+ |> send_resp(200, thumbnail_binary)
+ else
+ _ ->
+ fallback_on_preview_error(conn, media_proxy_url)
+ end
+ end
+
+ defp handle_video_preview(conn, media_proxy_url) do
+ with {:ok, thumbnail_binary} <-
+ MediaHelper.video_framegrab(media_proxy_url) do
+ conn
+ |> put_preview_response_headers()
+ |> send_resp(200, thumbnail_binary)
+ else
+ _ ->
+ fallback_on_preview_error(conn, media_proxy_url)
+ end
+ end
+
+ defp drop_static_param_and_redirect(conn) do
+ uri_without_static_param =
+ conn
+ |> current_url()
+ |> UriHelper.modify_uri_params(%{}, ["static"])
+
+ redirect(conn, external: uri_without_static_param)
+ end
+
+ defp fallback_on_preview_error(conn, media_proxy_url) do
+ redirect(conn, external: media_proxy_url)
+ end
+
+ defp put_preview_response_headers(
+ conn,
+ [content_type, filename] = _content_info \\ ["image/jpeg", "preview.jpg"]
+ ) do
+ conn
+ |> put_resp_header("content-type", content_type)
+ |> put_resp_header("content-disposition", "inline; filename=\"#{filename}\"")
+ |> put_resp_header("cache-control", ReverseProxy.default_cache_control_header())
+ end
+
+ defp thumbnail_max_dimensions do
+ config = media_preview_proxy_config()
+
+ thumbnail_max_width = Keyword.fetch!(config, :thumbnail_max_width)
+ thumbnail_max_height = Keyword.fetch!(config, :thumbnail_max_height)
+
+ {thumbnail_max_width, thumbnail_max_height}
+ end
+
+ defp min_content_length_for_preview do
+ Keyword.get(media_preview_proxy_config(), :min_content_length, 0)
+ end
+
+ defp media_preview_proxy_config do
+ Config.get!([:media_preview_proxy])
+ end
+
+ defp media_proxy_opts do
+ Config.get([:media_proxy, :proxy_opts], [])
end
end
diff --git a/lib/pleroma/web/metadata/feed.ex b/lib/pleroma/web/metadata/providers/feed.ex
similarity index 100%
rename from lib/pleroma/web/metadata/feed.ex
rename to lib/pleroma/web/metadata/providers/feed.ex
diff --git a/lib/pleroma/web/metadata/opengraph.ex b/lib/pleroma/web/metadata/providers/open_graph.ex
similarity index 100%
rename from lib/pleroma/web/metadata/opengraph.ex
rename to lib/pleroma/web/metadata/providers/open_graph.ex
diff --git a/lib/pleroma/web/metadata/provider.ex b/lib/pleroma/web/metadata/providers/provider.ex
similarity index 100%
rename from lib/pleroma/web/metadata/provider.ex
rename to lib/pleroma/web/metadata/providers/provider.ex
diff --git a/lib/pleroma/web/metadata/rel_me.ex b/lib/pleroma/web/metadata/providers/rel_me.ex
similarity index 100%
rename from lib/pleroma/web/metadata/rel_me.ex
rename to lib/pleroma/web/metadata/providers/rel_me.ex
diff --git a/lib/pleroma/web/metadata/restrict_indexing.ex b/lib/pleroma/web/metadata/providers/restrict_indexing.ex
similarity index 81%
rename from lib/pleroma/web/metadata/restrict_indexing.ex
rename to lib/pleroma/web/metadata/providers/restrict_indexing.ex
index f15607896..900c2434d 100644
--- a/lib/pleroma/web/metadata/restrict_indexing.ex
+++ b/lib/pleroma/web/metadata/providers/restrict_indexing.ex
@@ -10,7 +10,9 @@ defmodule Pleroma.Web.Metadata.Providers.RestrictIndexing do
"""
@impl true
- def build_tags(%{user: %{local: false}}) do
+ def build_tags(%{user: %{local: true, is_discoverable: true}}), do: []
+
+ def build_tags(_) do
[
{:meta,
[
@@ -19,7 +21,4 @@ def build_tags(%{user: %{local: false}}) do
], []}
]
end
-
- @impl true
- def build_tags(%{user: %{local: true}}), do: []
end
diff --git a/lib/pleroma/web/metadata/twitter_card.ex b/lib/pleroma/web/metadata/providers/twitter_card.ex
similarity index 100%
rename from lib/pleroma/web/metadata/twitter_card.ex
rename to lib/pleroma/web/metadata/providers/twitter_card.ex
diff --git a/lib/pleroma/web/metadata/utils.ex b/lib/pleroma/web/metadata/utils.ex
index 2f0dfb474..8a206e019 100644
--- a/lib/pleroma/web/metadata/utils.ex
+++ b/lib/pleroma/web/metadata/utils.ex
@@ -38,7 +38,7 @@ def scrub_html(content) when is_binary(content) do
def scrub_html(content), do: content
def attachment_url(url) do
- MediaProxy.url(url)
+ MediaProxy.preview_url(url)
end
def user_name_string(user) do
diff --git a/lib/pleroma/web/mongooseim/mongoose_im_controller.ex b/lib/pleroma/web/mongoose_im/mongoose_im_controller.ex
similarity index 93%
rename from lib/pleroma/web/mongooseim/mongoose_im_controller.ex
rename to lib/pleroma/web/mongoose_im/mongoose_im_controller.ex
index 6cbbe8fd8..2a5c7c356 100644
--- a/lib/pleroma/web/mongooseim/mongoose_im_controller.ex
+++ b/lib/pleroma/web/mongoose_im/mongoose_im_controller.ex
@@ -5,10 +5,10 @@
defmodule Pleroma.Web.MongooseIM.MongooseIMController do
use Pleroma.Web, :controller
- alias Pleroma.Plugs.AuthenticationPlug
- alias Pleroma.Plugs.RateLimiter
alias Pleroma.Repo
alias Pleroma.User
+ alias Pleroma.Web.Plugs.AuthenticationPlug
+ alias Pleroma.Web.Plugs.RateLimiter
plug(RateLimiter, [name: :authentication] when action in [:user_exists, :check_password])
plug(RateLimiter, [name: :authentication, params: ["user"]] when action == :check_password)
diff --git a/lib/pleroma/web/oauth.ex b/lib/pleroma/web/o_auth.ex
similarity index 100%
rename from lib/pleroma/web/oauth.ex
rename to lib/pleroma/web/o_auth.ex
diff --git a/lib/pleroma/web/oauth/app.ex b/lib/pleroma/web/o_auth/app.ex
similarity index 100%
rename from lib/pleroma/web/oauth/app.ex
rename to lib/pleroma/web/o_auth/app.ex
diff --git a/lib/pleroma/web/oauth/authorization.ex b/lib/pleroma/web/o_auth/authorization.ex
similarity index 100%
rename from lib/pleroma/web/oauth/authorization.ex
rename to lib/pleroma/web/o_auth/authorization.ex
diff --git a/lib/pleroma/web/oauth/fallback_controller.ex b/lib/pleroma/web/o_auth/fallback_controller.ex
similarity index 100%
rename from lib/pleroma/web/oauth/fallback_controller.ex
rename to lib/pleroma/web/o_auth/fallback_controller.ex
diff --git a/lib/pleroma/web/oauth/mfa_controller.ex b/lib/pleroma/web/o_auth/mfa_controller.ex
similarity index 100%
rename from lib/pleroma/web/oauth/mfa_controller.ex
rename to lib/pleroma/web/o_auth/mfa_controller.ex
diff --git a/lib/pleroma/web/oauth/mfa_view.ex b/lib/pleroma/web/o_auth/mfa_view.ex
similarity index 100%
rename from lib/pleroma/web/oauth/mfa_view.ex
rename to lib/pleroma/web/o_auth/mfa_view.ex
diff --git a/lib/pleroma/web/oauth/oauth_controller.ex b/lib/pleroma/web/o_auth/o_auth_controller.ex
similarity index 97%
rename from lib/pleroma/web/oauth/oauth_controller.ex
rename to lib/pleroma/web/o_auth/o_auth_controller.ex
index dd00600ea..d2f9d1ceb 100644
--- a/lib/pleroma/web/oauth/oauth_controller.ex
+++ b/lib/pleroma/web/o_auth/o_auth_controller.ex
@@ -8,7 +8,6 @@ defmodule Pleroma.Web.OAuth.OAuthController do
alias Pleroma.Helpers.UriHelper
alias Pleroma.Maps
alias Pleroma.MFA
- alias Pleroma.Plugs.RateLimiter
alias Pleroma.Registration
alias Pleroma.Repo
alias Pleroma.User
@@ -23,6 +22,7 @@ defmodule Pleroma.Web.OAuth.OAuthController do
alias Pleroma.Web.OAuth.Token
alias Pleroma.Web.OAuth.Token.Strategy.RefreshToken
alias Pleroma.Web.OAuth.Token.Strategy.Revoke, as: RevokeToken
+ alias Pleroma.Web.Plugs.RateLimiter
require Logger
@@ -31,7 +31,10 @@ defmodule Pleroma.Web.OAuth.OAuthController do
plug(:fetch_session)
plug(:fetch_flash)
- plug(:skip_plug, [Pleroma.Plugs.OAuthScopesPlug, Pleroma.Plugs.EnsurePublicOrAuthenticatedPlug])
+ plug(:skip_plug, [
+ Pleroma.Web.Plugs.OAuthScopesPlug,
+ Pleroma.Web.Plugs.EnsurePublicOrAuthenticatedPlug
+ ])
plug(RateLimiter, [name: :authentication] when action == :create_authorization)
@@ -119,7 +122,7 @@ defp handle_existing_authorization(
redirect_uri = redirect_uri(conn, redirect_uri)
url_params = %{access_token: token.token}
url_params = Maps.put_if_present(url_params, :state, params["state"])
- url = UriHelper.append_uri_params(redirect_uri, url_params)
+ url = UriHelper.modify_uri_params(redirect_uri, url_params)
redirect(conn, external: url)
else
conn
@@ -145,7 +148,10 @@ def create_authorization(
def after_create_authorization(%Plug.Conn{} = conn, %Authorization{} = auth, %{
"authorization" => %{"redirect_uri" => @oob_token_redirect_uri}
}) do
- render(conn, "oob_authorization_created.html", %{auth: auth})
+ # Enforcing the view to reuse the template when calling from other controllers
+ conn
+ |> put_view(OAuthView)
+ |> render("oob_authorization_created.html", %{auth: auth})
end
def after_create_authorization(%Plug.Conn{} = conn, %Authorization{} = auth, %{
@@ -158,7 +164,7 @@ def after_create_authorization(%Plug.Conn{} = conn, %Authorization{} = auth, %{
redirect_uri = redirect_uri(conn, redirect_uri)
url_params = %{code: auth.token}
url_params = Maps.put_if_present(url_params, :state, auth_attrs["state"])
- url = UriHelper.append_uri_params(redirect_uri, url_params)
+ url = UriHelper.modify_uri_params(redirect_uri, url_params)
redirect(conn, external: url)
else
conn
@@ -197,7 +203,7 @@ defp handle_create_authorization_error(
{:mfa_required, user, auth, _},
params
) do
- {:ok, token} = MFA.Token.create_token(user, auth)
+ {:ok, token} = MFA.Token.create(user, auth)
data = %{
"mfa_token" => token.token,
@@ -579,7 +585,7 @@ defp put_session_registration_id(%Plug.Conn{} = conn, registration_id),
do: put_session(conn, :registration_id, registration_id)
defp build_and_response_mfa_token(user, auth) do
- with {:ok, token} <- MFA.Token.create_token(user, auth) do
+ with {:ok, token} <- MFA.Token.create(user, auth) do
MFAView.render("mfa_response.json", %{token: token, user: user})
end
end
diff --git a/lib/pleroma/web/oauth/oauth_view.ex b/lib/pleroma/web/o_auth/o_auth_view.ex
similarity index 100%
rename from lib/pleroma/web/oauth/oauth_view.ex
rename to lib/pleroma/web/o_auth/o_auth_view.ex
diff --git a/lib/pleroma/web/oauth/scopes.ex b/lib/pleroma/web/o_auth/scopes.ex
similarity index 97%
rename from lib/pleroma/web/oauth/scopes.ex
rename to lib/pleroma/web/o_auth/scopes.ex
index 6f06f1431..90b9a0471 100644
--- a/lib/pleroma/web/oauth/scopes.ex
+++ b/lib/pleroma/web/o_auth/scopes.ex
@@ -7,7 +7,7 @@ defmodule Pleroma.Web.OAuth.Scopes do
Functions for dealing with scopes.
"""
- alias Pleroma.Plugs.OAuthScopesPlug
+ alias Pleroma.Web.Plugs.OAuthScopesPlug
@doc """
Fetch scopes from request params.
diff --git a/lib/pleroma/web/oauth/token.ex b/lib/pleroma/web/o_auth/token.ex
similarity index 86%
rename from lib/pleroma/web/oauth/token.ex
rename to lib/pleroma/web/o_auth/token.ex
index 08bb7326d..de37998f2 100644
--- a/lib/pleroma/web/oauth/token.ex
+++ b/lib/pleroma/web/o_auth/token.ex
@@ -50,7 +50,7 @@ def exchange_token(app, auth) do
true <- auth.app_id == app.id do
user = if auth.user_id, do: User.get_cached_by_id(auth.user_id), else: %User{}
- create_token(
+ create(
app,
user,
%{scopes: auth.scopes}
@@ -83,8 +83,22 @@ defp put_valid_until(changeset, attrs) do
|> validate_required([:valid_until])
end
- @spec create_token(App.t(), User.t(), map()) :: {:ok, Token} | {:error, Changeset.t()}
- def create_token(%App{} = app, %User{} = user, attrs \\ %{}) do
+ @spec create(App.t(), User.t(), map()) :: {:ok, Token} | {:error, Changeset.t()}
+ def create(%App{} = app, %User{} = user, attrs \\ %{}) do
+ with {:ok, token} <- do_create(app, user, attrs) do
+ if Pleroma.Config.get([:oauth2, :clean_expired_tokens]) do
+ Pleroma.Workers.PurgeExpiredToken.enqueue(%{
+ token_id: token.id,
+ valid_until: DateTime.from_naive!(token.valid_until, "Etc/UTC"),
+ mod: __MODULE__
+ })
+ end
+
+ {:ok, token}
+ end
+ end
+
+ defp do_create(app, user, attrs) do
%__MODULE__{user_id: user.id, app_id: app.id}
|> cast(%{scopes: attrs[:scopes] || app.scopes}, [:scopes])
|> validate_required([:scopes, :app_id])
@@ -105,11 +119,6 @@ def delete_user_token(%User{id: user_id}, token_id) do
|> Repo.delete_all()
end
- def delete_expired_tokens do
- Query.get_expired_tokens()
- |> Repo.delete_all()
- end
-
def get_user_tokens(%User{id: user_id}) do
Query.get_by_user(user_id)
|> Query.preload([:app])
diff --git a/lib/pleroma/web/oauth/token/query.ex b/lib/pleroma/web/o_auth/token/query.ex
similarity index 85%
rename from lib/pleroma/web/oauth/token/query.ex
rename to lib/pleroma/web/o_auth/token/query.ex
index 93d6e26ed..fd6d9b112 100644
--- a/lib/pleroma/web/oauth/token/query.ex
+++ b/lib/pleroma/web/o_auth/token/query.ex
@@ -33,12 +33,6 @@ def get_by_id(query \\ Token, id) do
from(q in query, where: q.id == ^id)
end
- @spec get_expired_tokens(query, DateTime.t() | nil) :: query
- def get_expired_tokens(query \\ Token, date \\ nil) do
- expired_date = date || Timex.now()
- from(q in query, where: fragment("?", q.valid_until) < ^expired_date)
- end
-
@spec get_by_user(query, String.t()) :: query
def get_by_user(query \\ Token, user_id) do
from(q in query, where: q.user_id == ^user_id)
diff --git a/lib/pleroma/web/oauth/token/strategy/refresh_token.ex b/lib/pleroma/web/o_auth/token/strategy/refresh_token.ex
similarity index 94%
rename from lib/pleroma/web/oauth/token/strategy/refresh_token.ex
rename to lib/pleroma/web/o_auth/token/strategy/refresh_token.ex
index debc29b0b..625b0fde2 100644
--- a/lib/pleroma/web/oauth/token/strategy/refresh_token.ex
+++ b/lib/pleroma/web/o_auth/token/strategy/refresh_token.ex
@@ -46,7 +46,7 @@ defp revoke_access_token(token) do
defp create_access_token({:error, error}, _), do: {:error, error}
defp create_access_token({:ok, token}, %{app: app, user: user} = token_params) do
- Token.create_token(app, user, add_refresh_token(token_params, token.refresh_token))
+ Token.create(app, user, add_refresh_token(token_params, token.refresh_token))
end
defp add_refresh_token(params, token) do
diff --git a/lib/pleroma/web/oauth/token/strategy/revoke.ex b/lib/pleroma/web/o_auth/token/strategy/revoke.ex
similarity index 100%
rename from lib/pleroma/web/oauth/token/strategy/revoke.ex
rename to lib/pleroma/web/o_auth/token/strategy/revoke.ex
diff --git a/lib/pleroma/web/oauth/token/utils.ex b/lib/pleroma/web/o_auth/token/utils.ex
similarity index 100%
rename from lib/pleroma/web/oauth/token/utils.ex
rename to lib/pleroma/web/o_auth/token/utils.ex
diff --git a/lib/pleroma/web/ostatus/ostatus_controller.ex b/lib/pleroma/web/o_status/o_status_controller.ex
similarity index 89%
rename from lib/pleroma/web/ostatus/ostatus_controller.ex
rename to lib/pleroma/web/o_status/o_status_controller.ex
index de1b0b3f0..668ae0ea4 100644
--- a/lib/pleroma/web/ostatus/ostatus_controller.ex
+++ b/lib/pleroma/web/o_status/o_status_controller.ex
@@ -5,28 +5,24 @@
defmodule Pleroma.Web.OStatus.OStatusController do
use Pleroma.Web, :controller
- alias Fallback.RedirectController
alias Pleroma.Activity
alias Pleroma.Object
- alias Pleroma.Plugs.RateLimiter
alias Pleroma.User
alias Pleroma.Web.ActivityPub.ActivityPubController
alias Pleroma.Web.ActivityPub.Visibility
alias Pleroma.Web.Endpoint
+ alias Pleroma.Web.Fallback.RedirectController
alias Pleroma.Web.Metadata.PlayerView
+ alias Pleroma.Web.Plugs.RateLimiter
alias Pleroma.Web.Router
- plug(Pleroma.Plugs.EnsureAuthenticatedPlug,
- unless_func: &Pleroma.Web.FederatingPlug.federating?/1
- )
-
plug(
RateLimiter,
[name: :ap_routes, params: ["uuid"]] when action in [:object, :activity]
)
plug(
- Pleroma.Plugs.SetFormatPlug
+ Pleroma.Web.Plugs.SetFormatPlug
when action in [:object, :activity, :notice]
)
@@ -37,14 +33,12 @@ def object(%{assigns: %{format: format}} = conn, _params)
ActivityPubController.call(conn, :object)
end
- def object(%{assigns: %{format: format}} = conn, _params) do
+ def object(conn, _params) do
with id <- Endpoint.url() <> conn.request_path,
{_, %Activity{} = activity} <-
{:activity, Activity.get_create_by_object_ap_id_with_object(id)},
{_, true} <- {:public?, Visibility.is_public?(activity)} do
- case format do
- _ -> redirect(conn, to: "/notice/#{activity.id}")
- end
+ redirect(conn, to: "/notice/#{activity.id}")
else
reason when reason in [{:public?, false}, {:activity, nil}] ->
{:error, :not_found}
@@ -59,13 +53,11 @@ def activity(%{assigns: %{format: format}} = conn, _params)
ActivityPubController.call(conn, :activity)
end
- def activity(%{assigns: %{format: format}} = conn, _params) do
+ def activity(conn, _params) do
with id <- Endpoint.url() <> conn.request_path,
{_, %Activity{} = activity} <- {:activity, Activity.normalize(id)},
{_, true} <- {:public?, Visibility.is_public?(activity)} do
- case format do
- _ -> redirect(conn, to: "/notice/#{activity.id}")
- end
+ redirect(conn, to: "/notice/#{activity.id}")
else
reason when reason in [{:public?, false}, {:activity, nil}] ->
{:error, :not_found}
@@ -119,6 +111,7 @@ def notice(%{assigns: %{format: format}} = conn, %{"id" => id}) do
def notice_player(conn, %{"id" => id}) do
with %Activity{data: %{"type" => "Create"}} = activity <- Activity.get_by_id_with_object(id),
true <- Visibility.is_public?(activity),
+ {_, true} <- {:visible?, Visibility.visible_for_user?(activity, _reading_user = nil)},
%Object{} = object <- Object.normalize(activity),
%{data: %{"attachment" => [%{"url" => [url | _]} | _]}} <- object,
true <- String.starts_with?(url["mediaType"], ["audio", "video"]) do
diff --git a/lib/pleroma/web/oauth/token/clean_worker.ex b/lib/pleroma/web/oauth/token/clean_worker.ex
deleted file mode 100644
index e3aa4eb7e..000000000
--- a/lib/pleroma/web/oauth/token/clean_worker.ex
+++ /dev/null
@@ -1,38 +0,0 @@
-# Pleroma: A lightweight social networking server
-# Copyright © 2017-2020 Pleroma Authors
-# SPDX-License-Identifier: AGPL-3.0-only
-
-defmodule Pleroma.Web.OAuth.Token.CleanWorker do
- @moduledoc """
- The module represents functions to clean an expired OAuth and MFA tokens.
- """
- use GenServer
-
- @ten_seconds 10_000
- @one_day 86_400_000
-
- alias Pleroma.MFA
- alias Pleroma.Web.OAuth
- alias Pleroma.Workers.BackgroundWorker
-
- def start_link(_), do: GenServer.start_link(__MODULE__, %{})
-
- def init(_) do
- Process.send_after(self(), :perform, @ten_seconds)
- {:ok, nil}
- end
-
- @doc false
- def handle_info(:perform, state) do
- BackgroundWorker.enqueue("clean_expired_tokens", %{})
- interval = Pleroma.Config.get([:oauth2, :clean_expired_tokens_interval], @one_day)
-
- Process.send_after(self(), :perform, interval)
- {:noreply, state}
- end
-
- def perform(:clean) do
- OAuth.Token.delete_expired_tokens()
- MFA.Token.delete_expired_tokens()
- end
-end
diff --git a/lib/pleroma/web/pleroma_api/controllers/account_controller.ex b/lib/pleroma/web/pleroma_api/controllers/account_controller.ex
index 563edded7..30cf83567 100644
--- a/lib/pleroma/web/pleroma_api/controllers/account_controller.ex
+++ b/lib/pleroma/web/pleroma_api/controllers/account_controller.ex
@@ -8,15 +8,20 @@ defmodule Pleroma.Web.PleromaAPI.AccountController do
import Pleroma.Web.ControllerHelper,
only: [json_response: 3, add_link_headers: 2, assign_account_by_id: 2]
- alias Pleroma.Plugs.EnsurePublicOrAuthenticatedPlug
- alias Pleroma.Plugs.OAuthScopesPlug
- alias Pleroma.Plugs.RateLimiter
alias Pleroma.User
alias Pleroma.Web.ActivityPub.ActivityPub
alias Pleroma.Web.MastodonAPI.StatusView
+ alias Pleroma.Web.Plugs.EnsurePublicOrAuthenticatedPlug
+ alias Pleroma.Web.Plugs.OAuthScopesPlug
+ alias Pleroma.Web.Plugs.RateLimiter
require Pleroma.Constants
+ plug(
+ Majic.Plug,
+ [pool: Pleroma.MajicPool] when action in [:update_avatar, :update_background, :update_banner]
+ )
+
plug(
OpenApiSpex.Plug.PutApiSpec,
[module: Pleroma.Web.ApiSpec] when action == :confirmation_resend
diff --git a/lib/pleroma/web/pleroma_api/controllers/backup_controller.ex b/lib/pleroma/web/pleroma_api/controllers/backup_controller.ex
new file mode 100644
index 000000000..dd0a2e22f
--- /dev/null
+++ b/lib/pleroma/web/pleroma_api/controllers/backup_controller.ex
@@ -0,0 +1,28 @@
+# Pleroma: A lightweight social networking server
+# Copyright © 2017-2020 Pleroma Authors
+# SPDX-License-Identifier: AGPL-3.0-only
+
+defmodule Pleroma.Web.PleromaAPI.BackupController do
+ use Pleroma.Web, :controller
+
+ alias Pleroma.User.Backup
+ alias Pleroma.Web.Plugs.OAuthScopesPlug
+
+ action_fallback(Pleroma.Web.MastodonAPI.FallbackController)
+ plug(OAuthScopesPlug, %{scopes: ["read:accounts"]} when action in [:index, :create])
+ plug(OpenApiSpex.Plug.CastAndValidate, render_error: Pleroma.Web.ApiSpec.RenderError)
+
+ defdelegate open_api_operation(action), to: Pleroma.Web.ApiSpec.PleromaBackupOperation
+
+ def index(%{assigns: %{user: user}} = conn, _params) do
+ backups = Backup.list(user)
+ render(conn, "index.json", backups: backups)
+ end
+
+ def create(%{assigns: %{user: user}} = conn, _params) do
+ with {:ok, _} <- Backup.create(user) do
+ backups = Backup.list(user)
+ render(conn, "index.json", backups: backups)
+ end
+ end
+end
diff --git a/lib/pleroma/web/pleroma_api/controllers/chat_controller.ex b/lib/pleroma/web/pleroma_api/controllers/chat_controller.ex
index e8a1746d4..77564b342 100644
--- a/lib/pleroma/web/pleroma_api/controllers/chat_controller.ex
+++ b/lib/pleroma/web/pleroma_api/controllers/chat_controller.ex
@@ -4,17 +4,18 @@
defmodule Pleroma.Web.PleromaAPI.ChatController do
use Pleroma.Web, :controller
+ import Pleroma.Web.ControllerHelper, only: [add_link_headers: 2]
+
alias Pleroma.Activity
alias Pleroma.Chat
alias Pleroma.Chat.MessageReference
alias Pleroma.Object
alias Pleroma.Pagination
- alias Pleroma.Plugs.OAuthScopesPlug
alias Pleroma.Repo
alias Pleroma.User
alias Pleroma.Web.CommonAPI
alias Pleroma.Web.PleromaAPI.Chat.MessageReferenceView
- alias Pleroma.Web.PleromaAPI.ChatView
+ alias Pleroma.Web.Plugs.OAuthScopesPlug
import Ecto.Query
@@ -47,7 +48,7 @@ def delete_message(%{assigns: %{user: %{id: user_id} = user}} = conn, %{
}) do
with %MessageReference{} = cm_ref <-
MessageReference.get_by_id(message_id),
- ^chat_id <- cm_ref.chat_id |> to_string(),
+ ^chat_id <- to_string(cm_ref.chat_id),
%Chat{user_id: ^user_id} <- Chat.get_by_id(chat_id),
{:ok, _} <- remove_or_delete(cm_ref, user) do
conn
@@ -68,38 +69,43 @@ defp remove_or_delete(
end
end
- defp remove_or_delete(cm_ref, _) do
- cm_ref
- |> MessageReference.delete()
- end
+ defp remove_or_delete(cm_ref, _), do: MessageReference.delete(cm_ref)
def post_chat_message(
- %{body_params: params, assigns: %{user: %{id: user_id} = user}} = conn,
- %{
- id: id
- }
+ %{body_params: params, assigns: %{user: user}} = conn,
+ %{id: id}
) do
- with %Chat{} = chat <- Repo.get_by(Chat, id: id, user_id: user_id),
+ with {:ok, chat} <- Chat.get_by_user_and_id(user, id),
%User{} = recipient <- User.get_cached_by_ap_id(chat.recipient),
{:ok, activity} <-
CommonAPI.post_chat_message(user, recipient, params[:content],
- media_id: params[:media_id]
+ media_id: params[:media_id],
+ idempotency_key: idempotency_key(conn)
),
message <- Object.normalize(activity, false),
cm_ref <- MessageReference.for_chat_and_object(chat, message) do
conn
|> put_view(MessageReferenceView)
|> render("show.json", chat_message_reference: cm_ref)
+ else
+ {:reject, message} ->
+ conn
+ |> put_status(:unprocessable_entity)
+ |> json(%{error: message})
+
+ {:error, message} ->
+ conn
+ |> put_status(:bad_request)
+ |> json(%{error: message})
end
end
- def mark_message_as_read(%{assigns: %{user: %{id: user_id}}} = conn, %{
- id: chat_id,
- message_id: message_id
- }) do
- with %MessageReference{} = cm_ref <-
- MessageReference.get_by_id(message_id),
- ^chat_id <- cm_ref.chat_id |> to_string(),
+ def mark_message_as_read(
+ %{assigns: %{user: %{id: user_id}}} = conn,
+ %{id: chat_id, message_id: message_id}
+ ) do
+ with %MessageReference{} = cm_ref <- MessageReference.get_by_id(message_id),
+ ^chat_id <- to_string(cm_ref.chat_id),
%Chat{user_id: ^user_id} <- Chat.get_by_id(chat_id),
{:ok, cm_ref} <- MessageReference.mark_as_read(cm_ref) do
conn
@@ -109,69 +115,60 @@ def mark_message_as_read(%{assigns: %{user: %{id: user_id}}} = conn, %{
end
def mark_as_read(
- %{
- body_params: %{last_read_id: last_read_id},
- assigns: %{user: %{id: user_id}}
- } = conn,
+ %{body_params: %{last_read_id: last_read_id}, assigns: %{user: user}} = conn,
%{id: id}
) do
- with %Chat{} = chat <- Repo.get_by(Chat, id: id, user_id: user_id),
- {_n, _} <-
- MessageReference.set_all_seen_for_chat(chat, last_read_id) do
- conn
- |> put_view(ChatView)
- |> render("show.json", chat: chat)
+ with {:ok, chat} <- Chat.get_by_user_and_id(user, id),
+ {_n, _} <- MessageReference.set_all_seen_for_chat(chat, last_read_id) do
+ render(conn, "show.json", chat: chat)
end
end
- def messages(%{assigns: %{user: %{id: user_id}}} = conn, %{id: id} = params) do
- with %Chat{} = chat <- Repo.get_by(Chat, id: id, user_id: user_id) do
- cm_refs =
+ def messages(%{assigns: %{user: user}} = conn, %{id: id} = params) do
+ with {:ok, chat} <- Chat.get_by_user_and_id(user, id) do
+ chat_message_refs =
chat
|> MessageReference.for_chat_query()
|> Pagination.fetch_paginated(params)
conn
+ |> add_link_headers(chat_message_refs)
|> put_view(MessageReferenceView)
- |> render("index.json", chat_message_references: cm_refs)
- else
- _ ->
- conn
- |> put_status(:not_found)
- |> json(%{error: "not found"})
+ |> render("index.json", chat_message_references: chat_message_refs)
end
end
- def index(%{assigns: %{user: %{id: user_id} = user}} = conn, _params) do
- blocked_ap_ids = User.blocked_users_ap_ids(user)
+ def index(%{assigns: %{user: %{id: user_id} = user}} = conn, params) do
+ exclude_users =
+ User.blocked_users_ap_ids(user) ++
+ if params[:with_muted], do: [], else: User.muted_users_ap_ids(user)
chats =
- from(c in Chat,
- where: c.user_id == ^user_id,
- where: c.recipient not in ^blocked_ap_ids,
- order_by: [desc: c.updated_at]
- )
+ user_id
+ |> Chat.for_user_query()
+ |> where([c], c.recipient not in ^exclude_users)
|> Repo.all()
- conn
- |> put_view(ChatView)
- |> render("index.json", chats: chats)
+ render(conn, "index.json", chats: chats)
end
- def create(%{assigns: %{user: user}} = conn, params) do
- with %User{ap_id: recipient} <- User.get_by_id(params[:id]),
+ def create(%{assigns: %{user: user}} = conn, %{id: id}) do
+ with %User{ap_id: recipient} <- User.get_cached_by_id(id),
{:ok, %Chat{} = chat} <- Chat.get_or_create(user.id, recipient) do
- conn
- |> put_view(ChatView)
- |> render("show.json", chat: chat)
+ render(conn, "show.json", chat: chat)
end
end
- def show(%{assigns: %{user: user}} = conn, params) do
- with %Chat{} = chat <- Repo.get_by(Chat, user_id: user.id, id: params[:id]) do
- conn
- |> put_view(ChatView)
- |> render("show.json", chat: chat)
+ def show(%{assigns: %{user: user}} = conn, %{id: id}) do
+ with {:ok, chat} <- Chat.get_by_user_and_id(user, id) do
+ render(conn, "show.json", chat: chat)
+ end
+ end
+
+ defp idempotency_key(conn) do
+ case get_req_header(conn, "idempotency-key") do
+ [key] -> key
+ _ -> nil
end
end
end
diff --git a/lib/pleroma/web/pleroma_api/controllers/conversation_controller.ex b/lib/pleroma/web/pleroma_api/controllers/conversation_controller.ex
index 3d007f324..df52b7566 100644
--- a/lib/pleroma/web/pleroma_api/controllers/conversation_controller.ex
+++ b/lib/pleroma/web/pleroma_api/controllers/conversation_controller.ex
@@ -8,9 +8,9 @@ defmodule Pleroma.Web.PleromaAPI.ConversationController do
import Pleroma.Web.ControllerHelper, only: [add_link_headers: 2]
alias Pleroma.Conversation.Participation
- alias Pleroma.Plugs.OAuthScopesPlug
alias Pleroma.Web.ActivityPub.ActivityPub
alias Pleroma.Web.MastodonAPI.StatusView
+ alias Pleroma.Web.Plugs.OAuthScopesPlug
plug(Pleroma.Web.ApiSpec.CastAndValidate)
plug(:put_view, Pleroma.Web.MastodonAPI.ConversationView)
diff --git a/lib/pleroma/web/pleroma_api/controllers/emoji_file_controller.ex b/lib/pleroma/web/pleroma_api/controllers/emoji_file_controller.ex
new file mode 100644
index 000000000..428c97de6
--- /dev/null
+++ b/lib/pleroma/web/pleroma_api/controllers/emoji_file_controller.ex
@@ -0,0 +1,137 @@
+# Pleroma: A lightweight social networking server
+# Copyright © 2017-2020 Pleroma Authors
+# SPDX-License-Identifier: AGPL-3.0-only
+
+defmodule Pleroma.Web.PleromaAPI.EmojiFileController do
+ use Pleroma.Web, :controller
+
+ alias Pleroma.Emoji.Pack
+ alias Pleroma.Web.ApiSpec
+
+ plug(Pleroma.Web.ApiSpec.CastAndValidate)
+
+ plug(
+ Pleroma.Web.Plugs.OAuthScopesPlug,
+ %{scopes: ["write"], admin: true}
+ when action in [
+ :create,
+ :update,
+ :delete
+ ]
+ )
+
+ defdelegate open_api_operation(action), to: ApiSpec.PleromaEmojiFileOperation
+
+ def create(%{body_params: params} = conn, %{name: pack_name}) do
+ filename = params[:filename] || get_filename(params[:file])
+ shortcode = params[:shortcode] || Path.basename(filename, Path.extname(filename))
+
+ with {:ok, pack} <- Pack.load_pack(pack_name),
+ {:ok, file} <- get_file(params[:file]),
+ {:ok, pack} <- Pack.add_file(pack, shortcode, filename, file) do
+ json(conn, pack.files)
+ else
+ {:error, :already_exists} ->
+ conn
+ |> put_status(:conflict)
+ |> json(%{error: "An emoji with the \"#{shortcode}\" shortcode already exists"})
+
+ {:error, :empty_values} ->
+ conn
+ |> put_status(:unprocessable_entity)
+ |> json(%{error: "pack name, shortcode or filename cannot be empty"})
+
+ {:error, _} = error ->
+ handle_error(conn, error, %{pack_name: pack_name})
+ end
+ end
+
+ def update(%{body_params: %{shortcode: shortcode} = params} = conn, %{name: pack_name}) do
+ new_shortcode = params[:new_shortcode]
+ new_filename = params[:new_filename]
+ force = params[:force]
+
+ with {:ok, pack} <- Pack.load_pack(pack_name),
+ {:ok, pack} <- Pack.update_file(pack, shortcode, new_shortcode, new_filename, force) do
+ json(conn, pack.files)
+ else
+ {:error, :already_exists} ->
+ conn
+ |> put_status(:conflict)
+ |> json(%{
+ error:
+ "New shortcode \"#{new_shortcode}\" is already used. If you want to override emoji use 'force' option"
+ })
+
+ {:error, :empty_values} ->
+ conn
+ |> put_status(:unprocessable_entity)
+ |> json(%{error: "new_shortcode or new_filename cannot be empty"})
+
+ {:error, _} = error ->
+ handle_error(conn, error, %{pack_name: pack_name, code: shortcode})
+ end
+ end
+
+ def delete(conn, %{name: pack_name, shortcode: shortcode}) do
+ with {:ok, pack} <- Pack.load_pack(pack_name),
+ {:ok, pack} <- Pack.delete_file(pack, shortcode) do
+ json(conn, pack.files)
+ else
+ {:error, :empty_values} ->
+ conn
+ |> put_status(:unprocessable_entity)
+ |> json(%{error: "pack name or shortcode cannot be empty"})
+
+ {:error, _} = error ->
+ handle_error(conn, error, %{pack_name: pack_name, code: shortcode})
+ end
+ end
+
+ defp handle_error(conn, {:error, :doesnt_exist}, %{code: emoji_code}) do
+ conn
+ |> put_status(:bad_request)
+ |> json(%{error: "Emoji \"#{emoji_code}\" does not exist"})
+ end
+
+ defp handle_error(conn, {:error, :not_found}, %{pack_name: pack_name}) do
+ conn
+ |> put_status(:not_found)
+ |> json(%{error: "pack \"#{pack_name}\" is not found"})
+ end
+
+ defp handle_error(conn, {:error, _}, _) do
+ render_error(
+ conn,
+ :internal_server_error,
+ "Unexpected error occurred while adding file to pack."
+ )
+ end
+
+ defp get_filename(%Plug.Upload{filename: filename}), do: filename
+ defp get_filename(url) when is_binary(url), do: Path.basename(url)
+
+ def get_file(%Plug.Upload{} = file), do: {:ok, file}
+
+ def get_file(url) when is_binary(url) do
+ with {:ok, %Tesla.Env{body: body, status: code, headers: headers}}
+ when code in 200..299 <- Pleroma.HTTP.get(url) do
+ path = Plug.Upload.random_file!("emoji")
+
+ content_type =
+ case List.keyfind(headers, "content-type", 0) do
+ {"content-type", value} -> value
+ nil -> nil
+ end
+
+ File.write(path, body)
+
+ {:ok,
+ %Plug.Upload{
+ filename: Path.basename(url),
+ path: path,
+ content_type: content_type
+ }}
+ end
+ end
+end
diff --git a/lib/pleroma/web/pleroma_api/controllers/emoji_pack_controller.ex b/lib/pleroma/web/pleroma_api/controllers/emoji_pack_controller.ex
index 657f46324..a9accc5af 100644
--- a/lib/pleroma/web/pleroma_api/controllers/emoji_pack_controller.ex
+++ b/lib/pleroma/web/pleroma_api/controllers/emoji_pack_controller.ex
@@ -1,3 +1,7 @@
+# Pleroma: A lightweight social networking server
+# Copyright © 2017-2020 Pleroma Authors
+# SPDX-License-Identifier: AGPL-3.0-only
+
defmodule Pleroma.Web.PleromaAPI.EmojiPackController do
use Pleroma.Web, :controller
@@ -6,7 +10,7 @@ defmodule Pleroma.Web.PleromaAPI.EmojiPackController do
plug(Pleroma.Web.ApiSpec.CastAndValidate)
plug(
- Pleroma.Plugs.OAuthScopesPlug,
+ Pleroma.Web.Plugs.OAuthScopesPlug,
%{scopes: ["write"], admin: true}
when action in [
:import_from_filesystem,
@@ -14,20 +18,21 @@ defmodule Pleroma.Web.PleromaAPI.EmojiPackController do
:download,
:create,
:update,
- :delete,
- :add_file,
- :update_file,
- :delete_file
+ :delete
]
)
- @skip_plugs [Pleroma.Plugs.OAuthScopesPlug, Pleroma.Plugs.EnsurePublicOrAuthenticatedPlug]
- plug(:skip_plug, @skip_plugs when action in [:index, :show, :archive])
+ @skip_plugs [
+ Pleroma.Web.Plugs.OAuthScopesPlug,
+ Pleroma.Web.Plugs.EnsurePublicOrAuthenticatedPlug
+ ]
+ plug(:skip_plug, @skip_plugs when action in [:index, :archive, :show])
defdelegate open_api_operation(action), to: Pleroma.Web.ApiSpec.PleromaEmojiPackOperation
- def remote(conn, %{url: url}) do
- with {:ok, packs} <- Pack.list_remote(url) do
+ def remote(conn, params) do
+ with {:ok, packs} <-
+ Pack.list_remote(url: params.url, page_size: params.page_size, page: params.page) do
json(conn, packs)
else
{:error, :not_shareable} ->
@@ -184,105 +189,6 @@ def update(%{body_params: %{metadata: metadata}} = conn, %{name: name}) do
end
end
- def add_file(%{body_params: params} = conn, %{name: name}) do
- filename = params[:filename] || get_filename(params[:file])
- shortcode = params[:shortcode] || Path.basename(filename, Path.extname(filename))
-
- with {:ok, pack} <- Pack.add_file(name, shortcode, filename, params[:file]) do
- json(conn, pack.files)
- else
- {:error, :already_exists} ->
- conn
- |> put_status(:conflict)
- |> json(%{error: "An emoji with the \"#{shortcode}\" shortcode already exists"})
-
- {:error, :not_found} ->
- conn
- |> put_status(:bad_request)
- |> json(%{error: "pack \"#{name}\" is not found"})
-
- {:error, :empty_values} ->
- conn
- |> put_status(:bad_request)
- |> json(%{error: "pack name, shortcode or filename cannot be empty"})
-
- {:error, _} ->
- render_error(
- conn,
- :internal_server_error,
- "Unexpected error occurred while adding file to pack."
- )
- end
- end
-
- def update_file(%{body_params: %{shortcode: shortcode} = params} = conn, %{name: name}) do
- new_shortcode = params[:new_shortcode]
- new_filename = params[:new_filename]
- force = params[:force]
-
- with {:ok, pack} <- Pack.update_file(name, shortcode, new_shortcode, new_filename, force) do
- json(conn, pack.files)
- else
- {:error, :doesnt_exist} ->
- conn
- |> put_status(:bad_request)
- |> json(%{error: "Emoji \"#{shortcode}\" does not exist"})
-
- {:error, :already_exists} ->
- conn
- |> put_status(:conflict)
- |> json(%{
- error:
- "New shortcode \"#{new_shortcode}\" is already used. If you want to override emoji use 'force' option"
- })
-
- {:error, :not_found} ->
- conn
- |> put_status(:bad_request)
- |> json(%{error: "pack \"#{name}\" is not found"})
-
- {:error, :empty_values} ->
- conn
- |> put_status(:bad_request)
- |> json(%{error: "new_shortcode or new_filename cannot be empty"})
-
- {:error, _} ->
- render_error(
- conn,
- :internal_server_error,
- "Unexpected error occurred while updating file in pack."
- )
- end
- end
-
- def delete_file(conn, %{name: name, shortcode: shortcode}) do
- with {:ok, pack} <- Pack.delete_file(name, shortcode) do
- json(conn, pack.files)
- else
- {:error, :doesnt_exist} ->
- conn
- |> put_status(:bad_request)
- |> json(%{error: "Emoji \"#{shortcode}\" does not exist"})
-
- {:error, :not_found} ->
- conn
- |> put_status(:bad_request)
- |> json(%{error: "pack \"#{name}\" is not found"})
-
- {:error, :empty_values} ->
- conn
- |> put_status(:bad_request)
- |> json(%{error: "pack name or shortcode cannot be empty"})
-
- {:error, _} ->
- render_error(
- conn,
- :internal_server_error,
- "Unexpected error occurred while removing file from pack."
- )
- end
- end
-
def import_from_filesystem(conn, _params) do
with {:ok, names} <- Pack.import_from_filesystem() do
json(conn, names)
@@ -298,7 +204,4 @@ def import_from_filesystem(conn, _params) do
|> json(%{error: "Error accessing emoji pack directory"})
end
end
-
- defp get_filename(%Plug.Upload{filename: filename}), do: filename
- defp get_filename(url) when is_binary(url), do: Path.basename(url)
end
diff --git a/lib/pleroma/web/pleroma_api/controllers/emoji_reaction_controller.ex b/lib/pleroma/web/pleroma_api/controllers/emoji_reaction_controller.ex
index 7f9254c13..ae199a50f 100644
--- a/lib/pleroma/web/pleroma_api/controllers/emoji_reaction_controller.ex
+++ b/lib/pleroma/web/pleroma_api/controllers/emoji_reaction_controller.ex
@@ -7,9 +7,9 @@ defmodule Pleroma.Web.PleromaAPI.EmojiReactionController do
alias Pleroma.Activity
alias Pleroma.Object
- alias Pleroma.Plugs.OAuthScopesPlug
alias Pleroma.Web.CommonAPI
alias Pleroma.Web.MastodonAPI.StatusView
+ alias Pleroma.Web.Plugs.OAuthScopesPlug
plug(Pleroma.Web.ApiSpec.CastAndValidate)
plug(OAuthScopesPlug, %{scopes: ["write:statuses"]} when action in [:create, :delete])
diff --git a/lib/pleroma/web/pleroma_api/controllers/instances_controller.ex b/lib/pleroma/web/pleroma_api/controllers/instances_controller.ex
new file mode 100644
index 000000000..9e97480df
--- /dev/null
+++ b/lib/pleroma/web/pleroma_api/controllers/instances_controller.ex
@@ -0,0 +1,21 @@
+# Pleroma: A lightweight social networking server
+# Copyright © 2017-2020 Pleroma Authors
+# SPDX-License-Identifier: AGPL-3.0-only
+
+defmodule Pleroma.Web.PleromaAPI.InstancesController do
+ use Pleroma.Web, :controller
+
+ alias Pleroma.Instances
+
+ plug(Pleroma.Web.ApiSpec.CastAndValidate)
+
+ defdelegate open_api_operation(action), to: Pleroma.Web.ApiSpec.PleromaInstancesOperation
+
+ def show(conn, _params) do
+ unreachable =
+ Instances.get_consistently_unreachable()
+ |> Map.new(fn {host, date} -> {host, to_string(date)} end)
+
+ json(conn, %{"unreachable" => unreachable})
+ end
+end
diff --git a/lib/pleroma/web/pleroma_api/controllers/mascot_controller.ex b/lib/pleroma/web/pleroma_api/controllers/mascot_controller.ex
index df6c50ca5..15210f1e6 100644
--- a/lib/pleroma/web/pleroma_api/controllers/mascot_controller.ex
+++ b/lib/pleroma/web/pleroma_api/controllers/mascot_controller.ex
@@ -5,10 +5,11 @@
defmodule Pleroma.Web.PleromaAPI.MascotController do
use Pleroma.Web, :controller
- alias Pleroma.Plugs.OAuthScopesPlug
alias Pleroma.User
alias Pleroma.Web.ActivityPub.ActivityPub
+ alias Pleroma.Web.Plugs.OAuthScopesPlug
+ plug(Majic.Plug, [pool: Pleroma.MajicPool] when action in [:update])
plug(Pleroma.Web.ApiSpec.CastAndValidate)
plug(OAuthScopesPlug, %{scopes: ["read:accounts"]} when action == :show)
plug(OAuthScopesPlug, %{scopes: ["write:accounts"]} when action != :show)
@@ -22,14 +23,15 @@ def show(%{assigns: %{user: user}} = conn, _params) do
@doc "PUT /api/v1/pleroma/mascot"
def update(%{assigns: %{user: user}, body_params: %{file: file}} = conn, _) do
- with {:ok, object} <- ActivityPub.upload(file, actor: User.ap_id(user)),
- # Reject if not an image
- %{type: "image"} = attachment <- render_attachment(object) do
+ with {:content_type, "image" <> _} <- {:content_type, file.content_type},
+ {:ok, object} <- ActivityPub.upload(file, actor: User.ap_id(user)) do
+ attachment = render_attachment(object)
{:ok, _user} = User.mascot_update(user, attachment)
json(conn, attachment)
else
- %{type: _} -> render_error(conn, :unsupported_media_type, "mascots can only be images")
+ {:content_type, _} ->
+ render_error(conn, :unsupported_media_type, "mascots can only be images")
end
end
diff --git a/lib/pleroma/web/pleroma_api/controllers/notification_controller.ex b/lib/pleroma/web/pleroma_api/controllers/notification_controller.ex
index 3ed8bd294..fa32aaa84 100644
--- a/lib/pleroma/web/pleroma_api/controllers/notification_controller.ex
+++ b/lib/pleroma/web/pleroma_api/controllers/notification_controller.ex
@@ -6,10 +6,14 @@ defmodule Pleroma.Web.PleromaAPI.NotificationController do
use Pleroma.Web, :controller
alias Pleroma.Notification
- alias Pleroma.Plugs.OAuthScopesPlug
plug(Pleroma.Web.ApiSpec.CastAndValidate)
- plug(OAuthScopesPlug, %{scopes: ["write:notifications"]} when action == :mark_as_read)
+
+ plug(
+ Pleroma.Web.Plugs.OAuthScopesPlug,
+ %{scopes: ["write:notifications"]} when action == :mark_as_read
+ )
+
plug(:put_view, Pleroma.Web.MastodonAPI.NotificationView)
defdelegate open_api_operation(action), to: Pleroma.Web.ApiSpec.PleromaNotificationOperation
diff --git a/lib/pleroma/web/pleroma_api/controllers/scrobble_controller.ex b/lib/pleroma/web/pleroma_api/controllers/scrobble_controller.ex
index e9a4fba92..632d65434 100644
--- a/lib/pleroma/web/pleroma_api/controllers/scrobble_controller.ex
+++ b/lib/pleroma/web/pleroma_api/controllers/scrobble_controller.ex
@@ -7,10 +7,10 @@ defmodule Pleroma.Web.PleromaAPI.ScrobbleController do
import Pleroma.Web.ControllerHelper, only: [add_link_headers: 2]
- alias Pleroma.Plugs.OAuthScopesPlug
alias Pleroma.User
alias Pleroma.Web.ActivityPub.ActivityPub
alias Pleroma.Web.CommonAPI
+ alias Pleroma.Web.Plugs.OAuthScopesPlug
plug(Pleroma.Web.ApiSpec.CastAndValidate)
diff --git a/lib/pleroma/web/pleroma_api/controllers/two_factor_authentication_controller.ex b/lib/pleroma/web/pleroma_api/controllers/two_factor_authentication_controller.ex
index b86791d09..eba452300 100644
--- a/lib/pleroma/web/pleroma_api/controllers/two_factor_authentication_controller.ex
+++ b/lib/pleroma/web/pleroma_api/controllers/two_factor_authentication_controller.ex
@@ -10,8 +10,8 @@ defmodule Pleroma.Web.PleromaAPI.TwoFactorAuthenticationController do
alias Pleroma.MFA
alias Pleroma.MFA.TOTP
- alias Pleroma.Plugs.OAuthScopesPlug
alias Pleroma.Web.CommonAPI.Utils
+ alias Pleroma.Web.Plugs.OAuthScopesPlug
plug(OAuthScopesPlug, %{scopes: ["read:security"]} when action in [:settings])
diff --git a/lib/pleroma/web/pleroma_api/controllers/user_import_controller.ex b/lib/pleroma/web/pleroma_api/controllers/user_import_controller.ex
new file mode 100644
index 000000000..7f089af1c
--- /dev/null
+++ b/lib/pleroma/web/pleroma_api/controllers/user_import_controller.ex
@@ -0,0 +1,61 @@
+# Pleroma: A lightweight social networking server
+# Copyright © 2017-2020 Pleroma Authors
+# SPDX-License-Identifier: AGPL-3.0-only
+
+defmodule Pleroma.Web.PleromaAPI.UserImportController do
+ use Pleroma.Web, :controller
+
+ require Logger
+
+ alias Pleroma.User
+ alias Pleroma.Web.ApiSpec
+ alias Pleroma.Web.Plugs.OAuthScopesPlug
+
+ plug(OAuthScopesPlug, %{scopes: ["follow", "write:follows"]} when action == :follow)
+ plug(OAuthScopesPlug, %{scopes: ["follow", "write:blocks"]} when action == :blocks)
+ plug(OAuthScopesPlug, %{scopes: ["follow", "write:mutes"]} when action == :mutes)
+
+ plug(OpenApiSpex.Plug.CastAndValidate)
+ defdelegate open_api_operation(action), to: ApiSpec.UserImportOperation
+
+ def follow(%{body_params: %{list: %Plug.Upload{path: path}}} = conn, _) do
+ follow(%Plug.Conn{conn | body_params: %{list: File.read!(path)}}, %{})
+ end
+
+ def follow(%{assigns: %{user: follower}, body_params: %{list: list}} = conn, _) do
+ identifiers =
+ list
+ |> String.split("\n")
+ |> Enum.map(&(&1 |> String.split(",") |> List.first()))
+ |> List.delete("Account address")
+ |> Enum.map(&(&1 |> String.trim() |> String.trim_leading("@")))
+ |> Enum.reject(&(&1 == ""))
+
+ User.Import.follow_import(follower, identifiers)
+ json(conn, "job started")
+ end
+
+ def blocks(%{body_params: %{list: %Plug.Upload{path: path}}} = conn, _) do
+ blocks(%Plug.Conn{conn | body_params: %{list: File.read!(path)}}, %{})
+ end
+
+ def blocks(%{assigns: %{user: blocker}, body_params: %{list: list}} = conn, _) do
+ User.Import.blocks_import(blocker, prepare_user_identifiers(list))
+ json(conn, "job started")
+ end
+
+ def mutes(%{body_params: %{list: %Plug.Upload{path: path}}} = conn, _) do
+ mutes(%Plug.Conn{conn | body_params: %{list: File.read!(path)}}, %{})
+ end
+
+ def mutes(%{assigns: %{user: user}, body_params: %{list: list}} = conn, _) do
+ User.Import.mutes_import(user, prepare_user_identifiers(list))
+ json(conn, "job started")
+ end
+
+ defp prepare_user_identifiers(list) do
+ list
+ |> String.split()
+ |> Enum.map(&String.trim_leading(&1, "@"))
+ end
+end
diff --git a/lib/pleroma/web/pleroma_api/views/backup_view.ex b/lib/pleroma/web/pleroma_api/views/backup_view.ex
new file mode 100644
index 000000000..af75876aa
--- /dev/null
+++ b/lib/pleroma/web/pleroma_api/views/backup_view.ex
@@ -0,0 +1,28 @@
+# Pleroma: A lightweight social networking server
+# Copyright © 2017-2020 Pleroma Authors
+# SPDX-License-Identifier: AGPL-3.0-only
+
+defmodule Pleroma.Web.PleromaAPI.BackupView do
+ use Pleroma.Web, :view
+
+ alias Pleroma.User.Backup
+ alias Pleroma.Web.CommonAPI.Utils
+
+ def render("show.json", %{backup: %Backup{} = backup}) do
+ %{
+ content_type: backup.content_type,
+ url: download_url(backup),
+ file_size: backup.file_size,
+ processed: backup.processed,
+ inserted_at: Utils.to_masto_date(backup.inserted_at)
+ }
+ end
+
+ def render("index.json", %{backups: backups}) do
+ render_many(backups, __MODULE__, "show.json")
+ end
+
+ def download_url(%Backup{file_name: file_name}) do
+ Pleroma.Web.Endpoint.url() <> "/media/backups/" <> file_name
+ end
+end
diff --git a/lib/pleroma/web/pleroma_api/views/chat/message_reference_view.ex b/lib/pleroma/web/pleroma_api/views/chat/message_reference_view.ex
index d4e08b50d..c058fb340 100644
--- a/lib/pleroma/web/pleroma_api/views/chat/message_reference_view.ex
+++ b/lib/pleroma/web/pleroma_api/views/chat/message_reference_view.ex
@@ -5,6 +5,7 @@
defmodule Pleroma.Web.PleromaAPI.Chat.MessageReferenceView do
use Pleroma.Web, :view
+ alias Pleroma.Maps
alias Pleroma.User
alias Pleroma.Web.CommonAPI.Utils
alias Pleroma.Web.MastodonAPI.StatusView
@@ -37,6 +38,7 @@ def render(
Pleroma.Web.RichMedia.Helpers.fetch_data_for_object(object)
)
}
+ |> put_idempotency_key()
end
def render("index.json", opts) do
@@ -47,4 +49,13 @@ def render("index.json", opts) do
Map.put(opts, :as, :chat_message_reference)
)
end
+
+ defp put_idempotency_key(data) do
+ with {:ok, idempotency_key} <- Cachex.get(:chat_message_id_idempotency_key_cache, data.id) do
+ data
+ |> Maps.put_if_present(:idempotency_key, idempotency_key)
+ else
+ _ -> data
+ end
+ end
end
diff --git a/lib/pleroma/web/pleroma_api/views/scrobble_view.ex b/lib/pleroma/web/pleroma_api/views/scrobble_view.ex
index bbff93abe..95bd4c368 100644
--- a/lib/pleroma/web/pleroma_api/views/scrobble_view.ex
+++ b/lib/pleroma/web/pleroma_api/views/scrobble_view.ex
@@ -10,14 +10,14 @@ defmodule Pleroma.Web.PleromaAPI.ScrobbleView do
alias Pleroma.Activity
alias Pleroma.HTML
alias Pleroma.Object
+ alias Pleroma.Web.CommonAPI
alias Pleroma.Web.CommonAPI.Utils
alias Pleroma.Web.MastodonAPI.AccountView
- alias Pleroma.Web.MastodonAPI.StatusView
def render("show.json", %{activity: %Activity{data: %{"type" => "Listen"}} = activity} = opts) do
object = Object.normalize(activity)
- user = StatusView.get_user(activity.data["actor"])
+ user = CommonAPI.get_user(activity.data["actor"])
created_at = Utils.to_masto_date(activity.data["published"])
%{
diff --git a/lib/pleroma/web/plug.ex b/lib/pleroma/web/plug.ex
new file mode 100644
index 000000000..840b35072
--- /dev/null
+++ b/lib/pleroma/web/plug.ex
@@ -0,0 +1,8 @@
+# Pleroma: A lightweight social networking server
+# Copyright © 2017-2020 Pleroma Authors
+# SPDX-License-Identifier: AGPL-3.0-only
+
+defmodule Pleroma.Web.Plug do
+ # Substitute for `call/2` which is defined with `use Pleroma.Web, :plug`
+ @callback perform(Plug.Conn.t(), Plug.opts()) :: Plug.Conn.t()
+end
diff --git a/lib/pleroma/plugs/admin_secret_authentication_plug.ex b/lib/pleroma/web/plugs/admin_secret_authentication_plug.ex
similarity index 89%
rename from lib/pleroma/plugs/admin_secret_authentication_plug.ex
rename to lib/pleroma/web/plugs/admin_secret_authentication_plug.ex
index 2e54df47a..d7d4e4092 100644
--- a/lib/pleroma/plugs/admin_secret_authentication_plug.ex
+++ b/lib/pleroma/web/plugs/admin_secret_authentication_plug.ex
@@ -2,12 +2,12 @@
# Copyright © 2017-2020 Pleroma Authors
# SPDX-License-Identifier: AGPL-3.0-only
-defmodule Pleroma.Plugs.AdminSecretAuthenticationPlug do
+defmodule Pleroma.Web.Plugs.AdminSecretAuthenticationPlug do
import Plug.Conn
- alias Pleroma.Plugs.OAuthScopesPlug
- alias Pleroma.Plugs.RateLimiter
alias Pleroma.User
+ alias Pleroma.Web.Plugs.OAuthScopesPlug
+ alias Pleroma.Web.Plugs.RateLimiter
def init(options) do
options
diff --git a/lib/pleroma/plugs/authentication_plug.ex b/lib/pleroma/web/plugs/authentication_plug.ex
similarity index 94%
rename from lib/pleroma/plugs/authentication_plug.ex
rename to lib/pleroma/web/plugs/authentication_plug.ex
index 057ea42f1..e2a8b1b69 100644
--- a/lib/pleroma/plugs/authentication_plug.ex
+++ b/lib/pleroma/web/plugs/authentication_plug.ex
@@ -2,8 +2,7 @@
# Copyright © 2017-2020 Pleroma Authors
# SPDX-License-Identifier: AGPL-3.0-only
-defmodule Pleroma.Plugs.AuthenticationPlug do
- alias Pleroma.Plugs.OAuthScopesPlug
+defmodule Pleroma.Web.Plugs.AuthenticationPlug do
alias Pleroma.User
import Plug.Conn
@@ -65,7 +64,7 @@ def call(
conn
|> assign(:user, auth_user)
- |> OAuthScopesPlug.skip_plug()
+ |> Pleroma.Web.Plugs.OAuthScopesPlug.skip_plug()
else
conn
end
diff --git a/lib/pleroma/plugs/basic_auth_decoder_plug.ex b/lib/pleroma/web/plugs/basic_auth_decoder_plug.ex
similarity index 92%
rename from lib/pleroma/plugs/basic_auth_decoder_plug.ex
rename to lib/pleroma/web/plugs/basic_auth_decoder_plug.ex
index af7ecb0d8..4dadfb000 100644
--- a/lib/pleroma/plugs/basic_auth_decoder_plug.ex
+++ b/lib/pleroma/web/plugs/basic_auth_decoder_plug.ex
@@ -2,7 +2,7 @@
# Copyright © 2017-2020 Pleroma Authors
# SPDX-License-Identifier: AGPL-3.0-only
-defmodule Pleroma.Plugs.BasicAuthDecoderPlug do
+defmodule Pleroma.Web.Plugs.BasicAuthDecoderPlug do
import Plug.Conn
def init(options) do
diff --git a/lib/pleroma/plugs/cache.ex b/lib/pleroma/web/plugs/cache.ex
similarity index 96%
rename from lib/pleroma/plugs/cache.ex
rename to lib/pleroma/web/plugs/cache.ex
index f65c2a189..6de01804a 100644
--- a/lib/pleroma/plugs/cache.ex
+++ b/lib/pleroma/web/plugs/cache.ex
@@ -2,19 +2,19 @@
# Copyright © 2017-2020 Pleroma Authors
# SPDX-License-Identifier: AGPL-3.0-only
-defmodule Pleroma.Plugs.Cache do
+defmodule Pleroma.Web.Plugs.Cache do
@moduledoc """
Caches successful GET responses.
To enable the cache add the plug to a router pipeline or controller:
- plug(Pleroma.Plugs.Cache)
+ plug(Pleroma.Web.Plugs.Cache)
## Configuration
To configure the plug you need to pass settings as the second argument to the `plug/2` macro:
- plug(Pleroma.Plugs.Cache, [ttl: nil, query_params: true])
+ plug(Pleroma.Web.Plugs.Cache, [ttl: nil, query_params: true])
Available options:
diff --git a/lib/pleroma/plugs/digest.ex b/lib/pleroma/web/plugs/digest_plug.ex
similarity index 100%
rename from lib/pleroma/plugs/digest.ex
rename to lib/pleroma/web/plugs/digest_plug.ex
diff --git a/lib/pleroma/plugs/ensure_authenticated_plug.ex b/lib/pleroma/web/plugs/ensure_authenticated_plug.ex
similarity index 94%
rename from lib/pleroma/plugs/ensure_authenticated_plug.ex
rename to lib/pleroma/web/plugs/ensure_authenticated_plug.ex
index 3fe550806..ea2af6881 100644
--- a/lib/pleroma/plugs/ensure_authenticated_plug.ex
+++ b/lib/pleroma/web/plugs/ensure_authenticated_plug.ex
@@ -2,7 +2,7 @@
# Copyright © 2017-2020 Pleroma Authors
# SPDX-License-Identifier: AGPL-3.0-only
-defmodule Pleroma.Plugs.EnsureAuthenticatedPlug do
+defmodule Pleroma.Web.Plugs.EnsureAuthenticatedPlug do
import Plug.Conn
import Pleroma.Web.TranslationHelpers
diff --git a/lib/pleroma/plugs/ensure_public_or_authenticated_plug.ex b/lib/pleroma/web/plugs/ensure_public_or_authenticated_plug.ex
similarity index 91%
rename from lib/pleroma/plugs/ensure_public_or_authenticated_plug.ex
rename to lib/pleroma/web/plugs/ensure_public_or_authenticated_plug.ex
index 7265bb87a..3bebdac6d 100644
--- a/lib/pleroma/plugs/ensure_public_or_authenticated_plug.ex
+++ b/lib/pleroma/web/plugs/ensure_public_or_authenticated_plug.ex
@@ -2,7 +2,7 @@
# Copyright © 2017-2020 Pleroma Authors
# SPDX-License-Identifier: AGPL-3.0-only
-defmodule Pleroma.Plugs.EnsurePublicOrAuthenticatedPlug do
+defmodule Pleroma.Web.Plugs.EnsurePublicOrAuthenticatedPlug do
import Pleroma.Web.TranslationHelpers
import Plug.Conn
diff --git a/lib/pleroma/plugs/ensure_user_key_plug.ex b/lib/pleroma/web/plugs/ensure_user_key_plug.ex
similarity index 87%
rename from lib/pleroma/plugs/ensure_user_key_plug.ex
rename to lib/pleroma/web/plugs/ensure_user_key_plug.ex
index 9795cdbde..70d3091f0 100644
--- a/lib/pleroma/plugs/ensure_user_key_plug.ex
+++ b/lib/pleroma/web/plugs/ensure_user_key_plug.ex
@@ -2,7 +2,7 @@
# Copyright © 2017-2020 Pleroma Authors
# SPDX-License-Identifier: AGPL-3.0-only
-defmodule Pleroma.Plugs.EnsureUserKeyPlug do
+defmodule Pleroma.Web.Plugs.EnsureUserKeyPlug do
import Plug.Conn
def init(opts) do
diff --git a/lib/pleroma/plugs/expect_authenticated_check_plug.ex b/lib/pleroma/web/plugs/expect_authenticated_check_plug.ex
similarity index 71%
rename from lib/pleroma/plugs/expect_authenticated_check_plug.ex
rename to lib/pleroma/web/plugs/expect_authenticated_check_plug.ex
index 66b8d5de5..0925ded4d 100644
--- a/lib/pleroma/plugs/expect_authenticated_check_plug.ex
+++ b/lib/pleroma/web/plugs/expect_authenticated_check_plug.ex
@@ -2,9 +2,9 @@
# Copyright © 2017-2020 Pleroma Authors
# SPDX-License-Identifier: AGPL-3.0-only
-defmodule Pleroma.Plugs.ExpectAuthenticatedCheckPlug do
+defmodule Pleroma.Web.Plugs.ExpectAuthenticatedCheckPlug do
@moduledoc """
- Marks `Pleroma.Plugs.EnsureAuthenticatedPlug` as expected to be executed later in plug chain.
+ Marks `Pleroma.Web.Plugs.EnsureAuthenticatedPlug` as expected to be executed later in plug chain.
No-op plug which affects `Pleroma.Web` operation (is checked with `PlugHelper.plug_called?/2`).
"""
diff --git a/lib/pleroma/plugs/expect_public_or_authenticated_check_plug.ex b/lib/pleroma/web/plugs/expect_public_or_authenticated_check_plug.ex
similarity index 70%
rename from lib/pleroma/plugs/expect_public_or_authenticated_check_plug.ex
rename to lib/pleroma/web/plugs/expect_public_or_authenticated_check_plug.ex
index ba0ef76bd..ace512a78 100644
--- a/lib/pleroma/plugs/expect_public_or_authenticated_check_plug.ex
+++ b/lib/pleroma/web/plugs/expect_public_or_authenticated_check_plug.ex
@@ -2,9 +2,9 @@
# Copyright © 2017-2020 Pleroma Authors
# SPDX-License-Identifier: AGPL-3.0-only
-defmodule Pleroma.Plugs.ExpectPublicOrAuthenticatedCheckPlug do
+defmodule Pleroma.Web.Plugs.ExpectPublicOrAuthenticatedCheckPlug do
@moduledoc """
- Marks `Pleroma.Plugs.EnsurePublicOrAuthenticatedPlug` as expected to be executed later in plug
+ Marks `Pleroma.Web.Plugs.EnsurePublicOrAuthenticatedPlug` as expected to be executed later in plug
chain.
No-op plug which affects `Pleroma.Web` operation (is checked with `PlugHelper.plug_called?/2`).
diff --git a/lib/pleroma/plugs/federating_plug.ex b/lib/pleroma/web/plugs/federating_plug.ex
similarity index 93%
rename from lib/pleroma/plugs/federating_plug.ex
rename to lib/pleroma/web/plugs/federating_plug.ex
index 09038f3c6..3c90a7644 100644
--- a/lib/pleroma/plugs/federating_plug.ex
+++ b/lib/pleroma/web/plugs/federating_plug.ex
@@ -2,7 +2,7 @@
# Copyright © 2017-2020 Pleroma Authors
# SPDX-License-Identifier: AGPL-3.0-only
-defmodule Pleroma.Web.FederatingPlug do
+defmodule Pleroma.Web.Plugs.FederatingPlug do
import Plug.Conn
def init(options) do
diff --git a/lib/pleroma/plugs/frontend_static.ex b/lib/pleroma/web/plugs/frontend_static.ex
similarity index 62%
rename from lib/pleroma/plugs/frontend_static.ex
rename to lib/pleroma/web/plugs/frontend_static.ex
index 11a0d5382..1b0b36813 100644
--- a/lib/pleroma/plugs/frontend_static.ex
+++ b/lib/pleroma/web/plugs/frontend_static.ex
@@ -2,7 +2,7 @@
# Copyright © 2017-2020 Pleroma Authors
# SPDX-License-Identifier: AGPL-3.0-only
-defmodule Pleroma.Plugs.FrontendStatic do
+defmodule Pleroma.Web.Plugs.FrontendStatic do
require Pleroma.Constants
@moduledoc """
@@ -34,22 +34,26 @@ def init(opts) do
end
def call(conn, opts) do
- frontend_type = Map.get(opts, :frontend_type, :primary)
- path = file_path("", frontend_type)
-
- if path do
- conn
- |> call_static(opts, path)
+ with false <- invalid_path?(conn.path_info),
+ frontend_type <- Map.get(opts, :frontend_type, :primary),
+ path when not is_nil(path) <- file_path("", frontend_type) do
+ call_static(conn, opts, path)
else
- conn
+ _ ->
+ conn
end
end
- defp call_static(conn, opts, from) do
- opts =
- opts
- |> Map.put(:from, from)
+ defp invalid_path?(list) do
+ invalid_path?(list, :binary.compile_pattern(["/", "\\", ":", "\0"]))
+ end
+ defp invalid_path?([h | _], _match) when h in [".", "..", ""], do: true
+ defp invalid_path?([h | t], match), do: String.contains?(h, match) or invalid_path?(t)
+ defp invalid_path?([], _match), do: false
+
+ defp call_static(conn, opts, from) do
+ opts = Map.put(opts, :from, from)
Plug.Static.call(conn, opts)
end
end
diff --git a/lib/pleroma/plugs/http_security_plug.ex b/lib/pleroma/web/plugs/http_security_plug.ex
similarity index 99%
rename from lib/pleroma/plugs/http_security_plug.ex
rename to lib/pleroma/web/plugs/http_security_plug.ex
index c363b193b..45aaf188e 100644
--- a/lib/pleroma/plugs/http_security_plug.ex
+++ b/lib/pleroma/web/plugs/http_security_plug.ex
@@ -2,7 +2,7 @@
# Copyright © 2017-2020 Pleroma Authors
# SPDX-License-Identifier: AGPL-3.0-only
-defmodule Pleroma.Plugs.HTTPSecurityPlug do
+defmodule Pleroma.Web.Plugs.HTTPSecurityPlug do
alias Pleroma.Config
import Plug.Conn
diff --git a/lib/pleroma/plugs/http_signature.ex b/lib/pleroma/web/plugs/http_signature_plug.ex
similarity index 100%
rename from lib/pleroma/plugs/http_signature.ex
rename to lib/pleroma/web/plugs/http_signature_plug.ex
diff --git a/lib/pleroma/plugs/idempotency_plug.ex b/lib/pleroma/web/plugs/idempotency_plug.ex
similarity index 97%
rename from lib/pleroma/plugs/idempotency_plug.ex
rename to lib/pleroma/web/plugs/idempotency_plug.ex
index f41397075..254a790b0 100644
--- a/lib/pleroma/plugs/idempotency_plug.ex
+++ b/lib/pleroma/web/plugs/idempotency_plug.ex
@@ -2,7 +2,7 @@
# Copyright © 2017-2020 Pleroma Authors
# SPDX-License-Identifier: AGPL-3.0-only
-defmodule Pleroma.Plugs.IdempotencyPlug do
+defmodule Pleroma.Web.Plugs.IdempotencyPlug do
import Phoenix.Controller, only: [json: 2]
import Plug.Conn
diff --git a/lib/pleroma/plugs/instance_static.ex b/lib/pleroma/web/plugs/instance_static.ex
similarity index 91%
rename from lib/pleroma/plugs/instance_static.ex
rename to lib/pleroma/web/plugs/instance_static.ex
index 0fb57e422..54b9175df 100644
--- a/lib/pleroma/plugs/instance_static.ex
+++ b/lib/pleroma/web/plugs/instance_static.ex
@@ -2,7 +2,7 @@
# Copyright © 2017-2020 Pleroma Authors
# SPDX-License-Identifier: AGPL-3.0-only
-defmodule Pleroma.Plugs.InstanceStatic do
+defmodule Pleroma.Web.Plugs.InstanceStatic do
require Pleroma.Constants
@moduledoc """
@@ -16,7 +16,7 @@ def file_path(path) do
instance_path =
Path.join(Pleroma.Config.get([:instance, :static_dir], "instance/static/"), path)
- frontend_path = Pleroma.Plugs.FrontendStatic.file_path(path, :primary)
+ frontend_path = Pleroma.Web.Plugs.FrontendStatic.file_path(path, :primary)
(File.exists?(instance_path) && instance_path) ||
(frontend_path && File.exists?(frontend_path) && frontend_path) ||
diff --git a/lib/pleroma/plugs/legacy_authentication_plug.ex b/lib/pleroma/web/plugs/legacy_authentication_plug.ex
similarity index 87%
rename from lib/pleroma/plugs/legacy_authentication_plug.ex
rename to lib/pleroma/web/plugs/legacy_authentication_plug.ex
index d346e01a6..2a54d0b59 100644
--- a/lib/pleroma/plugs/legacy_authentication_plug.ex
+++ b/lib/pleroma/web/plugs/legacy_authentication_plug.ex
@@ -2,10 +2,9 @@
# Copyright © 2017-2020 Pleroma Authors
# SPDX-License-Identifier: AGPL-3.0-only
-defmodule Pleroma.Plugs.LegacyAuthenticationPlug do
+defmodule Pleroma.Web.Plugs.LegacyAuthenticationPlug do
import Plug.Conn
- alias Pleroma.Plugs.OAuthScopesPlug
alias Pleroma.User
def init(options) do
@@ -29,7 +28,7 @@ def call(
conn
|> assign(:auth_user, user)
|> assign(:user, user)
- |> OAuthScopesPlug.skip_plug()
+ |> Pleroma.Web.Plugs.OAuthScopesPlug.skip_plug()
else
_ ->
conn
diff --git a/lib/pleroma/plugs/mapped_signature_to_identity_plug.ex b/lib/pleroma/web/plugs/mapped_signature_to_identity_plug.ex
similarity index 100%
rename from lib/pleroma/plugs/mapped_signature_to_identity_plug.ex
rename to lib/pleroma/web/plugs/mapped_signature_to_identity_plug.ex
diff --git a/lib/pleroma/plugs/oauth_plug.ex b/lib/pleroma/web/plugs/o_auth_plug.ex
similarity index 98%
rename from lib/pleroma/plugs/oauth_plug.ex
rename to lib/pleroma/web/plugs/o_auth_plug.ex
index 6fa71ef47..c7b58d90f 100644
--- a/lib/pleroma/plugs/oauth_plug.ex
+++ b/lib/pleroma/web/plugs/o_auth_plug.ex
@@ -2,7 +2,7 @@
# Copyright © 2017-2020 Pleroma Authors
# SPDX-License-Identifier: AGPL-3.0-only
-defmodule Pleroma.Plugs.OAuthPlug do
+defmodule Pleroma.Web.Plugs.OAuthPlug do
import Plug.Conn
import Ecto.Query
diff --git a/lib/pleroma/plugs/oauth_scopes_plug.ex b/lib/pleroma/web/plugs/o_auth_scopes_plug.ex
similarity index 94%
rename from lib/pleroma/plugs/oauth_scopes_plug.ex
rename to lib/pleroma/web/plugs/o_auth_scopes_plug.ex
index efc25b79f..cfc30837c 100644
--- a/lib/pleroma/plugs/oauth_scopes_plug.ex
+++ b/lib/pleroma/web/plugs/o_auth_scopes_plug.ex
@@ -2,7 +2,7 @@
# Copyright © 2017-2020 Pleroma Authors
# SPDX-License-Identifier: AGPL-3.0-only
-defmodule Pleroma.Plugs.OAuthScopesPlug do
+defmodule Pleroma.Web.Plugs.OAuthScopesPlug do
import Plug.Conn
import Pleroma.Web.Gettext
@@ -53,7 +53,7 @@ def drop_auth_info(conn) do
|> assign(:token, nil)
end
- @doc "Filters descendants of supported scopes"
+ @doc "Keeps those of `scopes` which are descendants of `supported_scopes`"
def filter_descendants(scopes, supported_scopes) do
Enum.filter(
scopes,
diff --git a/lib/pleroma/plugs/plug_helper.ex b/lib/pleroma/web/plugs/plug_helper.ex
similarity index 97%
rename from lib/pleroma/plugs/plug_helper.ex
rename to lib/pleroma/web/plugs/plug_helper.ex
index 9c67be8ef..b314e7596 100644
--- a/lib/pleroma/plugs/plug_helper.ex
+++ b/lib/pleroma/web/plugs/plug_helper.ex
@@ -2,7 +2,7 @@
# Copyright © 2017-2020 Pleroma Authors
# SPDX-License-Identifier: AGPL-3.0-only
-defmodule Pleroma.Plugs.PlugHelper do
+defmodule Pleroma.Web.Plugs.PlugHelper do
@moduledoc "Pleroma Plug helper"
@called_plugs_list_id :called_plugs
diff --git a/lib/pleroma/plugs/rate_limiter/rate_limiter.ex b/lib/pleroma/web/plugs/rate_limiter.ex
similarity index 93%
rename from lib/pleroma/plugs/rate_limiter/rate_limiter.ex
rename to lib/pleroma/web/plugs/rate_limiter.ex
index c51e2c634..a589610d1 100644
--- a/lib/pleroma/plugs/rate_limiter/rate_limiter.ex
+++ b/lib/pleroma/web/plugs/rate_limiter.ex
@@ -2,7 +2,7 @@
# Copyright © 2017-2020 Pleroma Authors
# SPDX-License-Identifier: AGPL-3.0-only
-defmodule Pleroma.Plugs.RateLimiter do
+defmodule Pleroma.Web.Plugs.RateLimiter do
@moduledoc """
## Configuration
@@ -35,8 +35,8 @@ defmodule Pleroma.Plugs.RateLimiter do
AllowedSyntax:
- plug(Pleroma.Plugs.RateLimiter, name: :limiter_name)
- plug(Pleroma.Plugs.RateLimiter, options) # :name is a required option
+ plug(Pleroma.Web.Plugs.RateLimiter, name: :limiter_name)
+ plug(Pleroma.Web.Plugs.RateLimiter, options) # :name is a required option
Allowed options:
@@ -46,11 +46,11 @@ defmodule Pleroma.Plugs.RateLimiter do
Inside a controller:
- plug(Pleroma.Plugs.RateLimiter, [name: :one] when action == :one)
- plug(Pleroma.Plugs.RateLimiter, [name: :two] when action in [:two, :three])
+ plug(Pleroma.Web.Plugs.RateLimiter, [name: :one] when action == :one)
+ plug(Pleroma.Web.Plugs.RateLimiter, [name: :two] when action in [:two, :three])
plug(
- Pleroma.Plugs.RateLimiter,
+ Pleroma.Web.Plugs.RateLimiter,
[name: :status_id_action, bucket_name: "status_id_action:fav_unfav", params: ["id"]]
when action in ~w(fav_status unfav_status)a
)
@@ -59,7 +59,7 @@ defmodule Pleroma.Plugs.RateLimiter do
pipeline :api do
...
- plug(Pleroma.Plugs.RateLimiter, name: :one)
+ plug(Pleroma.Web.Plugs.RateLimiter, name: :one)
...
end
"""
@@ -67,8 +67,8 @@ defmodule Pleroma.Plugs.RateLimiter do
import Plug.Conn
alias Pleroma.Config
- alias Pleroma.Plugs.RateLimiter.LimiterSupervisor
alias Pleroma.User
+ alias Pleroma.Web.Plugs.RateLimiter.LimiterSupervisor
require Logger
diff --git a/lib/pleroma/plugs/rate_limiter/limiter_supervisor.ex b/lib/pleroma/web/plugs/rate_limiter/limiter_supervisor.ex
similarity index 83%
rename from lib/pleroma/plugs/rate_limiter/limiter_supervisor.ex
rename to lib/pleroma/web/plugs/rate_limiter/limiter_supervisor.ex
index 884268d96..5642bb205 100644
--- a/lib/pleroma/plugs/rate_limiter/limiter_supervisor.ex
+++ b/lib/pleroma/web/plugs/rate_limiter/limiter_supervisor.ex
@@ -1,4 +1,8 @@
-defmodule Pleroma.Plugs.RateLimiter.LimiterSupervisor do
+# Pleroma: A lightweight social networking server
+# Copyright © 2017-2020 Pleroma Authors
+# SPDX-License-Identifier: AGPL-3.0-only
+
+defmodule Pleroma.Web.Plugs.RateLimiter.LimiterSupervisor do
use DynamicSupervisor
import Cachex.Spec
diff --git a/lib/pleroma/plugs/rate_limiter/supervisor.ex b/lib/pleroma/web/plugs/rate_limiter/supervisor.ex
similarity index 51%
rename from lib/pleroma/plugs/rate_limiter/supervisor.ex
rename to lib/pleroma/web/plugs/rate_limiter/supervisor.ex
index 9672f7876..a1c84063d 100644
--- a/lib/pleroma/plugs/rate_limiter/supervisor.ex
+++ b/lib/pleroma/web/plugs/rate_limiter/supervisor.ex
@@ -1,4 +1,8 @@
-defmodule Pleroma.Plugs.RateLimiter.Supervisor do
+# Pleroma: A lightweight social networking server
+# Copyright © 2017-2020 Pleroma Authors
+# SPDX-License-Identifier: AGPL-3.0-only
+
+defmodule Pleroma.Web.Plugs.RateLimiter.Supervisor do
use Supervisor
def start_link(opts) do
@@ -7,7 +11,7 @@ def start_link(opts) do
def init(_args) do
children = [
- Pleroma.Plugs.RateLimiter.LimiterSupervisor
+ Pleroma.Web.Plugs.RateLimiter.LimiterSupervisor
]
opts = [strategy: :one_for_one, name: Pleroma.Web.Streamer.Supervisor]
diff --git a/lib/pleroma/web/plugs/remote_ip.ex b/lib/pleroma/web/plugs/remote_ip.ex
new file mode 100644
index 000000000..401e2cbfa
--- /dev/null
+++ b/lib/pleroma/web/plugs/remote_ip.ex
@@ -0,0 +1,48 @@
+# Pleroma: A lightweight social networking server
+# Copyright © 2017-2020 Pleroma Authors
+# SPDX-License-Identifier: AGPL-3.0-only
+
+defmodule Pleroma.Web.Plugs.RemoteIp do
+ @moduledoc """
+ This is a shim to call [`RemoteIp`](https://git.pleroma.social/pleroma/remote_ip) but with runtime configuration.
+ """
+
+ alias Pleroma.Config
+ import Plug.Conn
+
+ @behaviour Plug
+
+ def init(_), do: nil
+
+ def call(%{remote_ip: original_remote_ip} = conn, _) do
+ if Config.get([__MODULE__, :enabled]) do
+ %{remote_ip: new_remote_ip} = conn = RemoteIp.call(conn, remote_ip_opts())
+ assign(conn, :remote_ip_found, original_remote_ip != new_remote_ip)
+ else
+ conn
+ end
+ end
+
+ defp remote_ip_opts do
+ headers = Config.get([__MODULE__, :headers], []) |> MapSet.new()
+ reserved = Config.get([__MODULE__, :reserved], [])
+
+ proxies =
+ Config.get([__MODULE__, :proxies], [])
+ |> Enum.concat(reserved)
+ |> Enum.map(&maybe_add_cidr/1)
+
+ {headers, proxies}
+ end
+
+ defp maybe_add_cidr(proxy) when is_binary(proxy) do
+ proxy =
+ cond do
+ "/" in String.codepoints(proxy) -> proxy
+ InetCidr.v4?(InetCidr.parse_address!(proxy)) -> proxy <> "/32"
+ InetCidr.v6?(InetCidr.parse_address!(proxy)) -> proxy <> "/128"
+ end
+
+ InetCidr.parse(proxy, true)
+ end
+end
diff --git a/lib/pleroma/plugs/session_authentication_plug.ex b/lib/pleroma/web/plugs/session_authentication_plug.ex
similarity index 89%
rename from lib/pleroma/plugs/session_authentication_plug.ex
rename to lib/pleroma/web/plugs/session_authentication_plug.ex
index 0f83a5e53..6e176d553 100644
--- a/lib/pleroma/plugs/session_authentication_plug.ex
+++ b/lib/pleroma/web/plugs/session_authentication_plug.ex
@@ -2,7 +2,7 @@
# Copyright © 2017-2020 Pleroma Authors
# SPDX-License-Identifier: AGPL-3.0-only
-defmodule Pleroma.Plugs.SessionAuthenticationPlug do
+defmodule Pleroma.Web.Plugs.SessionAuthenticationPlug do
import Plug.Conn
def init(options) do
diff --git a/lib/pleroma/plugs/set_format_plug.ex b/lib/pleroma/web/plugs/set_format_plug.ex
similarity index 92%
rename from lib/pleroma/plugs/set_format_plug.ex
rename to lib/pleroma/web/plugs/set_format_plug.ex
index c03fcb28d..c16d2f81d 100644
--- a/lib/pleroma/plugs/set_format_plug.ex
+++ b/lib/pleroma/web/plugs/set_format_plug.ex
@@ -2,7 +2,7 @@
# Copyright © 2017-2020 Pleroma Authors
# SPDX-License-Identifier: AGPL-3.0-only
-defmodule Pleroma.Plugs.SetFormatPlug do
+defmodule Pleroma.Web.Plugs.SetFormatPlug do
import Plug.Conn, only: [assign: 3, fetch_query_params: 1]
def init(_), do: nil
diff --git a/lib/pleroma/plugs/set_locale_plug.ex b/lib/pleroma/web/plugs/set_locale_plug.ex
similarity index 97%
rename from lib/pleroma/plugs/set_locale_plug.ex
rename to lib/pleroma/web/plugs/set_locale_plug.ex
index 9a21d0a9d..d9d24b93f 100644
--- a/lib/pleroma/plugs/set_locale_plug.ex
+++ b/lib/pleroma/web/plugs/set_locale_plug.ex
@@ -3,7 +3,7 @@
# SPDX-License-Identifier: AGPL-3.0-only
# NOTE: this module is based on https://github.com/smeevil/set_locale
-defmodule Pleroma.Plugs.SetLocalePlug do
+defmodule Pleroma.Web.Plugs.SetLocalePlug do
import Plug.Conn, only: [get_req_header: 2, assign: 3]
def init(_), do: nil
diff --git a/lib/pleroma/plugs/set_user_session_id_plug.ex b/lib/pleroma/web/plugs/set_user_session_id_plug.ex
similarity index 87%
rename from lib/pleroma/plugs/set_user_session_id_plug.ex
rename to lib/pleroma/web/plugs/set_user_session_id_plug.ex
index 730c4ac74..e520159e4 100644
--- a/lib/pleroma/plugs/set_user_session_id_plug.ex
+++ b/lib/pleroma/web/plugs/set_user_session_id_plug.ex
@@ -2,7 +2,7 @@
# Copyright © 2017-2020 Pleroma Authors
# SPDX-License-Identifier: AGPL-3.0-only
-defmodule Pleroma.Plugs.SetUserSessionIdPlug do
+defmodule Pleroma.Web.Plugs.SetUserSessionIdPlug do
import Plug.Conn
alias Pleroma.User
diff --git a/lib/pleroma/plugs/static_fe_plug.ex b/lib/pleroma/web/plugs/static_fe_plug.ex
similarity index 93%
rename from lib/pleroma/plugs/static_fe_plug.ex
rename to lib/pleroma/web/plugs/static_fe_plug.ex
index 143665c71..658a1052e 100644
--- a/lib/pleroma/plugs/static_fe_plug.ex
+++ b/lib/pleroma/web/plugs/static_fe_plug.ex
@@ -2,7 +2,7 @@
# Copyright © 2017-2020 Pleroma Authors
# SPDX-License-Identifier: AGPL-3.0-only
-defmodule Pleroma.Plugs.StaticFEPlug do
+defmodule Pleroma.Web.Plugs.StaticFEPlug do
import Plug.Conn
alias Pleroma.Web.StaticFE.StaticFEController
diff --git a/lib/pleroma/plugs/trailing_format_plug.ex b/lib/pleroma/web/plugs/trailing_format_plug.ex
similarity index 95%
rename from lib/pleroma/plugs/trailing_format_plug.ex
rename to lib/pleroma/web/plugs/trailing_format_plug.ex
index 8b4d5fc9f..e3f57c14a 100644
--- a/lib/pleroma/plugs/trailing_format_plug.ex
+++ b/lib/pleroma/web/plugs/trailing_format_plug.ex
@@ -2,7 +2,7 @@
# Copyright © 2017-2020 Pleroma Authors
# SPDX-License-Identifier: AGPL-3.0-only
-defmodule Pleroma.Plugs.TrailingFormatPlug do
+defmodule Pleroma.Web.Plugs.TrailingFormatPlug do
@moduledoc "Calls TrailingFormatPlug for specific paths. Ideally we would just do this in the router, but TrailingFormatPlug needs to be called before Plug.Parsers."
@behaviour Plug
diff --git a/lib/pleroma/plugs/uploaded_media.ex b/lib/pleroma/web/plugs/uploaded_media.ex
similarity index 98%
rename from lib/pleroma/plugs/uploaded_media.ex
rename to lib/pleroma/web/plugs/uploaded_media.ex
index 40984cfc0..402a8bb34 100644
--- a/lib/pleroma/plugs/uploaded_media.ex
+++ b/lib/pleroma/web/plugs/uploaded_media.ex
@@ -2,7 +2,7 @@
# Copyright © 2017-2020 Pleroma Authors
# SPDX-License-Identifier: AGPL-3.0-only
-defmodule Pleroma.Plugs.UploadedMedia do
+defmodule Pleroma.Web.Plugs.UploadedMedia do
@moduledoc """
"""
diff --git a/lib/pleroma/plugs/user_enabled_plug.ex b/lib/pleroma/web/plugs/user_enabled_plug.ex
similarity index 90%
rename from lib/pleroma/plugs/user_enabled_plug.ex
rename to lib/pleroma/web/plugs/user_enabled_plug.ex
index 23e800a74..fa28ee48b 100644
--- a/lib/pleroma/plugs/user_enabled_plug.ex
+++ b/lib/pleroma/web/plugs/user_enabled_plug.ex
@@ -2,7 +2,7 @@
# Copyright © 2017-2020 Pleroma Authors
# SPDX-License-Identifier: AGPL-3.0-only
-defmodule Pleroma.Plugs.UserEnabledPlug do
+defmodule Pleroma.Web.Plugs.UserEnabledPlug do
import Plug.Conn
alias Pleroma.User
diff --git a/lib/pleroma/plugs/user_fetcher_plug.ex b/lib/pleroma/web/plugs/user_fetcher_plug.ex
similarity index 91%
rename from lib/pleroma/plugs/user_fetcher_plug.ex
rename to lib/pleroma/web/plugs/user_fetcher_plug.ex
index 235c77d85..4039600da 100644
--- a/lib/pleroma/plugs/user_fetcher_plug.ex
+++ b/lib/pleroma/web/plugs/user_fetcher_plug.ex
@@ -2,7 +2,7 @@
# Copyright © 2017-2020 Pleroma Authors
# SPDX-License-Identifier: AGPL-3.0-only
-defmodule Pleroma.Plugs.UserFetcherPlug do
+defmodule Pleroma.Web.Plugs.UserFetcherPlug do
alias Pleroma.User
import Plug.Conn
diff --git a/lib/pleroma/plugs/user_is_admin_plug.ex b/lib/pleroma/web/plugs/user_is_admin_plug.ex
similarity index 91%
rename from lib/pleroma/plugs/user_is_admin_plug.ex
rename to lib/pleroma/web/plugs/user_is_admin_plug.ex
index 488a61d1d..531c965f0 100644
--- a/lib/pleroma/plugs/user_is_admin_plug.ex
+++ b/lib/pleroma/web/plugs/user_is_admin_plug.ex
@@ -2,7 +2,7 @@
# Copyright © 2017-2020 Pleroma Authors
# SPDX-License-Identifier: AGPL-3.0-only
-defmodule Pleroma.Plugs.UserIsAdminPlug do
+defmodule Pleroma.Web.Plugs.UserIsAdminPlug do
import Pleroma.Web.TranslationHelpers
import Plug.Conn
diff --git a/lib/pleroma/web/preload/instance.ex b/lib/pleroma/web/preload/providers/instance.ex
similarity index 79%
rename from lib/pleroma/web/preload/instance.ex
rename to lib/pleroma/web/preload/providers/instance.ex
index 50d1f3382..a549bb1eb 100644
--- a/lib/pleroma/web/preload/instance.ex
+++ b/lib/pleroma/web/preload/providers/instance.ex
@@ -3,15 +3,17 @@
# SPDX-License-Identifier: AGPL-3.0-only
defmodule Pleroma.Web.Preload.Providers.Instance do
- alias Pleroma.Plugs.InstanceStatic
alias Pleroma.Web.MastodonAPI.InstanceView
alias Pleroma.Web.Nodeinfo.Nodeinfo
+ alias Pleroma.Web.Plugs.InstanceStatic
alias Pleroma.Web.Preload.Providers.Provider
+ alias Pleroma.Web.TwitterAPI.UtilView
@behaviour Provider
@instance_url "/api/v1/instance"
@panel_url "/instance/panel.html"
@nodeinfo_url "/nodeinfo/2.0.json"
+ @fe_config_url "/api/pleroma/frontend_configurations"
@impl Provider
def generate_terms(_params) do
@@ -19,6 +21,7 @@ def generate_terms(_params) do
|> build_info_tag()
|> build_panel_tag()
|> build_nodeinfo_tag()
+ |> build_fe_config_tag()
end
defp build_info_tag(acc) do
@@ -47,4 +50,10 @@ defp build_nodeinfo_tag(acc) do
Map.put(acc, @nodeinfo_url, nodeinfo_data)
end
end
+
+ defp build_fe_config_tag(acc) do
+ fe_data = UtilView.render("frontend_configurations.json", %{})
+
+ Map.put(acc, @fe_config_url, fe_data)
+ end
end
diff --git a/lib/pleroma/web/preload/provider.ex b/lib/pleroma/web/preload/providers/provider.ex
similarity index 100%
rename from lib/pleroma/web/preload/provider.ex
rename to lib/pleroma/web/preload/providers/provider.ex
diff --git a/lib/pleroma/web/preload/timelines.ex b/lib/pleroma/web/preload/providers/timelines.ex
similarity index 100%
rename from lib/pleroma/web/preload/timelines.ex
rename to lib/pleroma/web/preload/providers/timelines.ex
diff --git a/lib/pleroma/web/preload/user.ex b/lib/pleroma/web/preload/providers/user.ex
similarity index 100%
rename from lib/pleroma/web/preload/user.ex
rename to lib/pleroma/web/preload/providers/user.ex
diff --git a/lib/pleroma/web/push/push.ex b/lib/pleroma/web/push.ex
similarity index 100%
rename from lib/pleroma/web/push/push.ex
rename to lib/pleroma/web/push.ex
diff --git a/lib/pleroma/web/push/impl.ex b/lib/pleroma/web/push/impl.ex
index 16368485e..da535aa68 100644
--- a/lib/pleroma/web/push/impl.ex
+++ b/lib/pleroma/web/push/impl.ex
@@ -19,7 +19,7 @@ defmodule Pleroma.Web.Push.Impl do
@types ["Create", "Follow", "Announce", "Like", "Move"]
@doc "Performs sending notifications for user subscriptions"
- @spec perform(Notification.t()) :: list(any) | :error
+ @spec perform(Notification.t()) :: list(any) | :error | {:error, :unknown_type}
def perform(
%{
activity: %{data: %{"type" => activity_type}} = activity,
@@ -64,20 +64,20 @@ def perform(_) do
@doc "Push message to web"
def push_message(body, sub, api_key, subscription) do
case WebPushEncryption.send_web_push(body, sub, api_key) do
- {:ok, %{status_code: code}} when 400 <= code and code < 500 ->
+ {:ok, %{status: code}} when code in 400..499 ->
Logger.debug("Removing subscription record")
Repo.delete!(subscription)
:ok
- {:ok, %{status_code: code}} when 200 <= code and code < 300 ->
+ {:ok, %{status: code}} when code in 200..299 ->
:ok
- {:ok, %{status_code: code}} ->
+ {:ok, %{status: code}} ->
Logger.error("Web Push Notification failed with code: #{code}")
:error
- _ ->
- Logger.error("Web Push Notification failed with unknown error")
+ error ->
+ Logger.error("Web Push Notification failed with #{inspect(error)}")
:error
end
end
diff --git a/lib/pleroma/web/rich_media/helpers.ex b/lib/pleroma/web/rich_media/helpers.ex
index bd7f03cbe..d67b594b5 100644
--- a/lib/pleroma/web/rich_media/helpers.ex
+++ b/lib/pleroma/web/rich_media/helpers.ex
@@ -57,7 +57,6 @@ defp get_tld(host) do
def fetch_data_for_object(object) do
with true <- Config.get([:rich_media, :enabled]),
- false <- object.data["sensitive"] || false,
{:ok, page_url} <-
HTML.extract_first_external_url_from_object(object),
:ok <- validate_page_url(page_url),
@@ -87,6 +86,50 @@ def perform(:fetch, %Activity{} = activity) do
def rich_media_get(url) do
headers = [{"user-agent", Pleroma.Application.user_agent() <> "; Bot"}]
- Pleroma.HTTP.get(url, headers, @options)
+ head_check =
+ case Pleroma.HTTP.head(url, headers, @options) do
+ # If the HEAD request didn't reach the server for whatever reason,
+ # we assume the GET that comes right after won't either
+ {:error, _} = e ->
+ e
+
+ {:ok, %Tesla.Env{status: 200, headers: headers}} ->
+ with :ok <- check_content_type(headers),
+ :ok <- check_content_length(headers),
+ do: :ok
+
+ _ ->
+ :ok
+ end
+
+ with :ok <- head_check, do: Pleroma.HTTP.get(url, headers, @options)
+ end
+
+ defp check_content_type(headers) do
+ case List.keyfind(headers, "content-type", 0) do
+ {_, content_type} ->
+ case Plug.Conn.Utils.media_type(content_type) do
+ {:ok, "text", "html", _} -> :ok
+ _ -> {:error, {:content_type, content_type}}
+ end
+
+ _ ->
+ :ok
+ end
+ end
+
+ @max_body @options[:max_body]
+ defp check_content_length(headers) do
+ case List.keyfind(headers, "content-length", 0) do
+ {_, maybe_content_length} ->
+ case Integer.parse(maybe_content_length) do
+ {content_length, ""} when content_length <= @max_body -> :ok
+ {_, ""} -> {:error, :body_too_large}
+ _ -> :ok
+ end
+
+ _ ->
+ :ok
+ end
end
end
diff --git a/lib/pleroma/web/rich_media/parser.ex b/lib/pleroma/web/rich_media/parser.ex
index 5727fda18..c70d2fdba 100644
--- a/lib/pleroma/web/rich_media/parser.ex
+++ b/lib/pleroma/web/rich_media/parser.ex
@@ -20,28 +20,61 @@ def parse(url) do
with {:ok, data} <- get_cached_or_parse(url),
{:ok, _} <- set_ttl_based_on_image(data, url) do
{:ok, data}
- else
- {:error, {:invalid_metadata, data}} = e ->
- Logger.debug(fn -> "Incomplete or invalid metadata for #{url}: #{inspect(data)}" end)
- e
-
- error ->
- Logger.error(fn -> "Rich media error for #{url}: #{inspect(error)}" end)
- error
end
end
defp get_cached_or_parse(url) do
- case Cachex.fetch!(:rich_media_cache, url, fn _ -> {:commit, parse_url(url)} end) do
- {:ok, _data} = res ->
- res
+ case Cachex.fetch(:rich_media_cache, url, fn ->
+ case parse_url(url) do
+ {:ok, _} = res ->
+ {:commit, res}
- {:error, _} = e ->
- ttl = Pleroma.Config.get([:rich_media, :failure_backoff], 60_000)
- Cachex.expire(:rich_media_cache, url, ttl)
- e
+ {:error, reason} = e ->
+ # Unfortunately we have to log errors here, instead of doing that
+ # along with ttl setting at the bottom. Otherwise we can get log spam
+ # if more than one process was waiting for the rich media card
+ # while it was generated. Ideally we would set ttl here as well,
+ # so we don't override it number_of_waiters_on_generation
+ # times, but one, obviously, can't set ttl for not-yet-created entry
+ # and Cachex doesn't support returning ttl from the fetch callback.
+ log_error(url, reason)
+ {:commit, e}
+ end
+ end) do
+ {action, res} when action in [:commit, :ok] ->
+ case res do
+ {:ok, _data} = res ->
+ res
+
+ {:error, reason} = e ->
+ if action == :commit, do: set_error_ttl(url, reason)
+ e
+ end
+
+ {:error, e} ->
+ {:error, {:cachex_error, e}}
end
end
+
+ defp set_error_ttl(_url, :body_too_large), do: :ok
+ defp set_error_ttl(_url, {:content_type, _}), do: :ok
+
+ # The TTL is not set for the errors above, since they are unlikely to change
+ # with time
+
+ defp set_error_ttl(url, _reason) do
+ ttl = Pleroma.Config.get([:rich_media, :failure_backoff], 60_000)
+ Cachex.expire(:rich_media_cache, url, ttl)
+ :ok
+ end
+
+ defp log_error(url, {:invalid_metadata, data}) do
+ Logger.debug(fn -> "Incomplete or invalid metadata for #{url}: #{inspect(data)}" end)
+ end
+
+ defp log_error(url, reason) do
+ Logger.warn(fn -> "Rich media error for #{url}: #{inspect(reason)}" end)
+ end
end
@doc """
diff --git a/lib/pleroma/web/rich_media/parser/ttl.ex b/lib/pleroma/web/rich_media/parser/ttl.ex
new file mode 100644
index 000000000..8353f0fff
--- /dev/null
+++ b/lib/pleroma/web/rich_media/parser/ttl.ex
@@ -0,0 +1,7 @@
+# Pleroma: A lightweight social networking server
+# Copyright © 2017-2020 Pleroma Authors
+# SPDX-License-Identifier: AGPL-3.0-only
+
+defmodule Pleroma.Web.RichMedia.Parser.TTL do
+ @callback ttl(Map.t(), String.t()) :: Integer.t() | nil
+end
diff --git a/lib/pleroma/web/rich_media/parsers/ttl/aws_signed_url.ex b/lib/pleroma/web/rich_media/parser/ttl/aws_signed_url.ex
similarity index 87%
rename from lib/pleroma/web/rich_media/parsers/ttl/aws_signed_url.ex
rename to lib/pleroma/web/rich_media/parser/ttl/aws_signed_url.ex
index c5aaea2d4..fc4ef79c0 100644
--- a/lib/pleroma/web/rich_media/parsers/ttl/aws_signed_url.ex
+++ b/lib/pleroma/web/rich_media/parser/ttl/aws_signed_url.ex
@@ -1,7 +1,11 @@
+# Pleroma: A lightweight social networking server
+# Copyright © 2017-2020 Pleroma Authors
+# SPDX-License-Identifier: AGPL-3.0-only
+
defmodule Pleroma.Web.RichMedia.Parser.TTL.AwsSignedUrl do
@behaviour Pleroma.Web.RichMedia.Parser.TTL
- @impl Pleroma.Web.RichMedia.Parser.TTL
+ @impl true
def ttl(data, _url) do
image = Map.get(data, :image)
diff --git a/lib/pleroma/web/rich_media/parsers/oembed_parser.ex b/lib/pleroma/web/rich_media/parsers/o_embed.ex
similarity index 100%
rename from lib/pleroma/web/rich_media/parsers/oembed_parser.ex
rename to lib/pleroma/web/rich_media/parsers/o_embed.ex
diff --git a/lib/pleroma/web/rich_media/parsers/ttl/ttl.ex b/lib/pleroma/web/rich_media/parsers/ttl/ttl.ex
deleted file mode 100644
index 6b3ec6d30..000000000
--- a/lib/pleroma/web/rich_media/parsers/ttl/ttl.ex
+++ /dev/null
@@ -1,3 +0,0 @@
-defmodule Pleroma.Web.RichMedia.Parser.TTL do
- @callback ttl(Map.t(), String.t()) :: {:ok, Integer.t()} | {:error, String.t()}
-end
diff --git a/lib/pleroma/web/router.ex b/lib/pleroma/web/router.ex
index c6433cc53..0f0538182 100644
--- a/lib/pleroma/web/router.ex
+++ b/lib/pleroma/web/router.ex
@@ -5,6 +5,26 @@
defmodule Pleroma.Web.Router do
use Pleroma.Web, :router
+ pipeline :accepts_html do
+ plug(:accepts, ["html"])
+ end
+
+ pipeline :accepts_html_xml do
+ plug(:accepts, ["html", "xml", "rss", "atom"])
+ end
+
+ pipeline :accepts_html_json do
+ plug(:accepts, ["html", "activity+json", "json"])
+ end
+
+ pipeline :accepts_html_xml_json do
+ plug(:accepts, ["html", "xml", "rss", "atom", "activity+json", "json"])
+ end
+
+ pipeline :accepts_xml_rss_atom do
+ plug(:accepts, ["xml", "rss", "atom"])
+ end
+
pipeline :browser do
plug(:accepts, ["html"])
plug(:fetch_session)
@@ -12,31 +32,31 @@ defmodule Pleroma.Web.Router do
pipeline :oauth do
plug(:fetch_session)
- plug(Pleroma.Plugs.OAuthPlug)
- plug(Pleroma.Plugs.UserEnabledPlug)
+ plug(Pleroma.Web.Plugs.OAuthPlug)
+ plug(Pleroma.Web.Plugs.UserEnabledPlug)
end
pipeline :expect_authentication do
- plug(Pleroma.Plugs.ExpectAuthenticatedCheckPlug)
+ plug(Pleroma.Web.Plugs.ExpectAuthenticatedCheckPlug)
end
pipeline :expect_public_instance_or_authentication do
- plug(Pleroma.Plugs.ExpectPublicOrAuthenticatedCheckPlug)
+ plug(Pleroma.Web.Plugs.ExpectPublicOrAuthenticatedCheckPlug)
end
pipeline :authenticate do
- plug(Pleroma.Plugs.OAuthPlug)
- plug(Pleroma.Plugs.BasicAuthDecoderPlug)
- plug(Pleroma.Plugs.UserFetcherPlug)
- plug(Pleroma.Plugs.SessionAuthenticationPlug)
- plug(Pleroma.Plugs.LegacyAuthenticationPlug)
- plug(Pleroma.Plugs.AuthenticationPlug)
+ plug(Pleroma.Web.Plugs.OAuthPlug)
+ plug(Pleroma.Web.Plugs.BasicAuthDecoderPlug)
+ plug(Pleroma.Web.Plugs.UserFetcherPlug)
+ plug(Pleroma.Web.Plugs.SessionAuthenticationPlug)
+ plug(Pleroma.Web.Plugs.LegacyAuthenticationPlug)
+ plug(Pleroma.Web.Plugs.AuthenticationPlug)
end
pipeline :after_auth do
- plug(Pleroma.Plugs.UserEnabledPlug)
- plug(Pleroma.Plugs.SetUserSessionIdPlug)
- plug(Pleroma.Plugs.EnsureUserKeyPlug)
+ plug(Pleroma.Web.Plugs.UserEnabledPlug)
+ plug(Pleroma.Web.Plugs.SetUserSessionIdPlug)
+ plug(Pleroma.Web.Plugs.EnsureUserKeyPlug)
end
pipeline :base_api do
@@ -50,25 +70,25 @@ defmodule Pleroma.Web.Router do
plug(:expect_public_instance_or_authentication)
plug(:base_api)
plug(:after_auth)
- plug(Pleroma.Plugs.IdempotencyPlug)
+ plug(Pleroma.Web.Plugs.IdempotencyPlug)
end
pipeline :authenticated_api do
plug(:expect_authentication)
plug(:base_api)
plug(:after_auth)
- plug(Pleroma.Plugs.EnsureAuthenticatedPlug)
- plug(Pleroma.Plugs.IdempotencyPlug)
+ plug(Pleroma.Web.Plugs.EnsureAuthenticatedPlug)
+ plug(Pleroma.Web.Plugs.IdempotencyPlug)
end
pipeline :admin_api do
plug(:expect_authentication)
plug(:base_api)
- plug(Pleroma.Plugs.AdminSecretAuthenticationPlug)
+ plug(Pleroma.Web.Plugs.AdminSecretAuthenticationPlug)
plug(:after_auth)
- plug(Pleroma.Plugs.EnsureAuthenticatedPlug)
- plug(Pleroma.Plugs.UserIsAdminPlug)
- plug(Pleroma.Plugs.IdempotencyPlug)
+ plug(Pleroma.Web.Plugs.EnsureAuthenticatedPlug)
+ plug(Pleroma.Web.Plugs.UserIsAdminPlug)
+ plug(Pleroma.Web.Plugs.IdempotencyPlug)
end
pipeline :mastodon_html do
@@ -80,7 +100,7 @@ defmodule Pleroma.Web.Router do
pipeline :pleroma_html do
plug(:browser)
plug(:authenticate)
- plug(Pleroma.Plugs.EnsureUserKeyPlug)
+ plug(Pleroma.Web.Plugs.EnsureUserKeyPlug)
end
pipeline :well_known do
@@ -129,16 +149,7 @@ defmodule Pleroma.Web.Router do
scope "/api/pleroma/admin", Pleroma.Web.AdminAPI do
pipe_through(:admin_api)
- post("/users/follow", AdminAPIController, :user_follow)
- post("/users/unfollow", AdminAPIController, :user_unfollow)
-
put("/users/disable_mfa", AdminAPIController, :disable_mfa)
- delete("/users", AdminAPIController, :user_delete)
- post("/users", AdminAPIController, :users_create)
- patch("/users/:nickname/toggle_activation", AdminAPIController, :user_toggle_activation)
- patch("/users/activate", AdminAPIController, :user_activate)
- patch("/users/deactivate", AdminAPIController, :user_deactivate)
- patch("/users/approve", AdminAPIController, :user_approve)
put("/users/tag", AdminAPIController, :tag_users)
delete("/users/tag", AdminAPIController, :untag_users)
@@ -161,6 +172,15 @@ defmodule Pleroma.Web.Router do
:right_delete_multiple
)
+ post("/users/follow", UserController, :follow)
+ post("/users/unfollow", UserController, :unfollow)
+ delete("/users", UserController, :delete)
+ post("/users", UserController, :create)
+ patch("/users/:nickname/toggle_activation", UserController, :toggle_activation)
+ patch("/users/activate", UserController, :activate)
+ patch("/users/deactivate", UserController, :deactivate)
+ patch("/users/approve", UserController, :approve)
+
get("/relay", RelayController, :index)
post("/relay", RelayController, :follow)
delete("/relay", RelayController, :unfollow)
@@ -175,12 +195,17 @@ defmodule Pleroma.Web.Router do
get("/users/:nickname/credentials", AdminAPIController, :show_user_credentials)
patch("/users/:nickname/credentials", AdminAPIController, :update_user_credentials)
- get("/users", AdminAPIController, :list_users)
- get("/users/:nickname", AdminAPIController, :user_show)
+ get("/users", UserController, :list)
+ get("/users/:nickname", UserController, :show)
get("/users/:nickname/statuses", AdminAPIController, :list_user_statuses)
+ get("/users/:nickname/chats", AdminAPIController, :list_user_chats)
get("/instances/:instance/statuses", AdminAPIController, :list_instance_statuses)
+ get("/instance_document/:name", InstanceDocumentController, :show)
+ patch("/instance_document/:name", InstanceDocumentController, :update)
+ delete("/instance_document/:name", InstanceDocumentController, :delete)
+
patch("/users/confirm_email", AdminAPIController, :confirm_email)
patch("/users/resend_confirmation_email", AdminAPIController, :resend_confirmation_email)
@@ -214,9 +239,29 @@ defmodule Pleroma.Web.Router do
get("/media_proxy_caches", MediaProxyCacheController, :index)
post("/media_proxy_caches/delete", MediaProxyCacheController, :delete)
post("/media_proxy_caches/purge", MediaProxyCacheController, :purge)
+
+ get("/chats/:id", ChatController, :show)
+ get("/chats/:id/messages", ChatController, :messages)
+ delete("/chats/:id/messages/:message_id", ChatController, :delete_message)
+
+ post("/backups", AdminAPIController, :create_backup)
end
scope "/api/pleroma/emoji", Pleroma.Web.PleromaAPI do
+ scope "/pack" do
+ pipe_through(:admin_api)
+
+ post("/", EmojiPackController, :create)
+ patch("/", EmojiPackController, :update)
+ delete("/", EmojiPackController, :delete)
+ end
+
+ scope "/pack" do
+ pipe_through(:api)
+
+ get("/", EmojiPackController, :show)
+ end
+
# Modifying packs
scope "/packs" do
pipe_through(:admin_api)
@@ -225,21 +270,17 @@ defmodule Pleroma.Web.Router do
get("/remote", EmojiPackController, :remote)
post("/download", EmojiPackController, :download)
- post("/:name", EmojiPackController, :create)
- patch("/:name", EmojiPackController, :update)
- delete("/:name", EmojiPackController, :delete)
-
- post("/:name/files", EmojiPackController, :add_file)
- patch("/:name/files", EmojiPackController, :update_file)
- delete("/:name/files", EmojiPackController, :delete_file)
+ post("/files", EmojiFileController, :create)
+ patch("/files", EmojiFileController, :update)
+ delete("/files", EmojiFileController, :delete)
end
# Pack info / downloading
scope "/packs" do
pipe_through(:api)
+
get("/", EmojiPackController, :index)
- get("/:name", EmojiPackController, :show)
- get("/:name/archive", EmojiPackController, :archive)
+ get("/archive", EmojiPackController, :archive)
end
end
@@ -260,14 +301,15 @@ defmodule Pleroma.Web.Router do
post("/delete_account", UtilController, :delete_account)
put("/notification_settings", UtilController, :update_notificaton_settings)
post("/disable_account", UtilController, :disable_account)
-
- post("/blocks_import", UtilController, :blocks_import)
- post("/follow_import", UtilController, :follow_import)
end
scope "/api/pleroma", Pleroma.Web.PleromaAPI do
pipe_through(:authenticated_api)
+ post("/mutes_import", UserImportController, :mutes)
+ post("/blocks_import", UserImportController, :blocks)
+ post("/follow_import", UserImportController, :follow)
+
get("/accounts/mfa", TwoFactorAuthenticationController, :settings)
get("/accounts/mfa/backup_codes", TwoFactorAuthenticationController, :backup_codes)
get("/accounts/mfa/setup/:method", TwoFactorAuthenticationController, :setup)
@@ -333,6 +375,9 @@ defmodule Pleroma.Web.Router do
put("/mascot", MascotController, :update)
post("/scrobble", ScrobbleController, :create)
+
+ get("/backups", BackupController, :index)
+ post("/backups", BackupController, :create)
end
scope [] do
@@ -353,6 +398,7 @@ defmodule Pleroma.Web.Router do
scope "/api/v1/pleroma", Pleroma.Web.PleromaAPI do
pipe_through(:api)
get("/accounts/:id/scrobbles", ScrobbleController, :index)
+ get("/federation_status", InstancesController, :show)
end
scope "/api/v1", Pleroma.Web.MastodonAPI do
@@ -546,30 +592,43 @@ defmodule Pleroma.Web.Router do
)
end
- pipeline :ostatus do
- plug(:accepts, ["html", "xml", "rss", "atom", "activity+json", "json"])
- plug(Pleroma.Plugs.StaticFEPlug)
- end
-
- pipeline :oembed do
- plug(:accepts, ["json", "xml"])
- end
-
scope "/", Pleroma.Web do
- pipe_through([:ostatus, :http_signature])
+ # Note: html format is supported only if static FE is enabled
+ # Note: http signature is only considered for json requests (no auth for non-json requests)
+ pipe_through([:accepts_html_json, :http_signature, Pleroma.Web.Plugs.StaticFEPlug])
get("/objects/:uuid", OStatus.OStatusController, :object)
get("/activities/:uuid", OStatus.OStatusController, :activity)
get("/notice/:id", OStatus.OStatusController, :notice)
- get("/notice/:id/embed_player", OStatus.OStatusController, :notice_player)
# Mastodon compatibility routes
get("/users/:nickname/statuses/:id", OStatus.OStatusController, :object)
get("/users/:nickname/statuses/:id/activity", OStatus.OStatusController, :activity)
+ end
+
+ scope "/", Pleroma.Web do
+ # Note: html format is supported only if static FE is enabled
+ # Note: http signature is only considered for json requests (no auth for non-json requests)
+ pipe_through([:accepts_html_xml_json, :http_signature, Pleroma.Web.Plugs.StaticFEPlug])
+
+ # Note: returns user _profile_ for json requests, redirects to user _feed_ for non-json ones
+ get("/users/:nickname", Feed.UserController, :feed_redirect, as: :user_feed)
+ end
+
+ scope "/", Pleroma.Web do
+ # Note: html format is supported only if static FE is enabled
+ pipe_through([:accepts_html_xml, Pleroma.Web.Plugs.StaticFEPlug])
get("/users/:nickname/feed", Feed.UserController, :feed, as: :user_feed)
- get("/users/:nickname", Feed.UserController, :feed_redirect, as: :user_feed)
+ end
+ scope "/", Pleroma.Web do
+ pipe_through(:accepts_html)
+ get("/notice/:id/embed_player", OStatus.OStatusController, :notice_player)
+ end
+
+ scope "/", Pleroma.Web do
+ pipe_through(:accepts_xml_rss_atom)
get("/tags/:tag", Feed.TagController, :feed, as: :tag_feed)
end
@@ -670,6 +729,8 @@ defmodule Pleroma.Web.Router do
end
scope "/proxy/", Pleroma.Web.MediaProxy do
+ get("/preview/:sig/:url", MediaProxyController, :preview)
+ get("/preview/:sig/:url/:filename", MediaProxyController, :preview)
get("/:sig/:url", MediaProxyController, :remote)
get("/:sig/:url/:filename", MediaProxyController, :remote)
end
@@ -715,7 +776,7 @@ defmodule Pleroma.Web.Router do
get("/check_password", MongooseIMController, :check_password)
end
- scope "/", Fallback do
+ scope "/", Pleroma.Web.Fallback do
get("/registration/:token", RedirectController, :registration_page)
get("/:maybe_nickname_or_id", RedirectController, :redirector_with_meta)
get("/api*path", RedirectController, :api_not_implemented)
diff --git a/lib/pleroma/web/static_fe/static_fe_controller.ex b/lib/pleroma/web/static_fe/static_fe_controller.ex
index a7a891b13..bdec0897a 100644
--- a/lib/pleroma/web/static_fe/static_fe_controller.ex
+++ b/lib/pleroma/web/static_fe/static_fe_controller.ex
@@ -17,12 +17,96 @@ defmodule Pleroma.Web.StaticFE.StaticFEController do
plug(:put_view, Pleroma.Web.StaticFE.StaticFEView)
plug(:assign_id)
- plug(Pleroma.Plugs.EnsureAuthenticatedPlug,
- unless_func: &Pleroma.Web.FederatingPlug.federating?/1
- )
-
@page_keys ["max_id", "min_id", "limit", "since_id", "order"]
+ @doc "Renders requested local public activity or public activities of requested user"
+ def show(%{assigns: %{notice_id: notice_id}} = conn, _params) do
+ with %Activity{local: true} = activity <-
+ Activity.get_by_id_with_object(notice_id),
+ true <- Visibility.is_public?(activity.object),
+ {_, true} <- {:visible?, Visibility.visible_for_user?(activity, _reading_user = nil)},
+ %User{} = user <- User.get_by_ap_id(activity.object.data["actor"]) do
+ meta = Metadata.build_tags(%{activity_id: notice_id, object: activity.object, user: user})
+
+ timeline =
+ activity.object.data["context"]
+ |> ActivityPub.fetch_activities_for_context(%{})
+ |> Enum.reverse()
+ |> Enum.map(&represent(&1, &1.object.id == activity.object.id))
+
+ render(conn, "conversation.html", %{activities: timeline, meta: meta})
+ else
+ %Activity{object: %Object{data: data}} ->
+ conn
+ |> put_status(:found)
+ |> redirect(external: data["url"] || data["external_url"] || data["id"])
+
+ _ ->
+ not_found(conn, "Post not found.")
+ end
+ end
+
+ def show(%{assigns: %{username_or_id: username_or_id}} = conn, params) do
+ with {_, %User{local: true} = user} <-
+ {:fetch_user, User.get_cached_by_nickname_or_id(username_or_id)},
+ {_, :visible} <- {:visibility, User.visible_for(user, _reading_user = nil)} do
+ meta = Metadata.build_tags(%{user: user})
+
+ params =
+ params
+ |> Map.take(@page_keys)
+ |> Map.new(fn {k, v} -> {String.to_existing_atom(k), v} end)
+
+ timeline =
+ user
+ |> ActivityPub.fetch_user_activities(_reading_user = nil, params)
+ |> Enum.map(&represent/1)
+
+ prev_page_id =
+ (params["min_id"] || params["max_id"]) &&
+ List.first(timeline) && List.first(timeline).id
+
+ next_page_id = List.last(timeline) && List.last(timeline).id
+
+ render(conn, "profile.html", %{
+ user: User.sanitize_html(user),
+ timeline: timeline,
+ prev_page_id: prev_page_id,
+ next_page_id: next_page_id,
+ meta: meta
+ })
+ else
+ _ ->
+ not_found(conn, "User not found.")
+ end
+ end
+
+ def show(%{assigns: %{object_id: _}} = conn, _params) do
+ url = Helpers.url(conn) <> conn.request_path
+
+ case Activity.get_create_by_object_ap_id_with_object(url) do
+ %Activity{} = activity ->
+ to = Helpers.o_status_path(Pleroma.Web.Endpoint, :notice, activity)
+ redirect(conn, to: to)
+
+ _ ->
+ not_found(conn, "Post not found.")
+ end
+ end
+
+ def show(%{assigns: %{activity_id: _}} = conn, _params) do
+ url = Helpers.url(conn) <> conn.request_path
+
+ case Activity.get_by_ap_id(url) do
+ %Activity{} = activity ->
+ to = Helpers.o_status_path(Pleroma.Web.Endpoint, :notice, activity)
+ redirect(conn, to: to)
+
+ _ ->
+ not_found(conn, "Post not found.")
+ end
+ end
+
defp get_title(%Object{data: %{"name" => name}}) when is_binary(name),
do: name
@@ -81,91 +165,6 @@ defp represent(%Activity{object: %Object{data: data}} = activity, selected) do
}
end
- def show(%{assigns: %{notice_id: notice_id}} = conn, _params) do
- with %Activity{local: true} = activity <-
- Activity.get_by_id_with_object(notice_id),
- true <- Visibility.is_public?(activity.object),
- %User{} = user <- User.get_by_ap_id(activity.object.data["actor"]) do
- meta = Metadata.build_tags(%{activity_id: notice_id, object: activity.object, user: user})
-
- timeline =
- activity.object.data["context"]
- |> ActivityPub.fetch_activities_for_context(%{})
- |> Enum.reverse()
- |> Enum.map(&represent(&1, &1.object.id == activity.object.id))
-
- render(conn, "conversation.html", %{activities: timeline, meta: meta})
- else
- %Activity{object: %Object{data: data}} ->
- conn
- |> put_status(:found)
- |> redirect(external: data["url"] || data["external_url"] || data["id"])
-
- _ ->
- not_found(conn, "Post not found.")
- end
- end
-
- def show(%{assigns: %{username_or_id: username_or_id}} = conn, params) do
- case User.get_cached_by_nickname_or_id(username_or_id) do
- %User{} = user ->
- meta = Metadata.build_tags(%{user: user})
-
- params =
- params
- |> Map.take(@page_keys)
- |> Map.new(fn {k, v} -> {String.to_existing_atom(k), v} end)
-
- timeline =
- user
- |> ActivityPub.fetch_user_activities(nil, params)
- |> Enum.map(&represent/1)
-
- prev_page_id =
- (params["min_id"] || params["max_id"]) &&
- List.first(timeline) && List.first(timeline).id
-
- next_page_id = List.last(timeline) && List.last(timeline).id
-
- render(conn, "profile.html", %{
- user: User.sanitize_html(user),
- timeline: timeline,
- prev_page_id: prev_page_id,
- next_page_id: next_page_id,
- meta: meta
- })
-
- _ ->
- not_found(conn, "User not found.")
- end
- end
-
- def show(%{assigns: %{object_id: _}} = conn, _params) do
- url = Helpers.url(conn) <> conn.request_path
-
- case Activity.get_create_by_object_ap_id_with_object(url) do
- %Activity{} = activity ->
- to = Helpers.o_status_path(Pleroma.Web.Endpoint, :notice, activity)
- redirect(conn, to: to)
-
- _ ->
- not_found(conn, "Post not found.")
- end
- end
-
- def show(%{assigns: %{activity_id: _}} = conn, _params) do
- url = Helpers.url(conn) <> conn.request_path
-
- case Activity.get_by_ap_id(url) do
- %Activity{} = activity ->
- to = Helpers.o_status_path(Pleroma.Web.Endpoint, :notice, activity)
- redirect(conn, to: to)
-
- _ ->
- not_found(conn, "Post not found.")
- end
- end
-
defp assign_id(%{path_info: ["notice", notice_id]} = conn, _opts),
do: assign(conn, :notice_id, notice_id)
diff --git a/lib/pleroma/web/streamer/streamer.ex b/lib/pleroma/web/streamer.ex
similarity index 78%
rename from lib/pleroma/web/streamer/streamer.ex
rename to lib/pleroma/web/streamer.ex
index d1d70e556..71fe27c89 100644
--- a/lib/pleroma/web/streamer/streamer.ex
+++ b/lib/pleroma/web/streamer.ex
@@ -15,6 +15,8 @@ defmodule Pleroma.Web.Streamer do
alias Pleroma.Web.ActivityPub.ActivityPub
alias Pleroma.Web.ActivityPub.Visibility
alias Pleroma.Web.CommonAPI
+ alias Pleroma.Web.OAuth.Token
+ alias Pleroma.Web.Plugs.OAuthScopesPlug
alias Pleroma.Web.StreamerView
@mix_env Mix.env()
@@ -26,53 +28,96 @@ def registry, do: @registry
@user_streams ["user", "user:notification", "direct", "user:pleroma_chat"]
@doc "Expands and authorizes a stream, and registers the process for streaming."
- @spec get_topic_and_add_socket(stream :: String.t(), User.t() | nil, Map.t() | nil) ::
+ @spec get_topic_and_add_socket(
+ stream :: String.t(),
+ User.t() | nil,
+ Token.t() | nil,
+ Map.t() | nil
+ ) ::
{:ok, topic :: String.t()} | {:error, :bad_topic} | {:error, :unauthorized}
- def get_topic_and_add_socket(stream, user, params \\ %{}) do
- case get_topic(stream, user, params) do
+ def get_topic_and_add_socket(stream, user, oauth_token, params \\ %{}) do
+ case get_topic(stream, user, oauth_token, params) do
{:ok, topic} -> add_socket(topic, user)
error -> error
end
end
@doc "Expand and authorizes a stream"
- @spec get_topic(stream :: String.t(), User.t() | nil, Map.t()) ::
+ @spec get_topic(stream :: String.t(), User.t() | nil, Token.t() | nil, Map.t()) ::
{:ok, topic :: String.t()} | {:error, :bad_topic}
- def get_topic(stream, user, params \\ %{})
+ def get_topic(stream, user, oauth_token, params \\ %{})
# Allow all public steams.
- def get_topic(stream, _, _) when stream in @public_streams do
+ def get_topic(stream, _user, _oauth_token, _params) when stream in @public_streams do
{:ok, stream}
end
# Allow all hashtags streams.
- def get_topic("hashtag", _, %{"tag" => tag}) do
+ def get_topic("hashtag", _user, _oauth_token, %{"tag" => tag} = _params) do
{:ok, "hashtag:" <> tag}
end
- # Expand user streams.
- def get_topic(stream, %User{} = user, _) when stream in @user_streams do
- {:ok, stream <> ":" <> to_string(user.id)}
+ # Allow remote instance streams.
+ def get_topic("public:remote", _user, _oauth_token, %{"instance" => instance} = _params) do
+ {:ok, "public:remote:" <> instance}
end
- def get_topic(stream, _, _) when stream in @user_streams do
+ def get_topic("public:remote:media", _user, _oauth_token, %{"instance" => instance} = _params) do
+ {:ok, "public:remote:media:" <> instance}
+ end
+
+ # Expand user streams.
+ def get_topic(
+ stream,
+ %User{id: user_id} = user,
+ %Token{user_id: token_user_id} = oauth_token,
+ _params
+ )
+ when stream in @user_streams and user_id == token_user_id do
+ # Note: "read" works for all user streams (not mentioning it since it's an ancestor scope)
+ required_scopes =
+ if stream == "user:notification" do
+ ["read:notifications"]
+ else
+ ["read:statuses"]
+ end
+
+ if OAuthScopesPlug.filter_descendants(required_scopes, oauth_token.scopes) == [] do
+ {:error, :unauthorized}
+ else
+ {:ok, stream <> ":" <> to_string(user.id)}
+ end
+ end
+
+ def get_topic(stream, _user, _oauth_token, _params) when stream in @user_streams do
{:error, :unauthorized}
end
# List streams.
- def get_topic("list", %User{} = user, %{"list" => id}) do
- if Pleroma.List.get(id, user) do
- {:ok, "list:" <> to_string(id)}
- else
- {:error, :bad_topic}
+ def get_topic(
+ "list",
+ %User{id: user_id} = user,
+ %Token{user_id: token_user_id} = oauth_token,
+ %{"list" => id}
+ )
+ when user_id == token_user_id do
+ cond do
+ OAuthScopesPlug.filter_descendants(["read", "read:lists"], oauth_token.scopes) == [] ->
+ {:error, :unauthorized}
+
+ Pleroma.List.get(id, user) ->
+ {:ok, "list:" <> to_string(id)}
+
+ true ->
+ {:error, :bad_topic}
end
end
- def get_topic("list", _, _) do
+ def get_topic("list", _user, _oauth_token, _params) do
{:error, :unauthorized}
end
- def get_topic(_, _, _) do
+ def get_topic(_stream, _user, _oauth_token, _params) do
{:error, :bad_topic}
end
diff --git a/lib/pleroma/web/templates/layout/app.html.eex b/lib/pleroma/web/templates/layout/app.html.eex
index 51603fe0c..3f28f1920 100644
--- a/lib/pleroma/web/templates/layout/app.html.eex
+++ b/lib/pleroma/web/templates/layout/app.html.eex
@@ -228,7 +228,7 @@
<%= Pleroma.Config.get([:instance, :name]) %>
- <%= render @view_module, @view_template, assigns %>
+ <%= @inner_content %>