Merge remote-tracking branch 'remotes/origin/develop' into auth-improvements

# Conflicts:
#	CHANGELOG.md
This commit is contained in:
Ivan Tashkinov 2020-12-07 21:45:04 +03:00
commit c8e7ace812
8 changed files with 602 additions and 125 deletions
CHANGELOG.md
docs
administration/CLI_tasks
configuration
lib
mix/tasks/pleroma
pleroma
test
mix/tasks/pleroma
pleroma/web/admin_api/controllers

View file

@ -24,6 +24,7 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/).
- Ability to view remote timelines, with ex. `/api/v1/timelines/public?instance=lain.com` and streams `public:remote` and `public:remote:media`. - Ability to view remote timelines, with ex. `/api/v1/timelines/public?instance=lain.com` and streams `public:remote` and `public:remote:media`.
- The site title is now injected as a `title` tag like preloads or metadata. - The site title is now injected as a `title` tag like preloads or metadata.
- Password reset tokens now are not accepted after a certain age. - Password reset tokens now are not accepted after a certain age.
- Mix tasks to help with displaying and removing ConfigDB entries. See `mix pleroma.config`
- OAuth form improvements: users are remembered by their cookie, the CSS is overridable by the admin, and the style has been improved. - OAuth form improvements: users are remembered by their cookie, the CSS is overridable by the admin, and the style has been improved.
<details> <details>

View file

@ -32,7 +32,7 @@
config :pleroma, configurable_from_database: false config :pleroma, configurable_from_database: false
``` ```
To delete transfered settings from database optional flag `-d` can be used. `<env>` is `prod` by default. To delete transferred settings from database optional flag `-d` can be used. `<env>` is `prod` by default.
=== "OTP" === "OTP"
```sh ```sh
@ -43,3 +43,111 @@ To delete transfered settings from database optional flag `-d` can be used. `<en
```sh ```sh
mix pleroma.config migrate_from_db [--env=<env>] [-d] mix pleroma.config migrate_from_db [--env=<env>] [-d]
``` ```
## Dump all of the config settings defined in the database
=== "OTP"
```sh
./bin/pleroma_ctl config dump
```
=== "From Source"
```sh
mix pleroma.config dump
```
## List individual configuration groups in the database
=== "OTP"
```sh
./bin/pleroma_ctl config groups
```
=== "From Source"
```sh
mix pleroma.config groups
```
## Dump the saved configuration values for a specific group or key
e.g., this shows all the settings under `config :pleroma`
=== "OTP"
```sh
./bin/pleroma_ctl config dump pleroma
```
=== "From Source"
```sh
mix pleroma.config dump pleroma
```
To get values under a specific key:
e.g., this shows all the settings under `config :pleroma, :instance`
=== "OTP"
```sh
./bin/pleroma_ctl config dump pleroma instance
```
=== "From Source"
```sh
mix pleroma.config dump pleroma instance
```
## Delete the saved configuration values for a specific group or key
e.g., this deletes all the settings under `config :tesla`
=== "OTP"
```sh
./bin/pleroma_ctl config delete [--force] tesla
```
=== "From Source"
```sh
mix pleroma.config delete [--force] tesla
```
To delete values under a specific key:
e.g., this deletes all the settings under `config :phoenix, :stacktrace_depth`
=== "OTP"
```sh
./bin/pleroma_ctl config delete [--force] phoenix stacktrace_depth
```
=== "From Source"
```sh
mix pleroma.config delete [--force] phoenix stacktrace_depth
```
## Remove all settings from the database
This forcibly removes all saved values in the database.
=== "OTP"
```sh
./bin/pleroma_ctl config [--force] reset
```
=== "From Source"
```sh
mix pleroma.config [--force] reset
```

View file

@ -5,50 +5,37 @@ The configuration of Pleroma has traditionally been managed with a config file,
## Migration to database config ## Migration to database config
1. Run the mix task to migrate to the database. You'll receive some debugging output and a few messages informing you of what happened. 1. Run the mix task to migrate to the database.
**Source:** **Source:**
``` ```
$ mix pleroma.config migrate_to_db $ mix pleroma.config migrate_to_db
``` ```
or or
**OTP:** **OTP:**
*Note: OTP users need Pleroma to be running for `pleroma_ctl` commands to work* *Note: OTP users need Pleroma to be running for `pleroma_ctl` commands to work*
``` ```
$ ./bin/pleroma_ctl config migrate_to_db $ ./bin/pleroma_ctl config migrate_to_db
``` ```
``` ```
10:04:34.155 [debug] QUERY OK source="config" db=1.6ms decode=2.0ms queue=33.5ms idle=0.0ms
SELECT c0."id", c0."key", c0."group", c0."value", c0."inserted_at", c0."updated_at" FROM "config" AS c0 []
Migrating settings from file: /home/pleroma/config/dev.secret.exs Migrating settings from file: /home/pleroma/config/dev.secret.exs
10:04:34.240 [debug] QUERY OK db=4.5ms queue=0.3ms idle=92.2ms
TRUNCATE config; []
10:04:34.244 [debug] QUERY OK db=2.8ms queue=0.3ms idle=97.2ms
ALTER SEQUENCE config_id_seq RESTART; []
10:04:34.256 [debug] QUERY OK source="config" db=0.8ms queue=1.4ms idle=109.8ms
SELECT c0."id", c0."key", c0."group", c0."value", c0."inserted_at", c0."updated_at" FROM "config" AS c0 WHERE ((c0."group" = $1) AND (c0."key" = $2)) [":pleroma", ":instance"]
10:04:34.292 [debug] QUERY OK db=2.6ms queue=1.7ms idle=137.7ms
INSERT INTO "config" ("group","key","value","inserted_at","updated_at") VALUES ($1,$2,$3,$4,$5) RETURNING "id" [":pleroma", ":instance", <<131, 108, 0, 0, 0, 1, 104, 2, 100, 0, 4, 110, 97, 109, 101, 109, 0, 0, 0, 7, 66, 108, 101, 114, 111, 109, 97, 106>>, ~N[2020-07-12 15:04:34], ~N[2020-07-12 15:04:34]]
Settings for key instance migrated. Settings for key instance migrated.
Settings for group :pleroma migrated. Settings for group :pleroma migrated.
``` ```
2. It is recommended to backup your config file now. 2. It is recommended to backup your config file now.
``` ```
cp config/dev.secret.exs config/dev.secret.exs.orig cp config/dev.secret.exs config/dev.secret.exs.orig
``` ```
3. Edit your Pleroma config to enable database configuration: 3. Edit your Pleroma config to enable database configuration:
``` ```
@ -76,17 +63,17 @@ The configuration of Pleroma has traditionally been managed with a config file,
config :pleroma, Pleroma.Web.Endpoint, config :pleroma, Pleroma.Web.Endpoint,
url: [host: "cool.pleroma.site", scheme: "https", port: 443] url: [host: "cool.pleroma.site", scheme: "https", port: 443]
config :pleroma, Pleroma.Repo, config :pleroma, Pleroma.Repo,
adapter: Ecto.Adapters.Postgres, adapter: Ecto.Adapters.Postgres,
username: "pleroma", username: "pleroma",
password: "MySecretPassword", password: "MySecretPassword",
database: "pleroma_prod", database: "pleroma_prod",
hostname: "localhost" hostname: "localhost"
config :pleroma, configurable_from_database: true config :pleroma, configurable_from_database: true
``` ```
5. Restart your instance and you can now access the Settings tab in AdminFE. 5. Restart your instance and you can now access the Settings tab in AdminFE.
@ -95,15 +82,15 @@ The configuration of Pleroma has traditionally been managed with a config file,
1. Run the mix task to migrate back from the database. You'll receive some debugging output and a few messages informing you of what happened. 1. Run the mix task to migrate back from the database. You'll receive some debugging output and a few messages informing you of what happened.
**Source:** **Source:**
``` ```
$ mix pleroma.config migrate_from_db $ mix pleroma.config migrate_from_db
``` ```
or or
**OTP:** **OTP:**
``` ```
$ ./bin/pleroma_ctl config migrate_from_db $ ./bin/pleroma_ctl config migrate_from_db
``` ```
@ -111,7 +98,7 @@ The configuration of Pleroma has traditionally been managed with a config file,
``` ```
10:26:30.593 [debug] QUERY OK source="config" db=9.8ms decode=1.2ms queue=26.0ms idle=0.0ms 10:26:30.593 [debug] QUERY OK source="config" db=9.8ms decode=1.2ms queue=26.0ms idle=0.0ms
SELECT c0."id", c0."key", c0."group", c0."value", c0."inserted_at", c0."updated_at" FROM "config" AS c0 [] SELECT c0."id", c0."key", c0."group", c0."value", c0."inserted_at", c0."updated_at" FROM "config" AS c0 []
10:26:30.659 [debug] QUERY OK source="config" db=1.1ms idle=80.7ms 10:26:30.659 [debug] QUERY OK source="config" db=1.1ms idle=80.7ms
SELECT c0."id", c0."key", c0."group", c0."value", c0."inserted_at", c0."updated_at" FROM "config" AS c0 [] SELECT c0."id", c0."key", c0."group", c0."value", c0."inserted_at", c0."updated_at" FROM "config" AS c0 []
Database configuration settings have been saved to config/dev.exported_from_db.secret.exs Database configuration settings have been saved to config/dev.exported_from_db.secret.exs
@ -124,30 +111,45 @@ The configuration of Pleroma has traditionally been managed with a config file,
## Debugging ## Debugging
### Clearing database config ### Clearing database config
You can clear the database config by truncating the `config` table in the database. e.g., You can clear the database config with the following command:
``` **Source:**
psql -d pleroma_dev
pleroma_dev=# TRUNCATE config; ```
TRUNCATE TABLE $ mix pleroma.config reset
``` ```
or
**OTP:**
```
$ ./bin/pleroma_ctl config reset
```
Additionally, every time you migrate the configuration to the database the config table is automatically truncated to ensure a clean migration. Additionally, every time you migrate the configuration to the database the config table is automatically truncated to ensure a clean migration.
### Manually removing a setting ### Manually removing a setting
If you encounter a situation where the server cannot run properly because of an invalid setting in the database and this is preventing you from accessing AdminFE, you can manually remove the offending setting if you know which one it is. If you encounter a situation where the server cannot run properly because of an invalid setting in the database and this is preventing you from accessing AdminFE, you can manually remove the offending setting if you know which one it is.
e.g., here is an example showing a minimal configuration in the database. Only the `config :pleroma, :instance` settings are in the table: e.g., here is an example showing a the removal of the `config :pleroma, :instance` settings:
``` **Source:**
psql -d pleroma_dev
pleroma_dev=# select * from config; ```
id | key | value | inserted_at | updated_at | group $ mix pleroma.config delete pleroma instance
----+-----------+------------------------------------------------------------+---------------------+---------------------+---------- Are you sure you want to continue? [n] y
1 | :instance | \x836c0000000168026400046e616d656d00000007426c65726f6d616a | 2020-07-12 15:33:29 | 2020-07-12 15:33:29 | :pleroma config :pleroma, :instance deleted from the ConfigDB.
(1 row) ```
pleroma_dev=# delete from config where key = ':instance' and group = ':pleroma';
DELETE 1 or
```
**OTP:**
```
$ ./bin/pleroma_ctl config delete pleroma instance
Are you sure you want to continue? [n] y
config :pleroma, :instance deleted from the ConfigDB.
```
Now the `config :pleroma, :instance` settings have been removed from the database. Now the `config :pleroma, :instance` settings have been removed from the database.

View file

@ -5,6 +5,7 @@
defmodule Mix.Tasks.Pleroma.Config do defmodule Mix.Tasks.Pleroma.Config do
use Mix.Task use Mix.Task
import Ecto.Query
import Mix.Pleroma import Mix.Pleroma
alias Pleroma.ConfigDB alias Pleroma.ConfigDB
@ -14,26 +15,199 @@ defmodule Mix.Tasks.Pleroma.Config do
@moduledoc File.read!("docs/administration/CLI_tasks/config.md") @moduledoc File.read!("docs/administration/CLI_tasks/config.md")
def run(["migrate_to_db"]) do def run(["migrate_to_db"]) do
start_pleroma() check_configdb(fn ->
migrate_to_db() start_pleroma()
migrate_to_db()
end)
end end
def run(["migrate_from_db" | options]) do def run(["migrate_from_db" | options]) do
check_configdb(fn ->
start_pleroma()
{opts, _} =
OptionParser.parse!(options,
strict: [env: :string, delete: :boolean],
aliases: [d: :delete]
)
migrate_from_db(opts)
end)
end
def run(["dump"]) do
check_configdb(fn ->
start_pleroma()
header = config_header()
settings =
ConfigDB
|> Repo.all()
|> Enum.sort()
unless settings == [] do
shell_info("#{header}")
Enum.each(settings, &dump(&1))
else
shell_error("No settings in ConfigDB.")
end
end)
end
def run(["dump", group, key]) do
check_configdb(fn ->
start_pleroma()
group = maybe_atomize(group)
key = maybe_atomize(key)
group
|> ConfigDB.get_by_group_and_key(key)
|> dump()
end)
end
def run(["dump", group]) do
check_configdb(fn ->
start_pleroma()
group = maybe_atomize(group)
dump_group(group)
end)
end
def run(["groups"]) do
check_configdb(fn ->
start_pleroma()
groups =
ConfigDB
|> distinct([c], true)
|> select([c], c.group)
|> Repo.all()
if length(groups) > 0 do
shell_info("The following configuration groups are set in ConfigDB:\r\n")
groups |> Enum.each(fn x -> shell_info("- #{x}") end)
shell_info("\r\n")
end
end)
end
def run(["reset", "--force"]) do
check_configdb(fn ->
start_pleroma()
truncatedb()
shell_info("The ConfigDB settings have been removed from the database.")
end)
end
def run(["reset"]) do
check_configdb(fn ->
start_pleroma()
shell_info("The following settings will be permanently removed:")
ConfigDB
|> Repo.all()
|> Enum.sort()
|> Enum.each(&dump(&1))
shell_error("\nTHIS CANNOT BE UNDONE!")
if shell_prompt("Are you sure you want to continue?", "n") in ~w(Yn Y y) do
truncatedb()
shell_info("The ConfigDB settings have been removed from the database.")
else
shell_error("No changes made.")
end
end)
end
def run(["delete", "--force", group, key]) do
start_pleroma() start_pleroma()
{opts, _} = group = maybe_atomize(group)
OptionParser.parse!(options, key = maybe_atomize(key)
strict: [env: :string, delete: :boolean],
aliases: [d: :delete]
)
migrate_from_db(opts) with true <- key_exists?(group, key) do
shell_info("The following settings will be removed from ConfigDB:\n")
group
|> ConfigDB.get_by_group_and_key(key)
|> dump()
delete_key(group, key)
else
_ ->
shell_error("No settings in ConfigDB for #{inspect(group)}, #{inspect(key)}. Aborting.")
end
end
def run(["delete", "--force", group]) do
start_pleroma()
group = maybe_atomize(group)
with true <- group_exists?(group) do
shell_info("The following settings will be removed from ConfigDB:\n")
dump_group(group)
delete_group(group)
else
_ -> shell_error("No settings in ConfigDB for #{inspect(group)}. Aborting.")
end
end
def run(["delete", group, key]) do
start_pleroma()
group = maybe_atomize(group)
key = maybe_atomize(key)
with true <- key_exists?(group, key) do
shell_info("The following settings will be removed from ConfigDB:\n")
group
|> ConfigDB.get_by_group_and_key(key)
|> dump()
if shell_prompt("Are you sure you want to continue?", "n") in ~w(Yn Y y) do
delete_key(group, key)
else
shell_error("No changes made.")
end
else
_ ->
shell_error("No settings in ConfigDB for #{inspect(group)}, #{inspect(key)}. Aborting.")
end
end
def run(["delete", group]) do
start_pleroma()
group = maybe_atomize(group)
with true <- group_exists?(group) do
shell_info("The following settings will be removed from ConfigDB:\n")
dump_group(group)
if shell_prompt("Are you sure you want to continue?", "n") in ~w(Yn Y y) do
delete_group(group)
else
shell_error("No changes made.")
end
else
_ -> shell_error("No settings in ConfigDB for #{inspect(group)}. Aborting.")
end
end end
@spec migrate_to_db(Path.t() | nil) :: any() @spec migrate_to_db(Path.t() | nil) :: any()
def migrate_to_db(file_path \\ nil) do def migrate_to_db(file_path \\ nil) do
with true <- Pleroma.Config.get([:configurable_from_database]), with :ok <- Pleroma.Config.DeprecationWarnings.warn() do
:ok <- Pleroma.Config.DeprecationWarnings.warn() do
config_file = config_file =
if file_path do if file_path do
file_path file_path
@ -47,16 +221,15 @@ def migrate_to_db(file_path \\ nil) do
do_migrate_to_db(config_file) do_migrate_to_db(config_file)
else else
:error -> deprecation_error() _ ->
_ -> migration_error() shell_error("Migration is not allowed until all deprecation warnings have been resolved.")
end end
end end
defp do_migrate_to_db(config_file) do defp do_migrate_to_db(config_file) do
if File.exists?(config_file) do if File.exists?(config_file) do
shell_info("Migrating settings from file: #{Path.expand(config_file)}") shell_info("Migrating settings from file: #{Path.expand(config_file)}")
Ecto.Adapters.SQL.query!(Repo, "TRUNCATE config;") truncatedb()
Ecto.Adapters.SQL.query!(Repo, "ALTER SEQUENCE config_id_seq RESTART;")
custom_config = custom_config =
config_file config_file
@ -80,52 +253,38 @@ defp create(group, settings) do
shell_info("Settings for key #{key} migrated.") shell_info("Settings for key #{key} migrated.")
end) end)
shell_info("Settings for group :#{group} migrated.") shell_info("Settings for group #{inspect(group)} migrated.")
end end
defp migrate_from_db(opts) do defp migrate_from_db(opts) do
if Pleroma.Config.get([:configurable_from_database]) do env = opts[:env] || Pleroma.Config.get(:env)
env = opts[:env] || Pleroma.Config.get(:env)
config_path = config_path =
if Pleroma.Config.get(:release) do if Pleroma.Config.get(:release) do
:config_path :config_path
|> Pleroma.Config.get() |> Pleroma.Config.get()
|> Path.dirname() |> Path.dirname()
else else
"config" "config"
end end
|> Path.join("#{env}.exported_from_db.secret.exs") |> Path.join("#{env}.exported_from_db.secret.exs")
file = File.open!(config_path, [:write, :utf8]) file = File.open!(config_path, [:write, :utf8])
IO.write(file, config_header()) IO.write(file, config_header())
ConfigDB ConfigDB
|> Repo.all() |> Repo.all()
|> Enum.each(&write_and_delete(&1, file, opts[:delete])) |> Enum.each(&write_and_delete(&1, file, opts[:delete]))
:ok = File.close(file) :ok = File.close(file)
System.cmd("mix", ["format", config_path]) System.cmd("mix", ["format", config_path])
shell_info( shell_info(
"Database configuration settings have been exported to config/#{env}.exported_from_db.secret.exs" "Database configuration settings have been exported to config/#{env}.exported_from_db.secret.exs"
)
else
migration_error()
end
end
defp migration_error do
shell_error(
"Migration is not allowed in config. You can change this behavior by setting `config :pleroma, configurable_from_database: true`"
) )
end end
defp deprecation_error do
shell_error("Migration is not allowed until all deprecation warnings have been resolved.")
end
if Code.ensure_loaded?(Config.Reader) do if Code.ensure_loaded?(Config.Reader) do
defp config_header, do: "import Config\r\n\r\n" defp config_header, do: "import Config\r\n\r\n"
defp read_file(config_file), do: Config.Reader.read_imports!(config_file) defp read_file(config_file), do: Config.Reader.read_imports!(config_file)
@ -150,8 +309,80 @@ defp write(config, file) do
defp delete(config, true) do defp delete(config, true) do
{:ok, _} = Repo.delete(config) {:ok, _} = Repo.delete(config)
shell_info("#{config.key} deleted from DB.")
shell_info(
"config #{inspect(config.group)}, #{inspect(config.key)} was deleted from the ConfigDB."
)
end end
defp delete(_config, _), do: :ok defp delete(_config, _), do: :ok
defp dump(%ConfigDB{} = config) do
value = inspect(config.value, limit: :infinity)
shell_info("config #{inspect(config.group)}, #{inspect(config.key)}, #{value}\r\n\r\n")
end
defp dump(_), do: :noop
defp dump_group(group) when is_atom(group) do
group
|> ConfigDB.get_all_by_group()
|> Enum.each(&dump/1)
end
defp group_exists?(group) do
group
|> ConfigDB.get_all_by_group()
|> Enum.any?()
end
defp key_exists?(group, key) do
group
|> ConfigDB.get_by_group_and_key(key)
|> is_nil
|> Kernel.!()
end
defp maybe_atomize(arg) when is_atom(arg), do: arg
defp maybe_atomize(":" <> arg), do: maybe_atomize(arg)
defp maybe_atomize(arg) when is_binary(arg) do
if ConfigDB.module_name?(arg) do
String.to_existing_atom("Elixir." <> arg)
else
String.to_atom(arg)
end
end
defp check_configdb(callback) do
with true <- Pleroma.Config.get([:configurable_from_database]) do
callback.()
else
_ ->
shell_error(
"ConfigDB not enabled. Please check the value of :configurable_from_database in your configuration."
)
end
end
defp delete_key(group, key) do
check_configdb(fn ->
ConfigDB.delete(%{group: group, key: key})
end)
end
defp delete_group(group) do
check_configdb(fn ->
group
|> ConfigDB.get_all_by_group()
|> Enum.each(&ConfigDB.delete/1)
end)
end
defp truncatedb do
Ecto.Adapters.SQL.query!(Repo, "TRUNCATE config;")
Ecto.Adapters.SQL.query!(Repo, "ALTER SEQUENCE config_id_seq RESTART;")
end
end end

View file

@ -6,7 +6,7 @@ defmodule Pleroma.ConfigDB do
use Ecto.Schema use Ecto.Schema
import Ecto.Changeset import Ecto.Changeset
import Ecto.Query, only: [select: 3] import Ecto.Query, only: [select: 3, from: 2]
import Pleroma.Web.Gettext import Pleroma.Web.Gettext
alias __MODULE__ alias __MODULE__
@ -41,8 +41,18 @@ def get_all_as_keyword do
end) end)
end end
@spec get_all_by_group(atom() | String.t()) :: [t()]
def get_all_by_group(group) do
from(c in ConfigDB, where: c.group == ^group) |> Repo.all()
end
@spec get_by_group_and_key(atom() | String.t(), atom() | String.t()) :: t() | nil
def get_by_group_and_key(group, key) do
get_by_params(%{group: group, key: key})
end
@spec get_by_params(map()) :: ConfigDB.t() | nil @spec get_by_params(map()) :: ConfigDB.t() | nil
def get_by_params(params), do: Repo.get_by(ConfigDB, params) def get_by_params(%{group: _, key: _} = params), do: Repo.get_by(ConfigDB, params)
@spec changeset(ConfigDB.t(), map()) :: Changeset.t() @spec changeset(ConfigDB.t(), map()) :: Changeset.t()
def changeset(config, params \\ %{}) do def changeset(config, params \\ %{}) do

View file

@ -7,6 +7,7 @@ defmodule Mix.Tasks.Pleroma.ConfigTest do
import Pleroma.Factory import Pleroma.Factory
alias Mix.Tasks.Pleroma.Config, as: MixTask
alias Pleroma.ConfigDB alias Pleroma.ConfigDB
alias Pleroma.Repo alias Pleroma.Repo
@ -22,30 +23,41 @@ defmodule Mix.Tasks.Pleroma.ConfigTest do
:ok :ok
end end
setup_all do: clear_config(:configurable_from_database, true) defp config_records do
ConfigDB
|> Repo.all()
|> Enum.sort()
end
defp insert_config_record(group, key, value) do
insert(:config,
group: group,
key: key,
value: value
)
end
test "error if file with custom settings doesn't exist" do test "error if file with custom settings doesn't exist" do
Mix.Tasks.Pleroma.Config.migrate_to_db("config/not_existance_config_file.exs") MixTask.migrate_to_db("config/non_existent_config_file.exs")
assert_receive {:mix_shell, :info, msg =
[ "To migrate settings, you must define custom settings in config/non_existent_config_file.exs."
"To migrate settings, you must define custom settings in config/not_existance_config_file.exs."
]}, assert_receive {:mix_shell, :info, [^msg]}, 15
15
end end
describe "migrate_to_db/1" do describe "migrate_to_db/1" do
setup do setup do
initial = Application.get_env(:quack, :level) clear_config(:configurable_from_database, true)
on_exit(fn -> Application.put_env(:quack, :level, initial) end) clear_config([:quack, :level])
end end
@tag capture_log: true @tag capture_log: true
test "config migration refused when deprecated settings are found" do test "config migration refused when deprecated settings are found" do
clear_config([:media_proxy, :whitelist], ["domain_without_scheme.com"]) clear_config([:media_proxy, :whitelist], ["domain_without_scheme.com"])
assert Repo.all(ConfigDB) == [] assert config_records() == []
Mix.Tasks.Pleroma.Config.migrate_to_db("test/fixtures/config/temp.secret.exs") MixTask.migrate_to_db("test/fixtures/config/temp.secret.exs")
assert_received {:mix_shell, :error, [message]} assert_received {:mix_shell, :error, [message]}
@ -54,9 +66,9 @@ test "config migration refused when deprecated settings are found" do
end end
test "filtered settings are migrated to db" do test "filtered settings are migrated to db" do
assert Repo.all(ConfigDB) == [] assert config_records() == []
Mix.Tasks.Pleroma.Config.migrate_to_db("test/fixtures/config/temp.secret.exs") MixTask.migrate_to_db("test/fixtures/config/temp.secret.exs")
config1 = ConfigDB.get_by_params(%{group: ":pleroma", key: ":first_setting"}) config1 = ConfigDB.get_by_params(%{group: ":pleroma", key: ":first_setting"})
config2 = ConfigDB.get_by_params(%{group: ":pleroma", key: ":second_setting"}) config2 = ConfigDB.get_by_params(%{group: ":pleroma", key: ":second_setting"})
@ -71,18 +83,19 @@ test "filtered settings are migrated to db" do
end end
test "config table is truncated before migration" do test "config table is truncated before migration" do
insert(:config, key: :first_setting, value: [key: "value", key2: ["Activity"]]) insert_config_record(:pleroma, :first_setting, key: "value", key2: ["Activity"])
assert Repo.aggregate(ConfigDB, :count, :id) == 1 assert length(config_records()) == 1
Mix.Tasks.Pleroma.Config.migrate_to_db("test/fixtures/config/temp.secret.exs") MixTask.migrate_to_db("test/fixtures/config/temp.secret.exs")
config = ConfigDB.get_by_params(%{group: ":pleroma", key: ":first_setting"}) config = ConfigDB.get_by_params(%{group: ":pleroma", key: ":first_setting"})
assert config.value == [key: "value", key2: [Repo]] assert config.value == [key: "value", key2: [Repo]]
end end
end end
describe "with deletion temp file" do describe "with deletion of temp file" do
setup do setup do
clear_config(:configurable_from_database, true)
temp_file = "config/temp.exported_from_db.secret.exs" temp_file = "config/temp.exported_from_db.secret.exs"
on_exit(fn -> on_exit(fn ->
@ -93,13 +106,13 @@ test "config table is truncated before migration" do
end end
test "settings are migrated to file and deleted from db", %{temp_file: temp_file} do test "settings are migrated to file and deleted from db", %{temp_file: temp_file} do
insert(:config, key: :setting_first, value: [key: "value", key2: ["Activity"]]) insert_config_record(:pleroma, :setting_first, key: "value", key2: ["Activity"])
insert(:config, key: :setting_second, value: [key: "value2", key2: [Repo]]) insert_config_record(:pleroma, :setting_second, key: "value2", key2: [Repo])
insert(:config, group: :quack, key: :level, value: :info) insert_config_record(:quack, :level, :info)
Mix.Tasks.Pleroma.Config.run(["migrate_from_db", "--env", "temp", "-d"]) MixTask.run(["migrate_from_db", "--env", "temp", "-d"])
assert Repo.all(ConfigDB) == [] assert config_records() == []
file = File.read!(temp_file) file = File.read!(temp_file)
assert file =~ "config :pleroma, :setting_first," assert file =~ "config :pleroma, :setting_first,"
@ -169,9 +182,9 @@ test "load a settings with large values and pass to file", %{temp_file: temp_fil
] ]
) )
Mix.Tasks.Pleroma.Config.run(["migrate_from_db", "--env", "temp", "-d"]) MixTask.run(["migrate_from_db", "--env", "temp", "-d"])
assert Repo.all(ConfigDB) == [] assert config_records() == []
assert File.exists?(temp_file) assert File.exists?(temp_file)
{:ok, file} = File.read(temp_file) {:ok, file} = File.read(temp_file)
@ -186,4 +199,114 @@ test "load a settings with large values and pass to file", %{temp_file: temp_fil
"#{header}\n\nconfig :pleroma, :instance,\n name: \"Pleroma\",\n email: \"example@example.com\",\n notify_email: \"noreply@example.com\",\n description: \"A Pleroma instance, an alternative fediverse server\",\n limit: 5000,\n chat_limit: 5000,\n remote_limit: 100_000,\n upload_limit: 16_000_000,\n avatar_upload_limit: 2_000_000,\n background_upload_limit: 4_000_000,\n banner_upload_limit: 4_000_000,\n poll_limits: %{\n max_expiration: 31_536_000,\n max_option_chars: 200,\n max_options: 20,\n min_expiration: 0\n },\n registrations_open: true,\n federating: true,\n federation_incoming_replies_max_depth: 100,\n federation_reachability_timeout_days: 7,\n federation_publisher_modules: [Pleroma.Web.ActivityPub.Publisher],\n allow_relay: true,\n public: true,\n quarantined_instances: [],\n managed_config: true,\n static_dir: \"instance/static/\",\n allowed_post_formats: [\"text/plain\", \"text/html\", \"text/markdown\", \"text/bbcode\"],\n autofollowed_nicknames: [],\n max_pinned_statuses: 1,\n attachment_links: false,\n max_report_comment_size: 1000,\n safe_dm_mentions: false,\n healthcheck: false,\n remote_post_retention_days: 90,\n skip_thread_containment: true,\n limit_to_local_content: :unauthenticated,\n user_bio_length: 5000,\n user_name_length: 100,\n max_account_fields: 10,\n max_remote_account_fields: 20,\n account_field_name_length: 512,\n account_field_value_length: 2048,\n external_user_synchronization: true,\n extended_nickname_format: true,\n multi_factor_authentication: [\n totp: [digits: 6, period: 30],\n backup_codes: [number: 2, length: 6]\n ]\n" "#{header}\n\nconfig :pleroma, :instance,\n name: \"Pleroma\",\n email: \"example@example.com\",\n notify_email: \"noreply@example.com\",\n description: \"A Pleroma instance, an alternative fediverse server\",\n limit: 5000,\n chat_limit: 5000,\n remote_limit: 100_000,\n upload_limit: 16_000_000,\n avatar_upload_limit: 2_000_000,\n background_upload_limit: 4_000_000,\n banner_upload_limit: 4_000_000,\n poll_limits: %{\n max_expiration: 31_536_000,\n max_option_chars: 200,\n max_options: 20,\n min_expiration: 0\n },\n registrations_open: true,\n federating: true,\n federation_incoming_replies_max_depth: 100,\n federation_reachability_timeout_days: 7,\n federation_publisher_modules: [Pleroma.Web.ActivityPub.Publisher],\n allow_relay: true,\n public: true,\n quarantined_instances: [],\n managed_config: true,\n static_dir: \"instance/static/\",\n allowed_post_formats: [\"text/plain\", \"text/html\", \"text/markdown\", \"text/bbcode\"],\n autofollowed_nicknames: [],\n max_pinned_statuses: 1,\n attachment_links: false,\n max_report_comment_size: 1000,\n safe_dm_mentions: false,\n healthcheck: false,\n remote_post_retention_days: 90,\n skip_thread_containment: true,\n limit_to_local_content: :unauthenticated,\n user_bio_length: 5000,\n user_name_length: 100,\n max_account_fields: 10,\n max_remote_account_fields: 20,\n account_field_name_length: 512,\n account_field_value_length: 2048,\n external_user_synchronization: true,\n extended_nickname_format: true,\n multi_factor_authentication: [\n totp: [digits: 6, period: 30],\n backup_codes: [number: 2, length: 6]\n ]\n"
end end
end end
describe "operations on database config" do
setup do: clear_config(:configurable_from_database, true)
test "dumping a specific group" do
insert_config_record(:pleroma, :instance, name: "Pleroma Test")
insert_config_record(:web_push_encryption, :vapid_details,
subject: "mailto:administrator@example.com",
public_key:
"BOsPL-_KjNnjj_RMvLeR3dTOrcndi4TbMR0cu56gLGfGaT5m1gXxSfRHOcC4Dd78ycQL1gdhtx13qgKHmTM5xAI",
private_key: "Ism6FNdS31nLCA94EfVbJbDdJXCxAZ8cZiB1JQPN_t4"
)
MixTask.run(["dump", "pleroma"])
assert_receive {:mix_shell, :info,
["config :pleroma, :instance, [name: \"Pleroma Test\"]\r\n\r\n"]}
refute_receive {
:mix_shell,
:info,
[
"config :web_push_encryption, :vapid_details, [subject: \"mailto:administrator@example.com\", public_key: \"BOsPL-_KjNnjj_RMvLeR3dTOrcndi4TbMR0cu56gLGfGaT5m1gXxSfRHOcC4Dd78ycQL1gdhtx13qgKHmTM5xAI\", private_key: \"Ism6FNdS31nLCA94EfVbJbDdJXCxAZ8cZiB1JQPN_t4\"]\r\n\r\n"
]
}
# Ensure operations work when using atom syntax
MixTask.run(["dump", ":pleroma"])
assert_receive {:mix_shell, :info,
["config :pleroma, :instance, [name: \"Pleroma Test\"]\r\n\r\n"]}
end
test "dumping a specific key in a group" do
insert_config_record(:pleroma, :instance, name: "Pleroma Test")
insert_config_record(:pleroma, Pleroma.Captcha, enabled: false)
MixTask.run(["dump", "pleroma", "Pleroma.Captcha"])
refute_receive {:mix_shell, :info,
["config :pleroma, :instance, [name: \"Pleroma Test\"]\r\n\r\n"]}
assert_receive {:mix_shell, :info,
["config :pleroma, Pleroma.Captcha, [enabled: false]\r\n\r\n"]}
end
test "dumps all configuration successfully" do
insert_config_record(:pleroma, :instance, name: "Pleroma Test")
insert_config_record(:pleroma, Pleroma.Captcha, enabled: false)
MixTask.run(["dump"])
assert_receive {:mix_shell, :info,
["config :pleroma, :instance, [name: \"Pleroma Test\"]\r\n\r\n"]}
assert_receive {:mix_shell, :info,
["config :pleroma, Pleroma.Captcha, [enabled: false]\r\n\r\n"]}
end
end
describe "when configdb disabled" do
test "refuses to dump" do
clear_config(:configurable_from_database, false)
insert_config_record(:pleroma, :instance, name: "Pleroma Test")
MixTask.run(["dump"])
msg =
"ConfigDB not enabled. Please check the value of :configurable_from_database in your configuration."
assert_receive {:mix_shell, :error, [^msg]}
end
end
describe "destructive operations" do
setup do: clear_config(:configurable_from_database, true)
setup do
insert_config_record(:pleroma, :instance, name: "Pleroma Test")
insert_config_record(:pleroma, Pleroma.Captcha, enabled: false)
insert_config_record(:pleroma2, :key2, z: 1)
assert length(config_records()) == 3
:ok
end
test "deletes group of settings" do
MixTask.run(["delete", "--force", "pleroma"])
assert [%ConfigDB{group: :pleroma2, key: :key2}] = config_records()
end
test "deletes specified key" do
MixTask.run(["delete", "--force", "pleroma", "Pleroma.Captcha"])
assert [
%ConfigDB{group: :pleroma, key: :instance},
%ConfigDB{group: :pleroma2, key: :key2}
] = config_records()
end
test "resets entire config" do
MixTask.run(["reset", "--force"])
assert config_records() == []
end
end
end end

View file

@ -162,7 +162,9 @@ test "with valid `admin_token` query parameter, skips OAuth scopes check" do
end end
end end
test "POST /api/pleroma/admin/config error", %{conn: conn} do test "POST /api/pleroma/admin/config with configdb disabled", %{conn: conn} do
clear_config(:configurable_from_database, false)
conn = conn =
conn conn
|> put_req_header("content-type", "application/json") |> put_req_header("content-type", "application/json")

View file

@ -60,7 +60,7 @@ test "GET /relay", %{conn: conn} do
conn = get(conn, "/api/pleroma/admin/relay") conn = get(conn, "/api/pleroma/admin/relay")
assert json_response_and_validate_schema(conn, 200)["relays"] == [ assert json_response_and_validate_schema(conn, 200)["relays"] |> Enum.sort() == [
%{ %{
"actor" => "http://mastodon.example.org/users/admin", "actor" => "http://mastodon.example.org/users/admin",
"followed_back" => true "followed_back" => true