config editing through database

This commit is contained in:
Alex S 2019-09-29 11:17:38 +03:00 committed by Alexander Strizhakov
parent e118d639a0
commit 2753285b77
No known key found for this signature in database
GPG key ID: 022896A53AEF1381
20 changed files with 1133 additions and 472 deletions

File diff suppressed because it is too large Load diff

View file

@ -669,7 +669,7 @@ Note: Available `:permission_group` is currently moderator and admin. 404 is ret
### Run mix task pleroma.config migrate_to_db ### Run mix task pleroma.config migrate_to_db
Copy settings on key `:pleroma` to DB. Copies `pleroma` environment settings to the database.
- Params: none - Params: none
- Response: - Response:
@ -682,7 +682,7 @@ Copy settings on key `:pleroma` to DB.
### Run mix task pleroma.config migrate_from_db ### Run mix task pleroma.config migrate_from_db
Copy all settings from DB to `config/prod.exported_from_db.secret.exs` with deletion from DB. Copies all settings from database to `config/{env}.exported_from_db.secret.exs` with deletion from the table. Where `{env}` is the environment in which `pleroma` is running.
- Params: none - Params: none
- Response: - Response:
@ -693,9 +693,9 @@ Copy all settings from DB to `config/prod.exported_from_db.secret.exs` with dele
## `GET /api/pleroma/admin/config` ## `GET /api/pleroma/admin/config`
### List config settings ### Get saved config settings
List config settings only works with `:pleroma => :instance => :dynamic_configuration` setting to `true`. **Only works when `:dynamic_configuration` is `true`.**
- Params: none - Params: none
- Response: - Response:
@ -704,9 +704,9 @@ List config settings only works with `:pleroma => :instance => :dynamic_configur
{ {
configs: [ configs: [
{ {
"group": string, "group": ":pleroma",
"key": string or string with leading `:` for atoms, "key": "Pleroma.Upload",
"value": string or {} or [] or {"tuple": []} "value": []
} }
] ]
} }
@ -716,44 +716,61 @@ List config settings only works with `:pleroma => :instance => :dynamic_configur
### Update config settings ### Update config settings
Updating config settings only works with `:pleroma => :instance => :dynamic_configuration` setting to `true`. **Only works when `:dynamic_configuration` is `true`.**
Module name can be passed as string, which starts with `Pleroma`, e.g. `"Pleroma.Upload"`.
Atom keys and values can be passed with `:` in the beginning, e.g. `":upload"`.
Tuples can be passed as `{"tuple": ["first_val", Pleroma.Module, []]}`.
`{"tuple": ["some_string", "Pleroma.Some.Module", []]}` will be converted to `{"some_string", Pleroma.Some.Module, []}`.
Keywords can be passed as lists with 2 child tuples, e.g.
`[{"tuple": ["first_val", Pleroma.Module]}, {"tuple": ["second_val", true]}]`.
If value contains list of settings `[subkey: val1, subkey2: val2, subkey3: val3]`, it's possible to remove only subkeys instead of all settings passing `subkeys` parameter. E.g.: Some modifications are necessary to save the config settings correctly:
{"group": "pleroma", "key": "some_key", "delete": "true", "subkeys": [":subkey", ":subkey3"]}.
Compile time settings (need instance reboot): - strings which start with `Pleroma.`, `Phoenix.`, `Tesla.` or strings like `Oban`, `Ueberauth` will be converted to modules;
- all settings by this keys: ```
"Pleroma.Upload" -> Pleroma.Upload
"Oban" -> Oban
```
- strings starting with `:` will be converted to atoms;
```
":pleroma" -> :pleroma
```
- objects with `tuple` key and array value will be converted to atoms;
```
{"tuple": ["string", "Pleroma.Upload", []]} -> {"string", Pleroma.Upload, []}
```
- arrays with *tuple objects* and 2 childs in array will be converted to keywords;
```
[{"tuple": [":key1", "value"]}, {"tuple": [":key2", "value"]}] -> [key1: "value", key2: "value"]
```
Most of the settings will be applied in `runtime`, this means that you don't need to restart the instance. But some settings are applied in `compile time` and require a reboot of the instance, such as:
- all settings inside these keys:
- `:hackney_pools` - `:hackney_pools`
- `:chat` - `:chat`
- `Pleroma.Web.Endpoint` - `Pleroma.Web.Endpoint`
- `Pleroma.Repo` - partially settings inside these keys:
- part settings: - `:seconds_valid` in `Pleroma.Captcha`
- `Pleroma.Captcha` -> `:seconds_valid` - `:proxy_remote` in `Pleroma.Upload`
- `Pleroma.Upload` -> `:proxy_remote` - `:upload_limit` in `:instance`
- `:instance` -> `:upload_limit`
- Params: - Params:
- `configs` => [ - `configs` - array of config objects
- `group` (string) - config object params:
- `key` (string or string with leading `:` for atoms) - `group` - string (**required**)
- `value` (string, [], {} or {"tuple": []}) - `key` - string (**required**)
- `delete` = true (optional, if parameter must be deleted) - `value` - string, [], {} or {"tuple": []} (**required**)
- `subkeys` [(string with leading `:` for atoms)] (optional, works only if `delete=true` parameter is passed, otherwise will be ignored) - `delete` - true (*optional*, if setting must be deleted)
] - `subkeys` - array of strings (*optional*, only works when `delete=true` parameter is passed, otherwise will be ignored)
- Request (example): *When a value have several nested settings, you can delete only some nested settings by passing a parameter `subkeys`, without deleting all settings by key.*
```
[subkey: val1, subkey2: val2, subkey3: val3] \\ initial value
{"group": ":pleroma", "key": "some_key", "delete": true, "subkeys": [":subkey", ":subkey3"]} \\ passing json for deletion
[subkey2: val2] \\ value after deletion
```
- Request:
```json ```json
{ {
configs: [ configs: [
{ {
"group": "pleroma", "group": ":pleroma",
"key": "Pleroma.Upload", "key": "Pleroma.Upload",
"value": [ "value": [
{"tuple": [":uploader", "Pleroma.Uploaders.Local"]}, {"tuple": [":uploader", "Pleroma.Uploaders.Local"]},
@ -784,14 +801,47 @@ Compile time settings (need instance reboot):
{ {
configs: [ configs: [
{ {
"group": string, "group": ":pleroma",
"key": string or string with leading `:` for atoms, "key": "Pleroma.Upload",
"value": string or {} or [] or {"tuple": []} "value": [...]
} }
] ]
} }
``` ```
## ` GET /api/pleroma/admin/config/descriptions`
### Get JSON with config descriptions.
Loads json generated from `config/descriptions.exs`.
- Params: none
- Response:
```json
[{
"group": ":pleroma", // string
"key": "ModuleName", // string
"type": "group", // string or list with possible values,
"description": "Upload general settings", // string
"children": [
{
"key": ":uploader", // string or module name `Pleroma.Upload`
"type": "module",
"description": "Module which will be used for uploads",
"suggestions": ["module1", "module2"]
},
{
"key": ":filters",
"type": ["list", "module"],
"description": "List of filter modules for uploads",
"suggestions": [
"module1", "module2", "module3"
]
}
]
}]
```
## `GET /api/pleroma/admin/moderation_log` ## `GET /api/pleroma/admin/moderation_log`
### Get moderation log ### Get moderation log

59
docs/admin/config.md Normal file
View file

@ -0,0 +1,59 @@
# Configuring instance
You can configure your instance from admin interface. You need account with admin rights and little change in config file, which will allow settings dynamic configuration from database.
```elixir
config :pleroma, :instance,
dynamic_configuration: true
```
## How it works
Settings are stored in database and are applied in `runtime` after each change. Most of the settings take effect immediately, except some, which need instance reboot. These settings are needed in `compile time`, that's why settings are duplicated to the file.
File with duplicated settings is located in `config/{env}.exported_from_db.exs`. For prod env it will be `config/prod.exported_from_db.exs`.
## How to set it up
You need to migrate your existing settings to the database. You can do this with mix task (all config files will remain untouched):
```bash
mix pleroma.config migrate_to_db
```
Now you can change settings in admin interface. After each save, settings are duplicated to the `config/{env}.exported_from_db.exs` file.
<span style="color:red">**ATTENTION**</span>
**<span style="color:red">Be careful while changing the settings. Every inaccurate configuration change can break the federation or the instance load.</span>**
*Compile time settings, which require instance reboot and can break instance loading:*
- all settings inside these keys:
- `:hackney_pools`
- `:chat`
- `Pleroma.Web.Endpoint`
- partially settings inside these keys:
- `:seconds_valid` in `Pleroma.Captcha`
- `:proxy_remote` in `Pleroma.Upload`
- `:upload_limit` in `:instance`
## How to remove it
1. Truncate or delete all values from `config` table
```sql
TRUNCATE TABLE config;
```
2. Delete `config/{env}.exported_from_db.exs`.
For `prod` env:
```bash
cd /opt/pleroma
cp config/prod.exported_from_db.exs config/exported_from_db.back
rm -rf config/prod.exported_from_db.exs
```
*If you don't want to backup settings, you can skip step with `cp` command.*
3. Set dynamic configuration to `false`.
```elixir
config :pleroma, :instance,
dynamic_configuration: false
```
4. Restart pleroma instance
```bash
sudo service pleroma restart
```

View file

@ -9,27 +9,29 @@ defmodule Mix.Tasks.Pleroma.Config do
alias Pleroma.Web.AdminAPI.Config alias Pleroma.Web.AdminAPI.Config
@shortdoc "Manages the location of the config" @shortdoc "Manages the location of the config"
@moduledoc File.read!("docs/administration/CLI_tasks/config.md") @moduledoc File.read!("docs/administration/CLI_tasks/config.md")
@groups [
:pleroma,
:logger,
:quack,
:mime,
:tesla,
:phoenix,
:cors_plug,
:auto_linker,
:esshd,
:ueberauth,
:prometheus,
:http_signatures,
:web_push_encryption,
:joken
]
def run(["migrate_to_db"]) do def run(["migrate_to_db"]) do
start_pleroma() start_pleroma()
if Pleroma.Config.get([:instance, :dynamic_configuration]) do if Pleroma.Config.get([:instance, :dynamic_configuration]) do
Application.get_all_env(:pleroma) Enum.each(@groups, &load_and_create(&1))
|> Enum.reject(fn {k, _v} -> k in [Pleroma.Repo, :env] end)
|> Enum.each(fn {k, v} ->
key = to_string(k) |> String.replace("Elixir.", "")
key =
if String.starts_with?(key, "Pleroma.") do
key
else
":" <> key
end
{:ok, _} = Config.update_or_create(%{group: "pleroma", key: key, value: v})
Mix.shell().info("#{key} is migrated.")
end)
Mix.shell().info("Settings migrated.")
else else
Mix.shell().info( Mix.shell().info(
"Migration is not allowed by config. You can change this behavior in instance settings." "Migration is not allowed by config. You can change this behavior in instance settings."
@ -37,38 +39,63 @@ def run(["migrate_to_db"]) do
end end
end end
def run(["migrate_from_db", env, delete?]) do def run(["migrate_from_db" | options]) do
start_pleroma() start_pleroma()
delete? = if delete? == "true", do: true, else: false {opts, _} =
OptionParser.parse!(options,
strict: [env: :string, delete_from_db: :boolean],
aliases: [d: :delete_from_db]
)
if Pleroma.Config.get([:instance, :dynamic_configuration]) do with {:active?, true} <- {:active?, Pleroma.Config.get([:instance, :dynamic_configuration])},
config_path = "config/#{env}.exported_from_db.secret.exs" env_path when is_binary(env_path) <- opts[:env],
config_path <- "config/#{env_path}.exported_from_db.secret.exs",
{:ok, file} = File.open(config_path, [:write, :utf8]) {:ok, file} <- File.open(config_path, [:write, :utf8]) do
IO.write(file, "use Mix.Config\r\n") IO.write(file, "use Mix.Config\r\n")
Repo.all(Config) Config
|> Enum.each(fn config -> |> Repo.all()
IO.write( |> Enum.each(&write_to_file_with_deletion(&1, file, opts[:delete_from_db]))
file,
"config :#{config.group}, #{config.key}, #{
inspect(Config.from_binary(config.value), limit: :infinity)
}\r\n\r\n"
)
if delete? do
{:ok, _} = Repo.delete(config)
Mix.shell().info("#{config.key} deleted from DB.")
end
end)
File.close(file) File.close(file)
System.cmd("mix", ["format", config_path]) System.cmd("mix", ["format", config_path])
else else
Mix.shell().info( {:active?, false} ->
"Migration is not allowed by config. You can change this behavior in instance settings." Mix.shell().info(
) "migration is not allowed by config. You can change this behavior in instance settings."
)
error ->
Mix.shell().info("error occuried while opening file. #{inspect(error)}")
end
end
defp load_and_create(group) do
group
|> Application.get_all_env()
|> Enum.reject(fn {k, _v} -> k in [Pleroma.Repo, :env] end)
|> Enum.each(fn {key, value} ->
key_str = inspect(key)
{:ok, _} = Config.update_or_create(%{group: ":#{group}", key: key_str, value: value})
Mix.shell().info("settings for key #{key_str} migrated.")
end)
Mix.shell().info("settings for group :#{group} migrated.")
end
defp write_to_file_with_deletion(config, file, with_deletion) do
IO.write(
file,
"config #{config.group}, #{config.key}, #{
inspect(Config.from_binary(config.value), limit: :infinity)
}\r\n\r\n"
)
if with_deletion do
{:ok, _} = Repo.delete(config)
Mix.shell().info("#{config.key} deleted from DB.")
end end
end end
end end

View file

@ -4,56 +4,59 @@
defmodule Pleroma.Config.TransferTask do defmodule Pleroma.Config.TransferTask do
use Task use Task
require Logger
alias Pleroma.Repo
alias Pleroma.Web.AdminAPI.Config alias Pleroma.Web.AdminAPI.Config
def start_link(_) do def start_link(_) do
load_and_update_env() load_and_update_env()
if Pleroma.Config.get(:env) == :test, do: Ecto.Adapters.SQL.Sandbox.checkin(Pleroma.Repo) if Pleroma.Config.get(:env) == :test, do: Ecto.Adapters.SQL.Sandbox.checkin(Repo)
:ignore :ignore
end end
def load_and_update_env do def load_and_update_env do
if Pleroma.Config.get([:instance, :dynamic_configuration]) and with true <- Pleroma.Config.get([:instance, :dynamic_configuration]),
Ecto.Adapters.SQL.table_exists?(Pleroma.Repo, "config") do true <- Ecto.Adapters.SQL.table_exists?(Repo, "config"),
for_restart = started_applications <- Application.started_applications() do
Pleroma.Repo.all(Config)
|> Enum.map(&update_env(&1))
# We need to restart applications for loaded settings take effect # We need to restart applications for loaded settings take effect
for_restart Config
|> Enum.reject(&(&1 in [:pleroma, :ok])) |> Repo.all()
|> Enum.each(fn app -> |> Enum.map(&update_env(&1))
Application.stop(app) |> Enum.uniq()
:ok = Application.start(app) # TODO: some problem with prometheus after restart!
end) |> Enum.reject(&(&1 in [:pleroma, nil, :prometheus]))
|> Enum.each(&restart(started_applications, &1))
end end
end end
defp update_env(setting) do defp update_env(setting) do
try do try do
key = key = Config.from_string(setting.key)
if String.starts_with?(setting.key, "Pleroma.") do group = Config.from_string(setting.group)
"Elixir." <> setting.key value = Config.from_binary(setting.value)
else
String.trim_leading(setting.key, ":")
end
group = String.to_existing_atom(setting.group) :ok = Application.put_env(group, key, value)
Application.put_env(
group,
String.to_existing_atom(key),
Config.from_binary(setting.value)
)
group group
rescue rescue
e -> e ->
require Logger
Logger.warn( Logger.warn(
"updating env causes error, key: #{inspect(setting.key)}, error: #{inspect(e)}" "updating env causes error, key: #{inspect(setting.key)}, error: #{inspect(e)}"
) )
nil
end
end
defp restart(started_applications, app) do
with {^app, _, _} <- List.keyfind(started_applications, app, 0),
:ok <- Application.stop(app) do
:ok = Application.start(app)
else
nil -> Logger.warn("#{app} is not started.")
error -> Logger.warn(inspect(error))
end end
end end
end end

View file

@ -6,68 +6,108 @@ def process(implementation, descriptions) do
implementation.process(descriptions) implementation.process(descriptions)
end end
@spec uploaders_list() :: [module()] @spec list_modules_in_dir(String.t(), String.t()) :: [module()]
def uploaders_list do def list_modules_in_dir(dir, start) do
{:ok, modules} = :application.get_key(:pleroma, :modules) with {:ok, files} <- File.ls(dir) do
files
Enum.filter(modules, fn module -> |> Enum.filter(&String.ends_with?(&1, ".ex"))
name_as_list = Module.split(module) |> Enum.map(fn filename ->
module = filename |> String.trim_trailing(".ex") |> Macro.camelize()
List.starts_with?(name_as_list, ["Pleroma", "Uploaders"]) and String.to_existing_atom(start <> module)
List.last(name_as_list) != "Uploader" end)
end) end
end end
@spec filters_list() :: [module()] @doc """
def filters_list do Converts:
{:ok, modules} = :application.get_key(:pleroma, :modules) - atoms to strings with leading `:`
- module names to strings, without leading `Elixir.`
Enum.filter(modules, fn module -> - add humanized labels to `keys` if label is not defined, e.g. `:instance` -> `Instance`
name_as_list = Module.split(module) """
@spec convert_to_strings([map()]) :: [map()]
List.starts_with?(name_as_list, ["Pleroma", "Upload", "Filter"]) def convert_to_strings(descriptions) do
end) Enum.map(descriptions, &format_entity(&1))
end end
@spec mrf_list() :: [module()] defp format_entity(entity) do
def mrf_list do entity
{:ok, modules} = :application.get_key(:pleroma, :modules) |> format_key()
|> Map.put(:group, atom_to_string(entity[:group]))
Enum.filter(modules, fn module -> |> format_children()
name_as_list = Module.split(module)
List.starts_with?(name_as_list, ["Pleroma", "Web", "ActivityPub", "MRF"]) and
length(name_as_list) > 4
end)
end end
@spec richmedia_parsers() :: [module()] defp format_key(%{key: key} = entity) do
def richmedia_parsers do entity
{:ok, modules} = :application.get_key(:pleroma, :modules) |> Map.put(:key, atom_to_string(key))
|> Map.put(:label, entity[:label] || humanize(key))
Enum.filter(modules, fn module ->
name_as_list = Module.split(module)
List.starts_with?(name_as_list, ["Pleroma", "Web", "RichMedia", "Parsers"]) and
length(name_as_list) == 5
end)
end end
defp format_key(%{group: group} = entity) do
Map.put(entity, :label, entity[:label] || humanize(group))
end
defp format_key(entity), do: entity
defp format_children(%{children: children} = entity) do
Map.put(entity, :children, Enum.map(children, &format_child(&1)))
end
defp format_children(entity), do: entity
defp format_child(%{suggestions: suggestions} = entity) do
entity
|> Map.put(:suggestions, format_suggestions(suggestions))
|> format_key()
|> format_children()
end
defp format_child(entity) do
entity
|> format_key()
|> format_children()
end
defp atom_to_string(entity) when is_binary(entity), do: entity
defp atom_to_string(entity) when is_atom(entity), do: inspect(entity)
defp humanize(entity) do
string = inspect(entity)
if String.starts_with?(string, ":"),
do: Phoenix.Naming.humanize(entity),
else: string
end
defp format_suggestions([]), do: []
defp format_suggestions([suggestion | tail]) do
[format_suggestion(suggestion) | format_suggestions(tail)]
end
defp format_suggestion(entity) when is_atom(entity) do
atom_to_string(entity)
end
defp format_suggestion([head | tail] = entity) when is_list(entity) do
[format_suggestion(head) | format_suggestions(tail)]
end
defp format_suggestion(entity) when is_tuple(entity) do
format_suggestions(Tuple.to_list(entity)) |> List.to_tuple()
end
defp format_suggestion(entity), do: entity
end end
defimpl Jason.Encoder, for: Tuple do defimpl Jason.Encoder, for: Tuple do
def encode(tuple, opts) do def encode(tuple, opts), do: Jason.Encode.list(Tuple.to_list(tuple), opts)
Jason.Encode.list(Tuple.to_list(tuple), opts)
end
end end
defimpl Jason.Encoder, for: [Regex, Function] do defimpl Jason.Encoder, for: [Regex, Function] do
def encode(term, opts) do def encode(term, opts), do: Jason.Encode.string(inspect(term), opts)
Jason.Encode.string(inspect(term), opts)
end
end end
defimpl String.Chars, for: Regex do defimpl String.Chars, for: Regex do
def to_string(term) do def to_string(term), do: inspect(term)
inspect(term)
end
end end

View file

@ -3,18 +3,22 @@ defmodule Pleroma.Docs.JSON do
@spec process(keyword()) :: {:ok, String.t()} @spec process(keyword()) :: {:ok, String.t()}
def process(descriptions) do def process(descriptions) do
config_path = "docs/generate_config.json" with path <- "docs/generated_config.json",
{:ok, file} <- File.open(path, [:write, :utf8]),
with {:ok, file} <- File.open(config_path, [:write, :utf8]), formatted_descriptions <-
json <- generate_json(descriptions), Pleroma.Docs.Generator.convert_to_strings(descriptions),
json <- Jason.encode!(formatted_descriptions),
:ok <- IO.write(file, json), :ok <- IO.write(file, json),
:ok <- File.close(file) do :ok <- File.close(file) do
{:ok, config_path} {:ok, path}
end end
end end
@spec generate_json([keyword()]) :: String.t() def compile do
def generate_json(descriptions) do with {config, _paths} <- Mix.Config.eval!("config/description.exs") do
Jason.encode!(descriptions) config[:pleroma][:config_description]
|> Pleroma.Docs.Generator.convert_to_strings()
|> Jason.encode!()
end
end end
end end

View file

@ -19,8 +19,7 @@ def filter(%{"type" => "Undo", "object" => child_message} = message) do
def filter(%{"type" => message_type} = message) do def filter(%{"type" => message_type} = message) do
with accepted_vocabulary <- Pleroma.Config.get([:mrf_vocabulary, :accept]), with accepted_vocabulary <- Pleroma.Config.get([:mrf_vocabulary, :accept]),
rejected_vocabulary <- Pleroma.Config.get([:mrf_vocabulary, :reject]), rejected_vocabulary <- Pleroma.Config.get([:mrf_vocabulary, :reject]),
true <- true <- accepted_vocabulary == [] || Enum.member?(accepted_vocabulary, message_type),
length(accepted_vocabulary) == 0 || Enum.member?(accepted_vocabulary, message_type),
false <- false <-
length(rejected_vocabulary) > 0 && Enum.member?(rejected_vocabulary, message_type), length(rejected_vocabulary) > 0 && Enum.member?(rejected_vocabulary, message_type),
{:ok, _} <- filter(message["object"]) do {:ok, _} <- filter(message["object"]) do

View file

@ -4,6 +4,9 @@
defmodule Pleroma.Web.AdminAPI.AdminAPIController do defmodule Pleroma.Web.AdminAPI.AdminAPIController do
use Pleroma.Web, :controller use Pleroma.Web, :controller
import Pleroma.Web.ControllerHelper, only: [json_response: 3]
alias Pleroma.Activity alias Pleroma.Activity
alias Pleroma.ModerationLog alias Pleroma.ModerationLog
alias Pleroma.Plugs.OAuthScopesPlug alias Pleroma.Plugs.OAuthScopesPlug
@ -25,10 +28,11 @@ defmodule Pleroma.Web.AdminAPI.AdminAPIController do
alias Pleroma.Web.MastodonAPI.StatusView alias Pleroma.Web.MastodonAPI.StatusView
alias Pleroma.Web.Router alias Pleroma.Web.Router
import Pleroma.Web.ControllerHelper, only: [json_response: 3]
require Logger require Logger
@descriptions_json Pleroma.Docs.JSON.compile()
@users_page_size 50
plug( plug(
OAuthScopesPlug, OAuthScopesPlug,
%{scopes: ["read:accounts"], admin: true} %{scopes: ["read:accounts"], admin: true}
@ -93,8 +97,6 @@ defmodule Pleroma.Web.AdminAPI.AdminAPIController do
when action in [:relay_follow, :relay_unfollow, :config_update] when action in [:relay_follow, :relay_unfollow, :config_update]
) )
@users_page_size 50
action_fallback(:errors) action_fallback(:errors)
def user_delete(%{assigns: %{user: admin}} = conn, %{"nickname" => nickname}) do def user_delete(%{assigns: %{user: admin}} = conn, %{"nickname" => nickname}) do
@ -782,10 +784,22 @@ def migrate_to_db(conn, _params) do
end end
def migrate_from_db(conn, _params) do def migrate_from_db(conn, _params) do
Mix.Tasks.Pleroma.Config.run(["migrate_from_db", Pleroma.Config.get(:env), "true"]) Mix.Tasks.Pleroma.Config.run([
"migrate_from_db",
"--env",
to_string(Pleroma.Config.get(:env)),
"-d"
])
json(conn, %{}) json(conn, %{})
end end
def config_descriptions(conn, _params) do
conn
|> Plug.Conn.put_resp_content_type("application/json")
|> Plug.Conn.send_resp(200, @descriptions_json)
end
def config_show(conn, _params) do def config_show(conn, _params) do
configs = Pleroma.Repo.all(Config) configs = Pleroma.Repo.all(Config)
@ -800,17 +814,27 @@ def config_update(conn, %{"configs" => configs}) do
updated = updated =
Enum.map(configs, fn Enum.map(configs, fn
%{"group" => group, "key" => key, "delete" => "true"} = params -> %{"group" => group, "key" => key, "delete" => "true"} = params ->
{:ok, config} = Config.delete(%{group: group, key: key, subkeys: params["subkeys"]}) with {:ok, config} <-
config Config.delete(%{group: group, key: key, subkeys: params["subkeys"]}) do
config
end
%{"group" => group, "key" => key, "value" => value} -> %{"group" => group, "key" => key, "value" => value} ->
{:ok, config} = Config.update_or_create(%{group: group, key: key, value: value}) with {:ok, config} <-
config Config.update_or_create(%{group: group, key: key, value: value}) do
config
end
end) end)
|> Enum.reject(&is_nil(&1)) |> Enum.reject(&is_nil(&1))
Pleroma.Config.TransferTask.load_and_update_env() Pleroma.Config.TransferTask.load_and_update_env()
Mix.Tasks.Pleroma.Config.run(["migrate_from_db", Pleroma.Config.get(:env), "false"])
Mix.Tasks.Pleroma.Config.run([
"migrate_from_db",
"--env",
to_string(Pleroma.Config.get(:env))
])
updated updated
else else
[] []

View file

@ -24,6 +24,8 @@ def get_by_params(params), do: Repo.get_by(Config, params)
@spec changeset(Config.t(), map()) :: Changeset.t() @spec changeset(Config.t(), map()) :: Changeset.t()
def changeset(config, params \\ %{}) do def changeset(config, params \\ %{}) do
params = Map.put(params, :value, transform(params[:value]))
config config
|> cast(params, [:key, :group, :value]) |> cast(params, [:key, :group, :value])
|> validate_required([:key, :group, :value]) |> validate_required([:key, :group, :value])
@ -33,42 +35,43 @@ def changeset(config, params \\ %{}) do
@spec create(map()) :: {:ok, Config.t()} | {:error, Changeset.t()} @spec create(map()) :: {:ok, Config.t()} | {:error, Changeset.t()}
def create(params) do def create(params) do
%Config{} %Config{}
|> changeset(Map.put(params, :value, transform(params[:value]))) |> changeset(params)
|> Repo.insert() |> Repo.insert()
end end
@spec update(Config.t(), map()) :: {:ok, Config} | {:error, Changeset.t()} @spec update(Config.t(), map()) :: {:ok, Config} | {:error, Changeset.t()}
def update(%Config{} = config, %{value: value}) do def update(%Config{} = config, %{value: value}) do
config config
|> change(value: transform(value)) |> changeset(%{value: value})
|> Repo.update() |> Repo.update()
end end
@spec update_or_create(map()) :: {:ok, Config.t()} | {:error, Changeset.t()} @spec update_or_create(map()) :: {:ok, Config.t()} | {:error, Changeset.t()}
def update_or_create(params) do def update_or_create(params) do
with %Config{} = config <- Config.get_by_params(Map.take(params, [:group, :key])) do search_opts = Map.take(params, [:group, :key])
with %Config{} = config <- Config.get_by_params(search_opts) do
Config.update(config, params) Config.update(config, params)
else else
nil -> Config.create(params) nil -> Config.create(params)
end end
end end
@spec delete(map()) :: {:ok, Config.t()} | {:error, Changeset.t()} @spec delete(map()) :: {:ok, Config.t()} | {:error, Changeset.t()} | {:ok, nil}
def delete(params) do def delete(params) do
with %Config{} = config <- Config.get_by_params(Map.delete(params, :subkeys)) do search_opts = Map.delete(params, :subkeys)
if params[:subkeys] do
updated_value =
Keyword.drop(
:erlang.binary_to_term(config.value),
Enum.map(params[:subkeys], &do_transform_string(&1))
)
Config.update(config, %{value: updated_value}) with %Config{} = config <- Config.get_by_params(search_opts),
else {config, sub_keys} when is_list(sub_keys) <- {config, params[:subkeys]},
old_value <- :erlang.binary_to_term(config.value),
keys <- Enum.map(sub_keys, &do_transform_string(&1)),
new_value <- Keyword.drop(old_value, keys) do
Config.update(config, %{value: new_value})
else
{config, nil} ->
Repo.delete(config) Repo.delete(config)
{:ok, nil} {:ok, nil}
end
else
nil -> nil ->
err = err =
dgettext("errors", "Config with params %{params} not found", params: inspect(params)) dgettext("errors", "Config with params %{params} not found", params: inspect(params))
@ -82,10 +85,22 @@ def from_binary(binary), do: :erlang.binary_to_term(binary)
@spec from_binary_with_convert(binary()) :: any() @spec from_binary_with_convert(binary()) :: any()
def from_binary_with_convert(binary) do def from_binary_with_convert(binary) do
from_binary(binary) binary
|> from_binary()
|> do_convert() |> do_convert()
end end
@spec from_string(String.t()) :: atom() | no_return()
def from_string(":" <> entity), do: String.to_existing_atom(entity)
def from_string(entity) when is_binary(entity) do
if is_module_name?(entity) do
String.to_existing_atom("Elixir.#{entity}")
else
entity
end
end
defp do_convert(entity) when is_list(entity) do defp do_convert(entity) when is_list(entity) do
for v <- entity, into: [], do: do_convert(v) for v <- entity, into: [], do: do_convert(v)
end end
@ -97,6 +112,7 @@ defp do_convert(entity) when is_map(entity) do
end end
defp do_convert({:dispatch, [entity]}), do: %{"tuple" => [":dispatch", [inspect(entity)]]} defp do_convert({:dispatch, [entity]}), do: %{"tuple" => [":dispatch", [inspect(entity)]]}
# TODO: will become useless after removing hackney
defp do_convert({:partial_chain, entity}), do: %{"tuple" => [":partial_chain", inspect(entity)]} defp do_convert({:partial_chain, entity}), do: %{"tuple" => [":partial_chain", inspect(entity)]}
defp do_convert(entity) when is_tuple(entity), defp do_convert(entity) when is_tuple(entity),
@ -105,21 +121,15 @@ defp do_convert(entity) when is_tuple(entity),
defp do_convert(entity) when is_boolean(entity) or is_number(entity) or is_nil(entity), defp do_convert(entity) when is_boolean(entity) or is_number(entity) or is_nil(entity),
do: entity do: entity
defp do_convert(entity) when is_atom(entity) do defp do_convert(entity) when is_atom(entity), do: inspect(entity)
string = to_string(entity)
if String.starts_with?(string, "Elixir."),
do: do_convert(string),
else: ":" <> string
end
defp do_convert("Elixir." <> module_name), do: module_name
defp do_convert(entity) when is_binary(entity), do: entity defp do_convert(entity) when is_binary(entity), do: entity
@spec transform(any()) :: binary() @spec transform(any()) :: binary() | no_return()
def transform(entity) when is_binary(entity) or is_map(entity) or is_list(entity) do def transform(entity) when is_binary(entity) or is_map(entity) or is_list(entity) do
:erlang.term_to_binary(do_transform(entity)) entity
|> do_transform()
|> :erlang.term_to_binary()
end end
def transform(entity), do: :erlang.term_to_binary(entity) def transform(entity), do: :erlang.term_to_binary(entity)
@ -131,6 +141,7 @@ defp do_transform(%{"tuple" => [":dispatch", [entity]]}) do
{:dispatch, [dispatch_settings]} {:dispatch, [dispatch_settings]}
end end
# TODO: will become useless after removing hackney
defp do_transform(%{"tuple" => [":partial_chain", entity]}) do defp do_transform(%{"tuple" => [":partial_chain", entity]}) do
{partial_chain, []} = do_eval(entity) {partial_chain, []} = do_eval(entity)
{:partial_chain, partial_chain} {:partial_chain, partial_chain}
@ -149,34 +160,63 @@ defp do_transform(entity) when is_list(entity) do
end end
defp do_transform(entity) when is_binary(entity) do defp do_transform(entity) when is_binary(entity) do
String.trim(entity) entity
|> String.trim()
|> do_transform_string() |> do_transform_string()
end end
defp do_transform(entity), do: entity defp do_transform(entity), do: entity
defp do_transform_string("~r/" <> pattern) do @delimiters ["/", "|", "\"", "'", {"(", ")"}, {"[", "]"}, {"{", "}"}, {"<", ">"}]
modificator = String.split(pattern, "/") |> List.last()
pattern = String.trim_trailing(pattern, "/" <> modificator)
case modificator do defp find_valid_delimiter([], _string, _),
"" -> ~r/#{pattern}/ do: raise(ArgumentError, message: "valid delimiter for Regex expression not found")
"i" -> ~r/#{pattern}/i
"u" -> ~r/#{pattern}/u defp find_valid_delimiter([{leading, closing} = delimiter | others], pattern, regex_delimiter)
"s" -> ~r/#{pattern}/s when is_tuple(delimiter) do
if String.contains?(pattern, closing) do
find_valid_delimiter(others, pattern, regex_delimiter)
else
{:ok, {leading, closing}}
end
end
defp find_valid_delimiter([delimiter | others], pattern, regex_delimiter) do
if String.contains?(pattern, delimiter) do
find_valid_delimiter(others, pattern, regex_delimiter)
else
{:ok, {delimiter, delimiter}}
end
end
@regex_parts ~r/^~r(?'delimiter'[\/|"'([{<]{1})(?'pattern'.+)[\/|"')\]}>]{1}(?'modifier'[uismxfU]*)/u
defp do_transform_string("~r" <> _pattern = regex) do
with %{"modifier" => modifier, "pattern" => pattern, "delimiter" => regex_delimiter} <-
Regex.named_captures(@regex_parts, regex),
{:ok, {leading, closing}} <- find_valid_delimiter(@delimiters, pattern, regex_delimiter),
{result, _} <- Code.eval_string("~r#{leading}#{pattern}#{closing}#{modifier}") do
result
end end
end end
defp do_transform_string(":" <> atom), do: String.to_atom(atom) defp do_transform_string(":" <> atom), do: String.to_atom(atom)
defp do_transform_string(value) do defp do_transform_string(value) do
if String.starts_with?(value, "Pleroma") or String.starts_with?(value, "Phoenix"), if is_module_name?(value) do
do: String.to_existing_atom("Elixir." <> value), String.to_existing_atom("Elixir." <> value)
else: value else
value
end
end
@spec is_module_name?(String.t()) :: boolean()
def is_module_name?(string) do
Regex.match?(~r/^(Pleroma|Phoenix|Tesla)\./, string) or string in ["Oban", "Ueberauth"]
end end
defp do_eval(entity) do defp do_eval(entity) do
cleaned_string = String.replace(entity, ~r/[^\w|^{:,[|^,|^[|^\]^}|^\/|^\.|^"]^\s/, "") cleaned_string = String.replace(entity, ~r/[^\w|^{:,[|^,|^[|^\]^}|^\/|^\.|^"]^\s/, "")
Code.eval_string(cleaned_string, [], requires: [], macros: []) Code.eval_string(cleaned_string)
end end
end end

View file

@ -195,6 +195,7 @@ defmodule Pleroma.Web.Router do
get("/config", AdminAPIController, :config_show) get("/config", AdminAPIController, :config_show)
post("/config", AdminAPIController, :config_update) post("/config", AdminAPIController, :config_update)
get("/config/descriptions", AdminAPIController, :config_descriptions)
get("/config/migrate_to_db", AdminAPIController, :migrate_to_db) get("/config/migrate_to_db", AdminAPIController, :migrate_to_db)
get("/config/migrate_from_db", AdminAPIController, :migrate_from_db) get("/config/migrate_from_db", AdminAPIController, :migrate_from_db)

View file

@ -14,14 +14,14 @@ test "transfer config values from db to env" do
refute Application.get_env(:idna, :test_key) refute Application.get_env(:idna, :test_key)
Pleroma.Web.AdminAPI.Config.create(%{ Pleroma.Web.AdminAPI.Config.create(%{
group: "pleroma", group: ":pleroma",
key: "test_key", key: ":test_key",
value: [live: 2, com: 3] value: [live: 2, com: 3]
}) })
Pleroma.Web.AdminAPI.Config.create(%{ Pleroma.Web.AdminAPI.Config.create(%{
group: "idna", group: ":idna",
key: "test_key", key: ":test_key",
value: [live: 15, com: 35] value: [live: 15, com: 35]
}) })
@ -38,14 +38,14 @@ test "transfer config values from db to env" do
test "non existing atom" do test "non existing atom" do
Pleroma.Web.AdminAPI.Config.create(%{ Pleroma.Web.AdminAPI.Config.create(%{
group: "pleroma", group: ":pleroma",
key: "undefined_atom_key", key: ":undefined_atom_key",
value: [live: 2, com: 3] value: [live: 2, com: 3]
}) })
assert ExUnit.CaptureLog.capture_log(fn -> assert ExUnit.CaptureLog.capture_log(fn ->
Pleroma.Config.TransferTask.start_link([]) Pleroma.Config.TransferTask.start_link([])
end) =~ end) =~
"updating env causes error, key: \"undefined_atom_key\", error: %ArgumentError{message: \"argument error\"}" "updating env causes error, key: \":undefined_atom_key\", error: %ArgumentError{message: \"argument error\"}"
end end
end end

View file

@ -0,0 +1,211 @@
defmodule Pleroma.Docs.GeneratorTest do
use ExUnit.Case, async: true
alias Pleroma.Docs.Generator
@descriptions [
%{
group: :pleroma,
key: Pleroma.Upload,
type: :group,
description: "",
children: [
%{
key: :uploader,
type: :module,
description: "",
suggestions:
Generator.list_modules_in_dir(
"lib/pleroma/upload/filter",
"Elixir.Pleroma.Upload.Filter."
)
},
%{
key: :filters,
type: {:list, :module},
description: "",
suggestions:
Generator.list_modules_in_dir(
"lib/pleroma/web/activity_pub/mrf",
"Elixir.Pleroma.Web.ActivityPub.MRF."
)
},
%{
key: Pleroma.Upload,
type: :string,
description: "",
suggestions: [""]
},
%{
key: :some_key,
type: :keyword,
description: "",
suggestions: [],
children: [
%{
key: :another_key,
type: :integer,
description: "",
suggestions: [5]
},
%{
key: :another_key_with_label,
label: "Another label",
type: :integer,
description: "",
suggestions: [7]
}
]
},
%{
key: :key1,
type: :atom,
description: "",
suggestions: [
:atom,
Pleroma.Upload,
{:tuple, "string", 8080},
[:atom, Pleroma.Upload, {:atom, Pleroma.Upload}]
]
},
%{
key: Pleroma.Upload,
label: "Special Label",
type: :string,
description: "",
suggestions: [""]
},
%{
group: {:subgroup, Swoosh.Adapters.SMTP},
key: :auth,
type: :atom,
description: "`Swoosh.Adapters.SMTP` adapter specific setting",
suggestions: [:always, :never, :if_available]
},
%{
key: "application/xml",
type: {:list, :string},
suggestions: ["xml"]
}
]
},
%{
group: :tesla,
key: :adapter,
type: :group,
description: ""
},
%{
group: :cors_plug,
type: :group,
children: [%{key: :key1, type: :string, suggestions: [""]}]
},
%{group: "Some string group", key: "Some string key", type: :group}
]
describe "convert_to_strings/1" do
test "group, key, label" do
[desc1, desc2 | _] = Generator.convert_to_strings(@descriptions)
assert desc1[:group] == ":pleroma"
assert desc1[:key] == "Pleroma.Upload"
assert desc1[:label] == "Pleroma.Upload"
assert desc2[:group] == ":tesla"
assert desc2[:key] == ":adapter"
assert desc2[:label] == "Adapter"
end
test "group without key" do
descriptions = Generator.convert_to_strings(@descriptions)
desc = Enum.at(descriptions, 2)
assert desc[:group] == ":cors_plug"
refute desc[:key]
assert desc[:label] == "Cors plug"
end
test "children key, label, type" do
[%{children: [child1, child2, child3, child4 | _]} | _] =
Generator.convert_to_strings(@descriptions)
assert child1[:key] == ":uploader"
assert child1[:label] == "Uploader"
assert child1[:type] == :module
assert child2[:key] == ":filters"
assert child2[:label] == "Filters"
assert child2[:type] == {:list, :module}
assert child3[:key] == "Pleroma.Upload"
assert child3[:label] == "Pleroma.Upload"
assert child3[:type] == :string
assert child4[:key] == ":some_key"
assert child4[:label] == "Some key"
assert child4[:type] == :keyword
end
test "child with predefined label" do
[%{children: children} | _] = Generator.convert_to_strings(@descriptions)
child = Enum.at(children, 5)
assert child[:key] == "Pleroma.Upload"
assert child[:label] == "Special Label"
end
test "subchild" do
[%{children: children} | _] = Generator.convert_to_strings(@descriptions)
child = Enum.at(children, 3)
%{children: [subchild | _]} = child
assert subchild[:key] == ":another_key"
assert subchild[:label] == "Another key"
assert subchild[:type] == :integer
end
test "subchild with predefined label" do
[%{children: children} | _] = Generator.convert_to_strings(@descriptions)
child = Enum.at(children, 3)
%{children: subchildren} = child
subchild = Enum.at(subchildren, 1)
assert subchild[:key] == ":another_key_with_label"
assert subchild[:label] == "Another label"
end
test "module suggestions" do
[%{children: [%{suggestions: suggestions} | _]} | _] =
Generator.convert_to_strings(@descriptions)
Enum.each(suggestions, fn suggestion ->
assert String.starts_with?(suggestion, "Pleroma.")
end)
end
test "atoms in suggestions with leading `:`" do
[%{children: children} | _] = Generator.convert_to_strings(@descriptions)
%{suggestions: suggestions} = Enum.at(children, 4)
assert Enum.at(suggestions, 0) == ":atom"
assert Enum.at(suggestions, 1) == "Pleroma.Upload"
assert Enum.at(suggestions, 2) == {":tuple", "string", 8080}
assert Enum.at(suggestions, 3) == [":atom", "Pleroma.Upload", {":atom", "Pleroma.Upload"}]
%{suggestions: suggestions} = Enum.at(children, 6)
assert Enum.at(suggestions, 0) == ":always"
assert Enum.at(suggestions, 1) == ":never"
assert Enum.at(suggestions, 2) == ":if_available"
end
test "group, key as string in main desc" do
descriptions = Generator.convert_to_strings(@descriptions)
desc = Enum.at(descriptions, 3)
assert desc[:group] == "Some string group"
assert desc[:key] == "Some string key"
end
test "key as string subchild" do
[%{children: children} | _] = Generator.convert_to_strings(@descriptions)
child = Enum.at(children, 7)
assert child[:key] == "application/xml"
end
end
end

View file

@ -377,8 +377,8 @@ def registration_factory do
def config_factory do def config_factory do
%Pleroma.Web.AdminAPI.Config{ %Pleroma.Web.AdminAPI.Config{
key: sequence(:key, &"some_key_#{&1}"), key: sequence(:key, &":some_key_#{&1}"),
group: "pleroma", group: ":pleroma",
value: value:
sequence( sequence(
:value, :value,

View file

@ -9,16 +9,14 @@ defmodule Mix.Tasks.Pleroma.ConfigTest do
setup_all do setup_all do
Mix.shell(Mix.Shell.Process) Mix.shell(Mix.Shell.Process)
temp_file = "config/temp.exported_from_db.secret.exs"
on_exit(fn -> on_exit(fn ->
Mix.shell(Mix.Shell.IO) Mix.shell(Mix.Shell.IO)
Application.delete_env(:pleroma, :first_setting) Application.delete_env(:pleroma, :first_setting)
Application.delete_env(:pleroma, :second_setting) Application.delete_env(:pleroma, :second_setting)
:ok = File.rm(temp_file)
end) end)
{:ok, temp_file: temp_file} :ok
end end
clear_config_all([:instance, :dynamic_configuration]) do clear_config_all([:instance, :dynamic_configuration]) do
@ -28,38 +26,44 @@ defmodule Mix.Tasks.Pleroma.ConfigTest do
test "settings are migrated to db" do test "settings are migrated to db" do
assert Repo.all(Config) == [] assert Repo.all(Config) == []
Application.put_env(:pleroma, :first_setting, key: "value", key2: [Pleroma.Repo]) Application.put_env(:pleroma, :first_setting, key: "value", key2: [Repo])
Application.put_env(:pleroma, :second_setting, key: "value2", key2: [Pleroma.Activity]) Application.put_env(:pleroma, :second_setting, key: "value2", key2: ["Activity"])
Mix.Tasks.Pleroma.Config.run(["migrate_to_db"]) Mix.Tasks.Pleroma.Config.run(["migrate_to_db"])
first_db = Config.get_by_params(%{group: "pleroma", key: ":first_setting"}) config1 = Config.get_by_params(%{group: ":pleroma", key: ":first_setting"})
second_db = Config.get_by_params(%{group: "pleroma", key: ":second_setting"}) config2 = Config.get_by_params(%{group: ":pleroma", key: ":second_setting"})
refute Config.get_by_params(%{group: "pleroma", key: "Pleroma.Repo"}) refute Config.get_by_params(%{group: ":pleroma", key: "Pleroma.Repo"})
assert Config.from_binary(first_db.value) == [key: "value", key2: [Pleroma.Repo]] assert Config.from_binary(config1.value) == [key: "value", key2: [Repo]]
assert Config.from_binary(second_db.value) == [key: "value2", key2: [Pleroma.Activity]] assert Config.from_binary(config2.value) == [key: "value2", key2: ["Activity"]]
end end
test "settings are migrated to file and deleted from db", %{temp_file: temp_file} do test "settings are migrated to file and deleted from db" do
env = "temp"
config_file = "config/#{env}.exported_from_db.secret.exs"
on_exit(fn ->
:ok = File.rm(config_file)
end)
Config.create(%{ Config.create(%{
group: "pleroma", group: ":pleroma",
key: ":setting_first", key: ":setting_first",
value: [key: "value", key2: [Pleroma.Activity]] value: [key: "value", key2: ["Activity"]]
}) })
Config.create(%{ Config.create(%{
group: "pleroma", group: ":pleroma",
key: ":setting_second", key: ":setting_second",
value: [key: "valu2", key2: [Pleroma.Repo]] value: [key: "value2", key2: [Repo]]
}) })
Mix.Tasks.Pleroma.Config.run(["migrate_from_db", "temp", "true"]) Mix.Tasks.Pleroma.Config.run(["migrate_from_db", "--env", env, "-d"])
assert Repo.all(Config) == [] assert Repo.all(Config) == []
assert File.exists?(temp_file)
{:ok, file} = File.read(temp_file)
file = File.read!(config_file)
assert file =~ "config :pleroma, :setting_first," assert file =~ "config :pleroma, :setting_first,"
assert file =~ "config :pleroma, :setting_second," assert file =~ "config :pleroma, :setting_second,"
end end

View file

@ -1950,6 +1950,7 @@ test "with settings in db", %{conn: conn} do
%{ %{
"configs" => [ "configs" => [
%{ %{
"group" => ":pleroma",
"key" => key1, "key" => key1,
"value" => _ "value" => _
}, },
@ -1995,15 +1996,15 @@ test "create new config setting in db", %{conn: conn} do
conn = conn =
post(conn, "/api/pleroma/admin/config", %{ post(conn, "/api/pleroma/admin/config", %{
configs: [ configs: [
%{group: "pleroma", key: "key1", value: "value1"}, %{group: ":pleroma", key: ":key1", value: "value1"},
%{ %{
group: "ueberauth", group: ":ueberauth",
key: "Ueberauth.Strategy.Twitter.OAuth", key: "Ueberauth.Strategy.Twitter.OAuth",
value: [%{"tuple" => [":consumer_secret", "aaaa"]}] value: [%{"tuple" => [":consumer_secret", "aaaa"]}]
}, },
%{ %{
group: "pleroma", group: ":pleroma",
key: "key2", key: ":key2",
value: %{ value: %{
":nested_1" => "nested_value1", ":nested_1" => "nested_value1",
":nested_2" => [ ":nested_2" => [
@ -2013,21 +2014,21 @@ test "create new config setting in db", %{conn: conn} do
} }
}, },
%{ %{
group: "pleroma", group: ":pleroma",
key: "key3", key: ":key3",
value: [ value: [
%{"nested_3" => ":nested_3", "nested_33" => "nested_33"}, %{"nested_3" => ":nested_3", "nested_33" => "nested_33"},
%{"nested_4" => true} %{"nested_4" => true}
] ]
}, },
%{ %{
group: "pleroma", group: ":pleroma",
key: "key4", key: ":key4",
value: %{":nested_5" => ":upload", "endpoint" => "https://example.com"} value: %{":nested_5" => ":upload", "endpoint" => "https://example.com"}
}, },
%{ %{
group: "idna", group: ":idna",
key: "key5", key: ":key5",
value: %{"tuple" => ["string", "Pleroma.Captcha.NotReal", []]} value: %{"tuple" => ["string", "Pleroma.Captcha.NotReal", []]}
} }
] ]
@ -2036,18 +2037,18 @@ test "create new config setting in db", %{conn: conn} do
assert json_response(conn, 200) == %{ assert json_response(conn, 200) == %{
"configs" => [ "configs" => [
%{ %{
"group" => "pleroma", "group" => ":pleroma",
"key" => "key1", "key" => ":key1",
"value" => "value1" "value" => "value1"
}, },
%{ %{
"group" => "ueberauth", "group" => ":ueberauth",
"key" => "Ueberauth.Strategy.Twitter.OAuth", "key" => "Ueberauth.Strategy.Twitter.OAuth",
"value" => [%{"tuple" => [":consumer_secret", "aaaa"]}] "value" => [%{"tuple" => [":consumer_secret", "aaaa"]}]
}, },
%{ %{
"group" => "pleroma", "group" => ":pleroma",
"key" => "key2", "key" => ":key2",
"value" => %{ "value" => %{
":nested_1" => "nested_value1", ":nested_1" => "nested_value1",
":nested_2" => [ ":nested_2" => [
@ -2057,21 +2058,21 @@ test "create new config setting in db", %{conn: conn} do
} }
}, },
%{ %{
"group" => "pleroma", "group" => ":pleroma",
"key" => "key3", "key" => ":key3",
"value" => [ "value" => [
%{"nested_3" => ":nested_3", "nested_33" => "nested_33"}, %{"nested_3" => ":nested_3", "nested_33" => "nested_33"},
%{"nested_4" => true} %{"nested_4" => true}
] ]
}, },
%{ %{
"group" => "pleroma", "group" => ":pleroma",
"key" => "key4", "key" => ":key4",
"value" => %{"endpoint" => "https://example.com", ":nested_5" => ":upload"} "value" => %{"endpoint" => "https://example.com", ":nested_5" => ":upload"}
}, },
%{ %{
"group" => "idna", "group" => ":idna",
"key" => "key5", "key" => ":key5",
"value" => %{"tuple" => ["string", "Pleroma.Captcha.NotReal", []]} "value" => %{"tuple" => ["string", "Pleroma.Captcha.NotReal", []]}
} }
] ]
@ -2101,8 +2102,8 @@ test "create new config setting in db", %{conn: conn} do
end end
test "update config setting & delete", %{conn: conn} do test "update config setting & delete", %{conn: conn} do
config1 = insert(:config, key: "keyaa1") config1 = insert(:config, key: ":keyaa1")
config2 = insert(:config, key: "keyaa2") config2 = insert(:config, key: ":keyaa2")
insert(:config, insert(:config,
group: "ueberauth", group: "ueberauth",
@ -2126,7 +2127,7 @@ test "update config setting & delete", %{conn: conn} do
assert json_response(conn, 200) == %{ assert json_response(conn, 200) == %{
"configs" => [ "configs" => [
%{ %{
"group" => "pleroma", "group" => ":pleroma",
"key" => config1.key, "key" => config1.key,
"value" => "another_value" "value" => "another_value"
} }
@ -2138,11 +2139,14 @@ test "update config setting & delete", %{conn: conn} do
end end
test "common config example", %{conn: conn} do test "common config example", %{conn: conn} do
adapter = Application.get_env(:tesla, :adapter)
on_exit(fn -> Application.put_env(:tesla, :adapter, adapter) end)
conn = conn =
post(conn, "/api/pleroma/admin/config", %{ post(conn, "/api/pleroma/admin/config", %{
configs: [ configs: [
%{ %{
"group" => "pleroma", "group" => ":pleroma",
"key" => "Pleroma.Captcha.NotReal", "key" => "Pleroma.Captcha.NotReal",
"value" => [ "value" => [
%{"tuple" => [":enabled", false]}, %{"tuple" => [":enabled", false]},
@ -2154,16 +2158,21 @@ test "common config example", %{conn: conn} do
%{"tuple" => [":regex1", "~r/https:\/\/example.com/"]}, %{"tuple" => [":regex1", "~r/https:\/\/example.com/"]},
%{"tuple" => [":regex2", "~r/https:\/\/example.com/u"]}, %{"tuple" => [":regex2", "~r/https:\/\/example.com/u"]},
%{"tuple" => [":regex3", "~r/https:\/\/example.com/i"]}, %{"tuple" => [":regex3", "~r/https:\/\/example.com/i"]},
%{"tuple" => [":regex4", "~r/https:\/\/example.com/s"]} %{"tuple" => [":regex4", "~r/https:\/\/example.com/s"]},
%{"tuple" => [":name", "Pleroma"]}
] ]
} },
%{"group" => ":tesla", "key" => ":adapter", "value" => "Tesla.Adapter.Httpc"}
] ]
}) })
assert Application.get_env(:tesla, :adapter) == Tesla.Adapter.Httpc
assert Pleroma.Config.get([Pleroma.Captcha.NotReal, :name]) == "Pleroma"
assert json_response(conn, 200) == %{ assert json_response(conn, 200) == %{
"configs" => [ "configs" => [
%{ %{
"group" => "pleroma", "group" => ":pleroma",
"key" => "Pleroma.Captcha.NotReal", "key" => "Pleroma.Captcha.NotReal",
"value" => [ "value" => [
%{"tuple" => [":enabled", false]}, %{"tuple" => [":enabled", false]},
@ -2175,9 +2184,11 @@ test "common config example", %{conn: conn} do
%{"tuple" => [":regex1", "~r/https:\\/\\/example.com/"]}, %{"tuple" => [":regex1", "~r/https:\\/\\/example.com/"]},
%{"tuple" => [":regex2", "~r/https:\\/\\/example.com/u"]}, %{"tuple" => [":regex2", "~r/https:\\/\\/example.com/u"]},
%{"tuple" => [":regex3", "~r/https:\\/\\/example.com/i"]}, %{"tuple" => [":regex3", "~r/https:\\/\\/example.com/i"]},
%{"tuple" => [":regex4", "~r/https:\\/\\/example.com/s"]} %{"tuple" => [":regex4", "~r/https:\\/\\/example.com/s"]},
%{"tuple" => [":name", "Pleroma"]}
] ]
} },
%{"group" => ":tesla", "key" => ":adapter", "value" => "Tesla.Adapter.Httpc"}
] ]
} }
end end
@ -2187,7 +2198,7 @@ test "tuples with more than two values", %{conn: conn} do
post(conn, "/api/pleroma/admin/config", %{ post(conn, "/api/pleroma/admin/config", %{
configs: [ configs: [
%{ %{
"group" => "pleroma", "group" => ":pleroma",
"key" => "Pleroma.Web.Endpoint.NotReal", "key" => "Pleroma.Web.Endpoint.NotReal",
"value" => [ "value" => [
%{ %{
@ -2251,7 +2262,7 @@ test "tuples with more than two values", %{conn: conn} do
assert json_response(conn, 200) == %{ assert json_response(conn, 200) == %{
"configs" => [ "configs" => [
%{ %{
"group" => "pleroma", "group" => ":pleroma",
"key" => "Pleroma.Web.Endpoint.NotReal", "key" => "Pleroma.Web.Endpoint.NotReal",
"value" => [ "value" => [
%{ %{
@ -2318,7 +2329,7 @@ test "settings with nesting map", %{conn: conn} do
post(conn, "/api/pleroma/admin/config", %{ post(conn, "/api/pleroma/admin/config", %{
configs: [ configs: [
%{ %{
"group" => "pleroma", "group" => ":pleroma",
"key" => ":key1", "key" => ":key1",
"value" => [ "value" => [
%{"tuple" => [":key2", "some_val"]}, %{"tuple" => [":key2", "some_val"]},
@ -2348,7 +2359,7 @@ test "settings with nesting map", %{conn: conn} do
%{ %{
"configs" => [ "configs" => [
%{ %{
"group" => "pleroma", "group" => ":pleroma",
"key" => ":key1", "key" => ":key1",
"value" => [ "value" => [
%{"tuple" => [":key2", "some_val"]}, %{"tuple" => [":key2", "some_val"]},
@ -2380,7 +2391,7 @@ test "value as map", %{conn: conn} do
post(conn, "/api/pleroma/admin/config", %{ post(conn, "/api/pleroma/admin/config", %{
configs: [ configs: [
%{ %{
"group" => "pleroma", "group" => ":pleroma",
"key" => ":key1", "key" => ":key1",
"value" => %{"key" => "some_val"} "value" => %{"key" => "some_val"}
} }
@ -2391,7 +2402,7 @@ test "value as map", %{conn: conn} do
%{ %{
"configs" => [ "configs" => [
%{ %{
"group" => "pleroma", "group" => ":pleroma",
"key" => ":key1", "key" => ":key1",
"value" => %{"key" => "some_val"} "value" => %{"key" => "some_val"}
} }
@ -2404,7 +2415,7 @@ test "dispatch setting", %{conn: conn} do
post(conn, "/api/pleroma/admin/config", %{ post(conn, "/api/pleroma/admin/config", %{
configs: [ configs: [
%{ %{
"group" => "pleroma", "group" => ":pleroma",
"key" => "Pleroma.Web.Endpoint.NotReal", "key" => "Pleroma.Web.Endpoint.NotReal",
"value" => [ "value" => [
%{ %{
@ -2437,7 +2448,7 @@ test "dispatch setting", %{conn: conn} do
assert json_response(conn, 200) == %{ assert json_response(conn, 200) == %{
"configs" => [ "configs" => [
%{ %{
"group" => "pleroma", "group" => ":pleroma",
"key" => "Pleroma.Web.Endpoint.NotReal", "key" => "Pleroma.Web.Endpoint.NotReal",
"value" => [ "value" => [
%{ %{
@ -2467,7 +2478,7 @@ test "queues key as atom", %{conn: conn} do
post(conn, "/api/pleroma/admin/config", %{ post(conn, "/api/pleroma/admin/config", %{
configs: [ configs: [
%{ %{
"group" => "oban", "group" => ":oban",
"key" => ":queues", "key" => ":queues",
"value" => [ "value" => [
%{"tuple" => [":federator_incoming", 50]}, %{"tuple" => [":federator_incoming", 50]},
@ -2485,7 +2496,7 @@ test "queues key as atom", %{conn: conn} do
assert json_response(conn, 200) == %{ assert json_response(conn, 200) == %{
"configs" => [ "configs" => [
%{ %{
"group" => "oban", "group" => ":oban",
"key" => ":queues", "key" => ":queues",
"value" => [ "value" => [
%{"tuple" => [":federator_incoming", 50]}, %{"tuple" => [":federator_incoming", 50]},
@ -2504,7 +2515,7 @@ test "queues key as atom", %{conn: conn} do
test "delete part of settings by atom subkeys", %{conn: conn} do test "delete part of settings by atom subkeys", %{conn: conn} do
config = config =
insert(:config, insert(:config,
key: "keyaa1", key: ":keyaa1",
value: :erlang.term_to_binary(subkey1: "val1", subkey2: "val2", subkey3: "val3") value: :erlang.term_to_binary(subkey1: "val1", subkey2: "val2", subkey3: "val3")
) )
@ -2524,8 +2535,8 @@ test "delete part of settings by atom subkeys", %{conn: conn} do
json_response(conn, 200) == %{ json_response(conn, 200) == %{
"configs" => [ "configs" => [
%{ %{
"group" => "pleroma", "group" => ":pleroma",
"key" => "keyaa1", "key" => ":keyaa1",
"value" => [%{"tuple" => [":subkey2", "val2"]}] "value" => [%{"tuple" => [":subkey2", "val2"]}]
} }
] ]
@ -3099,6 +3110,21 @@ test "it deletes the note", %{admin: admin, report_id: report_id} do
assert ReportNote |> Repo.all() |> length() == 1 assert ReportNote |> Repo.all() |> length() == 1
end end
end end
test "GET /api/pleroma/admin/config/descriptions", %{conn: conn} do
admin = insert(:user, is_admin: true)
conn =
assign(conn, :user, admin)
|> get("/api/pleroma/admin/config/descriptions")
assert [child | _others] = json_response(conn, 200)
assert child["children"]
assert child["key"]
assert String.starts_with?(child["group"], ":")
assert child["description"]
end
end end
# Needed for testing # Needed for testing

View file

@ -91,14 +91,26 @@ test "pleroma module" do
assert Config.from_binary(binary) == Pleroma.Bookmark assert Config.from_binary(binary) == Pleroma.Bookmark
end end
test "pleroma string" do
binary = Config.transform("Pleroma")
assert binary == :erlang.term_to_binary("Pleroma")
assert Config.from_binary(binary) == "Pleroma"
end
test "phoenix module" do test "phoenix module" do
binary = Config.transform("Phoenix.Socket.V1.JSONSerializer") binary = Config.transform("Phoenix.Socket.V1.JSONSerializer")
assert binary == :erlang.term_to_binary(Phoenix.Socket.V1.JSONSerializer) assert binary == :erlang.term_to_binary(Phoenix.Socket.V1.JSONSerializer)
assert Config.from_binary(binary) == Phoenix.Socket.V1.JSONSerializer assert Config.from_binary(binary) == Phoenix.Socket.V1.JSONSerializer
end end
test "tesla module" do
binary = Config.transform("Tesla.Adapter.Hackney")
assert binary == :erlang.term_to_binary(Tesla.Adapter.Hackney)
assert Config.from_binary(binary) == Tesla.Adapter.Hackney
end
test "sigil" do test "sigil" do
binary = Config.transform("~r/comp[lL][aA][iI][nN]er/") binary = Config.transform("~r[comp[lL][aA][iI][nN]er]")
assert binary == :erlang.term_to_binary(~r/comp[lL][aA][iI][nN]er/) assert binary == :erlang.term_to_binary(~r/comp[lL][aA][iI][nN]er/)
assert Config.from_binary(binary) == ~r/comp[lL][aA][iI][nN]er/ assert Config.from_binary(binary) == ~r/comp[lL][aA][iI][nN]er/
end end
@ -109,10 +121,10 @@ test "link sigil" do
assert Config.from_binary(binary) == ~r/https:\/\/example.com/ assert Config.from_binary(binary) == ~r/https:\/\/example.com/
end end
test "link sigil with u modifier" do test "link sigil with um modifiers" do
binary = Config.transform("~r/https:\/\/example.com/u") binary = Config.transform("~r/https:\/\/example.com/um")
assert binary == :erlang.term_to_binary(~r/https:\/\/example.com/u) assert binary == :erlang.term_to_binary(~r/https:\/\/example.com/um)
assert Config.from_binary(binary) == ~r/https:\/\/example.com/u assert Config.from_binary(binary) == ~r/https:\/\/example.com/um
end end
test "link sigil with i modifier" do test "link sigil with i modifier" do
@ -127,6 +139,12 @@ test "link sigil with s modifier" do
assert Config.from_binary(binary) == ~r/https:\/\/example.com/s assert Config.from_binary(binary) == ~r/https:\/\/example.com/s
end end
test "raise if valid delimiter not found" do
assert_raise ArgumentError, "valid delimiter for Regex expression not found", fn ->
Config.transform("~r/https://[]{}<>\"'()|example.com/s")
end
end
test "2 child tuple" do test "2 child tuple" do
binary = Config.transform(%{"tuple" => ["v1", ":v2"]}) binary = Config.transform(%{"tuple" => ["v1", ":v2"]})
assert binary == :erlang.term_to_binary({"v1", :v2}) assert binary == :erlang.term_to_binary({"v1", :v2})