Commit graph

213 commits

Author SHA1 Message Date
rinpatch
4128e3a84a HTTP: Implement max request limits 2020-07-15 15:26:35 +03:00
rinpatch
7738fbbaf5 Connection pool: implement logging and telemetry events 2020-07-15 15:26:25 +03:00
rinpatch
ec9d0d146b Connection pool: Fix race conditions in limit enforcement
Fixes race conditions in limit enforcement by putting worker processes
in a DynamicSupervisor
2020-07-15 15:17:27 +03:00
rinpatch
58a4f350a8 Refactor gun pooling and simplify adapter option insertion
This patch refactors gun pooling to use Elixir process registry and
simplifies adapter option insertion.

Having the pool use process registry instead of a GenServer has a number of advantages:
- Simpler code: the initial implementation adds about half the lines of code it deletes
- Concurrency: unlike a GenServer, ETS-based registry can handle multiple checkout/checkin
requests at the same time
- Precise and easy idle connection clousure: current proposal for closing idle connections in
the GenServer-based pool needs to filter through all connections once a minute and compare their
last active time with closing time. With Elixir process registry this can be done
by just using `Process.send_after`/`Process.cancel_timer` in the worker process.
- Lower memory footprint: In my tests `gun-memory-leak` branch uses about 290mb on peak load (250 connections)
and 235mb on idle (5-10 connections). Registry-based pool uses 210mb on idle and 240mb on peak load
2020-07-15 15:17:27 +03:00
rinpatch
e6ccc25565 Fix in-db configuration in dev environment
Previously, in-db configuration only worked when `warnings_as_errors`
was disabled because re-compiling scrubbers on application restart
created a warning about module conflicts. This patch fixes that
by enabling `ignore_module_conflict` option of the compiler at runtime,
and enables `warnings_as_errors` in prod since there is no reason
to keep it disabled anymore.
2020-07-14 13:16:54 +03:00
href
4347d2de5e Config/Docs: Expand behaviour suggestions at runtime 2020-07-12 17:23:33 +02:00
lain
48f98a2748 Merge branch 'mix-tasks-improvement' into 'develop'
Mix tasks improvement

See merge request pleroma/pleroma!2723
2020-07-10 08:58:44 +00:00
Mark Felder
d23804f191 Use the Pleroma.Config alias 2020-07-09 10:53:51 -05:00
Alexander Strizhakov
72ad3a66f4
don't fully start pleroma in mix tasks 2020-07-08 15:58:47 +03:00
lain
b7f297f0bd Merge branch 'issue/1798' into 'develop'
[#1798] fixed migration the settings to DB

See merge request pleroma/pleroma!2655
2020-06-23 10:54:15 +00:00
Maksim Pechnikov
b0a40fc2e4 added verify RUM settings before start app 2020-06-22 17:27:49 +03:00
Mark Felder
2731ea1334 Change references from "deleted_urls" to "banned_urls" as nothing is handled via media deletions anymore; all actions are manual operations by an admin to ban the url 2020-06-17 13:13:55 -05:00
href
f124f68205 Switch from gen_magic to majic, use Majic.Plug, remove Pleroma.MIME 2020-06-16 15:27:27 +02:00
href
ec1452fd1c Pleroma.MIME: use gen_magic 2020-06-16 15:15:50 +02:00
Maksim Pechnikov
2e8a236cef fix invalidates media url's 2020-06-14 21:02:57 +03:00
rinpatch
8308611279 Startup: suggest updating OTP when the version is too low for gun
I've seen quite a few people wonder what to do when presented with
this error message.
2020-05-12 17:30:39 +03:00
href
9491ba3e49 Streamer rework 2020-05-07 09:13:32 +00:00
Egor Kislitsyn
39a78998d0
Change Pleroma.CaptchaTest to be a regular module instead of GenServer 2020-04-29 21:26:07 +04:00
ef7d2b0f11 Merge branch 'gun' into 'develop'
Gun adapter

Closes #945

See merge request pleroma/pleroma!1861
2020-04-01 17:05:09 +00:00
Alexander Strizhakov
eb9744cade
activities generation tasks 2020-03-30 11:42:25 +03:00
Alexander Strizhakov
39ed608b13
Merge branch 'develop' into gun 2020-03-12 18:31:10 +03:00
Alexander Strizhakov
1306b92997
clean up 2020-03-12 18:29:07 +03:00
Alexander Strizhakov
fce090c1de
using Pleroma.Config instead of ets 2020-03-11 17:22:50 +03:00
Alexander Strizhakov
282a93554f
merging release default config on app start 2020-03-11 16:25:53 +03:00
Alexander Strizhakov
6b2fb9160c
otp version 2020-03-04 09:23:42 +03:00
Alexander Strizhakov
24d1ac125c
hiding raise error logic to otp_version module 2020-03-03 13:24:19 +03:00
Alexander Strizhakov
4c8569d403
otp_version refactor 2020-03-03 12:21:10 +03:00
Alexander Strizhakov
509c81e4b1
Merge branch 'develop' into gun 2020-03-03 10:08:07 +03:00
Haelwenn (lanodan) Monnier
6da6540036
Bump copyright years of files changed after 2020-01-07
Done via the following command:
git diff fcd5dd259a --stat --name-only | xargs sed -i '/Pleroma Authors/c# Copyright © 2017-2020 Pleroma Authors <https:\/\/pleroma.social\/>'
2020-03-02 06:08:45 +01:00
Alexander Strizhakov
514c899275
adding gun adapter 2020-02-18 08:19:01 +03:00
Maksim Pechnikov
2c40c8b4a2 Merge branch 'develop' into issue/1383 2020-02-03 21:42:36 +03:00
Egor Kislitsyn
6302b40791
Warn if HTTPSecurityPlug is disabled 2020-01-28 19:14:09 +04:00
Maksim Pechnikov
b8d5c09f48 Merge branch 'develop' into issue/1383 2020-01-22 09:30:30 +03:00
rinpatch
dc0498ab2b Check for unapplied migrations on startup
Closes #1328
2020-01-20 12:20:26 +03:00
Maksim Pechnikov
54c9db38c8 Merge branch 'develop' into issue/1383 2019-12-21 20:38:59 +03:00
Egor Kislitsyn
c098dec473 Merge branch 'develop' into feature/custom-runtime-modules 2019-12-10 00:16:43 +07:00
Maksim Pechnikov
2b821036c4 Merge branch 'develop' into issue/1383 2019-12-09 19:41:43 +03:00
Egor Kislitsyn
ed92784e7c Set Logger level to :info in prod 2019-12-09 19:11:54 +07:00
Egor Kislitsyn
84f891ea3e Add Pleroma.Utils.compile_dir/1 2019-12-09 18:23:07 +07:00
rinpatch
d6c89068f3 HTML: Compile Scrubbers on boot
This makes it possible to configure their behavior on OTP releases.
2019-12-08 20:35:41 +03:00
Maksim Pechnikov
5876a9cb79 Merge branch 'develop' into issue/1383 2019-12-08 19:52:46 +03:00
Egor Kislitsyn
a75d4a41e0 Add a test for custom runtime modules 2019-12-06 17:05:09 +07:00
Egor Kislitsyn
e4292cbfad Use Kernel.inspect/2 to print loaded custom modules 2019-12-06 15:16:39 +07:00
Egor Kislitsyn
157bceeda9 Move runtime configuration from :instance to :modules 2019-12-06 15:04:46 +07:00
Egor Kislitsyn
1216b546c6 Fix credo warning 2019-12-05 20:29:17 +07:00
Egor Kislitsyn
48ae3c4347 Add support for custom modules 2019-12-05 20:18:25 +07:00
lain
aa97fe8f17 ChatChannel: Ignore messages that are too long. 2019-12-04 12:48:34 +01:00
Maksim Pechnikov
b7c449118b Merge branch 'develop' into issue/1383 2019-11-28 21:27:01 +03:00
Maksim Pechnikov
a4f834a687 moved DigestEmail to Oban Periodic jobs 2019-11-27 15:59:13 +03:00
Maksim Pechnikov
c5766a8100 moved ActivityExpiration to Oban Periodic jobs 2019-11-27 13:36:03 +03:00
Maksim Pechnikov
6f202a401b moved ScheduledActivity to Oban Periodic jobs 2019-11-27 09:26:37 +03:00
Maksim Pechnikov
0e9be6bafa moved OAuth.Token.CleanWorker to Oban Periodic jobs 2019-11-25 14:32:20 +03:00
Sadposter
d3656c2725 add ability to set a custom user-agent string 2019-11-25 09:53:11 +00:00
Steven Fuchs
94627baa5c New rate limiter 2019-11-11 12:13:06 +00:00
Ariadne Conill
25b7ff56c3 application: don't start Federator.init/1 anymore 2019-10-18 14:50:09 +00:00
lain
27110793a7 Merge branch 'featrue/job-monitor' into 'develop'
Add Pleroma.JobQueueMonitor

Closes #1274

See merge request pleroma/pleroma!1721
2019-10-04 12:52:11 +00:00
Sergey Suprunenko
c541b83bef Track failed proxy urls and don't request them again 2019-10-01 20:00:27 +00:00
Egor Kislitsyn
75290cbfbd Add Pleroma.JobQueueMonitor 2019-09-26 18:49:57 +07:00
kaniini
3ff57f1fd3 Merge branch 'hex-flake-id' into 'develop'
Replace `Pleroma.FlakeId` with `flake_id` hex package

See merge request pleroma/pleroma!1692
2019-09-26 02:51:54 +00:00
Egor Kislitsyn
b5dfe83433 Replace Pleroma.FlakeId with flake_id hex package 2019-09-25 17:14:31 +07:00
Ekaterina Vaartis
ee620ecbf1 Add caching for emoji pack sharing 2019-09-19 00:16:33 +03:00
Steven Fuchs
80c5c3495b remove remaining errors from tests 2019-09-17 14:44:52 +00:00
Egor Kislitsyn
96816ceaa2 Revert "Merge branch 'revert-4fabf83a' into 'develop'"
This reverts commit fe7fd33126, reversing
changes made to 4fabf83ad0.
2019-09-16 17:55:04 +07:00
kaniini
c623b4324d Revert "Merge branch 'streamer-refactoring' into 'develop'"
This reverts merge request !1653
2019-09-16 09:09:21 +00:00
Steven Fuchs
aab264db82 Streamer refactoring 2019-09-16 07:44:03 +00:00
Ivan Tashkinov
6bcedb693c [#1149] Merge remote-tracking branch 'remotes/upstream/develop' into 1149-oban-job-queue
# Conflicts:
#	docs/config.md
#	mix.lock
2019-09-14 16:11:44 +03:00
minibikini
11e12b5761 Add Pleroma.Plugs.Cache 2019-09-09 18:53:08 +00:00
Ivan Tashkinov
b49085c156 [#1149] Refactoring: GenServer workers renamed to daemons, use Oban.Worker moved to helper. 2019-09-02 14:57:40 +03:00
Ivan Tashkinov
a90ea8ba15 [#1149] Addressed code review comments (code style, jobs pruning etc.). 2019-08-31 19:08:56 +03:00
Ivan Tashkinov
e890ea7e82 [#1149] Added Oban job for "activity_expiration". Merged remote-tracking branch 'remotes/upstream/develop' into 1149-oban-job-queue
# Conflicts:
#	config/config.exs
2019-08-31 14:25:43 +03:00
lain
1692fa8945 ActivityExpirationWorker: Fix merge issues. 2019-08-24 17:22:26 +02:00
lain
cc6c0b4ba6 Merge remote-tracking branch 'origin/develop' into sixohsix/pleroma-post_expiration 2019-08-24 15:48:33 +02:00
Ivan Tashkinov
581123f8bb [#1149] Introduced quantum job scheduler. Documentation & config changes. 2019-08-23 18:28:23 +03:00
Ivan Tashkinov
c29686309e [#1149] Upgraded oban from 0.6.0 to 0.7.1. 2019-08-23 09:23:10 +03:00
Ivan Tashkinov
256ff09aa8 [#1149] Merge remote-tracking branch 'remotes/upstream/develop' into 1149-oban-job-queue
# Conflicts:
#	lib/pleroma/application.ex
#	lib/pleroma/scheduled_activity_worker.ex
#	lib/pleroma/web/federator/retry_queue.ex
#	lib/pleroma/web/oauth/token/clean_worker.ex
#	test/user_test.exs
#	test/web/federator_test.exs
2019-08-22 20:59:58 +03:00
rinpatch
fba3c16d20 Fix OAuth cleanup worker unconditionally starting
!1576 removed enabled/disabled check from the worker, in favor of just
not starting it in application.ex if disabled. However a line
unconditionally starting the worker was removed
2019-08-15 20:36:20 +03:00
Ivan Tashkinov
a180c1360e [#1149] Oban mailer job. Adjusted tests. 2019-08-14 21:42:21 +03:00
stwf
c43152f6c1 fix formatting 2019-08-14 14:01:11 -04:00
stwf
15ef521009 Isolate OAuth.Token.CleanWorker 2019-08-14 11:57:50 -04:00
stwf
51bdf0cab6 use default child_specs 2019-08-14 11:55:17 -04:00
Ivan Tashkinov
267262491e Merge remote-tracking branch 'remotes/upstream/develop' into 1149-oban-job-queue 2019-08-09 20:09:56 +03:00
Ivan Tashkinov
23d279e03e [#1149] Replaced RetryQueue with oban-based retries. 2019-08-01 17:28:00 +03:00
Mike Verdone
378f5f0fbe Add activity expiration worker
This is a worker that runs every minute and deletes expired activities.
It's based heavily on the scheduled activities worker.
2019-07-24 13:14:52 +02:00
Roman Chvanikov
36049f08ef Merge develop 2019-07-20 01:03:25 +03:00
Ariadne Conill
d930e5d5c3 activitypub: introduce internal fetch service actor 2019-07-17 17:14:08 +00:00
Roman Chvanikov
e7c175c943 Use PleromaJobQueue for scheduling 2019-07-16 16:49:50 +03:00
Roman Chvanikov
eae991b06a merge develop 2019-07-12 18:08:27 +03:00
Alex S
beba7bbc85 removing synchronization worker 2019-07-10 17:42:18 +03:00
Roman Chvanikov
371d39e160 Merge develop 2019-07-09 21:21:09 +03:00
Alexander Strizhakov
d6b0fce6e9 Fix/1019 correct count remote users 2019-07-09 17:36:35 +00:00
Roman Chvanikov
657277ffc0 Resolve conflicts 2019-06-29 00:52:50 +03:00
Alexander Strizhakov
c2ca1f22a2 it is changed in compile time
we can't change module attributes and endpoint settings in runtime
2019-06-14 15:45:05 +00:00
lain
315f090f59 Prometheus: Remove flaky process collection NIF. 2019-06-13 20:30:04 +02:00
Roman Chvanikov
01fe5abad1 Resolve conflicts 2019-06-07 01:20:50 +03:00
rinpatch
92213fb87c Replace Mix.env with Pleroma.Config.get(:env)
Mix.env/0 is not availible in release environments such as distillery or
elixir's built-in releases.
2019-06-06 23:59:51 +03:00
rinpatch
9ae8f012a5 Switch to manual Supervisor child specifications instead of Supervisor.Spec
Supervisor.Spec is deprecated and causes warnings on Elixir master, see
https://hexdocs.pm/elixir/Supervisor.Spec.html
2019-06-06 16:37:44 +03:00
Roman Chvanikov
ce47017c89 Merge develop 2019-05-29 18:18:22 +03:00
Maksim
620908a2db [#699] add worker to clean expired oauth tokens 2019-05-22 15:44:50 +00:00
Aaron Tinio
f96e9b28bb Fix prometheus-ecto error when not configured 2019-05-21 07:30:18 +08:00
Roman Chvanikov
724311e151 Fix Credo warnings 2019-04-20 19:57:43 +07:00