cd1b4155d5
Merge branch 'oban-jobs-to-simple-tasks' into 'develop'
...
Moving some background jobs into simple tasks
Closes #1790
See merge request pleroma/pleroma!3129
2020-11-19 20:32:32 +00:00
rinpatch
2c55f7d7cb
Remove FedSockets
...
Current FedSocket implementation has a bunch of problems. It doesn't
have proper error handling (in case of an error the server just doesn't
respond until the connection is closed, while the client doesn't match
any error messages and just assumes there has been an error after 15s)
and the code is full of bad descisions (see: fetch registry which uses
uuids for no reason and waits for a response by recursively querying a
ets table until the value changes, or double JSON encoding).
Sometime ago I almost completed rewriting fedsockets from scrach to
adress these issues. However, while doing so, I realized that fedsockets
are just too overkill for what they were trying to accomplish, which is
reduce the overhead of federation by not signing every message.
This could be done without reimplementing failure states and endpoint
logic we already have with HTTP by, for example, using TLS cert auth,
or switching to a more performant signature algorithm. I opened
https://git.pleroma.social/pleroma/pleroma/-/issues/2262 for further
discussion on alternatives to fedsockets.
From discussions I had with other Pleroma developers it seems like they
would approve the descision to remove them as well,
therefore I am submitting this patch.
2020-11-17 17:28:30 +03:00
Alexander Strizhakov
8d218ebaf5
Moving some background jobs into simple tasks
...
- fetching activity data
- attachment prefetching
- using limiter to prevent overload
2020-11-11 13:39:49 +03:00
eugenijm
8e41baff40
Add idempotency_key to the chat_message entity.
2020-10-31 05:50:59 +03:00
Ivan Tashkinov
8a55de1d78
[ #3059 ] Fixed Phoenix 1.5 telemetry processing.
2020-10-22 13:54:15 +03:00
Ivan Tashkinov
9ea31b373f
Merge remote-tracking branch 'remotes/origin/develop' into chore/elixir-1.11
2020-10-17 17:53:47 +03:00
Mark Felder
b738f70953
Merge branch 'develop' into feature/gen-magic
2020-10-13 10:08:38 -05:00
Mark Felder
64553ebae2
Merge branch 'develop' into chore/elixir-1.11
2020-10-13 09:54:53 -05:00
Alexander Strizhakov
abc3c7689b
HTTPSecurityPlug module name and filename
2020-10-13 16:43:55 +03:00
Alexander Strizhakov
6a87f94ee2
renaming ratelimiter supervisor
2020-10-13 16:38:48 +03:00
Mark Felder
04b514c567
Merge branch 'develop' into feature/gen-magic
2020-10-09 11:26:10 -05:00
Mark Felder
8156940a49
Compatibility with phoenix_pubsub 2.0.0
2020-10-07 13:28:39 -05:00
Mark Felder
70880d54f8
@env is not used
2020-10-07 12:55:16 -05:00
Ivan Tashkinov
7d5c3883ac
[ #3031 ] Refactoring: moved system commands checks to ApplicationRequirements.
2020-09-29 16:28:06 +03:00
Ivan Tashkinov
4e4f771082
Adjusted MediaProxyControllerTest to gracefully fail on missing dependencies. Installation docs update.
...
Added ffmpeg/imagemagick checks to launch checks (if media preview proxy is enabled). Added documentation on installing optional media / graphics packages (imagemagick, ffmpeg, exiftool).
2020-09-26 19:32:16 +03:00
Steven Fuchs
f2ef9735c5
Federate data through persistent websocket connections
2020-09-18 11:58:22 +00:00
Mark Felder
55562ca936
Merge branch 'develop' into feature/gen-magic
2020-09-10 16:05:22 -05:00
Alexander Strizhakov
ee67c98e55
removing Stats worker from Oban cron jobs
2020-09-07 19:16:14 +03:00
Alexander Strizhakov
a11f23c130
user agent if Endpoint is not started yet
2020-09-02 15:45:47 +03:00
rinpatch
dc3a418c27
application.ex: disable warnings_as_errors at runtime
...
see changed files for rationale
2020-09-01 09:08:54 +03:00
Roman Chvanikov
7569f225f1
Move checks to application startup
2020-08-05 19:38:55 +03:00
rinpatch
4128e3a84a
HTTP: Implement max request limits
2020-07-15 15:26:35 +03:00
rinpatch
7738fbbaf5
Connection pool: implement logging and telemetry events
2020-07-15 15:26:25 +03:00
rinpatch
ec9d0d146b
Connection pool: Fix race conditions in limit enforcement
...
Fixes race conditions in limit enforcement by putting worker processes
in a DynamicSupervisor
2020-07-15 15:17:27 +03:00
rinpatch
58a4f350a8
Refactor gun pooling and simplify adapter option insertion
...
This patch refactors gun pooling to use Elixir process registry and
simplifies adapter option insertion.
Having the pool use process registry instead of a GenServer has a number of advantages:
- Simpler code: the initial implementation adds about half the lines of code it deletes
- Concurrency: unlike a GenServer, ETS-based registry can handle multiple checkout/checkin
requests at the same time
- Precise and easy idle connection clousure: current proposal for closing idle connections in
the GenServer-based pool needs to filter through all connections once a minute and compare their
last active time with closing time. With Elixir process registry this can be done
by just using `Process.send_after`/`Process.cancel_timer` in the worker process.
- Lower memory footprint: In my tests `gun-memory-leak` branch uses about 290mb on peak load (250 connections)
and 235mb on idle (5-10 connections). Registry-based pool uses 210mb on idle and 240mb on peak load
2020-07-15 15:17:27 +03:00
rinpatch
e6ccc25565
Fix in-db configuration in dev environment
...
Previously, in-db configuration only worked when `warnings_as_errors`
was disabled because re-compiling scrubbers on application restart
created a warning about module conflicts. This patch fixes that
by enabling `ignore_module_conflict` option of the compiler at runtime,
and enables `warnings_as_errors` in prod since there is no reason
to keep it disabled anymore.
2020-07-14 13:16:54 +03:00
href
4347d2de5e
Config/Docs: Expand behaviour suggestions at runtime
2020-07-12 17:23:33 +02:00
lain
48f98a2748
Merge branch 'mix-tasks-improvement' into 'develop'
...
Mix tasks improvement
See merge request pleroma/pleroma!2723
2020-07-10 08:58:44 +00:00
Mark Felder
d23804f191
Use the Pleroma.Config alias
2020-07-09 10:53:51 -05:00
Alexander Strizhakov
72ad3a66f4
don't fully start pleroma in mix tasks
2020-07-08 15:58:47 +03:00
lain
b7f297f0bd
Merge branch 'issue/1798' into 'develop'
...
[#1798 ] fixed migration the settings to DB
See merge request pleroma/pleroma!2655
2020-06-23 10:54:15 +00:00
Maksim Pechnikov
b0a40fc2e4
added verify RUM settings before start app
2020-06-22 17:27:49 +03:00
Mark Felder
2731ea1334
Change references from "deleted_urls" to "banned_urls" as nothing is handled via media deletions anymore; all actions are manual operations by an admin to ban the url
2020-06-17 13:13:55 -05:00
href
f124f68205
Switch from gen_magic to majic, use Majic.Plug, remove Pleroma.MIME
2020-06-16 15:27:27 +02:00
href
ec1452fd1c
Pleroma.MIME: use gen_magic
2020-06-16 15:15:50 +02:00
Maksim Pechnikov
2e8a236cef
fix invalidates media url's
2020-06-14 21:02:57 +03:00
rinpatch
8308611279
Startup: suggest updating OTP when the version is too low for gun
...
I've seen quite a few people wonder what to do when presented with
this error message.
2020-05-12 17:30:39 +03:00
href
9491ba3e49
Streamer rework
2020-05-07 09:13:32 +00:00
Egor Kislitsyn
39a78998d0
Change Pleroma.CaptchaTest to be a regular module instead of GenServer
2020-04-29 21:26:07 +04:00
ef7d2b0f11
Merge branch 'gun' into 'develop'
...
Gun adapter
Closes #945
See merge request pleroma/pleroma!1861
2020-04-01 17:05:09 +00:00
Alexander Strizhakov
eb9744cade
activities generation tasks
2020-03-30 11:42:25 +03:00
Alexander Strizhakov
39ed608b13
Merge branch 'develop' into gun
2020-03-12 18:31:10 +03:00
Alexander Strizhakov
1306b92997
clean up
2020-03-12 18:29:07 +03:00
Alexander Strizhakov
fce090c1de
using Pleroma.Config instead of ets
2020-03-11 17:22:50 +03:00
Alexander Strizhakov
282a93554f
merging release default config on app start
2020-03-11 16:25:53 +03:00
Alexander Strizhakov
6b2fb9160c
otp version
2020-03-04 09:23:42 +03:00
Alexander Strizhakov
24d1ac125c
hiding raise error logic to otp_version module
2020-03-03 13:24:19 +03:00
Alexander Strizhakov
4c8569d403
otp_version refactor
2020-03-03 12:21:10 +03:00
Alexander Strizhakov
509c81e4b1
Merge branch 'develop' into gun
2020-03-03 10:08:07 +03:00
Haelwenn (lanodan) Monnier
6da6540036
Bump copyright years of files changed after 2020-01-07
...
Done via the following command:
git diff fcd5dd259a
2020-03-02 06:08:45 +01:00
Alexander Strizhakov
514c899275
adding gun adapter
2020-02-18 08:19:01 +03:00
Maksim Pechnikov
2c40c8b4a2
Merge branch 'develop' into issue/1383
2020-02-03 21:42:36 +03:00
Egor Kislitsyn
6302b40791
Warn if HTTPSecurityPlug is disabled
2020-01-28 19:14:09 +04:00
Maksim Pechnikov
b8d5c09f48
Merge branch 'develop' into issue/1383
2020-01-22 09:30:30 +03:00
rinpatch
dc0498ab2b
Check for unapplied migrations on startup
...
Closes #1328
2020-01-20 12:20:26 +03:00
Maksim Pechnikov
54c9db38c8
Merge branch 'develop' into issue/1383
2019-12-21 20:38:59 +03:00
Egor Kislitsyn
c098dec473
Merge branch 'develop' into feature/custom-runtime-modules
2019-12-10 00:16:43 +07:00
Maksim Pechnikov
2b821036c4
Merge branch 'develop' into issue/1383
2019-12-09 19:41:43 +03:00
Egor Kislitsyn
ed92784e7c
Set Logger level to :info in prod
2019-12-09 19:11:54 +07:00
Egor Kislitsyn
84f891ea3e
Add Pleroma.Utils.compile_dir/1
2019-12-09 18:23:07 +07:00
rinpatch
d6c89068f3
HTML: Compile Scrubbers on boot
...
This makes it possible to configure their behavior on OTP releases.
2019-12-08 20:35:41 +03:00
Maksim Pechnikov
5876a9cb79
Merge branch 'develop' into issue/1383
2019-12-08 19:52:46 +03:00
Egor Kislitsyn
a75d4a41e0
Add a test for custom runtime modules
2019-12-06 17:05:09 +07:00
Egor Kislitsyn
e4292cbfad
Use Kernel.inspect/2 to print loaded custom modules
2019-12-06 15:16:39 +07:00
Egor Kislitsyn
157bceeda9
Move runtime configuration from :instance to :modules
2019-12-06 15:04:46 +07:00
Egor Kislitsyn
1216b546c6
Fix credo warning
2019-12-05 20:29:17 +07:00
Egor Kislitsyn
48ae3c4347
Add support for custom modules
2019-12-05 20:18:25 +07:00
lain
aa97fe8f17
ChatChannel: Ignore messages that are too long.
2019-12-04 12:48:34 +01:00
Maksim Pechnikov
b7c449118b
Merge branch 'develop' into issue/1383
2019-11-28 21:27:01 +03:00
Maksim Pechnikov
a4f834a687
moved DigestEmail to Oban Periodic jobs
2019-11-27 15:59:13 +03:00
Maksim Pechnikov
c5766a8100
moved ActivityExpiration to Oban Periodic jobs
2019-11-27 13:36:03 +03:00
Maksim Pechnikov
6f202a401b
moved ScheduledActivity to Oban Periodic jobs
2019-11-27 09:26:37 +03:00
Maksim Pechnikov
0e9be6bafa
moved OAuth.Token.CleanWorker to Oban Periodic jobs
2019-11-25 14:32:20 +03:00
Sadposter
d3656c2725
add ability to set a custom user-agent string
2019-11-25 09:53:11 +00:00
Steven Fuchs
94627baa5c
New rate limiter
2019-11-11 12:13:06 +00:00
Ariadne Conill
25b7ff56c3
application: don't start Federator.init/1 anymore
2019-10-18 14:50:09 +00:00
lain
27110793a7
Merge branch 'featrue/job-monitor' into 'develop'
...
Add Pleroma.JobQueueMonitor
Closes #1274
See merge request pleroma/pleroma!1721
2019-10-04 12:52:11 +00:00
Sergey Suprunenko
c541b83bef
Track failed proxy urls and don't request them again
2019-10-01 20:00:27 +00:00
Egor Kislitsyn
75290cbfbd
Add Pleroma.JobQueueMonitor
2019-09-26 18:49:57 +07:00
kaniini
3ff57f1fd3
Merge branch 'hex-flake-id' into 'develop'
...
Replace `Pleroma.FlakeId` with `flake_id` hex package
See merge request pleroma/pleroma!1692
2019-09-26 02:51:54 +00:00
Egor Kislitsyn
b5dfe83433
Replace Pleroma.FlakeId with flake_id hex package
2019-09-25 17:14:31 +07:00
Ekaterina Vaartis
ee620ecbf1
Add caching for emoji pack sharing
2019-09-19 00:16:33 +03:00
Steven Fuchs
80c5c3495b
remove remaining errors from tests
2019-09-17 14:44:52 +00:00
Egor Kislitsyn
96816ceaa2
Revert "Merge branch 'revert-4fabf83a' into 'develop'"
...
This reverts commit fe7fd331264fabf83ad0
2019-09-16 17:55:04 +07:00
kaniini
c623b4324d
Revert "Merge branch 'streamer-refactoring' into 'develop'"
...
This reverts merge request !1653
2019-09-16 09:09:21 +00:00
Steven Fuchs
aab264db82
Streamer refactoring
2019-09-16 07:44:03 +00:00
Ivan Tashkinov
6bcedb693c
[ #1149 ] Merge remote-tracking branch 'remotes/upstream/develop' into 1149-oban-job-queue
...
# Conflicts:
# docs/config.md
# mix.lock
2019-09-14 16:11:44 +03:00
minibikini
11e12b5761
Add Pleroma.Plugs.Cache
2019-09-09 18:53:08 +00:00
Ivan Tashkinov
b49085c156
[ #1149 ] Refactoring: GenServer workers renamed to daemons, use Oban.Worker moved to helper.
2019-09-02 14:57:40 +03:00
Ivan Tashkinov
a90ea8ba15
[ #1149 ] Addressed code review comments (code style, jobs pruning etc.).
2019-08-31 19:08:56 +03:00
Ivan Tashkinov
e890ea7e82
[ #1149 ] Added Oban job for "activity_expiration". Merged remote-tracking branch 'remotes/upstream/develop' into 1149-oban-job-queue
...
# Conflicts:
# config/config.exs
2019-08-31 14:25:43 +03:00
lain
1692fa8945
ActivityExpirationWorker: Fix merge issues.
2019-08-24 17:22:26 +02:00
lain
cc6c0b4ba6
Merge remote-tracking branch 'origin/develop' into sixohsix/pleroma-post_expiration
2019-08-24 15:48:33 +02:00
Ivan Tashkinov
581123f8bb
[ #1149 ] Introduced quantum job scheduler. Documentation & config changes.
2019-08-23 18:28:23 +03:00
Ivan Tashkinov
c29686309e
[ #1149 ] Upgraded oban from 0.6.0 to 0.7.1.
2019-08-23 09:23:10 +03:00
Ivan Tashkinov
256ff09aa8
[ #1149 ] Merge remote-tracking branch 'remotes/upstream/develop' into 1149-oban-job-queue
...
# Conflicts:
# lib/pleroma/application.ex
# lib/pleroma/scheduled_activity_worker.ex
# lib/pleroma/web/federator/retry_queue.ex
# lib/pleroma/web/oauth/token/clean_worker.ex
# test/user_test.exs
# test/web/federator_test.exs
2019-08-22 20:59:58 +03:00
rinpatch
fba3c16d20
Fix OAuth cleanup worker unconditionally starting
...
!1576 removed enabled/disabled check from the worker, in favor of just
not starting it in application.ex if disabled. However a line
unconditionally starting the worker was removed
2019-08-15 20:36:20 +03:00
Ivan Tashkinov
a180c1360e
[ #1149 ] Oban mailer job. Adjusted tests.
2019-08-14 21:42:21 +03:00
stwf
c43152f6c1
fix formatting
2019-08-14 14:01:11 -04:00
stwf
15ef521009
Isolate OAuth.Token.CleanWorker
2019-08-14 11:57:50 -04:00
