Commit graph

6466 commits

Author SHA1 Message Date
href
ce1a42bd04 Simplify TLS opts
- `verify_fun` is not useful now
- use `customize_check_hostname` (OTP 20+ so OK)
- `partial_chain` is useless as of OTP 21.1 (wasn't there, but hackney/..
uses it)
2020-07-15 15:26:35 +03:00
href
53ba6815b1 parentheses... 2020-07-15 15:26:35 +03:00
href
23d714ed30 Fix race in enforcer/reclaimer start 2020-07-15 15:26:35 +03:00
href
6a0f2bdf8c Ensure connections error get known by the caller 2020-07-15 15:26:35 +03:00
href
46dd276d68 ConnectionPool.Worker: Open gun conn in continue instead of init 2020-07-15 15:26:35 +03:00
rinpatch
a705637dcf Connection Pool: fix LRFU implementation to not actually be LRU
The numbers of the native time unit were so small the CRF was always 1,
making it an LRU. This commit switches the time to miliseconds and changes
the time delta multiplier to the one yielding mostly highest hit rates according
to the paper
2020-07-15 15:26:35 +03:00
rinpatch
9b73c35ca8 Request limiter setup: consider {:error, :existing} a success
When the application restarts (which happens after certain config
changes), the limiters are not destroyed, so `ConcurrentLimiter.new`
will produce {:error, :existing}
2020-07-15 15:26:35 +03:00
rinpatch
12fa5541f0 FollowRedirects: Unconditionally release the connection if there is an error
There is no need for streaming the body if there is no body
2020-07-15 15:26:35 +03:00
rinpatch
37f1e781cb Gun adapter helper: fix wildcard cert issues on OTP 23
See https://bugs.erlang.org/browse/ERL-1260 for more info.

The ssl match function is basically copied from mint, except
that `:string.lowercase/1` was replaced by `:string.casefold`.
It was a TODO in mint's code, so might as well do it since we don't need
to support OTP <20.

Closes 
2020-07-15 15:26:35 +03:00
rinpatch
007843b75e Add documentation for new connection pool settings and remove some
`:retry_timeout` and `:retry` got removed because reconnecting on failure is
something the new pool intentionally doesn't do.

`:max_overflow` had to go in favor of `:max_waiting`, I didn't reuse the key because
the settings are very different in their behaviour.

`:checkin_timeout` got removed in favor of `:connection_acquisition_wait`,
I didn't reuse the key because the settings are somewhat different.

I didn't do any migrations/deprecation warnings/changelog entries because
these settings were never in stable.
2020-07-15 15:26:35 +03:00
rinpatch
7882f28569 Use erlang monotonic time for CRF calculation 2020-07-15 15:26:35 +03:00
rinpatch
00926a63fb Adapter Helper: Use built-in ip address type 2020-07-15 15:26:35 +03:00
rinpatch
4128e3a84a HTTP: Implement max request limits 2020-07-15 15:26:35 +03:00
rinpatch
94c8f3cfaf Use a custom pool-aware FollowRedirects middleware 2020-07-15 15:26:35 +03:00
rinpatch
281ddd5e37 Connection pool: fix connections being supervised by gun_sup 2020-07-15 15:26:35 +03:00
rinpatch
1b15cb066c Connection pool: Add client death tracking
While running this in production I noticed a number of ghost
processes with all their clients dead before they released the connection,
so let's track them to log it and remove them from clients
2020-07-15 15:26:35 +03:00
rinpatch
e94ba05e52 Connection pool: Fix a possible infinite recursion if the pool is exhausted 2020-07-15 15:26:35 +03:00
rinpatch
7738fbbaf5 Connection pool: implement logging and telemetry events 2020-07-15 15:26:25 +03:00
rinpatch
0ffde499b8 Connection Pool: register workers using :via 2020-07-15 15:24:47 +03:00
rinpatch
ec9d0d146b Connection pool: Fix race conditions in limit enforcement
Fixes race conditions in limit enforcement by putting worker processes
in a DynamicSupervisor
2020-07-15 15:17:27 +03:00
rinpatch
d08b157699 Connection pool: check that there actually is a result
Sometimes connections died before being released to the pool, resulting
in MatchErrors
2020-07-15 15:17:27 +03:00
rinpatch
fffbcffb8c Connection Pool: don't enforce pool limits if no new connection needs to be opened 2020-07-15 15:17:27 +03:00
rinpatch
58a4f350a8 Refactor gun pooling and simplify adapter option insertion
This patch refactors gun pooling to use Elixir process registry and
simplifies adapter option insertion.

Having the pool use process registry instead of a GenServer has a number of advantages:
- Simpler code: the initial implementation adds about half the lines of code it deletes
- Concurrency: unlike a GenServer, ETS-based registry can handle multiple checkout/checkin
requests at the same time
- Precise and easy idle connection clousure: current proposal for closing idle connections in
the GenServer-based pool needs to filter through all connections once a minute and compare their
last active time with closing time. With Elixir process registry this can be done
by just using `Process.send_after`/`Process.cancel_timer` in the worker process.
- Lower memory footprint: In my tests `gun-memory-leak` branch uses about 290mb on peak load (250 connections)
and 235mb on idle (5-10 connections). Registry-based pool uses 210mb on idle and 240mb on peak load
2020-07-15 15:17:27 +03:00
Maksim Pechnikov
1dd767b8c7 Include port in host for signatures 2020-07-14 21:44:08 +03:00
Mark Felder
777a7edc6b Lint and fix test to match new log message 2020-07-14 13:15:37 -05:00
Mark Felder
37297a8482 Improve error messages 2020-07-14 13:12:16 -05:00
3f65f2ea79 Merge branch 'feature/1922-media-proxy-whitelist' into 'develop'
Support for hosts with scheme in MediaProxy whitelist setting

Closes 

See merge request 
2020-07-14 18:07:44 +00:00
63798e89de Merge branch 'fix/scrubber-recompile-warnings-as-errors' into 'develop'
Fix in-db configuration in dev environment

See merge request 
2020-07-14 18:05:36 +00:00
2909dc873b Merge branch '1940-admin-token-oauthless-auth' into 'develop'
[] Reinstated OAuth-less `admin_token` authentication

Closes 

See merge request 
2020-07-14 16:48:26 +00:00
Ivan Tashkinov
124b4709dc [] Added admin_token param (as admin_api_params/0) to existing Admin API OpenAPI operations. 2020-07-14 19:31:05 +03:00
246f49d685 Merge branch '1916-transmogrifier-skipping-oddities-in-address-fields' into 'develop'
Transmogrifier: filtering weirdness in address fields

Closes 

See merge request 
2020-07-14 15:17:25 +00:00
rinpatch
e6ccc25565 Fix in-db configuration in dev environment
Previously, in-db configuration only worked when `warnings_as_errors`
was disabled because re-compiling scrubbers on application restart
created a warning about module conflicts. This patch fixes that
by enabling `ignore_module_conflict` option of the compiler at runtime,
and enables `warnings_as_errors` in prod since there is no reason
to keep it disabled anymore.
2020-07-14 13:16:54 +03:00
Haelwenn (lanodan) Monnier
858d9fc7e8
MRF Policies: Return a {:reject, reason} instead of {:reject, nil} 2020-07-14 11:04:16 +02:00
Ivan Tashkinov
9b225db7d8 [] Applied rate limit for requests with bad admin_token. Added doc warnings on admin_token setting. 2020-07-14 11:58:41 +03:00
Ivan Tashkinov
cf3f8cb72a [] Reinstated OAuth-less admin_token authentication. Refactored UserIsAdminPlug (freed from checking admin scopes presence). 2020-07-19 21:35:57 +03:00
=
b221b640a2 Transmogrifier: filtering weirdness in address fields 2020-07-13 22:43:21 +03:00
Mark Felder
80c21100db Merge branch 'develop' into refactor/notification_settings 2020-07-13 13:32:21 -05:00
5d215fd81f Merge branch 'config-behaviours-runtime' into 'develop'
Config/Docs: Expand behaviour suggestions modules at runtime

Closes 

See merge request 
2020-07-13 15:45:25 +00:00
8f9ee694ed Merge branch 'features/profile-fields-emojo' into 'develop'
user: Add support for custom emojis in profile fields

See merge request 
2020-07-13 15:43:46 +00:00
Haelwenn (lanodan) Monnier
ce243b107f
Use Logger.info for {:reject, reason} 2020-07-13 15:26:31 +02:00
Haelwenn (lanodan) Monnier
2aac92e9e0
Transmogrifier.fix_in_reply_to/2: Use warn for non-fatal fail to get replied-to post 2020-07-13 15:26:30 +02:00
Haelwenn
f918b6f86d Merge branch '1937-renaming' into 'develop'
ActivityPub: Don't rename a clashing nickname with the same ap id.

Closes 

See merge request 
2020-07-13 12:06:43 +00:00
href
4347d2de5e Config/Docs: Expand behaviour suggestions at runtime 2020-07-12 17:23:33 +02:00
Alexander Strizhakov
b376442325
MediaProxy whitelist setting now supports hosts with scheme
added deprecation warning about using bare domains
2020-07-12 12:41:40 +03:00
Maksim Pechnikov
62fc8eab0d fix reset confirmation email in admin section 2020-07-11 07:20:35 +03:00
Mark Felder
b329f05ed6 Remove unused @types 2020-07-10 17:08:54 -05:00
Mark Felder
9e45672674 Add a moduledoc 2020-07-10 17:07:28 -05:00
Mark Felder
6167593881 Support Exiftool for stripping EXIF data
We really only want to strip location data anyway, and mogrify strips color profiles.
2020-07-10 16:46:26 -05:00
Mark Felder
3c0c1fd2ef Merge branch 'develop' into issue/1790-updated-oban 2020-07-10 11:34:53 -05:00
ce9514000d Merge branch 'proxy-range-and-chunk' into 'develop'
ReverseProxy: Streaming and disable encoding if Range

Closes  and 

See merge request 
2020-07-10 16:18:28 +00:00
Mark Felder
72b3dbf4d1 Credo line length complaint 2020-07-10 11:04:19 -05:00
href
a1dace088c ReverseProxy: Streaming and disable encoding if Range
Fixes 
Fixes 
2020-07-10 17:23:11 +02:00
lain
93e494ec21 ActivityPub: Don't rename a clashing nickname with the same ap id. 2020-07-10 14:10:44 +02:00
lain
b39eb6ecc5 Merge branch 'develop' of git.pleroma.social:pleroma/pleroma into chat-federation-information 2020-07-10 12:26:53 +02:00
Haelwenn
31a0ed5d01 Merge branch 'admin-api-user-credentials-for-remote-users-fix' into 'develop'
Admin API: fix `GET /api/pleroma/admin/users/:nickname/credentials` returning 404 when getting the credentials of a remote user while `:instance, :limit_to_local_content` is set to `:unauthenticated`

Closes admin-fe#107 and 

See merge request 
2020-07-10 09:22:14 +00:00
lain
48f98a2748 Merge branch 'mix-tasks-improvement' into 'develop'
Mix tasks improvement

See merge request 
2020-07-10 08:58:44 +00:00
Haelwenn
4ac6e6283f Merge branch 'develop' into 'admin-api-user-credentials-for-remote-users-fix'
# Conflicts:
#   CHANGELOG.md
2020-07-10 08:55:15 +00:00
Haelwenn
5ea6387572 Merge branch 'user-search-nickname-rank-boost' into 'develop'
Improved search results for localized nickname match

See merge request 
2020-07-10 08:49:14 +00:00
Haelwenn (lanodan) Monnier
cc7153cd82
user: Add support for custom emojis in profile fields 2020-07-09 19:21:58 +02:00
Mark Felder
2b979cc90c Add AdminFE reports URL to report emails 2020-07-09 11:55:40 -05:00
Mark Felder
d23804f191 Use the Pleroma.Config alias 2020-07-09 10:53:51 -05:00
Mark Felder
49c4e24953 Merge branch 'develop' into fix/csp-for-captcha 2020-07-09 09:08:59 -05:00
lain
8ca1f3e8c6 Merge branch '1723-token-fixes' into 'develop'
AccountController: Return scope in proper format.

Closes 

See merge request 
2020-07-09 13:10:05 +00:00
lain
f4469dc741 Merge branch 'features/favicons' into 'develop'
Add support for remote favicons

See merge request 
2020-07-09 13:09:31 +00:00
Haelwenn
bdc7b60610 Merge branch '1925-muting-reblogs' into 'develop'
AccountController: Fix muting / unmuting reblogs.

Closes 

See merge request 
2020-07-09 12:10:26 +00:00
eugenijm
c2be0da79f Admin API: fix GET /api/pleroma/admin/users/:nickname/credentials returning 404 when getting the credentials of a remote user while :instance, :limit_to_local_content is set to :unauthenticated 2020-07-09 14:57:40 +03:00
Haelwenn
68036f5a3b Merge branch 'follow-pipeline' into 'develop'
Handle `Follow` activities with the pipeline

See merge request 
2020-07-09 10:32:24 +00:00
lain
6d688f14fa Merge branch 'bugfix/status-deletion' into 'develop'
 fix activity deletion

See merge request 
2020-07-09 09:37:52 +00:00
Ivan Tashkinov
123352ffa1 Removed unused trigram index on users. Fixed users_fts_index usage. 2020-07-08 22:50:15 +03:00
lain
00e54f8fe7 ActivityPub: Remove follow and fix issues. 2020-07-08 17:07:24 +02:00
lain
9dda8b5427 CommonAPI: Switch to pipeline for following. 2020-07-08 15:40:56 +02:00
Alexander Strizhakov
b28cc15459
don't restart pleroma in mix tasks 2020-07-08 15:58:47 +03:00
Alexander Strizhakov
72ad3a66f4
don't fully start pleroma in mix tasks 2020-07-08 15:58:47 +03:00
lain
172f4aff8e Transmogrifier: Move following to the pipeline. 2020-07-08 14:30:53 +02:00
lain
29fa75d00d Notification: For follows, notify the followed. 2020-07-08 14:29:29 +02:00
lain
7d0e2b3fdd Merge branch 'develop' of git.pleroma.social:pleroma/pleroma into follow-pipeline 2020-07-08 13:23:07 +02:00
lain
cdc153db31 Merge branch 'develop' of git.pleroma.social:pleroma/pleroma into issue/1880 2020-07-08 13:21:58 +02:00
lain
6335b32aa8 Merge branch '1895-hashtag-timeline-restrict-unauthenticated-fix' into 'develop'
[] Made hashtag timeline respect `:restrict_unauthenticated` instance setting

Closes 

See merge request 
2020-07-08 10:55:03 +00:00
Ivan Tashkinov
704a383055 Improved search results for localized nickname match. Tweaked user search to rank nickname matches higher than name matches. 2020-07-08 13:14:18 +03:00
Ivan Tashkinov
a6495f4a68 [] credo fix. 2020-07-08 12:54:23 +03:00
lain
c0385cf47a AccountController: Fix muting / unmuting reblogs. 2020-07-08 11:52:29 +02:00
Ivan Tashkinov
3f8370a285 [] Applied code review suggestion. 2020-07-08 12:36:44 +03:00
Maksim Pechnikov
e341f81785 fixed delete Like activity in remove user 2020-07-08 09:06:23 +03:00
Haelwenn (lanodan) Monnier
31fef95e35
Add changelog and documentation 2020-07-08 06:29:24 +02:00
Haelwenn (lanodan) Monnier
312fc55f14
Add [:instances_favicons, :enabled] setting, defaults to false 2020-07-08 06:28:40 +02:00
Haelwenn (lanodan) Monnier
8c9df2d2e6
instance: Prevent loop of updates 2020-07-08 06:28:40 +02:00
Haelwenn (lanodan) Monnier
013e2c5057
Use instances table instead of Cachex 2020-07-08 06:28:39 +02:00
Haelwenn (lanodan) Monnier
6a679d80c9
Move get_favicon to Pleroma.Instances, use / 2020-07-08 06:28:39 +02:00
Haelwenn (lanodan) Monnier
f6d09fafee
Add support for remote favicons 2020-07-08 06:28:39 +02:00
Haelwenn
a8447c3803 Merge branch '1918-avatar-background-header-reset-2' into 'develop'
Resolve "Better support for resetting avatar/header/background"

Closes 

See merge request 
2020-07-08 03:25:30 +00:00
Haelwenn
db13acf242 Merge branch 'fix/twitter-rich-media-preview' into 'develop'
Add "Bot" to User Agent to coerce Twitter into serving OGP <meta> tags.

Closes 

See merge request 
2020-07-08 03:05:20 +00:00
lain
8939ff39e4 Merge branch 'remove-twitter-api' into 'develop'
Remove some TwitterAPI endpoints

See merge request 
2020-07-07 18:26:07 +00:00
Ivan Tashkinov
3e08e77151 [] Made hashtag timeline respect :restrict_unauthenticated instance setting. 2020-07-07 20:37:11 +03:00
Mark Felder
18438a9bf0 Add "Bot" to User Agent to coerce Twitter into serving OGP <meta> tags. 2020-07-07 11:21:05 -05:00
lain
c8dd973af5 AccountController: Remove unused update_? routes.
These were not documented and are also not used anymore.
2020-07-07 16:53:20 +02:00
lain
59cf78e412 AccountController: Allow removal / reset of user images. 2020-07-07 16:53:03 +02:00
lain
9ad305209a Merge branch 'bugfix/peertube-videos' into 'develop'
Fix getting videos from peertube

See merge request 
2020-07-07 09:14:50 +00:00
Haelwenn
fa0fa4552f Merge branch 'feature/1392-support-irreversible-filters' into 'develop'
Updates for Feature/1392 support irreversible filters

Closes 

See merge request 
2020-07-07 08:23:49 +00:00
Haelwenn (lanodan) Monnier
fbb9743a70
Fix getting videos from peertube 2020-07-07 09:38:38 +02:00
Haelwenn
d81acb5f62 Merge branch 'description-limits' into 'develop'
Add limits to image descriptions

See merge request 
2020-07-07 03:00:52 +00:00