Commit graph

700 commits

Author SHA1 Message Date
someone
cc4f20b130
mix pleroma.database set_text_search_config now runs concurrently and infinitely
Backport of: https://git.pleroma.social/pleroma/pleroma/-/merge_requests/3514
2021-08-28 18:29:43 +02:00
someone
4b940e441a mix pleroma.database set_text_search_config now runs concurrently and infinitely 2021-08-15 13:49:12 -04:00
sadposter
e56c319e8f merge 2021-06-27 22:31:37 +01:00
bc51dea425 Update lib/mix/tasks/pleroma/database.ex 2021-06-07 20:02:28 +00:00
faried nawaz
5be9d13981
a better query to delete from hashtags
old query:

Delete on hashtags  (cost=5089.81..5521.63 rows=6160 width=18)
   ->  Hash Semi Join  (cost=5089.81..5521.63 rows=6160 width=18)
         Hash Cond: (hashtags.id = ht.id)
         ->  Seq Scan on hashtags  (cost=0.00..317.28 rows=17528 width=14)
         ->  Hash  (cost=5012.81..5012.81 rows=6160 width=20)
               ->  Merge Anti Join  (cost=0.70..5012.81 rows=6160 width=20)
                     Merge Cond: (ht.id = hto.hashtag_id)
                     ->  Index Scan using hashtags_pkey on hashtags ht  (cost=0.29..610.53 rows=17528 width=14)
                     ->  Index Scan using hashtags_objects_pkey on hashtags_objects hto  (cost=0.42..3506.48 rows=68158 width=14)

new query:

Delete on hashtags ht  (cost=0.70..5012.81 rows=6160 width=12)
   ->  Merge Anti Join  (cost=0.70..5012.81 rows=6160 width=12)
         Merge Cond: (ht.id = hto.hashtag_id)
         ->  Index Scan using hashtags_pkey on hashtags ht  (cost=0.29..610.53 rows=17528 width=14)
         ->  Index Scan using hashtags_objects_pkey on hashtags_objects hto  (cost=0.42..3506.48 rows=68158 width=14)
2021-05-08 02:00:43 +05:00
faried nawaz
a0c9a2b4cc
mix prune_objects: remove unused hashtags after pruning remote objects 2021-05-08 02:00:42 +05:00
sadposter
5f62b55a6f Merge branch 'develop' of https://git.pleroma.social/pleroma/pleroma into develop 2021-03-23 23:01:07 +00:00
Alexander Strizhakov
4cd34d0197
suggestion 2021-03-23 17:27:02 +03:00
Alexander Strizhakov
03843a5386
migrating config to tmp folder 2021-03-23 14:29:02 +03:00
sadposter
292ddf730c Merge branch 'develop' of https://git.pleroma.social/pleroma/pleroma into develop 2021-03-03 13:51:07 +00:00
Ivan Tashkinov
882dd46843 Merge remote-tracking branch 'remotes/origin/develop' into feature/object-hashtags-rework 2021-03-02 08:26:30 +03:00
2217b35bd4 Merge branch 'ecto-rollback-in-test-env' into 'develop'
Possibility to run rollback in test env

See merge request pleroma/pleroma!3212
2021-02-26 16:47:53 +00:00
Ivan Tashkinov
40d4362261 [#3213] mix pleroma.database rollback tweaks. 2021-02-23 18:11:25 +03:00
sadposter
5f33699169 Merge branch 'develop' of https://git.pleroma.social/pleroma/pleroma into develop 2021-02-20 15:27:56 +00:00
Ivan Tashkinov
5992382cf8 Merge remote-tracking branch 'remotes/origin/develop' into feature/object-hashtags-rework
# Conflicts:
#	CHANGELOG.md
#	lib/mix/tasks/pleroma/database.ex
#	lib/pleroma/web/templates/feed/feed/_activity.rss.eex
2021-02-11 19:31:57 +03:00
rinpatch
f13f5d9303 OpenAPI spec task: Load pleroma application to get version info
For whatever reason it seems to pick up the version without loading the
app on my machine, but not on the CI.
2021-02-09 22:10:09 +03:00
593c0851d9 Merge branch 'develop' into 'new-user-emails' 2021-02-08 16:01:50 -06:00
Ivan Tashkinov
d1c6dd97aa [#3213] Partially addressed code review points.
migration rollback task changes, hashtags-related config handling tweaks, `hashtags.data` deletion (unused).
2021-02-07 22:24:12 +03:00
rinpatch
6e68058b63 Merge branch 'feat/allow_alt_text_search_config' into 'develop'
allow user defined text search config in database

See merge request pleroma/pleroma!3275
2021-02-06 09:42:20 +00:00
hyperion
8d4e0342e1 Update priv/repo/migrations/20190501125843_add_fts_index_to_objects.exs, priv/repo/optional_migrations/rum_indexing/20190510135645_add_fts_index_to_objects_two.exs files 2021-02-06 09:42:17 +00:00
af37a5c51a Also make this maybe_ for consistency 2021-02-04 14:33:49 -06:00
rinpatch
e2d7c4fd57 Merge branch 'feat/openapi-spec-export' into 'develop'
Improve OpenAPI spec and deploy it to api.pleroma.social

See merge request pleroma/pleroma!3296
2021-02-03 13:21:19 +00:00
rinpatch
9fcff7851f Improve OpenAPI spec and deploy it to api.pleroma.social
Too many changes in OpenAPI spec to describe each one, but
basically it is tag fixes, bringing consitency to operation summaries
and fixing some incorrect information.
2021-02-03 16:07:44 +03:00
Ivan Tashkinov
4e14945670 Merge remote-tracking branch 'remotes/origin/develop' into feature/object-hashtags-rework
# Conflicts:
#	CHANGELOG.md
#	lib/pleroma/web/activity_pub/activity_pub.ex
2021-02-03 09:31:38 +03:00
Ivan Tashkinov
108e90b18e [#3213] Explicitly defined PKs in hashtags_objects and data_migration_failed_ids. Added "pleroma.database rollback" task to revert a single migration. 2021-01-31 22:03:59 +03:00
aaceb042c5 fix format 2021-01-28 10:20:25 +00:00
1547a2fda4 mix: instance: ensure all needed folders are created before generating config 2021-01-28 09:39:53 +00:00
Ivan Tashkinov
694d98be55 Merge remote-tracking branch 'remotes/origin/develop' into feature/object-hashtags-rework 2021-01-25 21:24:07 +03:00
f868dcf3ac Ensure consistent behavior between pleroma.user activate/deactivate mix tasks 2021-01-25 10:48:28 -06:00
5e8da27e14 Provide pleroma.user mix task for both activate and deactivate 2021-01-25 10:36:51 -06:00
537ba1c5e0 Merge branch 'develop' into refactor/deactivated_user_field 2021-01-25 09:45:06 -06:00
sadposter
3aa25b008d Merge branch 'develop' of https://git.pleroma.social/pleroma/pleroma into develop 2021-01-23 22:38:07 +00:00
Ivan Tashkinov
2634a16b4c Merge remote-tracking branch 'remotes/origin/develop' into feature/object-hashtags-rework
# Conflicts:
#	CHANGELOG.md
2021-01-21 20:20:35 +03:00
2905df841b Merge branch 'update/oban' into 'develop'
Update Oban to 2.3.4

See merge request pleroma/pleroma!3256
2021-01-20 20:32:23 +00:00
28581e03ad Merge branch 'develop' into refactor/deactivated_user_field 2021-01-18 14:58:21 -06:00
Alexander Strizhakov
0e48c80d7f
start oban app in migrations and mix tasks 2021-01-17 09:58:41 +03:00
Mark Felder
2c0fe2ea9e Remove toggle_confirmation; require explicit state change
Also cosmetic changes to make the code clearer
2021-01-15 13:11:51 -06:00
Mark Felder
d36182c088 Change user.confirmation_pending field to user.is_confirmed 2021-01-15 12:44:41 -06:00
Mark Felder
ebd7d1365b Make setting user activation status explicit 2021-01-15 11:37:17 -06:00
Mark Felder
7516660753 Remove toggle_activation 2021-01-15 11:25:32 -06:00
Mark Felder
860b5c7804 Change user.deactivated field to user.is_active 2021-01-15 11:24:46 -06:00
Ivan Tashkinov
e350898828 Merge remote-tracking branch 'remotes/origin/develop' into feature/object-hashtags-rework 2021-01-13 22:11:16 +03:00
Ivan Tashkinov
3e4d84729a [#3213] Prototype of data migrations functionality / HashtagsTableMigrator. 2021-01-13 22:07:38 +03:00
Haelwenn (lanodan) Monnier
c4439c630f
Bump Copyright to 2021
grep -rl '# Copyright © .* Pleroma' * | xargs sed -i 's;Copyright © .* Pleroma .*;Copyright © 2017-2021 Pleroma Authors <https://pleroma.social/>;'
2021-01-13 07:49:50 +01:00
Ivan Tashkinov
8c972de045 [#3213] transfer_hashtags mix task refactoring. 2021-01-10 11:44:39 +03:00
Ivan Tashkinov
0d521022fe [#3213] Removed PK from hashtags_objects table. Improved hashtags_transfer mix task (logging of failed ids). 2021-01-07 12:20:29 +03:00
Ivan Tashkinov
367f0c31c3 [#3213] Added query options support for Repo.chunk_stream/4.
Used infinite timeout in transfer_hashtags select query.
2020-12-31 09:36:26 +03:00
Ivan Tashkinov
a25c1e8ec0 [#3213] Improved database.transfer_hashtags mix task: proper rollback, speedup. 2020-12-30 14:35:19 +03:00
sadposter
6cd0586ef9 Merge branch 'develop' of https://git.pleroma.social/pleroma/pleroma into develop 2020-12-29 13:16:14 +00:00
Haelwenn
3966add048 Revert "Merge branch 'features/hashtag-column' into 'develop'"
This reverts merge request !2824
2020-12-28 12:02:16 +00:00
Haelwenn (lanodan) Monnier
d0c2479710
pleroma.database fill_old_hashtags: Add month_limit argument 2020-12-28 11:05:25 +01:00
Ivan Tashkinov
cbb19d0e18 [#3213] Hashtag-filtering functions in ActivityPub. Mix task for migrating hashtags to hashtags table. 2020-12-26 22:20:55 +03:00
Alexander Strizhakov
538af14d52
possibility to run rollback in test env 2020-12-22 17:55:30 +03:00
Haelwenn (lanodan) Monnier
acb03d591b
Insert text representation of hashtags into object["hashtags"]
Includes a new mix task: pleroma.database fill_old_hashtags
2020-12-22 05:15:34 +01:00
Alex Gleason
80891e83d8
Merge remote-tracking branch 'upstream/develop' into registration-workflow 2020-12-17 09:05:36 -06:00
lain
8b8c317c0f Merge branch 'fix/2336-mastoapi-ws-posts-dropping' into 'develop'
Fix mastoapi ws posts dropping

Closes #2336

See merge request pleroma/pleroma!3198
2020-12-16 11:35:45 +00:00
Alexander Strizhakov
cebe3c7def Fix for dropping posts/notifs in WS when mix task is executed
- start oban in mix tasks with empty queues, plugins and crontab
- fix for update_users_following_followers_counts
- fix for removed logo.png
- typo in resend confirmation emails mix task docs
- fix for uploads mix task (start Majic.Pool)
- fix for creating user mix task (start :fast_html app)
2020-12-14 11:02:32 -06:00
rinpatch
7133c0c5ea instance.gen: Warn that stripping exif requires exiftool
And default to no if it is not installed

Closes #2343
2020-12-12 20:38:51 +03:00
rinpatch
6aece536eb instance.gen task: Only show files which will be actually overwritten 2020-12-12 20:35:38 +03:00
lain
ed76323776 Merge branch 'feature/configdb-mix-tasks' into 'develop'
Add mix tasks to give additional ConfigDB recovery and debugging options

See merge request pleroma/pleroma!3174
2020-12-07 18:06:06 +00:00
Mark Felder
93428d7c11 Print out settings that will be removed when specifying the group and key for consistency
Fix error message when specified key doesn't exist
2020-12-07 11:45:56 -06:00
Mark Felder
61494b5245 Formatting 2020-12-07 11:22:07 -06:00
d817bae802 Apply 1 suggestion(s) to 1 file(s) 2020-12-07 17:13:29 +00:00
Ivan Tashkinov
e00c667145 [#3174] Refactoring: ConfigDB fetching functions, ConfigDB tests.
Minor fixes.
2020-12-06 18:02:30 +03:00
Mark Felder
50aadc3d5c shell_yes?/1 was not showing the correct message and always defaults to yes which is dangerous 2020-12-04 11:49:56 -06:00
Mark Felder
9dfda37821 More compact representation 2020-12-04 11:37:49 -06:00
Mark Felder
3bf5c5b015 Ensure deleting entire group prints out settings that will be removed before actually removing them 2020-12-04 11:30:48 -06:00
Mark Felder
696d39c3dc Fix deleting an entire group. Also utilize Pleroma.ConfigDB.delete/1 2020-12-04 11:19:58 -06:00
Mark Felder
685e5c8509 Use Pleroma.ConfigDB.delete/1 instead of rolling our own 2020-12-04 11:09:13 -06:00
Mark Felder
4aad066091 Use Enum.any? to ensure we return true if there are results 2020-12-04 11:04:53 -06:00
Alexander Strizhakov
a02eb88396
config_db search methods 2020-12-03 19:34:23 +03:00
Mark Felder
7fd4f4908b dump_key/2 no longer used 2020-12-03 10:03:44 -06:00
60c4ac0f70 Apply 6 suggestion(s) to 1 file(s) 2020-12-03 16:03:14 +00:00
Mark Felder
95e908e4e2 Credo 2020-12-03 09:58:24 -06:00
Mark Felder
16bdc2bcd0 Make the --force flag for reset command consistent with the others and deduplicate db truncation 2020-12-02 16:34:23 -06:00
Mark Felder
e379ab8277 Add --force flag for delete and reset commands
Bunch of reorganization and consolidation
2020-12-02 16:24:32 -06:00
Mark Felder
20a911f9f7 Add comment for this mysterious behavior 2020-12-02 14:25:07 -06:00
Mark Felder
25fab7da69 No need for a separate functions here 2020-12-02 14:20:11 -06:00
Mark Felder
13947999ad Use a callback strategy to short circuit the functions and print a nice error 2020-12-02 12:33:34 -06:00
Mark Felder
a7b5280b5b Centralize check that configdb is enabled which now raises an exception 2020-12-02 10:43:22 -06:00
Mark Felder
53a5ec1952 Left public during debugging 2020-12-02 10:43:22 -06:00
Mark Felder
5135a8189f Use inspect instead of faking the output 2020-12-02 10:43:22 -06:00
Mark Felder
6334ba9ad3 Remove unnecessary keys command 2020-12-02 10:43:22 -06:00
Mark Felder
d0cb73527f Ensure scary warning starts on a new line 2020-12-02 10:43:22 -06:00
Mark Felder
0847e3e496 Print whole config when resetting and include a scary looking message. 2020-12-02 10:43:22 -06:00
Mark Felder
d4320e0daf Both are really atoms 2020-12-02 10:43:22 -06:00
Mark Felder
4bdfcf1682 Transform strings to atoms for all cases, including when the atom is a module like Pleroma.Emails.Mailer 2020-12-02 10:43:22 -06:00
Mark Felder
3df115b2b0 Support atoms and strings as args to the mix task
Improve output. Show the user what will be deleted before the prompt.
2020-12-02 10:43:22 -06:00
Mark Felder
ae7d37de06 Fix deletion regression due to strings instead of atoms
Improve message after successful deletion
2020-12-02 10:43:22 -06:00
Mark Felder
c6a0ca2213 Improve dumping groups and specific keys; add prompts for delete and reset 2020-12-02 10:43:22 -06:00
Mark Felder
67437feafc Support listing groups, listing keys in a group, and dumping the config based on group or specific key in that group 2020-12-02 10:43:22 -06:00
Mark Felder
a51da3c1d8 Sort output by group
Not the best sorting, but better than nothing.
2020-12-02 10:43:22 -06:00
Mark Felder
ada073f251 Rename keys to groups 2020-12-02 10:43:22 -06:00
Mark Felder
e8a4062d9d Document how to delete individual configuration groups and completely reset the config without SQL 2020-12-02 10:43:22 -06:00
Mark Felder
a82ba66662 Better deletion message 2020-12-02 10:43:22 -06:00
Mark Felder
6dcc36baa9 Add mix tasks to give additional recovery and debugging options
- pleroma.config dump: prints the entire config as it would be exported to the filesystem
- pleroma.config dump KEY: prints the configuration under a specific ConfigDB key in the database
- pleroma.config keylist: lists the available keys in ConfigDB
- pleroma.config keydel KEY: deletes ConfigDB entry stored under the key

This should prevent the need for users to manually execute SQL queries.
2020-12-02 10:43:22 -06:00
Mark Felder
751712d970 Prevent mix tasks from spewing any internal logging unless DEBUG is in the env
e.g., DEBUG=1 mix pleroma.config migrate_from_db
2020-11-25 12:52:39 -06:00
Alex Gleason
30ed7b502f
Merge remote-tracking branch 'upstream/develop' into registration-workflow 2020-11-21 10:23:53 -06:00
Alexander Strizhakov
0a5b22bc3b
start limiters in mix tasks 2020-11-20 11:37:01 +03:00
a1056aace1 Merge branch 'fix/mix-task-user-delete_activities' into 'develop'
Fix mix pleroma.user delete_activities task

See merge request pleroma/pleroma!3156
2020-11-18 18:57:47 +00:00
Mark Felder
97201f7e37 Need to start web_resp cache or mix task fails 2020-11-17 15:15:53 +00:00
Egor Kislitsyn
f69fe36ebf
Merge branch 'develop' into frontend-admin-api 2020-11-17 18:34:04 +04:00
sadposter
7947236575 Merge branch 'develop' of https://git.pleroma.social/pleroma/pleroma into develop 2020-11-15 19:40:23 +00:00
Alex Gleason
9546c1444c
Merge remote-tracking branch 'upstream/develop' into registration-workflow 2020-11-14 19:48:47 -06:00
Haelwenn (lanodan) Monnier
e2f573d68b
pleroma.instance: Fix Exiftool module name 2020-11-14 22:31:01 +01:00
Egor Kislitsyn
08cbd655d1
Merge branch 'develop' into frontend-admin-api 2020-11-11 19:53:40 +04:00
rinpatch
cc45c69fff Remove release_env
While taking a final look at instance.gen before releasing I noticed
that the release_env task outputs messages in broken english. Upon
further inspection it seems to have even more severe issues which, in
my opinion, warrant it's at least temporary removal:
- We do not explain what it actually does, anywhere. Neither the task
 docs nor instance.gen, nor installation instructions.
- It does not respect FHS on OTP releases (uses /opt/pleroma/config even
 though we store the config in /etc/pleroma/config.exs).
- It doesn't work on OTP releases, which is the main reason it exists.
Neither systemd nor openrc service files for OTP include it.
- It is not mentioned in install guides other than the ones for Debian
and OTP releases.
2020-11-08 11:56:09 +03:00
sadposter
ddf36b096e Merge branch 'develop' of https://git.pleroma.social/pleroma/pleroma into develop 2020-11-01 12:22:28 +00:00
Egor Kislitsyn
03e306785b
Add an API endpoint to install a new frontend 2020-10-27 19:20:21 +04:00
Maksim Pechnikov
2030ffd490 fix test 2020-10-15 22:31:52 +03:00
Maksim Pechnikov
cf53e300f8 added generate the release env to pleroma.instance gen 2020-10-15 22:31:00 +03:00
Maksim
e2333f757a Apply 1 suggestion(s) to 1 file(s) 2020-10-15 22:29:28 +03:00
Maksim
499df7b73a Apply 1 suggestion(s) to 1 file(s) 2020-10-15 22:29:18 +03:00
Maksim Pechnikov
14054cd004 update task messages 2020-10-15 22:28:49 +03:00
Maksim Pechnikov
20e68b30f0 added generated pleroma.env 2020-10-15 22:28:26 +03:00
Alex Gleason
dc38dc8472
Replace User.toggle_confirmation/1 with User.confirm/1, fixes #2235 2020-10-13 22:01:27 -05:00
Mark Felder
409f694e4f Merge branch 'develop' into refactor/locked_user_field 2020-10-13 09:54:29 -05:00
Mark Felder
9968b7efed Change user.locked field to user.is_locked 2020-10-13 09:31:13 -05:00
Alexander Strizhakov
6bf85440b3
mix tasks consistency 2020-10-13 16:33:24 +03:00
03855f9789 Merge branch 'improve/instance-gen' into 'develop'
#2159 Support enabling upload filters during instance gen

See merge request pleroma/pleroma!3070
2020-10-12 17:24:24 +00:00
Mark Felder
8539e386c3 Add missing Copyright headers 2020-10-12 12:00:50 -05:00
Mark Felder
6c61ef14c3 Support enabling upload filters during instance gen 2020-10-12 11:19:48 -05:00
sadposter
a9938611f7 Merge branch 'develop' of https://git.pleroma.social/pleroma/pleroma into develop 2020-10-02 15:09:47 +01:00
Maksim Pechnikov
de993b856b added force option to the unfollow operation 2020-09-28 09:16:42 +03:00
Mark Felder
8b84ca4901 Simplify the value comparison 2020-09-25 10:37:59 -05:00
Mark Felder
81faf540b3 Add some user feedback 2020-09-24 18:35:20 -05:00
Mark Felder
e33360fdb9 Cannot use batches with User.try_send_confirmation_email/1 2020-09-24 18:23:47 -05:00
Mark Felder
935ef21b02 Use the import 2020-09-24 16:47:34 -05:00
Mark Felder
8af8eb5ce7 Chase Pleroma.RepoStreamer.chunk_stream -> Pleroma.Repo.chunk_stream 2020-09-23 12:32:47 -05:00
Mark Felder
f3a1f9c3bb Merge branch 'develop' into feature/bulk-confirmation 2020-09-23 11:56:22 -05:00
Mark Felder
f7e40f7ef1 Deny ConfigDB migration when deprecated settings found 2020-09-17 09:32:50 -05:00
Maksim Pechnikov
599f8bb152 RepoStreamer.chunk_stream -> Repo.chunk_stream 2020-09-16 09:47:18 +03:00
sadposter
95a1e5bf5a Merge branch 'develop' of https://git.pleroma.social/pleroma/pleroma into develop 2020-09-15 17:21:18 +01:00
Alexander Strizhakov
15aece7238 remove validate_expires_at from enqueue method 2020-09-10 21:52:31 +03:00
Alexander Strizhakov
9bf1065a06 schedule activity expiration in Oban 2020-09-10 21:50:40 +03:00
Alexander Strizhakov
ab56dd54e7
use Pleroma.HTTP in emoji packs tasks 2020-09-10 11:23:39 +03:00
sadposter
d0ffefc270 Merge branch 'develop' of https://git.pleroma.social/pleroma/pleroma into develop 2020-09-09 09:35:48 +01:00
Mark Felder
d23d0c27c2 Handle possibility of user account in a bulk operation not having an email address 2020-09-08 16:48:54 -05:00
Mark Felder
75b6fef25d Add mix task for bulk [un]confirming the local instance users 2020-09-08 16:39:41 -05:00
Mark Felder
23ca5f75af Make it possible to bulk send confirmation emails to all unconfirmed users 2020-09-08 16:39:08 -05:00
Alexander Strizhakov
696bf09433
passing adapter options directly without adapter key 2020-09-07 19:59:17 +03:00
Alexander Strizhakov
a83916fdac
adapter options unification
not needed options deletion
2020-09-07 19:59:17 +03:00
Alexander Strizhakov
ee67c98e55
removing Stats worker from Oban cron jobs 2020-09-07 19:16:14 +03:00
sadposter
1c44d476e7 Merge branch 'develop' of https://git.pleroma.social/pleroma/pleroma into develop 2020-09-07 09:57:00 +01:00
lain
f26b580e80 Merge branch 'develop' of git.pleroma.social:pleroma/pleroma into seanking/pleroma-fix_install_fe_bug 2020-09-03 11:29:39 +02:00
Alexander Strizhakov
79f65b4374
correct pool and uniform headers format 2020-09-02 09:16:51 +03:00
Sean King
0a9c63fb43 Fix frontend install mix task bug 2020-09-01 12:20:32 -06:00
sadposter
7ac5f210ea Merge branch 'develop' of https://git.pleroma.social/pleroma/pleroma into develop 2020-08-30 02:05:25 +01:00
lain
c1d51944c7 Merge branch 'develop' of git.pleroma.social:pleroma/pleroma into frontend-bundles-downloads 2020-08-25 11:44:51 +02:00
d39abd02ac Merge branch 'relay-fix-admin-fe' into 'develop'
Relay fix for admin-fe

See merge request pleroma/pleroma!2902
2020-08-24 17:03:18 +00:00