Commit graph

51 commits

Author SHA1 Message Date
faried nawaz 5be9d13981
a better query to delete from hashtags
old query:

Delete on hashtags  (cost=5089.81..5521.63 rows=6160 width=18)
   ->  Hash Semi Join  (cost=5089.81..5521.63 rows=6160 width=18)
         Hash Cond: (hashtags.id = ht.id)
         ->  Seq Scan on hashtags  (cost=0.00..317.28 rows=17528 width=14)
         ->  Hash  (cost=5012.81..5012.81 rows=6160 width=20)
               ->  Merge Anti Join  (cost=0.70..5012.81 rows=6160 width=20)
                     Merge Cond: (ht.id = hto.hashtag_id)
                     ->  Index Scan using hashtags_pkey on hashtags ht  (cost=0.29..610.53 rows=17528 width=14)
                     ->  Index Scan using hashtags_objects_pkey on hashtags_objects hto  (cost=0.42..3506.48 rows=68158 width=14)

new query:

Delete on hashtags ht  (cost=0.70..5012.81 rows=6160 width=12)
   ->  Merge Anti Join  (cost=0.70..5012.81 rows=6160 width=12)
         Merge Cond: (ht.id = hto.hashtag_id)
         ->  Index Scan using hashtags_pkey on hashtags ht  (cost=0.29..610.53 rows=17528 width=14)
         ->  Index Scan using hashtags_objects_pkey on hashtags_objects hto  (cost=0.42..3506.48 rows=68158 width=14)
2021-05-08 02:00:43 +05:00
faried nawaz a0c9a2b4cc
mix prune_objects: remove unused hashtags after pruning remote objects 2021-05-08 02:00:42 +05:00
Ivan Tashkinov 40d4362261 [#3213] mix pleroma.database rollback tweaks. 2021-02-23 18:11:25 +03:00
Ivan Tashkinov 5992382cf8 Merge remote-tracking branch 'remotes/origin/develop' into feature/object-hashtags-rework
# Conflicts:
#	CHANGELOG.md
#	lib/mix/tasks/pleroma/database.ex
#	lib/pleroma/web/templates/feed/feed/_activity.rss.eex
2021-02-11 19:31:57 +03:00
Ivan Tashkinov d1c6dd97aa [#3213] Partially addressed code review points.
migration rollback task changes, hashtags-related config handling tweaks, `hashtags.data` deletion (unused).
2021-02-07 22:24:12 +03:00
hyperion 8d4e0342e1 Update priv/repo/migrations/20190501125843_add_fts_index_to_objects.exs, priv/repo/optional_migrations/rum_indexing/20190510135645_add_fts_index_to_objects_two.exs files 2021-02-06 09:42:17 +00:00
Ivan Tashkinov 108e90b18e [#3213] Explicitly defined PKs in hashtags_objects and data_migration_failed_ids. Added "pleroma.database rollback" task to revert a single migration. 2021-01-31 22:03:59 +03:00
Ivan Tashkinov e350898828 Merge remote-tracking branch 'remotes/origin/develop' into feature/object-hashtags-rework 2021-01-13 22:11:16 +03:00
Ivan Tashkinov 3e4d84729a [#3213] Prototype of data migrations functionality / HashtagsTableMigrator. 2021-01-13 22:07:38 +03:00
Haelwenn (lanodan) Monnier c4439c630f
Bump Copyright to 2021
grep -rl '# Copyright © .* Pleroma' * | xargs sed -i 's;Copyright © .* Pleroma .*;Copyright © 2017-2021 Pleroma Authors <https://pleroma.social/>;'
2021-01-13 07:49:50 +01:00
Ivan Tashkinov 8c972de045 [#3213] transfer_hashtags mix task refactoring. 2021-01-10 11:44:39 +03:00
Ivan Tashkinov 0d521022fe [#3213] Removed PK from hashtags_objects table. Improved hashtags_transfer mix task (logging of failed ids). 2021-01-07 12:20:29 +03:00
Ivan Tashkinov 367f0c31c3 [#3213] Added query options support for Repo.chunk_stream/4.
Used infinite timeout in transfer_hashtags select query.
2020-12-31 09:36:26 +03:00
Ivan Tashkinov a25c1e8ec0 [#3213] Improved database.transfer_hashtags mix task: proper rollback, speedup. 2020-12-30 14:35:19 +03:00
Haelwenn 3966add048 Revert "Merge branch 'features/hashtag-column' into 'develop'"
This reverts merge request !2824
2020-12-28 12:02:16 +00:00
Haelwenn (lanodan) Monnier d0c2479710
pleroma.database fill_old_hashtags: Add month_limit argument 2020-12-28 11:05:25 +01:00
Ivan Tashkinov cbb19d0e18 [#3213] Hashtag-filtering functions in ActivityPub. Mix task for migrating hashtags to hashtags table. 2020-12-26 22:20:55 +03:00
Haelwenn (lanodan) Monnier acb03d591b
Insert text representation of hashtags into object["hashtags"]
Includes a new mix task: pleroma.database fill_old_hashtags
2020-12-22 05:15:34 +01:00
Alexander Strizhakov cebe3c7def Fix for dropping posts/notifs in WS when mix task is executed
- start oban in mix tasks with empty queues, plugins and crontab
- fix for update_users_following_followers_counts
- fix for removed logo.png
- typo in resend confirmation emails mix task docs
- fix for uploads mix task (start Majic.Pool)
- fix for creating user mix task (start :fast_html app)
2020-12-14 11:02:32 -06:00
Maksim Pechnikov 599f8bb152 RepoStreamer.chunk_stream -> Repo.chunk_stream 2020-09-16 09:47:18 +03:00
Alexander Strizhakov 15aece7238 remove validate_expires_at from enqueue method 2020-09-10 21:52:31 +03:00
Alexander Strizhakov 9bf1065a06 schedule activity expiration in Oban 2020-09-10 21:50:40 +03:00
Alexander Strizhakov 29a7bcd5bb
reverting pinned posts in filtering 2020-08-12 20:01:21 +03:00
Alexander Strizhakov 3ab83f837e
don't load pinned activities in due_expirations 2020-08-12 19:46:47 +03:00
Alexander Strizhakov eec1ba232c
don't expire pinned posts 2020-08-12 15:15:17 +03:00
Mark Felder 724ed354f2 Ensure only Note objects are set to expire 2020-08-11 11:28:22 -05:00
Mark Felder cf4c97242b Ensure we only expire Create activities with the Mix task 2020-08-08 12:40:52 -05:00
Egor Kislitsyn e5557bf8ba
Add mix task to add expiration to all local statuses 2020-08-08 16:29:40 +04:00
Mark Felder 92fba24c74 Alpha sort 2020-05-27 17:17:06 -05:00
Mark Felder 30f96b19c1 Abstract out the database maintenance. I'd like to use this from AdminFE too. 2020-05-27 16:40:51 -05:00
Mark Felder 0d57e06626 Make clearer that this is time and resource consuming 2020-05-27 16:31:37 -05:00
Mark Felder 73ca57e4f1 Make it obvious a full vacuum can take a while 2020-05-27 16:27:29 -05:00
Mark Felder a2f57bd82b Permit easy access to vaccum full and analyze via a mix task 2020-05-27 16:27:07 -05:00
Mark Felder 05da5f5cca Update Copyrights 2020-03-03 16:44:49 -06:00
Egor Kislitsyn 4ea1a61b00
Merge branch 'develop' into refactor/following-relationships 2019-10-21 14:19:15 +07:00
rinpatch 39e996528c Fix a migration wiping user info of users that don't have any mutes
And introduce safe_jsonb_set
2019-10-18 15:22:07 +03:00
Egor Kislitsyn 059005ff82
Replace user.following with Pleroma.FollowingRelationship 2019-10-11 02:35:32 +07:00
rinpatch e00403af23 Mix tasks: derive moduledoc from doc files 2019-10-03 14:00:23 +03:00
rinpatch 6baa037903 Move database maintenance tasks docs to a separate file 2019-10-03 11:17:29 +03:00
Haelwenn (lanodan) Monnier 447514dfa2
Bump copyright years of files changed in 2019
Done via the following command:
git diff 1e6c102bfc --stat --name-only | cat - | xargs sed -i 's/2017-2018 Pleroma Authors/2017-2019 Pleroma Authors/'
2019-09-18 23:21:11 +02:00
Sergey Suprunenko af4cf35e20 Strip internal fields including likes from incoming and outgoing activities 2019-08-10 18:47:40 +00:00
Ariadne Conill b93498eb52 constants: add as_public constant and use it everywhere 2019-07-29 02:43:19 +00:00
rinpatch 8c7a382027 Rename Pleroma.Mix.Tasks.Common -> Mix.Pleroma and import it's functions
instead of aliasing

This seems to be the convention for functions that can be reused between
different mix tasks in all Elixir projects I've seen and it gets rid on
an error message when someone runs mix pleroma.common

Also in this commit by accident:
- Move benchmark task under a proper namespace
- Insert a space after the prompt
2019-06-20 02:08:02 +03:00
rinpatch 54e10a3e55 Disable timeouts for object pruning query 2019-05-22 20:10:52 +03:00
William Pitcock a023ca004c prune objects task: use Repo.delete_all() 2019-05-22 03:12:48 +00:00
William Pitcock 16b260fb19 add mix task to prune the object database using a configured retention period 2019-05-21 21:38:56 +00:00
Sergey Suprunenko 2c303afc8b Remove duplicated entries in users' following lists 2019-05-16 20:04:08 +00:00
Mark Felder efa61c1610 Move to pleroma.database mix task 2019-05-16 13:14:48 -05:00
rinpatch 73b8c5387b Add some logging 2019-04-19 00:17:37 +03:00
rinpatch 945325013a remove query timeouts 2019-04-18 23:58:59 +03:00