Commit graph

199 commits

Author SHA1 Message Date
noellabo
0b8c3df283 Fix status expires feature 2022-05-11 00:48:03 +09:00
noellabo
8029bd9afb Change to remove coalesce for null from conditional expression in statuses.expires_at 2022-05-11 00:48:03 +09:00
noellabo
788289011f Add optimizations to reduce the number of relationships queries 2022-05-11 00:48:03 +09:00
noellabo
4be8f12873 Fix a performance issue with status expiration 2022-05-11 00:48:03 +09:00
noellabo
ae60e0a7d7 Add emoji reaction 2022-05-11 00:48:03 +09:00
noellabo
fc8d4a7a0c Change account header file limit 2MB -> 4BM 2022-05-11 00:48:02 +09:00
noellabo
0d0d754b27 Improved timeline merge for account subscriptions and add option to subscribe to media only 2022-05-11 00:48:02 +09:00
noellabo
c3114f9a5e Add mutual-followers-only visibility 2022-05-11 00:48:02 +09:00
noellabo
a27fcf5e30 Add feature circle
Squashed commit of the following:

commit 7b2ba61c4841e23081552fb79270e4e430dd1fe0
Author: noellabo <noel.yoshiba@gmail.com>
Date:   Sat Sep 5 16:03:52 2020 +0900

    Add the ability to change to a new circle by replying to a circle

commit 7013a228c65c7bd147885de458b50095f3c24334
Author: noellabo <noel.yoshiba@gmail.com>
Date:   Sat Sep 5 16:10:57 2020 +0900

    fixup! add-limited-visibility-icon-to-status

commit 679aa8a7f9bef42ee5d0b326d9ae4925a1999939
Author: noellabo <noel.yoshiba@gmail.com>
Date:   Sat Sep 5 15:12:53 2020 +0900

    Fix 14666

commit b3addd8220d8bb3512ff345b32ca83c714dadd2a
Author: noellabo <noel.yoshiba@gmail.com>
Date:   Sat Sep 5 11:44:12 2020 +0900

    Add Japanese translation for circle

commit b7f4b773a0cd554084d5ad6a5923adb06b3acfc4
Author: noellabo <noel.yoshiba@gmail.com>
Date:   Sat Sep 5 11:40:12 2020 +0900

    Squashed commit of the following:

    commit b85a4685b27c49462288aba5f38723b91e936c4a
    Author: noellabo <noel.yoshiba@gmail.com>
    Date:   Sat Sep 5 10:50:03 2020 +0900

        Changed to remove restrictions on privacy options and allow users to switch circles when replying

    commit 0a8c0140c73d7c5333e4f8017964adb5061a7cf1
    Author: noellabo <noel.yoshiba@gmail.com>
    Date:   Sat Sep 5 09:33:07 2020 +0900

        Change limited visibility icon

    commit b64adf19788d828249408454ec6afa9beb3d4872
    Author: noellabo <noel.yoshiba@gmail.com>
    Date:   Mon Aug 31 06:50:56 2020 +0900

        Fix a change to limited-visibility-bearcaps replies

    commit ed361405b5e38857a2f42b0515a599ddcdd412cf
    Author: noellabo <noel.yoshiba@gmail.com>
    Date:   Thu Aug 27 15:53:18 2020 +0900

        Fix composer text when change visibility

    commit 4da3adddb6ffde43070d743e34c5b56e06579b30
    Author: noellabo <noel.yoshiba@gmail.com>
    Date:   Sat Aug 22 22:34:23 2020 +0900

        Fix wrong circle_id when changing visibility

    commit 752d7fc2a3c9e34fab9993d767f83c6eae7ba55a
    Author: noellabo <noel.yoshiba@gmail.com>
    Date:   Sun Aug 9 13:12:51 2020 +0900

        Add circle reply and redraft

    commit 5978bc04a24695edce6717bda89dcf6f861ef2c4
    Author: noellabo <noel.yoshiba@gmail.com>
    Date:   Mon Jul 27 01:07:52 2020 +0900

        Fix remove unused props

    commit 7970f69676c24b4aa9385fee8b1635c46ba52fcd
    Author: noellabo <noel.yoshiba@gmail.com>
    Date:   Sun Jul 26 21:17:07 2020 +0900

        Separate circle choice from privacy

    commit 36f6a684c0b0c895d4d0f1b9d09b05c91b104666
    Author: noellabo <noel.yoshiba@gmail.com>
    Date:   Thu Jul 23 10:54:25 2020 +0900

        Add UI for posting to circles

    commit 7ef48003c1407275663dd603b124d292db2aa93a
    Author: noellabo <noel.yoshiba@gmail.com>
    Date:   Fri Jul 24 12:55:10 2020 +0900

        Fix silent mention by circle

commit 7a1caed49333c3d3241301afb77639cdf1cabdc0
Author: noellabo <noel.yoshiba@gmail.com>
Date:   Sat Sep 5 11:38:10 2020 +0900

    Squashed commit of the following:

    commit dca71fab86c830932ca760b7d8b3f89cc25c453e
    Author: noellabo <noel.yoshiba@gmail.com>
    Date:   Sat Sep 5 09:31:26 2020 +0900

        Revert "Add focus setting when opening the circle column"

        This reverts commit 3a93ac99312a13b68b7edc2b81313fb0ffb7bcdc.

    commit 0a1bc8307bb699c7eb3024072ce14a440df1fc87
    Author: noellabo <noel.yoshiba@gmail.com>
    Date:   Sat Sep 5 09:31:11 2020 +0900

        Change limited visibility icon

    commit 9784f8b562f6592e9d9190ca29d2b2e870006d10
    Author: noellabo <noel.yoshiba@gmail.com>
    Date:   Thu Aug 13 21:52:07 2020 +0900

        Add focus setting when opening the circle column

    commit a84f680c167fab9276550850c60f9108d251144e
    Author: noellabo <noel.yoshiba@gmail.com>
    Date:   Thu Aug 13 15:55:27 2020 +0900

        Fix message

    commit e3f11c4adac57b6e6a15c981ed6f4721a1634212
    Author: noellabo <noel.yoshiba@gmail.com>
    Date:   Mon Jul 27 01:01:23 2020 +0900

        Fix light-theme

    commit d7d96eda5b86d3e3f654ce79888e7cf5aa535db5
    Author: noellabo <noel.yoshiba@gmail.com>
    Date:   Sun Jul 26 21:50:56 2020 +0900

        Fix circles loading in share page and followers search

    commit 10b821f7b8c0a87cea3df51f09deeadc2cb40b32
    Author: noellabo <noel.yoshiba@gmail.com>
    Date:   Fri Jul 24 14:08:00 2020 +0900

        Refactor list items

    commit e020072915572ce409039ccf799d08f8d8b5b393
    Author: noellabo <noel.yoshiba@gmail.com>
    Date:   Thu Jul 23 20:15:38 2020 +0900

        Fixed a bug that circle name change is not reflected in the list

    commit 735bc41161b4c09a8dafe2c0064096b3ca79f2a0
    Author: noellabo <noel.yoshiba@gmail.com>
    Date:   Wed Jul 22 08:49:47 2020 +0900

        Add UI for managing circle members

    commit d7c3145b8fa84be0631bf7f41bb229f3e6d03ff1
    Author: noellabo <noel.yoshiba@gmail.com>
    Date:   Wed Jul 22 07:34:52 2020 +0900

        Add the followers option to AccountSearchSercive

    commit 65e2b0c4299b72ede440b50089c1bd6afa6c9c05
    Author: noellabo <noel.yoshiba@gmail.com>
    Date:   Wed Jul 22 07:05:56 2020 +0900

        Add CircleSerializer

commit a639e1803abf5590068846dbe98bc5edfaa2ad82
Author: noellabo <noel.yoshiba@gmail.com>
Date:   Sat Sep 5 11:37:30 2020 +0900

    Squashed commit of the following:

    commit 9cb3fb9d980e3ee066083076f508c5ab1447176a
    Author: noellabo <noel.yoshiba@gmail.com>
    Date:   Sat Sep 5 07:15:19 2020 +0900

        Move the link to the mention list to the menu

    commit b32dd87b43f4e09b8e2c437f1fb5d3ebd6221215
    Author: noellabo <noel.yoshiba@gmail.com>
    Date:   Sat Sep 5 00:56:12 2020 +0900

        Change limited visibility icon

    commit 8db0d024119d1c2cef8de849f2501496a166a2dd
    Author: noellabo <noel.yoshiba@gmail.com>
    Date:   Tue Sep 1 01:42:13 2020 +0900

        Fix to disallow getting the list of mentions in limited replies

    commit 490a9d65a59a3dd0d86e81f6780e879dc4313dff
    Author: noellabo <noel.yoshiba@gmail.com>
    Date:   Fri Jul 24 11:36:24 2020 +0900

        Add column to list mentioned accounts of limited status

    commit 62a423ac2729c16f26fafe111f257bc373218df2
    Author: noellabo <noel.yoshiba@gmail.com>
    Date:   Thu Jul 23 13:30:17 2020 +0900

        Fix visibility compatibility more

    commit a5cfa54b259054f41e89037f299fa928a2361818
    Author: noellabo <noel.yoshiba@gmail.com>
    Date:   Mon Jul 20 05:39:49 2020 +0900

        Fix visibility compatibility

    commit 7900ca5650c77565b86ddc594a221dfa3b5321b4
    Author: noellabo <noel.yoshiba@gmail.com>
    Date:   Mon Jul 20 02:01:27 2020 +0900

        Add limited visibility icon to status

commit 66b83965ef068e9ee8c940249c68bcbde15731fe
Author: Eugen Rochko <eugen@zeonfederated.com>
Date:   Wed Aug 26 03:16:47 2020 +0200

    Add conversation-based forwarding for limited visibility statuses through bearcaps

commit 561abc65e0ace89318b3952047025b8d98515fbb
Author: Eugen Rochko <eugen@zeonfederated.com>
Date:   Sun Jul 19 02:05:16 2020 +0200

    Add REST API for managing and posting to circles

    Circles are the conceptual opposite of lists. A list is a subdivision
    of your follows, a circle is a subdivision of your followers. Posting
    to a circle means making content available to only some of your
    followers. Circles have been internally supported in Mastodon for
    the purposes of federation since #8950, this adds the REST API
    necessary for making use of them in Mastodon itsef.
2022-05-11 00:48:02 +09:00
noellabo
2d5a103176 Add group own post to group timeline 2022-05-11 00:48:02 +09:00
noellabo
110421c8a0 Add group timeline 2022-05-11 00:48:02 +09:00
noellabo
8e45e9bb82 Add favorite domains 2022-05-11 00:48:01 +09:00
Marek Lipka
145c0ccd2a Prevent stripping color profiles from JPEGs 2021-08-12 07:10:15 +09:00
noellabo
2b7d1d9941 Add show followed_by to WebUI 2021-08-12 07:10:15 +09:00
noellabo
f40706037b Add availability of follow-up delivery 2021-08-12 07:10:15 +09:00
noellabo
3bcbbb7463 Add hashtag follow and account subscribe support to list 2021-08-12 07:10:13 +09:00
noellabo
9fdfc2d8d5 Add account subscribe support to WebUI 2021-08-12 07:08:36 +09:00
noellabo
92a9a23eb6 Add subscribe features 2021-08-12 07:08:36 +09:00
noellabo
fd5b7c14fa Add favorite hashtags 2021-08-12 07:08:36 +09:00
Claire
5efb1ff337
Fix followers synchronization mechanism not working when URI has empty path (#16510)
* Fix followers synchronization mechanism not working when URI has empty path

To my knowledge, there is no current implementation on the fediverse
that can use bare domains (e.g., actor is at https://example.org instead of
something like https://example.org/actor) that also plans to support the
followers synchronization mechanism. However, Mastodon's current implementation
would exclude such accounts from followers list.

Also adds tests and rename them to reflect the proper method names.

* Move url prefix regexp to its own constant
2021-08-11 17:48:42 +02:00
Claire
4ac78e2a06
Add feature to automatically delete old toots (#16529)
* Add account statuses cleanup policy model

* Record last inspected toot to delete to speed up successive calls to statuses_to_delete

* Add service to cleanup a given account's statuses within a budget

* Add worker to go through account policies and delete old toots

* Fix last inspected status id logic

All existing statuses older or equal to last inspected status id must be
kept by the current policy. This is an invariant that must be kept so that
resuming deletion from the last inspected status remains sound.

* Add tests

* Refactor scheduler and add tests

* Add user interface

* Add support for discriminating based on boosts/favs

* Add UI support for min_reblogs and min_favs, rework UI

* Address first round of review comments

* Replace Snowflake#id_at_start with with_random parameter

* Add tests

* Add tests for StatusesCleanupController

* Rework settings page

* Adjust load-avoiding mechanisms

* Please CodeClimate
2021-08-09 23:11:50 +02:00
Claire
763ab0c7eb
Fix owned account notes not being deleted when an account is deleted (#16579)
* Add account_notes relationship

* Add tests

* Fix owned account notes not being deleted when an account is deleted

* Add post-migration to clean up orphaned account notes
2021-08-08 15:29:57 +02:00
Eugen Rochko
d174d12c83
Add authentication history (#16408) 2021-06-21 17:07:30 +02:00
Claire
526332c545
Fix account deletion sometimes failing because of optimistic locks (#16317)
* Fix account deletion sometimes failing because of optimistic locks

In some rare occasions[1], deleting accounts would fail with a
`StaleObjectError` exception.

Indeed, account deletion manually sets the `AccountStat` values without
handling cases where the optimistic locking on `AccountStat` would fail.

To my knowledge, with the rewrite of account counters in #15913, the
`DeleteAccountService` is now the only place that changes the counters in
a way that is not atomic.

Since in this specific case, we do not care about the previous values of the
account counters, it appears we don't need locking at all for this table
anymore.

[1]: https://discourse.joinmastodon.org/t/account-cant-be-deleted/3602

* Bump MAX_SUPPORTED_VERSION in maintenance script
2021-06-02 17:41:25 +02:00
Claire
76064e6608
Update fix-duplicates maintenance script to support latest migrations (#16231)
* Update maintenance script to support latest database migrations

* Update Account#merge_with!
2021-05-12 23:19:44 +02:00
Takeshi Umeda
c403c3695b
Fix to be able to redownload avatar and header (#16190)
* Fix to reset if header and avatar download fails

* Add RedownloadAvatarWorker and RedownloadHeaderWorker
2021-05-11 14:19:22 +02:00
Eugen Rochko
fab65848d2
Fix empty home feed before first follow has finished processing (#16152)
Change queue of merge worker from pull to default
2021-05-04 04:45:08 +02:00
Eugen Rochko
f7117646af
Add cold-start follow recommendations (#15945) 2021-04-12 12:37:14 +02:00
Claire
cbd0ee1d07
Update Mastodon to Rails 6.1 (#15910)
* Update devise-two-factor to unreleased fork for Rails 6 support

Update tests to match new `rotp` version.

* Update nsa gem to unreleased fork for Rails 6 support

* Update rails to 6.1.3 and rails-i18n to 6.0

* Update to unreleased fork of pluck_each for Ruby 6 support

* Run "rails app:update"

* Add missing ActiveStorage config file

* Use config.ssl_options instead of removed ApplicationController#force_ssl

Disabled force_ssl-related tests as they do not seem to be easily testable
anymore.

* Fix nonce directives by removing Rails 5 specific monkey-patching

* Fix fixture_file_upload deprecation warning

* Fix yield-based test failing with Rails 6

* Use Rails 6's index_with when possible

* Use ActiveRecord::Cache::Store#delete_multi from Rails 6

This will yield better performances when deleting an account

* Disable Rails 6.1's automatic preload link headers

Since Rails 6.1, ActionView adds preload links for javascript files
in the Links header per default.

In our case, that will bloat headers too much and potentially cause
issues with reverse proxies. Furhermore, we don't need those links,
as we already output them as HTML link tags.

* Switch to Rails 6.0 default config

* Switch to Rails 6.1 default config

* Do not include autoload paths in the load path
2021-03-24 10:44:31 +01:00
Eugen Rochko
af8fe6e1e9
WIP (#15222) 2021-03-19 17:15:36 +01:00
Claire
741d0952b1
Improve account counters handling (#15913)
* Improve account counters handling

* Use ActiveRecord::Base::sanitize_sql to pass values instead of interpolating them

Keep using string interpolation for `key` as it is safe and using
“ActiveRecord::Base::sanitize_sql_hash_for_assignment” would require stitching
bits of SQL in a way that is not more easily checked for safety.

* Add migration hook to catch PostgreSQL versions earlier than 9.5
2021-03-19 13:14:57 +01:00
Claire
1b02d29be5
Fix not being able to change world filter expiration back to “Never” (#15858)
Fixes #15849
2021-03-12 05:25:50 +01:00
Claire
acdeb162b8
Create instance actor if it hasn't been properly seeded (#15693)
An uncommon but somewhat difficult to digagnose issue is dealing with
improperly-seeded databases. In such cases, instance-signed fetches will
fail with a ActiveRecord::RecordNotFound error, usually caught and handled
as generic 404, leading people to think the remote resource itself has not
been found, while it's the local instance actor that does not exist.

This commit changes the code so that failure to find the instance actor
automatically creates a new one, so that improperly-seeded databases do
not cause any issue.
2021-02-09 18:12:54 +01:00
luigi
eb51e43fb4
Optimize some regex matching (#15528)
* Use Regex#match?

* Replace =~ too

* Avoid to call match? from Nil

* Keep value of Regexp.last_match
2021-01-22 10:09:08 +01:00
luigi
087ed84367
Optimize map { ... }.compact calls (#15513)
* Optimize map { ... }.compact

using Enumerable#filter_map, supported since Ruby 2.7

* Add poyfill for Enumerable#filter_map
2021-01-10 00:32:01 +01:00
ThibG
f1f96ebf02
Fix being able to import more than allowed number of follows (#15384)
* Fix being able to import more than allowed number of follows

Without this commit, if someone tries importing a second list of accounts to
follow before the first one has been processed, this will queue imports for
the two whole lists, even if they exceed the account's allowed number of
outgoing follows.

This commit changes it so the individual queued imports aren't exempt from
the follow limit check (they remain exempt from the rate-limiting check
though).

* Catch validation errors to not re-queue failed follows

Co-authored-by: Claire <claire.github-309c@sitedethib.com>
2020-12-26 23:52:46 +01:00
Eugen Rochko
216b85b053
Fix performance on instances list in admin UI (#15282)
- Reduce duplicate queries
- Remove n+1 queries
- Add accounts count to detailed view
- Add separate action log entry for updating existing domain blocks
2020-12-14 09:06:34 +01:00
ThibG
fbe7519243
Fix account merging in maintenance script (#15264)
Also include AccountNote and AccountDeletionRequest to the list of classes
needing to be reassigned the merged account.
2020-12-04 02:23:51 +01:00
ThibG
9136be480f
Fix followers hash cache not being invalidated on account merge (#15256)
Also clear relationships cache.
2020-12-02 21:20:00 +01:00
Eugen Rochko
f844386809
Add tootctl accounts merge (#15201)
* Add `tootctl accounts merge`

* Update lib/mastodon/accounts_cli.rb

Co-authored-by: Yamagishi Kazutoshi <ykzts@desire.sh>

Co-authored-by: Yamagishi Kazutoshi <ykzts@desire.sh>
2020-11-23 17:50:16 +01:00
ThibG
ca56527140
Add follower synchronization mechanism (#14510)
* Add support for followers synchronization on the receiving end

Check the `collectionSynchronization` attribute on `Create` and `Announce`
activities and synchronize followers from provided collection if possible.

* Add tests for followers synchronization on the receiving end

* Add support for follower synchronization on the sender's end

* Add tests for the sending end

* Switch from AS attributes to HTTP header

Replace the custom `collectionSynchronization` ActivityStreams attribute by
an HTTP header (`X-AS-Collection-Synchronization`) with the same syntax as
the `Signature` header and the following fields:
- `collectionId` to specify which collection to synchronize
- `digest` for the SHA256 hex-digest of the list of followers known on the
   receiving instance (where “receiving instance” is determined by accounts
   sharing the same host name for their ActivityPub actor `id`)
- `url` of a collection that should be fetched by the instance actor

Internally, move away from the webfinger-based `domain` attribute and use
account `uri` prefix to group accounts.

* Add environment variable to disable followers synchronization

Since the whole mechanism relies on some new preconditions that, in some
extremely rare cases, might not be met, add an environment variable
(DISABLE_FOLLOWERS_SYNCHRONIZATION) to disable the mechanism altogether and
avoid followers being incorrectly removed.

The current conditions are:
1. all managed accounts' actor `id` and inbox URL have the same URI scheme and
   netloc.
2. all accounts whose actor `id` or inbox URL share the same URI scheme and
   netloc as a managed account must be managed by the same Mastodon instance
   as well.

As far as Mastodon is concerned, breaking those preconditions require extensive
configuration changes in the reverse proxy and might also cause other issues.

Therefore, this environment variable provides a way out for people with highly
unusual configurations, and can be safely ignored for the overwhelming majority
of Mastodon administrators.

* Only set follower synchronization header on non-public statuses

This is to avoid unnecessary computations and allow Follow-related
activities to be handled by the usual codepath instead of going through
the synchronization mechanism (otherwise, any Follow/Undo/Accept activity
would trigger the synchronization mechanism even if processing the activity
itself would be enough to re-introduce synchronization)

* Change how ActivityPub::SynchronizeFollowersService handles follow requests

If the remote lists a local follower which we only know has sent a follow
request, consider the follow request as accepted instead of sending an Undo.

* Integrate review feeback

- rename X-AS-Collection-Synchronization to Collection-Synchronization
- various minor refactoring and code style changes

* Only select required fields when computing followers_hash

* Use actor URI rather than webfinger domain in synchronization endpoint

* Change hash computation to be a XOR of individual hashes

Makes it much easier to be memory-efficient, and avoid sorting discrepancy issues.

* Marginally improve followers_hash computation speed

* Further improve hash computation performances by using pluck_each
2020-10-21 18:04:09 +02:00
OSAMU SATO
96761752ec
Add duration parameter to muting. (#13831)
* Adding duration to muting.

* Remove useless checks
2020-10-13 01:01:14 +02:00
Eugen Rochko
5e1364c448
Add IP-based rules (#14963) 2020-10-12 16:33:49 +02:00
Eugen Rochko
974b1b79ce
Add option to be notified when a followed user posts (#13546)
* Add bell button

Fix #4890

* Remove duplicate type from post-deployment migration

* Fix legacy class type mappings

* Improve query performance with better index

* Fix validation

* Remove redundant index from notifications
2020-09-18 17:26:45 +02:00
Eugen Rochko
ed099d8bdc
Change account suspensions to be reversible by default (#14726) 2020-09-15 14:37:58 +02:00
tateisu
fa0c71f0d9
allow pagination by min_id and max_id (#14776)
* allow pagination by min_id and max_id

* also AccountConversation allows min_id,max_id pair

* also home,list TL allows min_id,max_id pair
2020-09-12 17:09:49 +02:00
Akihiko Odaki
ae871c4d46
Make Array-creation behavior of Paginable more predictable (#14687)
* Make Array-creation behavior of Paginable more predictable

Paginable.paginate_by_id usually returns ActiveRecord::Relation, but it
returns an Array if min_id option is present. The behavior caused problems
fixed with the following commits:
- 552e886b64
- b63ede5005
- 64ef37b89d

To prevent from recurring similar problems, this commit introduces two
changes:
- The scope now always returns an Array whether min_id option is present
  or not.
- The scope is renamed to to_a_paginated_by_id to clarify it returns an
  Array.

* Transform Paginable.to_a_paginated_by_id from a scope to a class method

https://api.rubyonrails.org/classes/ActiveRecord/Scoping/Named/ClassMethods.html#method-i-scope
> The method is intended to return an ActiveRecord::Relation object, which
> is composable with other scopes.

Paginable.to_a_paginated_by_id returns an Array and is not appropriate
as a scope.
2020-08-31 12:47:09 +02:00
Takeshi Umeda
7540e235a2
Fix movie width and frame_rate returning nil (#14357)
* Fix movie width and frame_rate returning nil

* Add StreamValidationError and raise

* Fix code style
2020-07-19 22:28:27 +02:00
abcang
6a96af4d20
Fix rubocop warning (#14288)
* Fix rubocop warning

* use limit variable

* use ContextCreatingMethods option
2020-07-14 19:05:07 +02:00
Eugen Rochko
7aaf2b44ec
Fix remote files not using Content-Type header, streaming (#14184) 2020-06-30 23:58:02 +02:00