Commit graph

721 commits

Author SHA1 Message Date
David Yip
42e8c8eb0e Merge tag 'v2.0.0' into gs-master 2017-10-18 11:52:04 -05:00
aschmitz
554c2fd8af Clean up reblog tracking keys, related improvements (#5428)
* Clean up reblog-tracking sets from FeedManager

Builds on #5419, with a few minor optimizations and cleanup of sets
after they are no longer needed.

* Update tests, fix multiply-reblogged case

Previously, we would have lost the fact that a given status was
reblogged if the displayed reblog of it was removed, now we don't.

Also added tests to make sure FeedManager#trim cleans up our reblog
tracking keys, fixed up FeedCleanupScheduler to use the right loop,
and fixed the test for it.
2017-10-17 11:45:06 +02:00
Ondřej Hruška
8b9ee5f16b
Merge remote-tracking branch 'upstream/master' into upstream-merge-again 2017-10-16 21:13:47 +02:00
Eugen Rochko
34118169ac Keep references to all reblogs of a status on home feed (#5419)
* Keep references to all reblogs of a status on home feed

When inserting reblog: Add to set of reblogs of this status on
the feed, if original status was present in the feed, add it to
that set as well.

When removing a reblog: Remove it from that set. Take random
remaining item from the set. If one exists, re-insert it into feed,
otherwise do not re-insert anything.

Fix #4210

* When original is removed, toss out reblog references
2017-10-16 20:44:31 +02:00
David Yip
dbb1fce94d Merge remote-tracking branch 'upstream/master' into gs-master 2017-10-16 09:23:59 -05:00
unarist
6f490b4bfe Fix un-reblogged status being at wrong position in the home timeline (#5418)
We've changed un-reblogging behavior when we implement Snowflake, to insert un-reblogged status at the position reblogging status existed.

However, our API expects home timeline is ordered by status ids, and max_id/since_id filters by zset score. Due to this, un-reblogged status appears as a last item of result set, and timeline expansion may skips many statuses.

So this reverts that change...reblogged status inserted at corresponding position to its id.
2017-10-16 15:58:23 +02:00
Yamagishi Kazutoshi
d5b767c374 Replace JavaScript Testing Framework from Mocha to Jest (#5412) 2017-10-16 09:33:08 +02:00
David Yip
6cd5b3bbe5 Merge remote-tracking branch 'origin/master' into gs-master 2017-10-16 01:29:02 -05:00
unarist
8125fdc19f Use atomUri in Undo activity of Announce (#5376)
This allows deletion of reblogs which delivered before with OStatus URI.
2017-10-14 14:42:09 +02:00
Eugen Rochko
3283868e28 Improve spec of Feed and UserTrackingConcern (#5367) 2017-10-13 11:00:11 +02:00
Lex Alexander
b8bae96647 Retoot count increases without reason (#5363)
* Retoot count increases without reason

-The store_uri method for Statuses was being called on after_create and causing reblogs to be incremented twice.
-This calls it when the transaction is finished by using after_create_commit.
-Fixes #4916.

* Added test case for after_create_commit callback for checking reblog count.

* Rewrote test to keep original, but added one for only the after_create_commit callback.
2017-10-13 02:52:09 +02:00
kibigo!
8d6b9ba494 Merge upstream 2.0ish #165 2017-10-11 10:43:10 -07:00
Akihiko Odaki
cc796298c9 Fix pagination in Api::V1::BlocksController (#5285) 2017-10-09 17:30:31 +02:00
Eugen Rochko
0717d9b3e6 Set snowflake IDs for backdated statuses (#5260)
- Rename Mastodon::TimestampIds into Mastodon::Snowflake for clarity
- Skip for statuses coming from inbox, aka delivered in real-time
- Skip for statuses that claim to be from the future
2017-10-08 17:34:34 +02:00
nullkal
633426b261 Add moderation note (#5240)
* Add moderation note

* Add frozen_string_literal

* Make rspec pass
2017-10-07 20:26:43 +02:00
Eugen Rochko
f486ef2666 Redesign public hashtag pages (#5237) 2017-10-07 20:00:35 +02:00
Eugen Rochko
3a3475450e Encode custom emojis as resolveable objects in ActivityPub (#5243)
* Encode custom emojis as resolveable objects in ActivityPub

* Improve code style
2017-10-07 17:43:42 +02:00
Nolan Lawson
057db0ecd0 Update emoji-mart to v2.1.1 (#5256) 2017-10-07 12:17:02 +02:00
Nolan Lawson
72d939b69f Fix thinking_face emoji autocomplete (#5238) 2017-10-06 12:03:13 +02:00
Ondřej Hruška
dcd8ff5308
fix haml quotes in a bid to make travis less grumpy 2017-10-06 09:07:04 +02:00
Nolan Lawson
fd7f0732fe Compress and combine emoji data (#5229) 2017-10-06 03:42:34 +02:00
ThibG
2559d9166c Fix regression in FetchRemoteResourceService (#5217)
* Fix regression in FetchRemoteResourceService

* Update specs to match interface changes made in #5114
2017-10-05 00:21:44 +02:00
utam0k
b3af3f9f8c Implement EmailBlackList (#5109)
* Implement BlacklistedEmailDomain

* Use Faker::Internet.domain_name

* Remove note column

* Add frozen_string_literal comment

* Delete unnecessary codes

* Sort alphabetically

* Change of wording

* Rename BlacklistedEmailDomain to EmailDomainBlock
2017-10-04 15:16:10 +02:00
Yamagishi Kazutoshi
178f718a9b Separate notifications preferences from general preferences (#4447)
* Separate notifications preferences from general preferences

* Refine settings/notifications/show

* remove preferences.notifications
2017-10-04 10:22:52 +02:00
aschmitz
468523f4ad Non-Serial ("Snowflake") IDs (#4801)
* Use non-serial IDs

This change makes a number of nontrivial tweaks to the data model in
Mastodon:

* All IDs are now 8 byte integers (rather than mixed 4- and 8-byte)
* IDs are now assigned as:
  * Top 6 bytes: millisecond-resolution time from epoch
  * Bottom 2 bytes: serial (within the millisecond) sequence number
  * See /lib/tasks/db.rake's `define_timestamp_id` for details, but
    note that the purpose of these changes is to make it difficult to
    determine the number of objects in a table from the ID of any
    object.
* The Redis sorted set used for the feed will have values used to look
  up toots, rather than scores. This is almost always the same as the
  existing behavior, except in the case of boosted toots. This change
  was made because Redis stores scores as double-precision floats,
  which cannot store the new ID format exactly. Note that this doesn't
  cause problems with sorting/pagination, because ZREVRANGEBYSCORE
  sorts lexicographically when scores are tied. (This will still cause
  sorting issues when the ID gains a new significant digit, but that's
  extraordinarily uncommon.)

Note a couple of tradeoffs have been made in this commit:

* lib/tasks/db.rake is used to enforce many/most column constraints,
  because this commit seems likely to take a while to bring upstream.
  Enforcing a post-migrate hook is an easier way to maintain the code
  in the interim.
* Boosted toots will appear in the timeline as many times as they have
  been boosted. This is a tradeoff due to the way the feed is saved in
  Redis at the moment, but will be handled by a future commit.

This would effectively close Mastodon's #1059, as it is a
snowflake-like system of generating IDs. However, given how involved
the changes were simply within Mastodon, it may have unexpected
interactions with some clients, if they store IDs as doubles
(or as 4-byte integers). This was a problem that Twitter ran into with
their "snowflake" transition, particularly in JavaScript clients that
treated IDs as JS integers, rather than strings. It therefore would be
useful to test these changes at least in the web interface and popular
clients before pushing them to all users.

* Fix JavaScript interface with long IDs

Somewhat predictably, the JS interface handled IDs as numbers, which in
JS are IEEE double-precision floats. This loses some precision when
working with numbers as large as those generated by the new ID scheme,
so we instead handle them here as strings. This is relatively simple,
and doesn't appear to have caused any problems, but should definitely
be tested more thoroughly than the built-in tests. Several days of use
appear to support this working properly.

BREAKING CHANGE:

The major(!) change here is that IDs are now returned as strings by the
REST endpoints, rather than as integers. In practice, relatively few
changes were required to make the existing JS UI work with this change,
but it will likely hit API clients pretty hard: it's an entirely
different type to consume. (The one API client I tested, Tusky, handles
this with no problems, however.)

Twitter ran into this issue when introducing Snowflake IDs, and decided
to instead introduce an `id_str` field in JSON responses. I have opted
to *not* do that, and instead force all IDs to 64-bit integers
represented by strings in one go. (I believe Twitter exacerbated their
problem by rolling out the changes three times: once for statuses, once
for DMs, and once for user IDs, as well as by leaving an integer ID
value in JSON. As they said, "If you’re using the `id` field with JSON
in a Javascript-related language, there is a very high likelihood that
the integers will be silently munged by Javascript interpreters. In most
cases, this will result in behavior such as being unable to load or
delete a specific direct message, because the ID you're sending to the
API is different than the actual identifier associated with the
message." [1]) However, given that this is a significant change for API
users, alternatives or a transition time may be appropriate.

1: https://blog.twitter.com/developer/en_us/a/2011/direct-messages-going-snowflake-on-sep-30-2011.html

* Restructure feed pushes/unpushes

This was necessary because the previous behavior used Redis zset scores
to identify statuses, but those are IEEE double-precision floats, so we
can't actually use them to identify all 64-bit IDs. However, it leaves
the code in a much better state for refactoring reblog handling /
coalescing.

Feed-management code has been consolidated in FeedManager, including:

* BatchedRemoveStatusService no longer directly manipulates feed zsets
* RemoveStatusService no longer directly manipulates feed zsets
* PrecomputeFeedService has moved its logic to FeedManager#populate_feed

(PrecomputeFeedService largely made lots of calls to FeedManager, but
didn't follow the normal adding-to-feed process.)

This has the effect of unifying all of the feed push/unpush logic in
FeedManager, making it much more tractable to update it in the future.

Due to some additional checks that must be made during, for example,
batch status removals, some Redis pipelining has been removed. It does
not appear that this should cause significantly increased load, but if
necessary, some optimizations are possible in batch cases. These were
omitted in the pursuit of simplicity, but a batch_push and batch_unpush
would be possible in the future.

Tests were added to verify that pushes happen under expected conditions,
and to verify reblog behavior (both on pushing and unpushing). In the
case of unpushing, this includes testing behavior that currently leads
to confusion such as Mastodon's #2817, but this codifies that the
behavior is currently expected.

* Rubocop fixes

I could swear I made these changes already, but I must have lost them
somewhere along the line.

* Address review comments

This addresses the first two comments from review of this feature:

https://github.com/tootsuite/mastodon/pull/4801#discussion_r139336735
https://github.com/tootsuite/mastodon/pull/4801#discussion_r139336931

This adds an optional argument to FeedManager#key, the subtype of feed
key to generate. It also tests to ensure that FeedManager's settings are
such that reblogs won't be tracked forever.

* Hardcode IdToBigints migration columns

This addresses a comment during review:
https://github.com/tootsuite/mastodon/pull/4801#discussion_r139337452

This means we'll need to make sure that all _id columns going forward
are bigints, but that should happen automatically in most cases.

* Additional fixes for stringified IDs in JSON

These should be the last two. These were identified using eslint to try
to identify any plain casts to JavaScript numbers. (Some such casts are
legitimate, but these were not.)

Adding the following to .eslintrc.yml will identify casts to numbers:

~~~
  no-restricted-syntax:
  - warn
  - selector: UnaryExpression[operator='+'] > :not(Literal)
    message: Avoid the use of unary +
  - selector: CallExpression[callee.name='Number']
    message: Casting with Number() may coerce string IDs to numbers
~~~

The remaining three casts appear legitimate: two casts to array indices,
one in a server to turn an environment variable into a number.

* Only implement timestamp IDs for Status IDs

Per discussion in #4801, this is only being merged in for Status IDs at
this point. We do this in a migration, as there is no longer use for
a post-migration hook. We keep the initialization of the timestamp_id
function as a Rake task, as it is also needed after db:schema:load (as
db/schema.rb doesn't store Postgres functions).

* Change internal streaming payloads to stringified IDs as well

This is equivalent to 591a9af356faf2d5c7e66e3ec715502796c875cd from
#5019, with an extra change for the addition to FeedManager#unpush.

* Ensure we have a status_id_seq sequence

Apparently this is not a given when specifying a custom ID function,
so now we ensure it gets created. This uses the generic version of this
function to more easily support adding additional tables with timestamp
IDs in the future, although it would be possible to cut this down to a
less generic version if necessary. It is only run during db:schema:load
or the relevant migration, so the overhead is extraordinarily minimal.

* Transition reblogs to new Redis format

This provides a one-way migration to transition old Redis reblog entries
into the new format, with a separate tracking entry for reblogs.

It is not invertible because doing so could (if timestamp IDs are used)
require a database query for each status in each users' feed, which is
likely to be a significant toll on major instances.

* Address review comments from @akihikodaki

No functional changes.

* Additional review changes

* Heredoc cleanup

* Run db:schema:load hooks for test in development

This matches the behavior in Rails'
ActiveRecord::Tasks::DatabaseTasks.each_current_configuration, which
would otherwise break `rake db:setup` in development.

It also moves some functionality out to a library, which will be a good
place to put additional related functionality in the near future.
2017-10-04 09:56:37 +02:00
Akihiko Odaki
63f0979799 Validate id of ActivityPub representations (#5114)
Additionally, ActivityPub::FetchRemoteStatusService no longer parses
activities.
OStatus::Activity::Creation no longer delegates to ActivityPub because
the provided ActivityPub representations are not signed while OStatus
representations are.
2017-10-04 01:13:48 +02:00
ThibG
dfaa219f88 Fix HTTP responses for salmon and ActivityPub inbox processing (#5200)
* Return sensible HTTP status for ActivityPub inbox processing

* Return sensible HTTP status for salmon slap processing

* Return additional information to debug signature verification failures
2017-10-03 23:21:19 +02:00
Nolan Lawson
813c5f2f52 Add spec for emoji_index_light.js (#5199) 2017-10-03 20:54:38 +02:00
Nolan Lawson
334a446313 Fix emoji sequence bug in substring-trie (#5191)
Fixes #5188
2017-10-03 13:11:22 +02:00
Eugen Rochko
47ecd652d3 Make Chrome splash screen same color as web UI's background color (#5169) 2017-10-02 01:23:32 +02:00
Akihiko Odaki
cdacac8c6c Fix order of paginated accounts in FollowerDomainsController and spec (#3357)
* Fix order of paginated accounts in FollowerDomainsController

Unordered pagination could result in unexpected behavior.

* Cover Settings::FollowerDomainsController more
2017-09-30 23:06:09 +02:00
Eugen Rochko
eb605141ff Fix #5104 - GET /api/v1/apps/verify_credentials to confirm app works (#5112) 2017-09-30 22:05:42 +02:00
Eugen Rochko
ebb8c89207 Upgrade to React 16 (#5119)
* Upgrade to React 16.0.0

* Disable some uncritical tests while chai-enzyme remains incompatible
2017-09-30 04:29:56 +02:00
Eugen Rochko
f4ca116ea8 After 7 days of repeated delivery failures, give up on inbox (#5131)
- A successful delivery cancels it out
- An incoming delivery from account of the inbox cancels it out
2017-09-29 03:16:20 +02:00
Eugen Rochko
887cd94e96 Increase attachment descriptions to 420 characters (#5139)
Blaze it
2017-09-29 02:30:00 +02:00
Eugen Rochko
4ec1771165 Add ability to specify alternative text for media attachments (#5123)
* Fix #117 - Add ability to specify alternative text for media attachments

- POST /api/v1/media accepts `description` straight away
- PUT /api/v1/media/:id to update `description` (only for unattached ones)
- Serialized as `name` of Document object in ActivityPub
- Uploads form adjusted for better performance and description input

* Add tests

* Change undo button blend mode to difference
2017-09-28 15:31:31 +02:00
Ondřej Hruška
83bda6c1a8
Merge commit 'b9f59ebcc68e9da0a7158741a1a2ef3564e1321e' into merging-upstream 2017-09-28 09:18:35 +02:00
Ondřej Hruška
9330ea1f4d
Merge commit '4aea3f88a6d30f102a79c2da7fcfac96465ba1a8' into merging-upstream 2017-09-28 09:12:17 +02:00
Eugen Rochko
e528114c53 Follow-up to #4582 and #5027, removing dead code (#5101) 2017-09-26 01:06:27 +02:00
Eugen Rochko
cf7fbf2c56 Fix #5059 - Stop processing payload if it's from local account (#5100) 2017-09-26 01:06:13 +02:00
Eugen Rochko
510df0ac55 Fix emojify() by generating a mapping to existing Twemoji files (#5080)
A new rake task emojis:generate downloads a full list of valid
unicode sequences from unicode.org and checks it against existing
Twemoji files, finally generating a map from each sequence to the
existing file (e.g. when there's multiple ways an emoji can be
expressed). The map is dumped into app/javascript/mastodon/emoji_map.json

That file is loaded by emojione_light.js (now a misnomer) which
decorates it further with shortcodes taken from emoji-mart's index.
2017-09-25 18:36:33 +02:00
Akihiko Odaki
98936bfcdf Add missing validations in ActivityPub::Activity::Create (#5096) 2017-09-25 18:33:11 +02:00
Eugen Rochko
1e02ba111a Add emoji autosuggest (#5053)
* Add emoji autosuggest

Some credit goes to glitch-soc/mastodon#149

* Remove server-side shortcode->unicode conversion

* Insert shortcode when suggestion is custom emoji

* Remove remnant of server-side emojis

* Update style of autosuggestions

* Fix wrong emoji filenames generated in autosuggest item

* Do not lazy load emoji picker, as that no longer works

* Fix custom emoji autosuggest

* Fix multiple "Custom" categories getting added to emoji index, only add once
2017-09-23 14:47:32 +02:00
Eugen Rochko
293972f716 New API: GET /api/v1/custom_emojis to get a server's custom emojis (#5051) 2017-09-23 01:57:23 +02:00
Eugen Rochko
846cd4e838 Switch from EmojiOne to Twemoji, different emoji picker (#5046)
* Switch from EmojiOne to Twemoji, different emoji picker

* Make emoji-mart use a local spritesheet

* Fix emojify test

* yarn manage:translations
2017-09-23 01:41:00 +02:00
Akihiko Odaki
0de82dd316 Do not filter statuses with unknown languages (#5045) 2017-09-22 19:33:17 +02:00
Eugen Rochko
034fab39ab Make dropdowns render into portal, expand animation (#5018)
* Make dropdowns render into portal, expand animation

* Improve actions modal style
2017-09-22 04:59:17 +02:00
aschmitz
669fe9ee06 Change IDs to strings rather than numbers in API JSON output (#5019)
* Fix JavaScript interface with long IDs

Somewhat predictably, the JS interface handled IDs as numbers, which in
JS are IEEE double-precision floats. This loses some precision when
working with numbers as large as those generated by the new ID scheme,
so we instead handle them here as strings. This is relatively simple,
and doesn't appear to have caused any problems, but should definitely
be tested more thoroughly than the built-in tests. Several days of use
appear to support this working properly.

BREAKING CHANGE:

The major(!) change here is that IDs are now returned as strings by the
REST endpoints, rather than as integers. In practice, relatively few
changes were required to make the existing JS UI work with this change,
but it will likely hit API clients pretty hard: it's an entirely
different type to consume. (The one API client I tested, Tusky, handles
this with no problems, however.)

Twitter ran into this issue when introducing Snowflake IDs, and decided
to instead introduce an `id_str` field in JSON responses. I have opted
to *not* do that, and instead force all IDs to 64-bit integers
represented by strings in one go. (I believe Twitter exacerbated their
problem by rolling out the changes three times: once for statuses, once
for DMs, and once for user IDs, as well as by leaving an integer ID
value in JSON. As they said, "If you’re using the `id` field with JSON
in a Javascript-related language, there is a very high likelihood that
the integers will be silently munged by Javascript interpreters. In most
cases, this will result in behavior such as being unable to load or
delete a specific direct message, because the ID you're sending to the
API is different than the actual identifier associated with the
message." [1]) However, given that this is a significant change for API
users, alternatives or a transition time may be appropriate.

1: https://blog.twitter.com/developer/en_us/a/2011/direct-messages-going-snowflake-on-sep-30-2011.html

* Additional fixes for stringified IDs in JSON

These should be the last two. These were identified using eslint to try
to identify any plain casts to JavaScript numbers. (Some such casts are
legitimate, but these were not.)

Adding the following to .eslintrc.yml will identify casts to numbers:

~~~
  no-restricted-syntax:
  - warn
  - selector: UnaryExpression[operator='+'] > :not(Literal)
    message: Avoid the use of unary +
  - selector: CallExpression[callee.name='Number']
    message: Casting with Number() may coerce string IDs to numbers
~~~

The remaining three casts appear legitimate: two casts to array indices,
one in a server to turn an environment variable into a number.

* Back out RelationshipsController Change

This was made to make a test a bit less flakey, but has nothing to
do with this branch.

* Change internal streaming payloads to stringified IDs as well

Per
https://github.com/tootsuite/mastodon/pull/5019#issuecomment-330736452
we need these changes to send deleted status IDs as strings, not
integers.
2017-09-20 14:53:48 +02:00
Akihiko Odaki
bb4d005a83 Introduce OStatus::TagManager (#5008) 2017-09-19 18:08:08 +02:00
Eugen Rochko
81cec35dbf Custom emoji (#4988)
* Custom emoji

- In OStatus: `<link rel="emoji" name="coolcat" href="http://..." />`
- In ActivityPub: `{ type: "Emoji", name: ":coolcat:", href: "http://..." }`
- In REST API: Status object includes `emojis` array (`shortcode`, `url`)
- Domain blocks with reject media stop emojis
- Emoji file up to 50KB
- Web UI handles custom emojis
- Static pages render custom emojis as `<img />` tags

Side effects:

- Undo #4500 optimization, as I needed to modify it to restore
  shortcode handling in emojify()
- Formatter#plaintext should now make sure stripped out line-breaks
  and paragraphs are replaced with newlines

* Fix emoji at the start not being converted
2017-09-19 02:42:40 +02:00
unarist
dd6f9a1b82 Validate uri presence for remote status (#4985) 2017-09-17 15:21:57 +02:00
unarist
1eab53ee10 Fix an error when actor json couldn't be fetched in ResolveRemoteAccountService (#4979)
* Fix an error when actor json couldn't be fetched in ResolveRemoteAccountService

* Add specs
2017-09-17 11:54:23 +02:00
Akihiko Odaki
198a9a4fa4 Remove local_only scope in Status (#4977) 2017-09-17 05:29:43 +02:00
unarist
ec36df97c4 Escape URL parts on formatting local status (#4975) 2017-09-16 21:33:52 +02:00
Eugen Rochko
ad8e856a5b Fix error when following locked accounts (#4896)
(cherry picked from commit 0ef9d45d05)
2017-09-16 11:10:33 -05:00
Akihiko Odaki
48d77ea1eb Fix filterable_languages method of SettingsHelper (#4966) 2017-09-16 14:59:41 +02:00
Eugen Rochko
472df24579 When web UI URL used while logged out, redirect to static page (#4954) 2017-09-15 00:57:08 +02:00
ふぁぼ原
3816943e6b Enable to recognize most kinds of characters as URL paths (#4941) 2017-09-14 18:03:20 +02:00
Surinna Curtis
d880b3182b minor code style fixes oops 2017-09-13 21:47:30 -05:00
Surinna Curtis
83b3a0389c Add more specs for the /api/v1/mutes/details endpoint 2017-09-13 21:47:30 -05:00
Surinna Curtis
70592cdaba Add a /api/v1/mutes/details route that just returns the array of mutes. 2017-09-13 21:47:30 -05:00
Surinna Curtis
500c465226 specs for MuteService notifications params 2017-09-13 21:47:30 -05:00
Surinna Curtis
eaaf2170fe API support for muting notifications (and specs) 2017-09-13 21:47:30 -05:00
Surinna Curtis
5c2e1869f0 specs testing that hide_notifications in mutes actually hides notifications 2017-09-13 21:47:30 -05:00
Surinna Curtis
0f2af2a974 Add specs for how mute! interacts with muting_notifications? 2017-09-13 21:47:30 -05:00
Eugen Rochko
9239e4ce4d Uploads for admin site settings (#4913)
* Improve OpenGraph tags for about pages

* Add thumbnail admin setting

* Fix error

* Fix up
2017-09-14 00:04:30 +02:00
ThibG
550ff677da Fix ActivityPub handling of replies with WEB_DOMAIN (#4895) (#4904)
* Fix ActivityPub handling of replies when LOCAL_DOMAIN ≠ WEB_DOMAIN (#4895)

For all intents and purposes, `local_url?` is used to check if an URL refers
to the Web UI or the various API endpoints of the local instances. Those things
reside on `WEB_DOMAIN` and not `LOCAL_DOMAIN`.

* Change local_url? spec, as all URLs handled by Mastodon are based on WEB_DOMAIN
2017-09-13 14:22:16 +02:00
Eugen Rochko
8f8e677630 Clean up and improve generated OpenGraph tags (#4901)
- Return all images as og:image
- Return videos as og:image (preview) and og:video
- Return profile:username on profiles
2017-09-12 05:39:38 +02:00
Eugen Rochko
0ef9d45d05 Fix error when following locked accounts (#4896) 2017-09-11 23:50:37 +02:00
Surinna Curtis
3cac5bc2c3 Fix a spuriously failing spec that assumed we required short bios like upstream 2017-09-10 19:43:52 -05:00
David Yip
932571fa22 Merge tag 'v1.6.0' into sync/upstream 2017-09-10 13:04:27 -05:00
Eugen Rochko
c450ddb613 Fix POST /api/v1/follows error when already following (#4878) 2017-09-10 15:09:06 +02:00
Eugen Rochko
7d7844a47f Default follows for new users (#4871)
When a new user confirms their e-mail, bootstrap their home timeline
by automatically following a set of accounts. By default, all local
admin accounts (that are unlocked). Can be customized by new admin
setting (comma-separated usernames, local and unlocked only)
2017-09-10 09:58:38 +02:00
David Yip
67ad453373 Merge tag 'v1.6.0rc4' into sync/upstream-1.6.0rc4
Conflicts:
      app/javascript/mastodon/features/getting_started/index.js
      app/javascript/packs/public.js
      app/javascript/styles/components.scss
2017-09-09 23:56:21 -05:00
David Yip
514fc908a3 Merge tag 'v1.6.0rc3' into sync/upstream 2017-09-09 14:28:08 -05:00
David Yip
b9f7bc149b Merge branch 'origin/master' into sync/upstream
Conflicts:
	app/javascript/mastodon/components/status_list.js
	app/javascript/mastodon/features/notifications/index.js
	app/javascript/mastodon/features/ui/components/modal_root.js
	app/javascript/mastodon/features/ui/components/onboarding_modal.js
	app/javascript/mastodon/features/ui/index.js
	app/javascript/styles/about.scss
	app/javascript/styles/accounts.scss
	app/javascript/styles/components.scss
	app/presenters/instance_presenter.rb
	app/services/post_status_service.rb
	app/services/reblog_service.rb
	app/views/about/more.html.haml
	app/views/about/show.html.haml
	app/views/accounts/_header.html.haml
	config/webpack/loaders/babel.js
	spec/controllers/api/v1/accounts/credentials_controller_spec.rb
2017-09-09 14:27:47 -05:00
Eugen Rochko
90712d4293 Fix errors preventing UnsubscribeService from working (#4866) 2017-09-09 17:36:27 +02:00
unarist
a12572e074 Handle stream_entry URL correctly in ActivityPub (#4854)
In before, the method uses stream_entry id as status id, so replied status was wrongly selected.

This PR uses StatusFinder which was introduced with `Api::Web::EmbedsController`.
2017-09-08 18:20:03 +02:00
Eugen Rochko
1caf11ddcc Fix language filter codes (#4841)
* Fix language filter codes

CLD3 returns BCP-47 language identifier, filter settings expect
identifiers in the ISO 639-1 format. Convert between formats,
and exclude duplicate languages from filter choices (zh-CN->zh)

* Fix zh name
2017-09-08 12:32:22 +02:00
Eugen Rochko
a4caa7eb62 Fetch statuses/following/followers numbers from ActivityPub collections (#4840) 2017-09-08 12:00:17 +02:00
voidSatisfaction
8185f98872 Feat add validation for report comment: characters under 1000 valid (#4833) 2017-09-07 09:55:42 +02:00
Eugen Rochko
e7adbf572a Switch to static URIs, new URI format in both protocols for new statuses (#4815)
* Decouple Status#local? from uri being nil

* Replace on-the-fly URI generation with stored URIs

- Generate URI in after_save hook for local statuses
- Use static value in TagManager when available, fallback to tag format
- Make TagManager use ActivityPub::TagManager to understand new format
- Adjust tests

* Use other heuristic for locality of old statuses, do not perform long query

* Exclude tombstone stream entries from Atom feed

* Prevent nil statuses from landing in Pubsubhubbub::DistributionWorker

* Fix URI not being saved (#4818)

* Add more specs for Status

* Save generated uri immediately

and also fix method order to minimize diff.

* Fix alternate HTML URL in Atom

* Fix tests

* Remove not-null constraint from statuses migration to speed it up
2017-09-06 19:01:28 +02:00
Eugen Rochko
9b994c4aee Fix #4794 - Fake instant follow in API response when account is believed unlocked (#4799) 2017-09-05 17:48:13 +02:00
nullkal
2293466edd Show pinned statuses only in the top of the profile page (#4803)
* Show pinned statuses only in the top of the profile page

* Refactor AccountsController#show_pinned_statuses?
2017-09-04 12:53:18 +02:00
abcang
6ec1aa372d Validate data of Imports (#4782) 2017-09-02 20:45:42 +02:00
aschmitz
e18ed4bbc7 Fix StatusLengthValidator tests
This fixes the StatusLengthValidator tests to use
StatusLengthValidator::MAX_CHARS to derive the length of the strings
tested, and so makes them resilient to changes in MAX_CHARS (such as the
one that made these tests fail, with it being changed from 500 to 512).

This seems a bit odd for some types of tests, but appears to make sense
here, where the testing goes beyond `status.length > MAX_CHARS`.
2017-09-02 11:37:40 -05:00
unarist
6a4e2db661 Raise an error for remote url in StatusFinder (#4776)
* Raise an error for remote url in StatusFinder

Previous implementation had allowed remote url with status id which also exists on local.

Then that bug leads /api/web/embed to return wrong embed url.

* Fix oembed_controller_spec
2017-09-02 17:42:47 +02:00
Eugen Rochko
7dc5035031 Make PreviewCard records reuseable between statuses (#4642)
* Make PreviewCard records reuseable between statuses

**Warning!** Migration truncates preview_cards tablec

* Allow a wider thumbnail for link preview, display it in horizontal layout (#4648)

* Delete preview cards files before truncating

* Rename old table instead of truncating it

* Add mastodon:maintenance:remove_deprecated_preview_cards

* Ignore deprecated_preview_cards in schema definition

* Fix null behaviour
2017-09-01 16:20:16 +02:00
Eugen Rochko
9a5ae09620 Remove identity context from output of LinkedDataSignature (#4753) 2017-08-31 21:32:09 +02:00
unarist
f7937d903c Don't process ActivityPub payload if signature is invalid (#4752)
* Don't process ActivityPub payload if signature is invalid

* Fix style issue
2017-08-31 17:18:49 +02:00
Eugen Rochko
7b8f262840 Forward ActivityPub creates that reply to local statuses (#4709)
* Forward ActivityPub creates that reply to local statuses

* Fix test

* Fix wrong signers
2017-08-30 15:37:02 +02:00
Eugen Rochko
e95bdec7c5 Update status embeds (#4742)
- Use statuses controller for embeds instead of stream entries controller
- Prefer /@:username/:id/embed URL for embeds
- Use /@:username as author_url in OEmbed
- Add follow link to embeds which opens web intent in new window
- Use redis cache in development
- Cache entire embed
2017-08-30 10:23:43 +02:00
Eugen Rochko
4c76402ba1 Serialize ActivityPub alternate link into OStatus deletes, handle it (#4730)
Requires moving Atom rendering from DistributionWorker (where
`stream_entry.status` is already nil) to inline (where
`stream_entry.status.destroyed?` is true) and distributing that.

Unfortunately, such XML renderings can no longer be easily chained
together into one payload of n items.
2017-08-29 16:11:05 +02:00
unarist
938cd2875b Fix Delete activity handling when the status has been reblogged (#4729) 2017-08-28 22:08:11 +02:00
unarist
7876aed134 Fix deletion of status which has been reblogged (#4728) 2017-08-28 21:38:59 +02:00
abcang
ce9a5f358e rescue HTTP::ConnectionError in RemoteFollowController#create (#4726) 2017-08-28 19:12:09 +02:00
Eugen Rochko
00840f4f2e Add handling of Linked Data Signatures in payloads (#4687)
* Add handling of Linked Data Signatures in payloads

* Add a way to sign JSON, fix canonicalization of signature options

* Fix signatureValue encoding, send out signed JSON when distributing

* Add missing security context
2017-08-26 13:47:38 +02:00
nullkal
c2af138113 Allow multiple pinned statuses to be shown and make them be ordered b… (#4690)
* Allow multiple pinned statuses to be shown and make them be ordered by pinned date

* Set timestamps NOT NULL

* Make single-line pinned_statuses

* Spec for pinned_statuses

* Remove redundant empty line
2017-08-25 18:50:52 +02:00
Eugen Rochko
9caa90025f Pinned statuses (#4675)
* Pinned statuses

* yarn manage:translations
2017-08-25 01:41:18 +02:00
Eugen Rochko
cf615abbf9 Add configuration to disable private status federation over PuSH (#4582) 2017-08-24 17:51:32 +02:00