* add youtube oembed endpoint
* add check for oembed endpoint
* change unless for a more readable if
* clear blank lines
* endpoint via https
* Fix string literal in condition
* use cache for endpoints
* use cache for endpoints
* clean up and adding check
* clean up and remove redundant return
* add html check
* add false to return
* use double quotes
* use double quotes
* Clean up
The @property attribute in HTML is a space-separated list of values.
This change normalizes whitespace and finds the desired value in
the list instead of requiring an exact single-value match.
More details:
https://www.ctrl.blog/entry/rdfa-socialmedia-metadata.html
* Add failing test for windows-1251 link cards
* Ignore low-confidence CharlockHolmes guesses
Fixes#9466
* Fix no method error when charlock holmes cannot detect charset
* Verify link ownership with rel="me"
* Add explanation about verification to UI
* Perform link verifications
* Add click-to-copy widget for verification HTML
* Redesign edit profile page
* Redesign forms
* Improve responsive design of settings pages
* Restore landing page sign-up form
* Fix typo
* Support <link> tags, add spec
* Fix links not being verified on first discovery and passive updates
* No need to re-require sidekiq plugins, they are required via Gemfile
* Add derailed_benchmarks tool, no need to require TTY gems in Gemfile
* Replace ruby-oembed with FetchOEmbedService
Reduce startup by 45382 allocated objects
* Remove preloaded JSON-LD in favour of caching HTTP responses
Reduce boot RAM by about 6 MiB
* Fix tests
* Fix test suite by stubbing out JSON-LD contexts
to_s method of HTTP::Response keeps blocking while it receives the whole
content, no matter how it is big. This means it may waste time to receive
unacceptably large files. It may also consume memory and disk in the
process. This solves the inefficency by checking response length while
receiving.
HTTP connections must be explicitly closed in many cases, and letting
perform method close connections makes its callers less redundant and
prevent them from forgetting to close connections.
Avoid error when the service returns a mostly valid oembed, but has no
url in it, causing a MethodError: undefined method `url'
for #<OEmbed::Response::Photo:0x000056505def9620>
Specifically, this fixes status length calculation to be same as JS side.
BTW, since this pattern used in not only preview card fetching, we
should extract it (with twitter-regex?) and write tests I think.
* If OEmbed response doesn't have a required property `type`, ignore it.
e.g. `NoMethodError: undefined method 'type' for ...`
* If we failed to detect encoding, fallback to default behavior of Nokogiri.
e.g. `KeyError: key not found: :encoding`
* Improve error handling on LinkCrawlWorker
* Ignore TimeoutError and InvalidURIError too
* Record errors to debug log
* Enable dead job queue on LinkCrawlWorker
Since most of acceptable errors were already ignored, only our side issue should go to dead job queue.
* Ignore all http gem errors
* Support OpenGraph video embeds
It's not really OpenGraph, it's twitter:player property, but it's
not OEmbed so that fits. For example, this allows Twitch clips to
be displayed as embeds.
Also, fixesglitch-soc/mastodon#135
* Fix invalid OpenGraph cards being saved through attaching and
revisit URLs after 14 days
* Make PreviewCard records reuseable between statuses
**Warning!** Migration truncates preview_cards tablec
* Allow a wider thumbnail for link preview, display it in horizontal layout (#4648)
* Delete preview cards files before truncating
* Rename old table instead of truncating it
* Add mastodon:maintenance:remove_deprecated_preview_cards
* Ignore deprecated_preview_cards in schema definition
* Fix null behaviour
* Add Request class with HTTP signature generator
Spec: https://tools.ietf.org/html/draft-cavage-http-signatures-06
* Add HTTP signature verification concern
* Add test for SignatureVerification concern
* Add basic test for Request class
* Make PuSH subscribe/unsubscribe requests use new Request class
Accidentally fix lease_seconds not being set and sent properly, and
change the new minimum subscription duration to 1 day
* Make all PuSH workers use new Request class
* Make Salmon sender use new Request class
* Make FetchLinkService use new Request class
* Make FetchAtomService use the new Request class
* Make Remotable use the new Request class
* Make ResolveRemoteAccountService use the new Request class
* Add more tests
* Allow +-30 seconds window for signed request to remain valid
* Disable time window validation for signed requests, restore 7 days
as PuSH subscription duration (which was previous default due to a bug)
* Specs for language detection
* Use CharlockHolmes instead of NKF
* Correct mistakes
* Correct style
* Set hint_enc instead of falling back and strip_tags
* Improve specs
* Add dependencies
TagManager.local_url? was sometimes called with an URI with a nil host,
leading to a crash in TagManager.local_url?. This fixes moves the
already-existing uri.host.blank? check in front to avoid this case.
* OEmbed support for PreviewCard
* Improve ProviderDiscovery code failure treatment
* Do not crawl links if there is a content warning, since those
don't display a link card anyway
* Reset db schema
* Fresh migrate
* Fix rubocop style issues
Fix#1681 - return existing access token when applicable instead of creating new
* Fix test
* Extract http client to helper
* Improve oembed controller
* Fix#2119 - Whenever about to send a HTTP request, normalize the URI
* Add test for IDN request in FetchLinkCardService
* Perform IDN normalization on domains before they are stored in the DB
