Commit graph

103 commits

Author SHA1 Message Date
Alexander Strizhakov
8d218ebaf5
Moving some background jobs into simple tasks
- fetching activity data
- attachment prefetching
- using limiter to prevent overload
2020-11-11 13:39:49 +03:00
Alexander Strizhakov
fc7151a9c4
more files renamings 2020-10-13 16:38:19 +03:00
Alexander Strizhakov
103f3dcb9e
rich media parser ttl files consistency 2020-10-13 16:38:15 +03:00
Mark Felder
8539e386c3 Add missing Copyright headers 2020-10-12 12:00:50 -05:00
Mark Felder
ba7f9459b4 Revert Rich Media censorship for sensitive statuses
The #NSFW hashtag test was broken anyway.
2020-09-28 18:22:59 -05:00
rinpatch
db80b9d630 RichMedia: Fix log spam on failures and resetting TTL on cached errors 2020-09-17 16:56:39 +03:00
rinpatch
bb407edce4 RichMedia: fix a compilation error due to nonexistent variable
No idea why this passed Gitlab CI
2020-09-14 15:46:00 +03:00
rinpatch
f70335002d RichMedia: Do a HEAD request to check content type/length
This shouldn't be too expensive, since the connections are pooled,
but it should save us some bandwidth since we won't fetch non-html
files and files that are too large for us to process (especially
since you can't cancel a request without closing the connection
with HTTP1).
2020-09-14 14:45:58 +03:00
rinpatch
f66a15c4a5 RichMedia parser: do not set a cache TTL for unchanging errors 2020-09-14 14:44:25 +03:00
Alexander Strizhakov
696bf09433
passing adapter options directly without adapter key 2020-09-07 19:59:17 +03:00
Alexander Strizhakov
a83916fdac
adapter options unification
not needed options deletion
2020-09-07 19:59:17 +03:00
lain
fdab01ab56 Merge branch 'fix/rich-media-fake-statuses' into 'develop'
Rich Media: Do not cache URLs for preview statuses

Closes #1987

See merge request pleroma/pleroma!2956
2020-09-07 10:19:19 +00:00
rinpatch
170599c390 RichMedia: do not log webpages missing metadata as errors
Also fixes the return value of Parser.parse on errors, previously
was just `:ok` due to the logger call in the end
2020-09-05 22:05:35 +03:00
rinpatch
e198ba492e Rich Media: Do not cache URLs for preview statuses
Closes #1987
2020-09-05 20:53:46 +03:00
rinpatch
19691389b9 Rich media: Add failure tracking 2020-09-02 14:59:52 +03:00
rinpatch
47ff425cfd Merge branch 'fix/2047-rich-media-parser' into 'develop'
RichMedia parser fix

Closes #2047

See merge request pleroma/pleroma!2941
2020-09-02 09:38:43 +00:00
Alexander Strizhakov
79f65b4374
correct pool and uniform headers format 2020-09-02 09:16:51 +03:00
Alexander Strizhakov
03d06062ab
don't fail on url fetch 2020-09-01 19:39:07 +03:00
Mark Felder
016d8d6c56 Consolidate construction of Rich Media Parser HTTP requests 2020-08-03 12:37:31 -05:00
lain
781b270863 ChatMessageReferenceView: Display preview cards. 2020-07-30 19:57:26 +02:00
lain
5b1eeb06d8 Revert "Merge branch 'revert-2b5d9eb1' into 'develop'"
This reverts merge request !2784
2020-07-21 22:18:17 +00:00
lain
696c13ce54 Revert "Merge branch 'linkify' into 'develop'"
This reverts merge request !2677
2020-07-21 22:17:34 +00:00
Alex Gleason
38425ebdbf
Merge remote-tracking branch 'upstream/develop' into linkify 2020-07-16 14:51:36 -05:00
Mark Felder
18438a9bf0 Add "Bot" to User Agent to coerce Twitter into serving OGP <meta> tags. 2020-07-07 11:21:05 -05:00
Alex Gleason
8daacc9114
AutoLinker --> Linkify, update to latest version
https://git.pleroma.social/pleroma/elixir-libraries/linkify
2020-06-30 16:39:15 -05:00
Egor Kislitsyn
58e4e3db8b
Merge remote-tracking branch 'origin/develop' into merge-ogp-twitter-parsers 2020-06-15 16:03:40 +04:00
Egor Kislitsyn
520367d6fd Fix atom leak in Rich Media Parser 2020-06-13 12:08:46 +03:00
Egor Kislitsyn
09d31d24de
Return an empty map from Pleroma.Web.RichMedia.Parsers.OGP.parse/2 2020-06-12 18:39:51 +04:00
Egor Kislitsyn
2419776e19
Deprecate Pleroma.Web.RichMedia.Parsers.OGP 2020-06-11 23:11:46 +04:00
Egor Kislitsyn
1f35acce54
Merge OGP parser with TwitterCard 2020-06-11 17:57:31 +04:00
Mark Felder
3bf78f2be7 Fix Oban not receiving :ok from RichMediaHelper job 2020-04-14 11:43:53 -05:00
Mark Felder
4427161ca3 Merge branch 'develop' into gun 2020-03-03 17:15:49 -06:00
Mark Felder
05da5f5cca Update Copyrights 2020-03-03 16:44:49 -06:00
Alexander Strizhakov
509c81e4b1
Merge branch 'develop' into gun 2020-03-03 10:08:07 +03:00
Haelwenn (lanodan) Monnier
6da6540036
Bump copyright years of files changed after 2020-01-07
Done via the following command:
git diff fcd5dd259a --stat --name-only | xargs sed -i '/Pleroma Authors/c# Copyright © 2017-2020 Pleroma Authors <https:\/\/pleroma.social\/>'
2020-03-02 06:08:45 +01:00
Mark Felder
cf94349287 Merge branch 'develop' into gun 2020-02-18 09:06:27 -06:00
Alexander Strizhakov
514c899275
adding gun adapter 2020-02-18 08:19:01 +03:00
rinpatch
472132215e Use floki's new APIs for parsing fragments 2020-02-16 01:55:26 +03:00
237b2068f9 Revert "Merge branch 'feat/floki-fasthtml' into 'develop'"
This reverts merge request !2194
2020-02-11 16:55:18 +00:00
rinpatch
ea1631d7e6 Make Floki use fast_html 2020-02-11 16:17:21 +03:00
Alexander Strizhakov
1f4fbe9d98
title parse improvement 2020-01-29 11:13:34 +03:00
Alexander Strizhakov
7bd4c14581
meta tag parser respect first title header 2020-01-28 19:29:27 +03:00
Maksim Pechnikov
b4cf74c106 added prepare html for RichMedia.Parser 2019-09-15 14:53:58 +03:00
Maksim
139b196bc0 [#1150] fixed parser TwitterCard 2019-08-06 20:19:28 +00:00
Ariadne Conill
b93498eb52 constants: add as_public constant and use it everywhere 2019-07-29 02:43:19 +00:00
Ariadne Conill
d3bdb8e704 rich media: parser: splice the given URL into the result 2019-07-23 23:51:29 +00:00
rinpatch
3368174785 Fix rich media parser failing when no TTL can be found by image TTL
setters
2019-07-21 18:22:22 +03:00
Sachin Joshi
de9906ad56 change the structure of image ttl parsar 2019-07-19 11:43:42 +05:45
Sachin Joshi
18234cc44e add the rich media ttl based on image exp time 2019-07-17 00:20:34 +05:45
Alex S
f4447d82b8 parsers configurable 2019-07-14 09:21:56 +03:00