Alexander Strizhakov
8d218ebaf5
Moving some background jobs into simple tasks
...
- fetching activity data
- attachment prefetching
- using limiter to prevent overload
2020-11-11 13:39:49 +03:00
Alexander Strizhakov
fc7151a9c4
more files renamings
2020-10-13 16:38:19 +03:00
Alexander Strizhakov
103f3dcb9e
rich media parser ttl files consistency
2020-10-13 16:38:15 +03:00
Mark Felder
8539e386c3
Add missing Copyright headers
2020-10-12 12:00:50 -05:00
Mark Felder
ba7f9459b4
Revert Rich Media censorship for sensitive statuses
...
The #NSFW hashtag test was broken anyway.
2020-09-28 18:22:59 -05:00
rinpatch
db80b9d630
RichMedia: Fix log spam on failures and resetting TTL on cached errors
2020-09-17 16:56:39 +03:00
rinpatch
bb407edce4
RichMedia: fix a compilation error due to nonexistent variable
...
No idea why this passed Gitlab CI
2020-09-14 15:46:00 +03:00
rinpatch
f70335002d
RichMedia: Do a HEAD request to check content type/length
...
This shouldn't be too expensive, since the connections are pooled,
but it should save us some bandwidth since we won't fetch non-html
files and files that are too large for us to process (especially
since you can't cancel a request without closing the connection
with HTTP1).
2020-09-14 14:45:58 +03:00
rinpatch
f66a15c4a5
RichMedia parser: do not set a cache TTL for unchanging errors
2020-09-14 14:44:25 +03:00
Alexander Strizhakov
696bf09433
passing adapter options directly without adapter key
2020-09-07 19:59:17 +03:00
Alexander Strizhakov
a83916fdac
adapter options unification
...
not needed options deletion
2020-09-07 19:59:17 +03:00
lain
fdab01ab56
Merge branch 'fix/rich-media-fake-statuses' into 'develop'
...
Rich Media: Do not cache URLs for preview statuses
Closes #1987
See merge request pleroma/pleroma!2956
2020-09-07 10:19:19 +00:00
rinpatch
170599c390
RichMedia: do not log webpages missing metadata as errors
...
Also fixes the return value of Parser.parse on errors, previously
was just `:ok` due to the logger call in the end
2020-09-05 22:05:35 +03:00
rinpatch
e198ba492e
Rich Media: Do not cache URLs for preview statuses
...
Closes #1987
2020-09-05 20:53:46 +03:00
rinpatch
19691389b9
Rich media: Add failure tracking
2020-09-02 14:59:52 +03:00
rinpatch
47ff425cfd
Merge branch 'fix/2047-rich-media-parser' into 'develop'
...
RichMedia parser fix
Closes #2047
See merge request pleroma/pleroma!2941
2020-09-02 09:38:43 +00:00
Alexander Strizhakov
79f65b4374
correct pool and uniform headers format
2020-09-02 09:16:51 +03:00
Alexander Strizhakov
03d06062ab
don't fail on url fetch
2020-09-01 19:39:07 +03:00
Mark Felder
016d8d6c56
Consolidate construction of Rich Media Parser HTTP requests
2020-08-03 12:37:31 -05:00
lain
781b270863
ChatMessageReferenceView: Display preview cards.
2020-07-30 19:57:26 +02:00
lain
5b1eeb06d8
Revert "Merge branch 'revert-2b5d9eb1' into 'develop'"
...
This reverts merge request !2784
2020-07-21 22:18:17 +00:00
lain
696c13ce54
Revert "Merge branch 'linkify' into 'develop'"
...
This reverts merge request !2677
2020-07-21 22:17:34 +00:00
Alex Gleason
38425ebdbf
Merge remote-tracking branch 'upstream/develop' into linkify
2020-07-16 14:51:36 -05:00
Mark Felder
18438a9bf0
Add "Bot" to User Agent to coerce Twitter into serving OGP <meta> tags.
2020-07-07 11:21:05 -05:00
Alex Gleason
8daacc9114
AutoLinker --> Linkify, update to latest version
...
https://git.pleroma.social/pleroma/elixir-libraries/linkify
2020-06-30 16:39:15 -05:00
Egor Kislitsyn
58e4e3db8b
Merge remote-tracking branch 'origin/develop' into merge-ogp-twitter-parsers
2020-06-15 16:03:40 +04:00
Egor Kislitsyn
520367d6fd
Fix atom leak in Rich Media Parser
2020-06-13 12:08:46 +03:00
Egor Kislitsyn
09d31d24de
Return an empty map from Pleroma.Web.RichMedia.Parsers.OGP.parse/2
2020-06-12 18:39:51 +04:00
Egor Kislitsyn
2419776e19
Deprecate Pleroma.Web.RichMedia.Parsers.OGP
2020-06-11 23:11:46 +04:00
Egor Kislitsyn
1f35acce54
Merge OGP parser with TwitterCard
2020-06-11 17:57:31 +04:00
Mark Felder
3bf78f2be7
Fix Oban not receiving :ok from RichMediaHelper job
2020-04-14 11:43:53 -05:00
Mark Felder
4427161ca3
Merge branch 'develop' into gun
2020-03-03 17:15:49 -06:00
Mark Felder
05da5f5cca
Update Copyrights
2020-03-03 16:44:49 -06:00
Alexander Strizhakov
509c81e4b1
Merge branch 'develop' into gun
2020-03-03 10:08:07 +03:00
Haelwenn (lanodan) Monnier
6da6540036
Bump copyright years of files changed after 2020-01-07
...
Done via the following command:
git diff fcd5dd259a
2020-03-02 06:08:45 +01:00
Mark Felder
cf94349287
Merge branch 'develop' into gun
2020-02-18 09:06:27 -06:00
Alexander Strizhakov
514c899275
adding gun adapter
2020-02-18 08:19:01 +03:00
rinpatch
472132215e
Use floki's new APIs for parsing fragments
2020-02-16 01:55:26 +03:00
237b2068f9
Revert "Merge branch 'feat/floki-fasthtml' into 'develop'"
...
This reverts merge request !2194
2020-02-11 16:55:18 +00:00
rinpatch
ea1631d7e6
Make Floki use fast_html
2020-02-11 16:17:21 +03:00
Alexander Strizhakov
1f4fbe9d98
title parse improvement
2020-01-29 11:13:34 +03:00
Alexander Strizhakov
7bd4c14581
meta tag parser respect first title header
2020-01-28 19:29:27 +03:00
Maksim Pechnikov
b4cf74c106
added prepare html for RichMedia.Parser
2019-09-15 14:53:58 +03:00
Maksim
139b196bc0
[ #1150 ] fixed parser TwitterCard
2019-08-06 20:19:28 +00:00
Ariadne Conill
b93498eb52
constants: add as_public constant and use it everywhere
2019-07-29 02:43:19 +00:00
Ariadne Conill
d3bdb8e704
rich media: parser: splice the given URL into the result
2019-07-23 23:51:29 +00:00
rinpatch
3368174785
Fix rich media parser failing when no TTL can be found by image TTL
...
setters
2019-07-21 18:22:22 +03:00
Sachin Joshi
de9906ad56
change the structure of image ttl parsar
2019-07-19 11:43:42 +05:45
Sachin Joshi
18234cc44e
add the rich media ttl based on image exp time
2019-07-17 00:20:34 +05:45
Alex S
f4447d82b8
parsers configurable
2019-07-14 09:21:56 +03:00
