AkkomaGang/akkoma
AkkomaGang/akkoma
15
66
Fork 94
Code Issues 180 Pull requests 28 Projects 2 Releases 14 Packages Wiki Activity

[bug] Fetch rich media failed, blocked timeline for certain URLs #533

New issue
Open
opened 2023-05-05 03:19:18 +00:00 by nninja · 7 comments
nninja commented 2023-05-05 03:19:18 +00:00
Copy link

Your setup

From source

Extra details

Debian 11

Version

2023.04

PostgreSQL version

13

What were you trying to do?

This could be related to the "home/public timeline" failed to load issues e.g. #487 #508

Several days ago, my instance's public timeline didn't load for all users. I deleted one post and timeline worked again.

The deleted post also wouldn't load in Admin-FE. Nothing special in the post, except an URL (NSFW): https://www.reinderdijkhuis.com/greyfriars/2022/11/12/nudist-conversation-enders/

In the screenshot below, I pasted the same URL in Akkoma-FE's compose field, with Preview opened. Note that the preview took forever to load, and eventually failed with a message, JSON.parse: unexpected character at line 1 column 1 of the JSON data

image

What did you expect to happen?

Fetching rich media doesn't block forever when JSON parse errors out

What actually happened?

No response

Logs

See screenshot above

Severity

I cannot use it as easily as I'd like

Have you searched for this issue?

  • I have double-checked and have not found this issue mentioned anywhere.
### Your setup From source ### Extra details Debian 11 ### Version 2023.04 ### PostgreSQL version 13 ### What were you trying to do? This could be related to the "home/public timeline" failed to load issues e.g. #487 #508 Several days ago, my instance's public timeline didn't load for all users. I deleted one post and timeline worked again. The deleted post also wouldn't load in Admin-FE. Nothing special in the post, except an URL (NSFW): https://www.reinderdijkhuis.com/greyfriars/2022/11/12/nudist-conversation-enders/ In the screenshot below, I pasted the same URL in Akkoma-FE's compose field, with Preview opened. Note that the preview took forever to load, and eventually failed with a message, `JSON.parse: unexpected character at line 1 column 1 of the JSON data` ![image](/attachments/95e64f3d-8dd5-46c8-8c35-d969711ef74b) ### What did you expect to happen? Fetching rich media doesn't block forever when JSON parse errors out ### What actually happened? _No response_ ### Logs ```shell See screenshot above ``` ### Severity I cannot use it as easily as I'd like ### Have you searched for this issue? - [x] I have double-checked and have not found this issue mentioned anywhere.
image.png
94 KiB
👍 3
nninja added the
bug
 label 2023-05-05 03:19:18 +00:00
Clovis commented 2023-09-23 15:44:01 +00:00
Contributor
Copy link

Had the same issue on my instance, finding the "faulty" toot was hard.
This issue can be concerning since legit users can toot bad links that can crash timelines for all users without even knowing.
Here the issue was caused by a bad redirect.

Sep 22 09:27:58 NOM_HOTE_ANONYME docker[1723]: ** (ArgumentError) scheme is required for url: wp-json/oembed/1.0/embedde4c.json?url=http%3A%2F%2Fdm1r.fr%2F
Sep 22 09:27:58 NOM_HOTE_ANONYME docker[1723]:     (finch 0.14.0) lib/finch/request.ex:135: Finch.Request.parse_url/1
Sep 22 09:27:58 NOM_HOTE_ANONYME docker[1723]:     (finch 0.14.0) lib/finch/request.ex:103: Finch.Request.build/5
Sep 22 09:27:58 NOM_HOTE_ANONYME docker[1723]:     (tesla 1.4.4) lib/tesla/adapter/finch.ex:84: Tesla.Adapter.Finch.request/6
Sep 22 09:27:58 NOM_HOTE_ANONYME docker[1723]:     (tesla 1.4.4) lib/tesla/adapter/finch.ex:63: Tesla.Adapter.Finch.call/2
Sep 22 09:27:58 NOM_HOTE_ANONYME docker[1723]:     (tesla 1.4.4) lib/tesla/middleware/telemetry.ex:97: Tesla.Middleware.Telemetry.call/3
Sep 22 09:27:58 NOM_HOTE_ANONYME docker[1723]:     (tesla 1.4.4) lib/tesla/middleware/follow_redirects.ex:46: Tesla.Middleware.FollowRedirects.redirect/3
Sep 22 09:27:58 NOM_HOTE_ANONYME docker[1723]:     (pleroma 3.9.3-0-g8e7e2d9) lib/pleroma/web/rich_media/helpers.ex:84: Pleroma.Web.RichMedia.Helpers.rich_media_get/1
Sep 22 09:27:58 NOM_HOTE_ANONYME docker[1723]:     (pleroma 3.9.3-0-g8e7e2d9) lib/pleroma/web/rich_media/parsers/o_embed.ex:25: Pleroma.Web.RichMedia.Parsers.OEmbed.get_oembed_data/1    
Sep 22 09:27:58 NOM_HOTE_ANONYME docker[1723]: Function: #Function<6.51738268/0 in Pleroma.Web.RichMedia.Parser.parse_with_timeout/1>
Sep 22 09:27:58 NOM_HOTE_ANONYME docker[1723]:     Args: []
Had the same issue on my instance, finding the "faulty" toot was hard. This issue can be concerning since legit users can toot bad links that can crash timelines for all users without even knowing. Here the issue was caused by a bad redirect. ``` Sep 22 09:27:58 NOM_HOTE_ANONYME docker[1723]: ** (ArgumentError) scheme is required for url: wp-json/oembed/1.0/embedde4c.json?url=http%3A%2F%2Fdm1r.fr%2F Sep 22 09:27:58 NOM_HOTE_ANONYME docker[1723]: (finch 0.14.0) lib/finch/request.ex:135: Finch.Request.parse_url/1 Sep 22 09:27:58 NOM_HOTE_ANONYME docker[1723]: (finch 0.14.0) lib/finch/request.ex:103: Finch.Request.build/5 Sep 22 09:27:58 NOM_HOTE_ANONYME docker[1723]: (tesla 1.4.4) lib/tesla/adapter/finch.ex:84: Tesla.Adapter.Finch.request/6 Sep 22 09:27:58 NOM_HOTE_ANONYME docker[1723]: (tesla 1.4.4) lib/tesla/adapter/finch.ex:63: Tesla.Adapter.Finch.call/2 Sep 22 09:27:58 NOM_HOTE_ANONYME docker[1723]: (tesla 1.4.4) lib/tesla/middleware/telemetry.ex:97: Tesla.Middleware.Telemetry.call/3 Sep 22 09:27:58 NOM_HOTE_ANONYME docker[1723]: (tesla 1.4.4) lib/tesla/middleware/follow_redirects.ex:46: Tesla.Middleware.FollowRedirects.redirect/3 Sep 22 09:27:58 NOM_HOTE_ANONYME docker[1723]: (pleroma 3.9.3-0-g8e7e2d9) lib/pleroma/web/rich_media/helpers.ex:84: Pleroma.Web.RichMedia.Helpers.rich_media_get/1 Sep 22 09:27:58 NOM_HOTE_ANONYME docker[1723]: (pleroma 3.9.3-0-g8e7e2d9) lib/pleroma/web/rich_media/parsers/o_embed.ex:25: Pleroma.Web.RichMedia.Parsers.OEmbed.get_oembed_data/1 Sep 22 09:27:58 NOM_HOTE_ANONYME docker[1723]: Function: #Function<6.51738268/0 in Pleroma.Web.RichMedia.Parser.parse_with_timeout/1> Sep 22 09:27:58 NOM_HOTE_ANONYME docker[1723]: Args: [] ```
floatingghost commented 2023-09-23 16:06:00 +00:00
Owner
Copy link

the above log is from an old version, please check if it occurs on the most recent release

the above log is from an old version, please check if it occurs on the most recent release
Clovis commented 2023-09-24 14:32:56 +00:00
Contributor
Copy link

Here is a log from v3.10.4 :)

[error] Task #PID<0.2385.0> started from #PID<0.2379.0> terminating
** (ArgumentError) scheme is required for url: wp-json/oembed/1.0/embedde4c.json?url=http%3A%2F%2Fdm1r.fr%2F
    (finch 0.16.0) lib/finch/request.ex:135: Finch.Request.parse_url/1
    (finch 0.16.0) lib/finch/request.ex:103: Finch.Request.build/5
    (tesla 1.7.0) lib/tesla/adapter/finch.ex:84: Tesla.Adapter.Finch.request/6
    (tesla 1.7.0) lib/tesla/adapter/finch.ex:63: Tesla.Adapter.Finch.call/2
    (tesla 1.7.0) lib/tesla/middleware/telemetry.ex:99: Tesla.Middleware.Telemetry.call/3
    (tesla 1.7.0) lib/tesla/middleware/follow_redirects.ex:46: Tesla.Middleware.FollowRedirects.redirect/3
    (pleroma 3.10.4-16-g3d17b943-upstream+dev) lib/pleroma/web/rich_media/helpers.ex:84: Pleroma.Web.RichMedia.Helpers.rich_media_get/1
    (pleroma 3.10.4-16-g3d17b943-upstream+dev) lib/pleroma/web/rich_media/parsers/o_embed.ex:25: Pleroma.Web.RichMedia.Parsers.OEmbed.get_oembed_data/1
    (pleroma 3.10.4-16-g3d17b943-upstream+dev) lib/pleroma/web/rich_media/parsers/o_embed.ex:9: Pleroma.Web.RichMedia.Parsers.OEmbed.parse/2
    (pleroma 3.10.4-16-g3d17b943-upstream+dev) lib/pleroma/web/rich_media/parser.ex:161: anonymous fn/3 in Pleroma.Web.RichMedia.Parser.maybe_parse/1
    (elixir 1.14.0) lib/enum.ex:4751: Enumerable.List.reduce/3
    (elixir 1.14.0) lib/enum.ex:2514: Enum.reduce_while/3
    (pleroma 3.10.4-16-g3d17b943-upstream+dev) lib/pleroma/web/rich_media/parser.ex:137: Pleroma.Web.RichMedia.Parser.parse_url/1
    (elixir 1.14.0) lib/task/supervised.ex:89: Task.Supervised.invoke_mfa/2
    (elixir 1.14.0) lib/task/supervised.ex:34: Task.Supervised.reply/4
    (stdlib 4.2) proc_lib.erl:240: :proc_lib.init_p_do_apply/3
Function: #Function<6.51738268/0 in Pleroma.Web.RichMedia.Parser.parse_with_timeout/1>

I mitigated the issue with this fix, but I'm absolutely not satisfied with it

80519fe492

Here is a log from v3.10.4 :) ``` [error] Task #PID<0.2385.0> started from #PID<0.2379.0> terminating ** (ArgumentError) scheme is required for url: wp-json/oembed/1.0/embedde4c.json?url=http%3A%2F%2Fdm1r.fr%2F (finch 0.16.0) lib/finch/request.ex:135: Finch.Request.parse_url/1 (finch 0.16.0) lib/finch/request.ex:103: Finch.Request.build/5 (tesla 1.7.0) lib/tesla/adapter/finch.ex:84: Tesla.Adapter.Finch.request/6 (tesla 1.7.0) lib/tesla/adapter/finch.ex:63: Tesla.Adapter.Finch.call/2 (tesla 1.7.0) lib/tesla/middleware/telemetry.ex:99: Tesla.Middleware.Telemetry.call/3 (tesla 1.7.0) lib/tesla/middleware/follow_redirects.ex:46: Tesla.Middleware.FollowRedirects.redirect/3 (pleroma 3.10.4-16-g3d17b943-upstream+dev) lib/pleroma/web/rich_media/helpers.ex:84: Pleroma.Web.RichMedia.Helpers.rich_media_get/1 (pleroma 3.10.4-16-g3d17b943-upstream+dev) lib/pleroma/web/rich_media/parsers/o_embed.ex:25: Pleroma.Web.RichMedia.Parsers.OEmbed.get_oembed_data/1 (pleroma 3.10.4-16-g3d17b943-upstream+dev) lib/pleroma/web/rich_media/parsers/o_embed.ex:9: Pleroma.Web.RichMedia.Parsers.OEmbed.parse/2 (pleroma 3.10.4-16-g3d17b943-upstream+dev) lib/pleroma/web/rich_media/parser.ex:161: anonymous fn/3 in Pleroma.Web.RichMedia.Parser.maybe_parse/1 (elixir 1.14.0) lib/enum.ex:4751: Enumerable.List.reduce/3 (elixir 1.14.0) lib/enum.ex:2514: Enum.reduce_while/3 (pleroma 3.10.4-16-g3d17b943-upstream+dev) lib/pleroma/web/rich_media/parser.ex:137: Pleroma.Web.RichMedia.Parser.parse_url/1 (elixir 1.14.0) lib/task/supervised.ex:89: Task.Supervised.invoke_mfa/2 (elixir 1.14.0) lib/task/supervised.ex:34: Task.Supervised.reply/4 (stdlib 4.2) proc_lib.erl:240: :proc_lib.init_p_do_apply/3 Function: #Function<6.51738268/0 in Pleroma.Web.RichMedia.Parser.parse_with_timeout/1> ``` I mitigated the issue with this fix, but I'm absolutely not satisfied with it https://github.com/BDX-town/Akkoma/commit/80519fe49232962c8fde2288942d56d3ed1b401d
ahfrom commented 2024-05-12 09:22:00 +00:00
Copy link

Here is a log from 3.13.1. The issue is preventing me from fetching older posts.

URL: https://www.weaselhat.com/2015/03/16/a-refinement-type-by-any-other-name/

09:03:04.184 [error] Task #PID<0.5259.0> started from #PID<0.5258.0> terminating
** (ArgumentError) scheme is required for url: wp-json/oembed/1.0/embed?url=http:%252F%252Fwww.weaselhat.com%252F2015%252F03%252F16%252Fa-refinement-type-by-any-other-name%252F
    (finch 0.16.0) lib/finch/request.ex:135: Finch.Request.parse_url/1
    (finch 0.16.0) lib/finch/request.ex:103: Finch.Request.build/5
    (tesla 1.9.0) lib/tesla/adapter/finch.ex:66: Tesla.Adapter.Finch.call/2
    (tesla 1.9.0) lib/tesla/middleware/telemetry.ex:99: Tesla.Middleware.Telemetry.call/3
    (tesla 1.9.0) lib/tesla/middleware/follow_redirects.ex:46: Tesla.Middleware.FollowRedirects.redirect/3
    (pleroma 3.13.1) lib/pleroma/web/rich_media/helpers.ex:84: Pleroma.Web.RichMedia.Helpers.rich_media_get/1
    (pleroma 3.13.1) lib/pleroma/web/rich_media/parsers/o_embed.ex:25: Pleroma.Web.RichMedia.Parsers.OEmbed.get_oembed_data/1
    (pleroma 3.13.1) lib/pleroma/web/rich_media/parsers/o_embed.ex:9: Pleroma.Web.RichMedia.Parsers.OEmbed.parse/2
Function: #Function<6.88132293/0 in Pleroma.Web.RichMedia.Parser.parse_with_timeout/1>
    Args: []
Here is a log from 3.13.1. The issue is preventing me from fetching older posts. URL: https://www.weaselhat.com/2015/03/16/a-refinement-type-by-any-other-name/ ``` 09:03:04.184 [error] Task #PID<0.5259.0> started from #PID<0.5258.0> terminating ** (ArgumentError) scheme is required for url: wp-json/oembed/1.0/embed?url=http:%252F%252Fwww.weaselhat.com%252F2015%252F03%252F16%252Fa-refinement-type-by-any-other-name%252F (finch 0.16.0) lib/finch/request.ex:135: Finch.Request.parse_url/1 (finch 0.16.0) lib/finch/request.ex:103: Finch.Request.build/5 (tesla 1.9.0) lib/tesla/adapter/finch.ex:66: Tesla.Adapter.Finch.call/2 (tesla 1.9.0) lib/tesla/middleware/telemetry.ex:99: Tesla.Middleware.Telemetry.call/3 (tesla 1.9.0) lib/tesla/middleware/follow_redirects.ex:46: Tesla.Middleware.FollowRedirects.redirect/3 (pleroma 3.13.1) lib/pleroma/web/rich_media/helpers.ex:84: Pleroma.Web.RichMedia.Helpers.rich_media_get/1 (pleroma 3.13.1) lib/pleroma/web/rich_media/parsers/o_embed.ex:25: Pleroma.Web.RichMedia.Parsers.OEmbed.get_oembed_data/1 (pleroma 3.13.1) lib/pleroma/web/rich_media/parsers/o_embed.ex:9: Pleroma.Web.RichMedia.Parsers.OEmbed.parse/2 Function: #Function<6.88132293/0 in Pleroma.Web.RichMedia.Parser.parse_with_timeout/1> Args: [] ```
floatingghost commented 2024-05-12 10:13:49 +00:00
Owner
Copy link

that is a completely different issue - What's happening there is that a remote instance is giving you a post URL that isn't actually a URL but a weird wordpress thing

the spec does not allow that and failing is the correct behaviour

that is a completely different issue - What's happening there is that a remote instance is giving you a post URL that isn't actually a URL but a weird wordpress thing the spec does not allow that and failing is the correct behaviour
👍 1
ahfrom commented 2024-05-12 10:56:15 +00:00
Copy link

@floatingghost Sorry! I shouldn't pollute this thread then. It seems to be failing very hard though? In any context where this post occurs, I'm getting nothing else too.
https://recurse.social/@lindsey/112425593807393381

@floatingghost Sorry! I shouldn't pollute this thread then. It seems to be failing very hard though? In any context where this post occurs, I'm getting nothing else too. https://recurse.social/@lindsey/112425593807393381
Clovis commented 2024-05-12 17:11:05 +00:00
Contributor
Copy link

It does seem that failure is the expected outcome here. However, the timeline cannot be properly displayed to users as long as the problematic URL remains in the status that needs to be displayed. Once the faulty status is deleted, the timeline functions correctly again.

I apologize for adding to this thread if you feel my message is off-topic. In my opinion, the symptoms appeared similar enough, but I might have misunderstood things completely!

It does seem that failure is the expected outcome here. However, the timeline cannot be properly displayed to users as long as the problematic URL remains in the status that needs to be displayed. Once the faulty status is deleted, the timeline functions correctly again. I apologize for adding to this thread if you feel my message is off-topic. In my opinion, the symptoms appeared similar enough, but I might have misunderstood things completely!
Sign in to join this conversation.
No Branch/Tag specified
Branches Tags
develop
better-stats
stable
translations
customizable-docker-db
static-adminfe
bad-uploader-config-warning
docker
user-expiry-backfill
config-db-keys
ensure-scrubbers-loaded
otp26
code-shaking-does-not-go-brr
elixir1.15
circleci
backup-fixes
mod-task
policy
mrf-coolness
frontend-switcher-9000
contentMap
language-on-posts
rabbit
credo-on-pr
unify-http
normalise-markup-by-default
buildx
v3.13.3
v3.13.2
v2.6.3
v3.13.1
v3.13.0
snac-3.12.1.2
v3.12.2
snac-3.12.1.1
v3.12.1
v3.12.0
v3.11.0
v2.6.2
v2.6.1
2.6.1
v2.6.0
v2.5.5
v3.10.4
v3.10.3
v3.10.2
v3.10.1
v3.10.0
v2.5.4
v2.5.3
v3.9.3
v3.9.2
v3.9.1
V3.9.0
2023.05
v3.8.0
v3.7.1
v3.7.0
v3.6.0
v3.5.0
v2.4.5
v3.4.0
v3.3.1
v2.4.4
v3.3.0
v3.2.3
v3.2.2
v3.2.1
stable-202209
v3.2.0
v3.1.0
stable-2022.07
v3.0.0
v0.0.1
v2.5.2-6
v2.5.2-5
v2.5.2-4
v2.5.2-3
v2.5.2-2
v2.5.2-1
v2.5.2
v2.5.1
v2.5.0-8
v2.5.0-7
v2.5.0-6
v2.5.0-5
v2.5.0-4
v2.5.0-3
v2.5.0-2
v2.5.0-1
v2.5.0
v2.4.3
pre-rebase
v2.4.2
v2.4.1
v2.4.0
soapbox-v1.1.1
soapbox-v1.1.0
soapbox-v1.0.0
v2.3.0
v2.2.2
v2.2.1
v2.2.0
v2.1.2
v2.1.1
v2.1.0
v2.0.7
v2.0.6
v2.0.5
v2.0.4
v2.0.3
v2.0.2
v2.0.1
v2.0.0
v1.1.9
v1.1.8
v1.1.7
v1.1.6
v1.1.5
v1.1.4
v1.1.3
v1.1.2
v1.1.1
v1.1.0
v1.0.91
v1.0.90
v1.0.7
v1.0.6
v1.0.5
v1.0.4
v1.0.3
v1.0.2
v1.0.1
v1.0.0
v0.9.0
Labels
Clear labels   
approved, awaiting change

   
bug

Something is not working

  
configuration

This requires config changes

  
documentation

This is about human-readable docs

  
duplicate

This issue or pull request already exists

  
enhancement

New feature

  
extremely low priority

   
feature request

Something new?

  
Fix it yourself

Support has been attempted

  
help wanted

Need some help

  
invalid

Something is wrong

  
mastodon_api

   
needs docs

   
needs tests

   
not a bug

   
planned

   
pleroma_api

   
privacy

   
question

More information is needed

  
static_fe

   
triage

   
wontfix

This won't be fixed

No labels
approved, awaiting change
bug
configuration
documentation
duplicate
enhancement
extremely low priority
feature request
Fix it yourself
help wanted
invalid
mastodon_api
needs docs
needs tests
not a bug
planned
pleroma_api
privacy
question
static_fe
triage
wontfix
Milestone
Clear milestone
No items
No milestone
Projects
Clear projects
No items
No project
Assignees
Clear assignees
No assignees
4 participants
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference: AkkomaGang/akkoma#533
No description provided.
Delete branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?