Possible link preview enhancements #320

New Issue

Open

opened 2022-11-27 23:15:13 +00:00 by luna · 1 comment

luna commented

2022-11-27 23:15:13 +00:00

Contributor

It is a known issue that link previews on the fediverse are weird. They can become DDoS vectors as the network instance count increases, and while a Hug-of-Death is expected if your link is interesting enough (HN frontpage, for example), at the moment it's all automated.

I wouldn't say this is that big of an issue at the moment, but I do want to bring into question what could we do about it.

One proposal I have in my mind is:

Extend AS2 vocabulary with Link Preview metadata for each status.
Instances would receive it and show on their respective APIs.
To prevent malicious attached previews, double-check with the upstream link after a random amount of minutes from the status being received.
- If it matches, increase some form of trustworthiness score to the instance.
- If it doesn't, decrease the score accordingly.
If an instance's score on their attached previews gets too low, deny their previews from appearing in the API, instead relying on upstream.

There are drawbacks, such as this is an extension and we'd want it to be specified in a way that other AP implementations could take it (which might be hard because there's some 3 different ways you can create previews), which is why I'm proposing to see what comes out (I looked a bit after writing this and it looks like Mastodon could go down the same path with https://github.com/mastodon/mastodon/issues/12738, but the status is unknown on that proposal).

It is a known issue that [link previews on the fediverse are weird](https://jort.link). They can become DDoS vectors as the network instance count increases, and while a Hug-of-Death is expected if your link is interesting enough (HN frontpage, for example), at the moment it's all automated. I wouldn't say this is that big of an issue *at the moment*, but I do want to bring into question what could we do about it. One proposal I have in my mind is: - Extend AS2 vocabulary with Link Preview metadata for each status. - Instances would receive it and show on their respective APIs. - To prevent malicious attached previews, double-check with the upstream link after a random amount of minutes from the status being received. - If it matches, increase some form of trustworthiness score to the instance. - If it doesn't, decrease the score accordingly. - If an instance's score on their attached previews gets too low, deny their previews from appearing in the API, instead relying on upstream. There are drawbacks, such as this is an extension and we'd want it to be specified in a way that other AP implementations could take it (which might be hard because there's some 3 different ways you can create previews), which is why I'm proposing to see what comes out (I looked a bit after writing this and it looks like Mastodon could go down the same path with https://github.com/mastodon/mastodon/issues/12738, but the status is unknown on that proposal).

tnhh commented

2022-11-28 17:29:42 +00:00

This is probably bigger than just link previews, but one could imagine some sort of distributed reputation scheme, as used to be popular in peer-to-peer networks. See for instance https://web.archive.org/web/20090311041434id_/http://mailman.cs.indiana.edu/~minaxi/pubs/reputation.pdf

Another thought: might it be possible to have an allowlist for the metadata parser rather than a blocklist? I'm thinking of disabling link previews on my instance because I don't want to impose on small websites. But (in my mind at least) it isn't too much of an imposition to allow link previews for large websites.

This sounds worth exploring. But this would check every link preview so the DDoS would still occur, just over a longer timescale. Perhaps instances with higher trustworthiness could be checked less frequently. This is probably bigger than just link previews, but one could imagine some sort of distributed reputation scheme, as used to be popular in peer-to-peer networks. See for instance https://web.archive.org/web/20090311041434id_/http://mailman.cs.indiana.edu/~minaxi/pubs/reputation.pdf Another thought: might it be possible to have an allowlist for the metadata parser rather than a blocklist? I'm thinking of disabling link previews on my instance because I don't want to impose on small websites. But (in my mind at least) it isn't too much of an imposition to allow link previews for large websites.

No Milestone

No project

No Assignees

2 Participants

Notifications

Due Date

The due date is invalid or out of range. Please use the format 'yyyy-mm-dd'.

No due date set.

Dependencies

No dependencies set.

Reference: AkkomaGang/akkoma#320

No description provided.