luna commented

Contributor

I'll be taking the same text I made for Pleroma's proposal as I want to upstream this to both projects. Feel free to show feedback.

Problem Statement

I've been recently thinking about authorized_fetch_mode and how broken my federation became when I first enabled it years ago. Nowadays, with the recent network influx, I'm thinking of enabling it again, but I worry about how unreachable my instance would become if I enabled it, I don't have that sort of data to really decide if I should enable it or not.

Suggestion

Part 1: Create a new attribute under the Instance entity, called has_request_signatures. While reachable refers to the overall health of the network connection, has_request_signatures would be set on requests that contain valid HTTP Signatures (possibly on Pleroma.Web.Plugs.HTTPSignaturePlug?).

Part 2: An instance admin can then (with possibly a separate script or mix task) calculate the set of instances that they would not be able to see if authorized_fetch_mode is enabled, it would be a more informed decision than the current one of taking guesses.

I'll be taking the same text I made for [Pleroma's proposal](https://git.pleroma.social/pleroma/pleroma/-/issues/2987) as I want to upstream this to both projects. Feel free to show feedback. --- # Problem Statement I've been recently thinking about `authorized_fetch_mode` and how broken my federation became when I first enabled it years ago. Nowadays, with the recent network influx, I'm thinking of enabling it again, but I worry about **how** unreachable my instance would become if I enabled it, I don't have that sort of data to really decide if I should enable it or not. # Suggestion Part 1: Create a new attribute under the `Instance` entity, called `has_request_signatures`. While `reachable` refers to the overall health of the network connection, `has_request_signatures` would be set on requests that contain valid HTTP Signatures (possibly on `Pleroma.Web.Plugs.HTTPSignaturePlug`?). Part 2: An instance admin can then (with possibly a separate script or `mix` task) calculate the set of instances that they would not be able to see if `authorized_fetch_mode` is enabled, it would be a more informed decision than the current one of taking guesses.

👍 1

nocebo commented

2022-11-24 00:05:43 +00:00

Contributor

just as a note, http request signatures are required in practice (and thus universally supported) by all software for push federation

you would want to specifically record signed fetches, which only started being supported by fedi software since introduction in mastodon 3.0 (iirc)

just as a note, http request signatures are required in practice (and thus universally supported) by all software for push federation you would want to specifically record signed *fetches*, which only started being supported by fedi software since introduction in mastodon 3.0 (iirc)

luna commented

2022-11-24 02:06:49 +00:00

Author

Contributor

Indeed, that's why I suggested implementing the has_request_signatures recording on the HTTP Signature plug, as that's the one that also holds the enforcement of such through authorized_fetch_mode. I got the core part of this proposal finished, will be testing in production soon to find out some interesting numbers.

Indeed, that's why I suggested implementing the `has_request_signatures` recording on the HTTP Signature plug, as that's the one that also holds the enforcement of such through `authorized_fetch_mode`. I got [the core part of this proposal](https://git.pleroma.social/luna/pleroma/-/commit/05875345c28661e30228ac15954b2d892250a400) finished, will be testing in production soon to find out some interesting numbers.

👍 1

luna commented

2022-11-24 04:07:40 +00:00

Author

Contributor

Some hours after applying the patch to my instance and I can see some results: 165 instances that confirmed themselves as having signed fetches, while 1302 instances missing (could be either because they're unreachable, as my instance is quite old, or they haven't gotten around to fetching any of my posts yet). Even with those remarks, the numbers are interesting.

Some hours after applying the patch to my instance and I can see some results: 165 instances that *confirmed* themselves as having signed fetches, while 1302 instances missing (could be either because they're unreachable, as my instance is quite old, or they haven't gotten around to fetching any of my posts yet). Even with those remarks, the numbers are interesting.

screenie-2022-11-24-01_05_57.png

3.9 KiB

floatingghost commented

2022-11-24 11:23:09 +00:00

Owner

my extremely hot take is that authorized fetch mode should be default - it's de facto for most instances anyhow

but in lieu of that, this is not a bad stopgap

the one thing that does concern me is that the patch as-is runs an SQL query on every request, which could be a spam vector - you might want to consider adding a cache for instances to prevent this, and also consider if we'd ever need to go "backwards", so to say

that is, would the state ever realistically go true -> false?

my extremely hot take is that authorized fetch mode should be default - it's de facto for most instances anyhow but in lieu of that, this is not a bad stopgap the one thing that does concern me is that the patch as-is runs an SQL query on every request, which could be a spam vector - you might want to consider adding a cache for instances to prevent this, and also consider if we'd ever need to go "backwards", so to say that is, would the state ever realistically go true -> false?

👍 1

luna commented

2022-11-24 16:38:27 +00:00

Author

Contributor

you might want to consider adding a cache for instances to prevent this

Agreed, will be adding to the patch.

that is, would the state ever realistically go true -> false?

Maybe if an instance has switched their actor keys, maybe? In theory that would mean it's a different instance (say, someone bought the domain after some years of an instance being dead), and so our statistics for that domain should be reset.

We wouldn't be able to detect if an instance has turned off their signed fetches after they've turned it on, as we wouldn't be able to assert their identity at that rate (and I don't think reverse DNS of the requester's IP is a good idea to infer identity).

> you might want to consider adding a cache for instances to prevent this Agreed, will be adding to the patch. > that is, would the state ever realistically go true -> false? Maybe if an instance has switched their actor keys, maybe? In theory that would mean it's a different instance (say, someone bought the domain after some years of an instance being dead), and so our statistics for that domain should be reset. We wouldn't be able to detect if an instance has turned off their signed fetches *after* they've turned it on, as we wouldn't be able to assert their identity at that rate (and I don't think reverse DNS of the requester's IP is a good idea to infer identity).

luna referenced a pull request that will close this issue

2022-11-25 15:20:38 +00:00

Add Signed Fetch Statistics #312

floatingghost closed this issue

2022-11-26 19:22:56 +00:00

floatingghost referenced this issue from a commit

2022-11-26 19:22:57 +00:00

Add Signed Fetch Statistics (#312)

luna referenced this issue

2022-11-26 20:09:25 +00:00

Reset instance data based on heuristics #315

Instance HTTP Signatures Statistics #304

Problem Statement

Suggestion