receiver_worker: prevent duplicate jobs #886

Merged

Oneric merged 1 commit from Oneric/akkoma:receive_dedupe into develop

2025-05-09 19:13:15 +00:00

Oneric commented

2025-03-18 00:31:12 +00:00

Owner

Logically this somewhat depends on the first commit from #882, though if merged earlier the only effect is a less comprehensive duplicate filtering.

E.g. *oma federates (most) follower-only posts multiple times
to each personal inbox. This commonly leads to race conditions
with jobs of several copies running at the same time and getting
past the initial "already known" check but then later all but
one will crash with an exception from the unique db index.

Since the only special thing we do with copies anyway is to discard them,
just don't create such duplicate jobs in the first place.
For the same reason and since failed jobs don't count towards
duplicates, this should have virtually no effect on federation.

Logically this somewhat depends on the first commit from #882, though if merged earlier the only effect is a less comprehensive duplicate filtering. ---- E.g. \*oma federates (most) follower-only posts multiple times to each personal inbox. This commonly leads to race conditions with jobs of several copies running at the same time and getting past the initial "already known" check but then later all but one will crash with an exception from the unique db index. Since the only special thing we do with copies anyway is to discard them, just don't create such duplicate jobs in the first place. For the same reason and since failed jobs don't count towards duplicates, this should have virtually no effect on federation.

Oneric added 1 commit 2025-03-18 00:31:12 +00:00

receiver_worker: prevent duplicate jobs

ci/woodpecker/pr/build-amd64 Pipeline is pending approval

Details

ci/woodpecker/pr/build-arm64 Pipeline is pending approval

Details

ci/woodpecker/pr/docs Pipeline is pending approval

Details

ci/woodpecker/pr/lint Pipeline is pending approval

Details

ci/woodpecker/pr/test/1 Pipeline is pending approval

Details

ci/woodpecker/pr/test/2 Pipeline is pending approval

Details

1cb08fd306

E.g. \*oma federates (most) follower-only posts multiple times
to each personal inbox. This commonly leads to race conditions
with jobs of several copies running at the same time and getting
past the initial "already known" check but then later all but
one will crash with an exception from the unique db index.

Since the only special thing we do with copies anyway is to discard them,
just don't create such duplicate jobs in the first place.
For the same reason and since failed jobs don't count towards
duplicates, this should have virtually no effect on federation.

Oneric force-pushed receive_dedupe from 1cb08fd306 to 195042bdc9

2025-03-18 02:47:25 +00:00

Compare