Handle failed fetches a bit better #743

Merged
floatingghost merged 34 commits from failed-fetch-processing into develop 2024-04-19 11:25:14 +00:00

34 Commits

Author SHA1 Message Date
Floatingghost 370576474c only consider :op and :id args in duplicate checks
ci/woodpecker/push/build-amd64 Pipeline is pending Details
ci/woodpecker/push/build-arm64 Pipeline is pending Details
ci/woodpecker/push/docs Pipeline is pending Details
ci/woodpecker/push/lint Pipeline is pending Details
ci/woodpecker/push/test Pipeline is pending Details
ci/woodpecker/pr/lint Pipeline was successful Details
ci/woodpecker/pr/test Pipeline was successful Details
ci/woodpecker/pr/build-amd64 unknown status Details
ci/woodpecker/pr/build-arm64 unknown status Details
ci/woodpecker/pr/docs unknown status Details
2024-04-19 11:39:27 +01:00
Floatingghost 123db1abc4 Merge branch 'develop' into failed-fetch-processing
ci/woodpecker/push/build-amd64 Pipeline is pending Details
ci/woodpecker/push/build-arm64 Pipeline is pending Details
ci/woodpecker/push/docs Pipeline is pending Details
ci/woodpecker/push/lint Pipeline is pending Details
ci/woodpecker/push/test Pipeline is pending Details
ci/woodpecker/pr/lint Pipeline was successful Details
ci/woodpecker/pr/test Pipeline was successful Details
ci/woodpecker/pr/build-amd64 unknown status Details
ci/woodpecker/pr/build-arm64 unknown status Details
ci/woodpecker/pr/docs unknown status Details
2024-04-16 12:35:54 +01:00
Floatingghost b2c29527fb make xmerl shut up about markup
ci/woodpecker/push/build-amd64 Pipeline is pending Details
ci/woodpecker/push/build-arm64 Pipeline is pending Details
ci/woodpecker/push/docs Pipeline is pending Details
ci/woodpecker/push/lint Pipeline is pending Details
ci/woodpecker/push/test Pipeline is pending Details
ci/woodpecker/pr/lint Pipeline was successful Details
ci/woodpecker/pr/test Pipeline failed Details
ci/woodpecker/pr/build-amd64 unknown status Details
ci/woodpecker/pr/build-arm64 unknown status Details
ci/woodpecker/pr/docs unknown status Details
2024-04-16 10:19:30 +01:00
Floatingghost d2cee15c15 mix format says no
ci/woodpecker/push/build-amd64 Pipeline is pending Details
ci/woodpecker/push/build-arm64 Pipeline is pending Details
ci/woodpecker/push/docs Pipeline is pending Details
ci/woodpecker/push/lint Pipeline is pending Details
ci/woodpecker/push/test Pipeline is pending Details
ci/woodpecker/pr/lint Pipeline was successful Details
ci/woodpecker/pr/test Pipeline was successful Details
ci/woodpecker/pr/build-arm64 unknown status Details
ci/woodpecker/pr/docs unknown status Details
ci/woodpecker/pr/build-amd64 unknown status Details
2024-04-16 03:07:28 +01:00
Floatingghost d70fa16383 oban options should be a keyword list
ci/woodpecker/push/build-amd64 Pipeline is pending Details
ci/woodpecker/push/build-arm64 Pipeline is pending Details
ci/woodpecker/push/docs Pipeline is pending Details
ci/woodpecker/push/lint Pipeline is pending Details
ci/woodpecker/push/test Pipeline is pending Details
ci/woodpecker/pr/lint Pipeline failed Details
ci/woodpecker/pr/test unknown status Details
ci/woodpecker/pr/build-arm64 unknown status Details
ci/woodpecker/pr/build-amd64 unknown status Details
ci/woodpecker/pr/docs unknown status Details
2024-04-16 02:58:50 +01:00
Floatingghost 5043571084 Enable oban job uniqueness
ci/woodpecker/push/build-amd64 Pipeline is pending Details
ci/woodpecker/push/build-arm64 Pipeline is pending Details
ci/woodpecker/push/docs Pipeline is pending Details
ci/woodpecker/push/lint Pipeline is pending Details
ci/woodpecker/push/test Pipeline is pending Details
ci/woodpecker/pr/build-amd64 Pipeline is pending Details
ci/woodpecker/pr/build-arm64 Pipeline is pending Details
ci/woodpecker/pr/docs Pipeline is pending Details
ci/woodpecker/pr/lint Pipeline is pending Details
ci/woodpecker/pr/test Pipeline is pending Details
by default just prevent job floods with a 1-seconds
uniqueness check, but override in RemoteFetcherWorker
for 5 minute uniqueness check over all states

:infinity is an option we can go for maybe at some point,
but that would prevent any refetches so maybe not idk.
2024-04-16 02:53:24 +01:00
Floatingghost 1896ff1ab0 changelog entry
ci/woodpecker/push/build-amd64 Pipeline is pending Details
ci/woodpecker/push/build-arm64 Pipeline is pending Details
ci/woodpecker/push/docs Pipeline is pending Details
ci/woodpecker/push/lint Pipeline is pending Details
ci/woodpecker/push/test Pipeline is pending Details
ci/woodpecker/pr/build-amd64 Pipeline is pending Details
ci/woodpecker/pr/build-arm64 Pipeline is pending Details
ci/woodpecker/pr/docs Pipeline is pending Details
ci/woodpecker/pr/lint Pipeline is pending Details
ci/woodpecker/pr/test Pipeline is pending Details
2024-04-16 02:35:59 +01:00
Floatingghost b7dd739de1 Make sure we return the right format for oban 2024-04-16 02:35:21 +01:00
Floatingghost 2fc25980d1 fix pattern matching in fetch errors
ci/woodpecker/push/build-amd64 Pipeline is pending Details
ci/woodpecker/push/build-arm64 Pipeline is pending Details
ci/woodpecker/push/docs Pipeline is pending Details
ci/woodpecker/push/lint Pipeline is pending Details
ci/woodpecker/push/test Pipeline is pending Details
ci/woodpecker/pr/lint Pipeline was successful Details
ci/woodpecker/pr/test Pipeline was successful Details
ci/woodpecker/pr/build-arm64 unknown status Details
ci/woodpecker/pr/build-amd64 unknown status Details
ci/woodpecker/pr/docs unknown status Details
2024-04-13 23:55:26 +01:00
Floatingghost 18442dcc7e Fix quote test 2024-04-13 23:05:52 +01:00
Floatingghost 33fb74043d Bring our adjustments into line with atom-failure 2024-04-13 22:56:04 +01:00
Floatingghost 49ed27cd96 require logger 2024-04-13 22:25:31 +01:00
Floatingghost 7f6e35ece4 formatting 2024-04-12 20:33:33 +01:00
Mark Felder 2e369aef71 Allow the Remote Fetcher to attempt fetching an unreachable instance 2024-04-12 20:33:21 +01:00
Mark Felder fed7a78c77 Oban jobs should be discarded on permanent errors 2024-04-12 20:33:17 +01:00
Mark Felder c0532bcae0 Handle 401s as I have observed it in the wild 2024-04-12 20:33:11 +01:00
Mark Felder f31b262aec Improve test descriptions 2024-04-12 20:32:38 +01:00
Mark Felder ff515c05c3 Prevent requeuing Remote Fetcher jobs that exceed thread depth 2024-04-12 20:32:31 +01:00
Mark Felder 7e5004b3e2 Leverage existing atoms as return errors for the object fetcher 2024-04-12 20:32:13 +01:00
Mark Felder 53a9413b95 Formatting 2024-04-12 20:31:40 +01:00
Mark Felder d69cba1b93 Remove duplicate log messages from Transmogrifier
Object fetch errors are logged in the fetcher module
2024-04-12 20:31:31 +01:00
Mark Felder 3c54f407c5 Conslidate log messages for object fetcher failures and leverage Logger.metadata 2024-04-12 20:30:38 +01:00
Mark Felder 825ae46bfa Set Logger level to error 2024-04-12 20:29:33 +01:00
Mark Felder 331710b6bb RemoteFetcherWorker Oban job tests 2024-04-12 20:29:28 +01:00
Mark Felder eeed051a0f Fix detection of user follower collection being private
We were overzealous with matching on a raw error from the object fetch that should have never been relied on like this. If we can't fetch successfully we should assume that the collection is private.

Building a more expressive and universal error struct to match on may be something to consider.
2024-04-12 20:29:11 +01:00
Mark Felder 30d63aaa6e Revert "Mark instances as unreachable when returning a 403 from an object fetch"
This reverts commit d472bafec19cee269e7c943bafae7c805785acd7.
2024-04-12 20:28:56 +01:00
Mark Felder e2b04fac5a Skip remote fetch jobs for unreachable instances 2024-04-12 20:28:36 +01:00
Mark Felder 6d368808d3 Remove mistaken duplicate fetch 2024-04-12 20:28:31 +01:00
Mark Felder 160d113b30 Changelogs 2024-04-12 20:28:26 +01:00
Mark Felder 132036f951 Cancel remote fetch jobs for deleted objects 2024-04-12 20:28:21 +01:00
Mark Felder 4ff22a409a Consolidate the HTTP status code checking into the private get_object/1 2024-04-12 20:28:16 +01:00
Mark Felder 4c29366fe5 Mark instances as unreachable when returning a 403 from an object fetch
This is a definite sign the instance is blocked and they are enforcing authorized_fetch
2024-04-12 20:27:33 +01:00
Mark Felder ac4cc619ea Fix Transmogrifier tests
These tests relied on the removed Fetcher.fetch_object_from_id!/2 function injecting the error tuple into a log message with the exact words "Object containment failed."

We will keep this behavior by generating a similar log message, but perhaps this should do a better job of matching on the error tuple returned by Transmogrifier.handle_incoming/1
2024-04-12 20:26:56 +01:00
Mark Felder c241b5b09f Remove Fetcher.fetch_object_from_id!/2
It was only being called once and can be replaced with a case statement.
2024-04-12 20:26:28 +01:00