370576474c
only consider :op and :id args in duplicate checks
ci/woodpecker/push/build-amd64 Pipeline is pending
ci/woodpecker/push/build-arm64 Pipeline is pending
ci/woodpecker/push/docs Pipeline is pending
ci/woodpecker/push/lint Pipeline is pending
ci/woodpecker/push/test Pipeline is pending
ci/woodpecker/pr/lint Pipeline was successful
ci/woodpecker/pr/test Pipeline was successful
ci/woodpecker/pr/build-amd64 unknown status
ci/woodpecker/pr/build-arm64 unknown status
ci/woodpecker/pr/docs unknown status
2024-04-19 11:39:27 +01:00
123db1abc4
Merge branch 'develop' into failed-fetch-processing
ci/woodpecker/push/build-amd64 Pipeline is pending
ci/woodpecker/push/build-arm64 Pipeline is pending
ci/woodpecker/push/docs Pipeline is pending
ci/woodpecker/push/lint Pipeline is pending
ci/woodpecker/push/test Pipeline is pending
ci/woodpecker/pr/lint Pipeline was successful
ci/woodpecker/pr/test Pipeline was successful
ci/woodpecker/pr/build-amd64 unknown status
ci/woodpecker/pr/build-arm64 unknown status
ci/woodpecker/pr/docs unknown status
2024-04-16 12:35:54 +01:00
b2c29527fb
make xmerl shut up about markup
ci/woodpecker/push/build-amd64 Pipeline is pending
ci/woodpecker/push/build-arm64 Pipeline is pending
ci/woodpecker/push/docs Pipeline is pending
ci/woodpecker/push/lint Pipeline is pending
ci/woodpecker/push/test Pipeline is pending
ci/woodpecker/pr/lint Pipeline was successful
ci/woodpecker/pr/test Pipeline failed
ci/woodpecker/pr/build-amd64 unknown status
ci/woodpecker/pr/build-arm64 unknown status
ci/woodpecker/pr/docs unknown status
2024-04-16 10:19:30 +01:00
d2cee15c15
mix format says no
ci/woodpecker/push/build-amd64 Pipeline is pending
ci/woodpecker/push/build-arm64 Pipeline is pending
ci/woodpecker/push/docs Pipeline is pending
ci/woodpecker/push/lint Pipeline is pending
ci/woodpecker/push/test Pipeline is pending
ci/woodpecker/pr/lint Pipeline was successful
ci/woodpecker/pr/test Pipeline was successful
ci/woodpecker/pr/build-arm64 unknown status
ci/woodpecker/pr/docs unknown status
ci/woodpecker/pr/build-amd64 unknown status
2024-04-16 03:07:28 +01:00
d70fa16383
oban options should be a keyword list
ci/woodpecker/push/build-amd64 Pipeline is pending
ci/woodpecker/push/build-arm64 Pipeline is pending
ci/woodpecker/push/docs Pipeline is pending
ci/woodpecker/push/lint Pipeline is pending
ci/woodpecker/push/test Pipeline is pending
ci/woodpecker/pr/lint Pipeline failed
ci/woodpecker/pr/test unknown status
ci/woodpecker/pr/build-arm64 unknown status
ci/woodpecker/pr/build-amd64 unknown status
ci/woodpecker/pr/docs unknown status
2024-04-16 02:58:50 +01:00
5043571084
Enable oban job uniqueness
...
ci/woodpecker/push/build-amd64 Pipeline is pending
ci/woodpecker/push/build-arm64 Pipeline is pending
ci/woodpecker/push/docs Pipeline is pending
ci/woodpecker/push/lint Pipeline is pending
ci/woodpecker/push/test Pipeline is pending
ci/woodpecker/pr/build-amd64 Pipeline is pending
ci/woodpecker/pr/build-arm64 Pipeline is pending
ci/woodpecker/pr/docs Pipeline is pending
ci/woodpecker/pr/lint Pipeline is pending
ci/woodpecker/pr/test Pipeline is pending
by default just prevent job floods with a 1-seconds
uniqueness check, but override in RemoteFetcherWorker
for 5 minute uniqueness check over all states
:infinity is an option we can go for maybe at some point,
but that would prevent any refetches so maybe not idk.
2024-04-16 02:53:24 +01:00
1896ff1ab0
changelog entry
ci/woodpecker/push/build-amd64 Pipeline is pending
ci/woodpecker/push/test Pipeline is pending
ci/woodpecker/pr/build-amd64 Pipeline is pending
ci/woodpecker/pr/build-arm64 Pipeline is pending
ci/woodpecker/push/build-arm64 Pipeline is pending
ci/woodpecker/push/docs Pipeline is pending
ci/woodpecker/push/lint Pipeline is pending
ci/woodpecker/pr/docs Pipeline is pending
ci/woodpecker/pr/lint Pipeline is pending
ci/woodpecker/pr/test Pipeline is pending
2024-04-16 02:35:59 +01:00
b7dd739de1
Make sure we return the right format for oban
2024-04-16 02:35:21 +01:00
2fc25980d1
fix pattern matching in fetch errors
ci/woodpecker/push/build-amd64 Pipeline is pending
ci/woodpecker/push/build-arm64 Pipeline is pending
ci/woodpecker/push/docs Pipeline is pending
ci/woodpecker/push/lint Pipeline is pending
ci/woodpecker/push/test Pipeline is pending
ci/woodpecker/pr/lint Pipeline was successful
ci/woodpecker/pr/test Pipeline was successful
ci/woodpecker/pr/build-arm64 unknown status
ci/woodpecker/pr/build-amd64 unknown status
ci/woodpecker/pr/docs unknown status
2024-04-13 23:55:26 +01:00
18442dcc7e
Fix quote test
2024-04-13 23:05:52 +01:00
33fb74043d
Bring our adjustments into line with atom-failure
2024-04-13 22:56:04 +01:00
49ed27cd96
require logger
2024-04-13 22:25:31 +01:00
7f6e35ece4
formatting
2024-04-12 20:33:33 +01:00
2e369aef71
Allow the Remote Fetcher to attempt fetching an unreachable instance
2024-04-12 20:33:21 +01:00
fed7a78c77
Oban jobs should be discarded on permanent errors
2024-04-12 20:33:17 +01:00
c0532bcae0
Handle 401s as I have observed it in the wild
2024-04-12 20:33:11 +01:00
f31b262aec
Improve test descriptions
2024-04-12 20:32:38 +01:00
ff515c05c3
Prevent requeuing Remote Fetcher jobs that exceed thread depth
2024-04-12 20:32:31 +01:00
7e5004b3e2
Leverage existing atoms as return errors for the object fetcher
2024-04-12 20:32:13 +01:00
53a9413b95
Formatting
2024-04-12 20:31:40 +01:00
d69cba1b93
Remove duplicate log messages from Transmogrifier
...
Object fetch errors are logged in the fetcher module
2024-04-12 20:31:31 +01:00
3c54f407c5
Conslidate log messages for object fetcher failures and leverage Logger.metadata
2024-04-12 20:30:38 +01:00
825ae46bfa
Set Logger level to error
2024-04-12 20:29:33 +01:00
331710b6bb
RemoteFetcherWorker Oban job tests
2024-04-12 20:29:28 +01:00
eeed051a0f
Fix detection of user follower collection being private
...
We were overzealous with matching on a raw error from the object fetch that should have never been relied on like this. If we can't fetch successfully we should assume that the collection is private.
Building a more expressive and universal error struct to match on may be something to consider.
2024-04-12 20:29:11 +01:00
30d63aaa6e
Revert "Mark instances as unreachable when returning a 403 from an object fetch"
...
This reverts commit d472bafec19cee269e7c943bafae7c805785acd7.
2024-04-12 20:28:56 +01:00
e2b04fac5a
Skip remote fetch jobs for unreachable instances
2024-04-12 20:28:36 +01:00
6d368808d3
Remove mistaken duplicate fetch
2024-04-12 20:28:31 +01:00
160d113b30
Changelogs
2024-04-12 20:28:26 +01:00
132036f951
Cancel remote fetch jobs for deleted objects
2024-04-12 20:28:21 +01:00
4ff22a409a
Consolidate the HTTP status code checking into the private get_object/1
2024-04-12 20:28:16 +01:00
4c29366fe5
Mark instances as unreachable when returning a 403 from an object fetch
...
This is a definite sign the instance is blocked and they are enforcing authorized_fetch
2024-04-12 20:27:33 +01:00
ac4cc619ea
Fix Transmogrifier tests
...
These tests relied on the removed Fetcher.fetch_object_from_id!/2 function injecting the error tuple into a log message with the exact words "Object containment failed."
We will keep this behavior by generating a similar log message, but perhaps this should do a better job of matching on the error tuple returned by Transmogrifier.handle_incoming/1
2024-04-12 20:26:56 +01:00
c241b5b09f
Remove Fetcher.fetch_object_from_id!/2
...
It was only being called once and can be replaced with a case statement.
2024-04-12 20:26:28 +01:00
