8878 commits

Author	SHA1	Message	Date
Oneric	34a48cb87f	scheduled_activity: mark private functions as private ... And remove unused due_activities/1	2024-05-22 20:18:08 +02:00
floatingghost	76ded10a70	Merge pull request 'Backoff on HTTP requests when 429 is recieved' (#762) from backoff-http into develop ... Reviewed-on: #762	2024-05-11 04:38:47 +00:00
Floatingghost	4457928e32	duct-tape fix for #438 ... we really need to make this less manual	2024-05-11 05:30:18 +01:00
Floatingghost	bd74693db6	additionally support retry-after values	2024-05-06 23:34:48 +01:00
Floatingghost	010e8c7bb2	where were you when lint fail	2024-04-26 19:28:01 +01:00
Floatingghost	f531484063	Merge branch 'develop' into backoff-http	2024-04-26 19:06:18 +01:00
Floatingghost	ec7e9da734	Correct ttl syntax for new cachex	2024-04-26 19:05:12 +01:00
FloatingGhost	3c384c1b76	Add ratelimit backoff to HTTP get	2024-04-26 19:01:12 +01:00
FloatingGhost	2437a3e9ba	add test for backoff	2024-04-26 19:01:01 +01:00
FloatingGhost	ad7dcf38a8	Add HTTP backoff cache to respect 429s	2024-04-26 19:00:35 +01:00
Floatingghost	828158ef49	Merge remote-tracking branch 'oneric/fedfix-public-ld' into develop	2024-04-26 18:49:31 +01:00
Oneric	5ee0fb18cb	exiftool: make stripped tags configurable	2024-04-26 18:57:24 +02:00
Oneric	a95af3ee4c	exiftool: strip all non-essential tags ... Documentation was already clear on this only stripping GPS tags. But there are more potentially sensitive metadata tags (e.g. author and possibly description) and the name alone suggests a broader effect. Thus change the filter to strip all metadata except for colourspace info and orientation (technically it strips everything and then readds selected tags). Explicitly stripping CommonIFD0 is needed since -all does not modify IFD0 due to TIFF storing some actual image data there. CommonIFD0 then strips a bunch of commonly used actual metadata tags from IFD0, to my understanding leaving TIFF image data and custom metadata tags intact.	2024-04-25 23:00:42 +02:00
Oneric	163cb1d5e0	exiftool: strip JXL and HEIC ... As of exiftool 12.57 both formats are supported, but EXIF data is optional for JXL and if exiftool doesn’t find a preexisting metadata chunk it will create one and treat it as a minor error resulting in a non-zero exit code. Setting -ignoreMinorErrors avoids failing on such uploads.	2024-04-25 23:00:42 +02:00
Oneric	b0a46c1e2e	Normalise public adressing to fix federation ... Due to JSON-LD compaction the full address of public scope may also occur in shorter forms and the spec requires us to treat them all equivalently. To save us the pain of repeatedly checking for all variants internally, normalise inbound data to just one form. See note at: https://www.w3.org/TR/activitypub/#public-addressing This needs to happen very early, even before the other addressing fixes else an earlier validator will reject the object. This in turn required to move the list-tpye normalisation earlier as well, but since I was unsure about putting empty lists into the data when no such field existed before, I excluded this case and thus the later fixing had to be kept as well. Fixes: #670	2024-04-25 18:45:16 +02:00
floatingghost	b1c6621e66	Merge pull request 'Read image description from EXIF data' (#744) from timorl/akkoma:elseinspe into develop ... Reviewed-on: #744	2024-04-25 12:52:31 +00:00
floatingghost	764dbeded4	Merge pull request 'Accept all standard actor types' (#751) from Oneric/akkoma:all-actor-types into develop ... Reviewed-on: #751	2024-04-24 17:09:02 +00:00
floatingghost	80e1c094c7	Merge pull request 'Don't strip newlines in pre' (#709) from snan/akkoma:pre into develop ... Reviewed-on: #709	2024-04-24 17:00:34 +00:00
floatingghost	4a0e90e8a8	Merge pull request 'ReceiverWorker: Make sure non-{:ok, _} is returned as {:error, …}' (#753) from Oneric/akkoma:receive-worker-return into develop ... Reviewed-on: #753	2024-04-24 17:00:18 +00:00
Oneric	83f75c3e93	Accept all standard actor types	2024-04-23 18:14:34 +02:00
Floatingghost	92168fa5a1	Merge remote-tracking branch 'origin/develop' into who-wants-to-yeet-c2s-i-want-to-yeet-c2s	2024-04-23 14:37:05 +01:00
Floatingghost	3e199242b0	remove upload_media from AP representation	2024-04-23 14:35:52 +01:00
Haelwenn (lanodan) Monnier	0c2f200b4d	ReceiverWorker: Make sure non-{:ok, _} is returned as {:error, …} ... Otherwise an error like `{:signature, {:error, {:error, :not_found}}}` ends up considered a success. Cherry-picked-from: a299ddb10e	2024-04-21 20:58:06 +02:00
timorl	9da0fe930e	Format, but this time with a non-ancient version of elixir	2024-04-19 18:07:50 +02:00
timorl	2a9db73b4c	Merge branch 'develop' into elseinspe	2024-04-19 17:11:55 +02:00
Floatingghost	370576474c	only consider :op and :id args in duplicate checks	2024-04-19 11:39:27 +01:00
Floatingghost	1ed975636b	Keep READ endpoints, purge WRITE	2024-04-19 11:06:01 +01:00
timorl	cd7af81896	Rename StripLocation to StripMetadata for temporal-proofing reasons	2024-04-16 20:37:00 +02:00
Floatingghost	ddb8a5ef73	yeet AP C2S support ... literally nothing uses C2S AP, and it's another route into core systems which requires analysis and maintenance. A second API is just extra surface for potentially bad things so let's take it out back and obliterate it	2024-04-16 13:55:03 +01:00
Floatingghost	123db1abc4	Merge branch 'develop' into failed-fetch-processing	2024-04-16 12:35:54 +01:00
Floatingghost	b2c29527fb	make xmerl shut up about markup	2024-04-16 10:19:30 +01:00
timorl	59d32c10d9	Formatting	2024-04-16 08:02:13 +02:00
Floatingghost	d2cee15c15	mix format says no	2024-04-16 03:07:28 +01:00
Floatingghost	d70fa16383	oban options should be a keyword list	2024-04-16 02:58:50 +01:00
Floatingghost	5043571084	Enable oban job uniqueness ... by default just prevent job floods with a 1-seconds uniqueness check, but override in RemoteFetcherWorker for 5 minute uniqueness check over all states :infinity is an option we can go for maybe at some point, but that would prevent any refetches so maybe not idk.	2024-04-16 02:53:24 +01:00
Floatingghost	b7dd739de1	Make sure we return the right format for oban	2024-04-16 02:35:21 +01:00
timorl	b144218dce	Merge branch 'develop' into elseinspe	2024-04-14 20:31:33 +02:00
Floatingghost	2fc25980d1	fix pattern matching in fetch errors	2024-04-13 23:55:26 +01:00
floatingghost	c1f0b6b875	Merge pull request 'Accept body parameters for /api/pleroma/notification_settings' (#738) from Oneric/akkoma:notif-setting-parameters into develop ... Reviewed-on: #738	2024-04-13 22:55:02 +00:00
Floatingghost	33fb74043d	Bring our adjustments into line with atom-failure	2024-04-13 22:56:04 +01:00
Floatingghost	49ed27cd96	require logger	2024-04-13 22:25:31 +01:00
Mark Felder	2e369aef71	Allow the Remote Fetcher to attempt fetching an unreachable instance	2024-04-12 20:33:21 +01:00
Mark Felder	fed7a78c77	Oban jobs should be discarded on permanent errors	2024-04-12 20:33:17 +01:00
Mark Felder	c0532bcae0	Handle 401s as I have observed it in the wild	2024-04-12 20:33:11 +01:00
Mark Felder	ff515c05c3	Prevent requeuing Remote Fetcher jobs that exceed thread depth	2024-04-12 20:32:31 +01:00
Mark Felder	7e5004b3e2	Leverage existing atoms as return errors for the object fetcher	2024-04-12 20:32:13 +01:00
Mark Felder	53a9413b95	Formatting	2024-04-12 20:31:40 +01:00
Mark Felder	d69cba1b93	Remove duplicate log messages from Transmogrifier ... Object fetch errors are logged in the fetcher module	2024-04-12 20:31:31 +01:00
Mark Felder	3c54f407c5	Conslidate log messages for object fetcher failures and leverage Logger.metadata	2024-04-12 20:30:38 +01:00
Mark Felder	825ae46bfa	Set Logger level to error	2024-04-12 20:29:33 +01:00