9303 commits

Author	SHA1	Message	Date
Floatingghost	828158ef49	Merge remote-tracking branch 'oneric/fedfix-public-ld' into develop	2024-04-26 18:49:31 +01:00
Oneric	5ee0fb18cb	exiftool: make stripped tags configurable	2024-04-26 18:57:24 +02:00
Oneric	a95af3ee4c	exiftool: strip all non-essential tags ... Documentation was already clear on this only stripping GPS tags. But there are more potentially sensitive metadata tags (e.g. author and possibly description) and the name alone suggests a broader effect. Thus change the filter to strip all metadata except for colourspace info and orientation (technically it strips everything and then readds selected tags). Explicitly stripping CommonIFD0 is needed since -all does not modify IFD0 due to TIFF storing some actual image data there. CommonIFD0 then strips a bunch of commonly used actual metadata tags from IFD0, to my understanding leaving TIFF image data and custom metadata tags intact.	2024-04-25 23:00:42 +02:00
Oneric	163cb1d5e0	exiftool: strip JXL and HEIC ... As of exiftool 12.57 both formats are supported, but EXIF data is optional for JXL and if exiftool doesn’t find a preexisting metadata chunk it will create one and treat it as a minor error resulting in a non-zero exit code. Setting -ignoreMinorErrors avoids failing on such uploads.	2024-04-25 23:00:42 +02:00
Oneric	b0a46c1e2e	Normalise public adressing to fix federation ... Due to JSON-LD compaction the full address of public scope may also occur in shorter forms and the spec requires us to treat them all equivalently. To save us the pain of repeatedly checking for all variants internally, normalise inbound data to just one form. See note at: https://www.w3.org/TR/activitypub/#public-addressing This needs to happen very early, even before the other addressing fixes else an earlier validator will reject the object. This in turn required to move the list-tpye normalisation earlier as well, but since I was unsure about putting empty lists into the data when no such field existed before, I excluded this case and thus the later fixing had to be kept as well. Fixes: #670	2024-04-25 18:45:16 +02:00
floatingghost	b1c6621e66	Merge pull request 'Read image description from EXIF data' (#744) from timorl/akkoma:elseinspe into develop ... Reviewed-on: #744	2024-04-25 12:52:31 +00:00
floatingghost	764dbeded4	Merge pull request 'Accept all standard actor types' (#751) from Oneric/akkoma:all-actor-types into develop ... Reviewed-on: #751	2024-04-24 17:09:02 +00:00
floatingghost	80e1c094c7	Merge pull request 'Don't strip newlines in pre' (#709) from snan/akkoma:pre into develop ... Reviewed-on: #709	2024-04-24 17:00:34 +00:00
floatingghost	4a0e90e8a8	Merge pull request 'ReceiverWorker: Make sure non-{:ok, _} is returned as {:error, …}' (#753) from Oneric/akkoma:receive-worker-return into develop ... Reviewed-on: #753	2024-04-24 17:00:18 +00:00
Oneric	83f75c3e93	Accept all standard actor types	2024-04-23 18:14:34 +02:00
Floatingghost	92168fa5a1	Merge remote-tracking branch 'origin/develop' into who-wants-to-yeet-c2s-i-want-to-yeet-c2s	2024-04-23 14:37:05 +01:00
Floatingghost	3e199242b0	remove upload_media from AP representation	2024-04-23 14:35:52 +01:00
Haelwenn (lanodan) Monnier	0c2f200b4d	ReceiverWorker: Make sure non-{:ok, _} is returned as {:error, …} ... Otherwise an error like `{:signature, {:error, {:error, :not_found}}}` ends up considered a success. Cherry-picked-from: a299ddb10e	2024-04-21 20:58:06 +02:00
timorl	09d3ccf770	Read description before stripping metadata	2024-04-19 20:51:54 +02:00
timorl	9da0fe930e	Format, but this time with a non-ancient version of elixir	2024-04-19 18:07:50 +02:00
timorl	2a9db73b4c	Merge branch 'develop' into elseinspe	2024-04-19 17:11:55 +02:00
Floatingghost	370576474c	only consider :op and :id args in duplicate checks	2024-04-19 11:39:27 +01:00
Floatingghost	1ed975636b	Keep READ endpoints, purge WRITE	2024-04-19 11:06:01 +01:00
timorl	cd7af81896	Rename StripLocation to StripMetadata for temporal-proofing reasons	2024-04-16 20:37:00 +02:00
Floatingghost	ddb8a5ef73	yeet AP C2S support ... literally nothing uses C2S AP, and it's another route into core systems which requires analysis and maintenance. A second API is just extra surface for potentially bad things so let's take it out back and obliterate it	2024-04-16 13:55:03 +01:00
Floatingghost	123db1abc4	Merge branch 'develop' into failed-fetch-processing	2024-04-16 12:35:54 +01:00
Floatingghost	b2c29527fb	make xmerl shut up about markup	2024-04-16 10:19:30 +01:00
timorl	59d32c10d9	Formatting	2024-04-16 08:02:13 +02:00
Floatingghost	d2cee15c15	mix format says no	2024-04-16 03:07:28 +01:00
Floatingghost	d70fa16383	oban options should be a keyword list	2024-04-16 02:58:50 +01:00
Floatingghost	5043571084	Enable oban job uniqueness ... by default just prevent job floods with a 1-seconds uniqueness check, but override in RemoteFetcherWorker for 5 minute uniqueness check over all states :infinity is an option we can go for maybe at some point, but that would prevent any refetches so maybe not idk.	2024-04-16 02:53:24 +01:00
Floatingghost	b7dd739de1	Make sure we return the right format for oban	2024-04-16 02:35:21 +01:00
timorl	b144218dce	Merge branch 'develop' into elseinspe	2024-04-14 20:31:33 +02:00
Floatingghost	2fc25980d1	fix pattern matching in fetch errors	2024-04-13 23:55:26 +01:00
floatingghost	c1f0b6b875	Merge pull request 'Accept body parameters for /api/pleroma/notification_settings' (#738) from Oneric/akkoma:notif-setting-parameters into develop ... Reviewed-on: #738	2024-04-13 22:55:02 +00:00
Floatingghost	33fb74043d	Bring our adjustments into line with atom-failure	2024-04-13 22:56:04 +01:00
Floatingghost	49ed27cd96	require logger	2024-04-13 22:25:31 +01:00
Mark Felder	2e369aef71	Allow the Remote Fetcher to attempt fetching an unreachable instance	2024-04-12 20:33:21 +01:00
Mark Felder	fed7a78c77	Oban jobs should be discarded on permanent errors	2024-04-12 20:33:17 +01:00
Mark Felder	c0532bcae0	Handle 401s as I have observed it in the wild	2024-04-12 20:33:11 +01:00
Mark Felder	ff515c05c3	Prevent requeuing Remote Fetcher jobs that exceed thread depth	2024-04-12 20:32:31 +01:00
Mark Felder	7e5004b3e2	Leverage existing atoms as return errors for the object fetcher	2024-04-12 20:32:13 +01:00
Mark Felder	53a9413b95	Formatting	2024-04-12 20:31:40 +01:00
Mark Felder	d69cba1b93	Remove duplicate log messages from Transmogrifier ... Object fetch errors are logged in the fetcher module	2024-04-12 20:31:31 +01:00
Mark Felder	3c54f407c5	Conslidate log messages for object fetcher failures and leverage Logger.metadata	2024-04-12 20:30:38 +01:00
Mark Felder	825ae46bfa	Set Logger level to error	2024-04-12 20:29:33 +01:00
Mark Felder	eeed051a0f	Fix detection of user follower collection being private ... We were overzealous with matching on a raw error from the object fetch that should have never been relied on like this. If we can't fetch successfully we should assume that the collection is private. Building a more expressive and universal error struct to match on may be something to consider.	2024-04-12 20:29:11 +01:00
Mark Felder	30d63aaa6e	Revert "Mark instances as unreachable when returning a 403 from an object fetch" ... This reverts commit d472bafec19cee269e7c943bafae7c805785acd7.	2024-04-12 20:28:56 +01:00
Mark Felder	e2b04fac5a	Skip remote fetch jobs for unreachable instances	2024-04-12 20:28:36 +01:00
Mark Felder	6d368808d3	Remove mistaken duplicate fetch	2024-04-12 20:28:31 +01:00
Mark Felder	132036f951	Cancel remote fetch jobs for deleted objects	2024-04-12 20:28:21 +01:00
Mark Felder	4ff22a409a	Consolidate the HTTP status code checking into the private get_object/1	2024-04-12 20:28:16 +01:00
Mark Felder	4c29366fe5	Mark instances as unreachable when returning a 403 from an object fetch ... This is a definite sign the instance is blocked and they are enforcing authorized_fetch	2024-04-12 20:27:33 +01:00
Mark Felder	ac4cc619ea	Fix Transmogrifier tests ... These tests relied on the removed Fetcher.fetch_object_from_id!/2 function injecting the error tuple into a log message with the exact words "Object containment failed." We will keep this behavior by generating a similar log message, but perhaps this should do a better job of matching on the error tuple returned by Transmogrifier.handle_incoming/1	2024-04-12 20:26:56 +01:00
Mark Felder	c241b5b09f	Remove Fetcher.fetch_object_from_id!/2 ... It was only being called once and can be replaced with a case statement.	2024-04-12 20:26:28 +01:00