Commit Graph

282 Commits

Author SHA1 Message Date
floatingghost b1c6621e66 Merge pull request 'Read image description from EXIF data' (#744) from timorl/akkoma:elseinspe into develop
ci/woodpecker/push/build-amd64 Pipeline is pending Details
ci/woodpecker/push/build-arm64 Pipeline is pending Details
ci/woodpecker/push/docs Pipeline is pending Details
ci/woodpecker/push/lint Pipeline is pending Details
ci/woodpecker/push/test Pipeline is pending Details
Reviewed-on: #744
2024-04-25 12:52:31 +00:00
Oneric 83f75c3e93 Accept all standard actor types
ci/woodpecker/pr/lint Pipeline was successful Details
ci/woodpecker/pr/test Pipeline was successful Details
ci/woodpecker/pr/build-arm64 unknown status Details
ci/woodpecker/pr/build-amd64 unknown status Details
ci/woodpecker/pr/docs unknown status Details
2024-04-23 18:14:34 +02:00
timorl 2a9db73b4c
Merge branch 'develop' into elseinspe 2024-04-19 17:11:55 +02:00
timorl b144218dce
Merge branch 'develop' into elseinspe
ci/woodpecker/pr/lint Pipeline failed Details
ci/woodpecker/pr/test unknown status Details
ci/woodpecker/pr/build-arm64 unknown status Details
ci/woodpecker/pr/build-amd64 unknown status Details
ci/woodpecker/pr/docs unknown status Details
2024-04-14 20:31:33 +02:00
Floatingghost 18442dcc7e Fix quote test 2024-04-13 23:05:52 +01:00
Oneric 8684964c5d Only allow exact id matches
This protects us from falling for obvious spoofs as from the current
upload exploit (unfortunately we can’t reasonably do anything about
spoofs with exact matches as was possible via emoji and proxy).

Such objects being invalid is supported by the spec, sepcifically
sections 3.1 and 3.2: https://www.w3.org/TR/activitypub/#obj-id

Anonymous objects are not relevant here (they can only exists within
parent objects iiuc) and neither is client-to-server or transient objects
(as those cannot be fetched in the first place).
This leaves us with the requirement for `id` to (a) exist and
(b) be a publicly dereferencable URI from the originating server.
This alone does not yet demand strict equivalence, but the spec then
further explains objects ought to be fetchable _via their ID_.
Meaning an object not retrievable via its ID, is invalid.

This reading is supported by the fact, e.g. GoToSocial (recently) and
Mastodon (for 6+ years) do already implement such strict ID checks,
additionally proving this doesn’t cause federation issues in practice.

However, apart from canonical IDs there can also be additional display
URLs. *omas first redirect those to their canonical location, but *keys
and Mastodon directly serve the AP representation without redirects.

Mastodon and GTS deal with this in two different ways,
but both constitute an effective countermeasure:
 - Mastodon:
   Unless it already is a known AP id, two fetches occur.
   The first fetch just reads the `id` property and then refetches from
   the id. The last fetch requires the returned id to exactly match the
   URL the content was fetched from. (This can be optimised by skipping
   the second fetch if it already matches)
   05eda8d193/app/helpers/jsonld_helper.rb (L168)
   63f0979799

 - GTS:
   Only does a single fetch and then checks if _either_ the id
   _or_ url property (which can be an object) match the original fetch
   URL. This relies on implementations always including their display URL
   as "url" if differing from the id. For actors this is true for all
   investigated implementations, for posts only Mastodon includes an
   "url", but it is also the only one with a differing display URL.
   2bafd7daf5 (diff-943bbb02c8ac74ac5dc5d20807e561dcdfaebdc3b62b10730f643a20ac23c24fR222)

Albeit Mastodon’s refetch offers higher compatibility with theoretical
implmentations using either multiple different display URL or not
denoting any of them as "url" at all, for now we chose to adopt a
GTS-like refetch-free approach to avoid additional implementation
concerns wrt to whether redirects should be allowed when fetching a
canonical AP id and potential for accidentally loosening some checks
(e.g. cross-domain refetches) for one of the fetches.
This may be reconsidered in the future.
2024-03-25 14:05:05 -01:00
Oneric f07eb4cb55 Sanity check fetched user data
In order to properly process incoming notes we need
to be able to map the key id back to an actor.
Also, check collections actually belong to the same server.

Key ids of Hubzilla and Bridgy samples were updated to what
modern versions of those output. If anything still uses the
old format, we would not be able to verify their posts anyway.
2024-03-25 14:05:05 -01:00
Oneric 59a142e0b0 Never fetch resource from ourselves
If it’s not already in the database,
it must be counterfeit (or just not exists at all)

Changed test URLs were only ever used from "local: false" users anyway.
2024-03-25 14:05:05 -01:00
FloatingGhost 7825798e32 Add XML matcher 2023-08-07 11:12:14 +01:00
mae d868348fac Completely disable xml entity resolution
ci/woodpecker/pr/build-amd64 Pipeline is pending Details
ci/woodpecker/pr/build-arm64 Pipeline is pending Details
ci/woodpecker/pr/docs Pipeline is pending Details
ci/woodpecker/pr/test Pipeline is pending Details
2023-08-05 12:32:05 +00:00
FloatingGhost 9c7409808f Add unit test for external entity loading
ci/woodpecker/push/build-amd64 Pipeline is pending Details
ci/woodpecker/push/build-arm64 Pipeline is pending Details
ci/woodpecker/push/docs Pipeline is pending Details
ci/woodpecker/push/test Pipeline is pending Details
2023-08-04 22:24:32 +01:00
Hélène 3227ebf1e1 CommonFixes: more predictable context generation
`context` fields for objects and activities can now be generated based
on the object/activity `inReplyTo` field or its ActivityPub ID, as a
fallback method in cases where `context` fields are missing for incoming
activities and objects.
2023-06-14 16:22:26 +00:00
ilja b4952a81fe Interpret `\n` as newline for MFM
Markdown doesn't generally consider `\n` a newline,
but Misskey does for MFM.

Now we do to for MFM (and not for Markdown) :)
2023-02-18 19:56:11 +01:00
FloatingGhost ff5793198f add inbound language test
ci/woodpecker/push/woodpecker Pipeline is pending Details
2023-01-11 15:42:13 +00:00
FloatingGhost f752126427 Remove quack, ensure adapter is finch
ci/woodpecker/push/woodpecker Pipeline is pending Details
2022-12-11 23:22:35 +00:00
floatingghost e1e0d5d759 microblogpub federation fixes (#288)
ci/woodpecker/push/woodpecker Pipeline is pending Details
Co-authored-by: FloatingGhost <hannah@coffee-and-dreams.uk>
Reviewed-on: #288
2022-11-18 11:14:35 +00:00
Ilja 66a04cead3 Descriptions from exif data with only whitespeces are considered empty
ci/woodpecker/pr/woodpecker Pipeline is pending Details
I noticed that pictures taken with Ubuntu-Touch have whitespace in one of the fields
This should just be ignored imo
2022-10-23 14:46:22 +02:00
Ilja 338612d72b Use EXIF data of image to prefill image description
During attachment upload Pleroma returns a "description" field.

* This MR allows Pleroma to read the EXIF data during upload and return the description to the FE using this field.
    * If a description is already present (e.g. because a previous module added it), it will use that
    * Otherwise it will read from the EXIF data. First it will check -ImageDescription, if that's empty, it will check -iptc:Caption-Abstract
    * If no description is found, it will simply return nil, which is the default value
* When people set up a new instance, they will be asked if they want to read metadata and this module will be activated if so

There was an Exiftool module, which has now been renamed to Exiftool.StripLocation
2022-10-23 14:46:16 +02:00
FloatingGhost b4261b0335 Use set of pregenerated RSA keys
ci/woodpecker/push/woodpecker Pipeline was successful Details
ci/woodpecker/pr/woodpecker Pipeline was successful Details
Randomness is a huge resource sink, so let's just use
a some that we made earlier
2022-09-11 20:14:58 +01:00
floatingghost aaf78e2b52 only put linked mfm in source (#171)
ci/woodpecker/push/woodpecker Pipeline was successful Details
Reviewed-on: #171
2022-08-17 09:35:11 +00:00
Hélène c1e15ff6f8 Transmogrifier: fix reply context fixing
ci/woodpecker/push/woodpecker Pipeline is pending Details
ci/woodpecker/pr/woodpecker Pipeline is pending Details
Incoming Pleroma replies to a Misskey thread were rejected due to a
broken context fix, which caused them to not be visible until a
non-Pleroma user interacted with the replies.

This fix properly sets the post-fix object context to its parent Create
activity as well, if it was changed.
2022-08-04 12:57:48 +01:00
Joel Beckmeyer e26388a01c Support reaching user@sub.domain.tld at user@domain.tld (#134)
ci/woodpecker/push/woodpecker Pipeline was successful Details
Reviewed-on: #134
Co-authored-by: Joel Beckmeyer <joel@beckmeyer.us>
Co-committed-by: Joel Beckmeyer <joel@beckmeyer.us>
2022-08-02 13:54:22 +00:00
floatingghost a3501cab86 ensure quote fetching obeys max thread distance (#119)
ci/woodpecker/push/test Pipeline was successful Details
ci/woodpecker/push/docs Pipeline was successful Details
ci/woodpecker/push/release Pipeline was successful Details
Reviewed-on: #119
2022-07-26 17:28:47 +00:00
FloatingGhost 0a55c37182 don't error out if the featured collection has a string ID
ci/woodpecker/push/test Pipeline was successful Details
ci/woodpecker/push/docs Pipeline was successful Details
ci/woodpecker/pr/release Pipeline was successful Details
ci/woodpecker/pr/docs Pipeline was successful Details
ci/woodpecker/push/release Pipeline was successful Details
ci/woodpecker/pr/test Pipeline was successful Details
2022-07-26 15:08:35 +01:00
floatingghost 1419eee5df Quote posting (#113)
ci/woodpecker/push/docs Pipeline is pending Details
ci/woodpecker/push/release Pipeline is pending Details
ci/woodpecker/push/test Pipeline is pending Details
Reviewed-on: #113
2022-07-25 16:30:06 +00:00
floatingghost c4e9c4bc95 extend custom runtime system (#108)
ci/woodpecker/push/test Pipeline was successful Details
ci/woodpecker/push/docs Pipeline was successful Details
ci/woodpecker/push/release Pipeline was successful Details
Reviewed-on: #108
2022-07-24 16:42:43 +00:00
FloatingGhost 0f132b802d purge chat and shout endpoints
ci/woodpecker/push/docs Pipeline was successful Details
ci/woodpecker/push/release Pipeline was successful Details
ci/woodpecker/push/test Pipeline was successful Details
ci/woodpecker/pr/release Pipeline was successful Details
ci/woodpecker/pr/docs Pipeline was successful Details
ci/woodpecker/pr/test Pipeline was successful Details
2022-07-21 11:29:28 +01:00
floatingghost 8215434c65 also resolve @full@tags in mfm (#62)
ci/woodpecker/push/lint Pipeline was successful Details
ci/woodpecker/push/test Pipeline failed Details
ci/woodpecker/push/release Pipeline was successful Details
Reviewed-on: #62
2022-07-11 13:48:43 +00:00
floatingghost ff6c8455fb [#58] ensure all users are linked in MFM content (#61)
ci/woodpecker/push/lint Pipeline was successful Details
ci/woodpecker/push/test Pipeline was successful Details
ci/woodpecker/push/release Pipeline was successful Details
Reviewed-on: #61
2022-07-11 11:48:29 +00:00
floatingghost 5ad256f170 [#58] pre-link MFM content (#59)
ci/woodpecker/push/lint Pipeline was successful Details
ci/woodpecker/push/test Pipeline was successful Details
ci/woodpecker/push/release Pipeline was successful Details
Reviewed-on: #59
2022-07-10 17:06:25 +00:00
sfr 058bf96798 implement Move activities (#45)
ci/woodpecker/push/lint Pipeline was successful Details
ci/woodpecker/push/test Pipeline failed Details
ci/woodpecker/push/release Pipeline was successful Details
Reviewed-on: #45
Co-authored-by: sfr <sol@solfisher.com>
Co-committed-by: sfr <sol@solfisher.com>
2022-07-04 16:29:39 +00:00
FloatingGhost 05081cd81b Add collection fetching module 2022-07-03 19:20:59 +01:00
FloatingGhost 0a3a552696 Add support for a `first` reference in pinned objects 2022-07-03 17:25:20 +01:00
FloatingGhost 4da9a12bf8 Add test for friendica featured collection 2022-07-03 16:59:12 +01:00
floatingghost 3f7c64f30b Fix broken attachments from owncast (#31)
ci/woodpecker/push/lint Pipeline was successful Details
ci/woodpecker/push/test Pipeline failed Details
ci/woodpecker/push/release Pipeline was successful Details
Reviewed-on: #31
2022-07-01 11:14:55 +00:00
FloatingGhost 142646426e fix emoji tests 2022-06-11 14:08:54 +01:00
FloatingGhost 25b4395830 just drop unknown tags 2022-01-07 20:14:04 +00:00
FloatingGhost 680c5d8d89 Add compatibility with bookwyrm's weird entities 2022-01-07 16:51:04 +00:00
Haelwenn 173e977e28 Merge branch 'features/ingestion-page' into 'develop'
Pipeline Ingestion: Page

See merge request pleroma/pleroma!3097
2021-07-12 05:05:49 +00:00
Mark Felder 1c4c73c6a0 Add test for AnalyzeMetadata upload filter fetching dimensions from a video 2021-06-08 14:02:56 -05:00
Alex Gleason fe4c4a7178
MRF: create MRF.Policy behaviour separate from MRF module
Speeds up recompilation by reducing compile-time deps
2021-06-07 14:22:08 -05:00
Haelwenn (lanodan) Monnier eb7313b0d3
Pipeline Ingestion: Page 2021-06-04 20:06:33 +02:00
Haelwenn c4b4258374 Merge branch 'features/validators-note' into 'develop'
Pipeline Ingestion: Note

Closes #290

See merge request pleroma/pleroma!2984
2021-06-01 01:51:38 +00:00
Mark Felder fe40f6f291 Switch from the deprecated "use Mix.config" to "import Config" 2021-05-20 14:02:58 -05:00
rinpatch 79376b4afb Merge branch 'feature/521-pinned-post-federation' into 'develop'
Pinned posts federation

Closes #521

See merge request pleroma/pleroma!3312
2021-04-16 09:53:47 +00:00
Haelwenn (lanodan) Monnier c944932674
Pipeline Ingestion: Note 2021-04-05 19:19:11 +02:00
Alexander Strizhakov 4d046afd27
tests for release config provider 2021-03-27 09:05:33 +03:00
Alexander Strizhakov 8f0778166c
moving fixture into mastodon folder 2021-03-25 13:03:41 +03:00
Alexander Strizhakov 16c96966e9
not needed 2021-03-25 13:03:41 +03:00
Alexander Strizhakov d1d2744ee3
featured_address valition in AddRemoveValidator 2021-03-25 13:03:40 +03:00