Commit graph

288 commits

Author SHA1 Message Date
16bed0562d Fix tests 2024-06-09 18:28:00 +01:00
765c7e98d2 Respect the TTL returned in OpenGraph tags 2024-06-09 17:36:15 +01:00
f15eded3e1 Add extra test case for nonsense field, increase timeouts
Some checks failed
ci/woodpecker/push/build-amd64 Pipeline is pending
ci/woodpecker/push/build-arm64 Pipeline is pending
ci/woodpecker/push/docs Pipeline is pending
ci/woodpecker/push/lint Pipeline is pending
ci/woodpecker/push/test Pipeline is pending
ci/woodpecker/pr/lint Pipeline was successful
ci/woodpecker/pr/test Pipeline was successful
ci/woodpecker/pr/build-arm64 unknown status
ci/woodpecker/pr/build-amd64 unknown status
ci/woodpecker/pr/docs unknown status
2024-05-27 02:09:48 +01:00
da67e69af5 Allow for attachment to be a single object in user data
Some checks failed
ci/woodpecker/push/build-amd64 Pipeline is pending
ci/woodpecker/push/build-arm64 Pipeline is pending
ci/woodpecker/push/docs Pipeline is pending
ci/woodpecker/push/lint Pipeline is pending
ci/woodpecker/push/test Pipeline is pending
ci/woodpecker/pr/lint Pipeline was successful
ci/woodpecker/pr/test Pipeline was successful
ci/woodpecker/pr/build-arm64 unknown status
ci/woodpecker/pr/build-amd64 unknown status
ci/woodpecker/pr/docs unknown status
2024-05-26 17:09:26 +01:00
lain
50403351f4 add impostor test for webfinger 2024-05-22 19:17:34 +01:00
Alex Gleason
a953b1d927 Prevent spoofing webfinger 2024-05-22 19:08:37 +01:00
b1c6621e66 Merge pull request 'Read image description from EXIF data' (#744) from timorl/akkoma:elseinspe into develop
Some checks are pending
ci/woodpecker/push/build-amd64 Pipeline is pending
ci/woodpecker/push/build-arm64 Pipeline is pending
ci/woodpecker/push/docs Pipeline is pending
ci/woodpecker/push/lint Pipeline is pending
ci/woodpecker/push/test Pipeline is pending
Reviewed-on: #744
2024-04-25 12:52:31 +00:00
83f75c3e93 Accept all standard actor types
Some checks failed
ci/woodpecker/pr/lint Pipeline was successful
ci/woodpecker/pr/test Pipeline was successful
ci/woodpecker/pr/build-arm64 unknown status
ci/woodpecker/pr/build-amd64 unknown status
ci/woodpecker/pr/docs unknown status
2024-04-23 18:14:34 +02:00
timorl
2a9db73b4c
Merge branch 'develop' into elseinspe 2024-04-19 17:11:55 +02:00
timorl
b144218dce
Merge branch 'develop' into elseinspe
Some checks failed
ci/woodpecker/pr/lint Pipeline failed
ci/woodpecker/pr/test unknown status
ci/woodpecker/pr/build-arm64 unknown status
ci/woodpecker/pr/build-amd64 unknown status
ci/woodpecker/pr/docs unknown status
2024-04-14 20:31:33 +02:00
18442dcc7e Fix quote test 2024-04-13 23:05:52 +01:00
8684964c5d Only allow exact id matches
This protects us from falling for obvious spoofs as from the current
upload exploit (unfortunately we can’t reasonably do anything about
spoofs with exact matches as was possible via emoji and proxy).

Such objects being invalid is supported by the spec, sepcifically
sections 3.1 and 3.2: https://www.w3.org/TR/activitypub/#obj-id

Anonymous objects are not relevant here (they can only exists within
parent objects iiuc) and neither is client-to-server or transient objects
(as those cannot be fetched in the first place).
This leaves us with the requirement for `id` to (a) exist and
(b) be a publicly dereferencable URI from the originating server.
This alone does not yet demand strict equivalence, but the spec then
further explains objects ought to be fetchable _via their ID_.
Meaning an object not retrievable via its ID, is invalid.

This reading is supported by the fact, e.g. GoToSocial (recently) and
Mastodon (for 6+ years) do already implement such strict ID checks,
additionally proving this doesn’t cause federation issues in practice.

However, apart from canonical IDs there can also be additional display
URLs. *omas first redirect those to their canonical location, but *keys
and Mastodon directly serve the AP representation without redirects.

Mastodon and GTS deal with this in two different ways,
but both constitute an effective countermeasure:
 - Mastodon:
   Unless it already is a known AP id, two fetches occur.
   The first fetch just reads the `id` property and then refetches from
   the id. The last fetch requires the returned id to exactly match the
   URL the content was fetched from. (This can be optimised by skipping
   the second fetch if it already matches)
   05eda8d193/app/helpers/jsonld_helper.rb (L168)
   63f0979799

 - GTS:
   Only does a single fetch and then checks if _either_ the id
   _or_ url property (which can be an object) match the original fetch
   URL. This relies on implementations always including their display URL
   as "url" if differing from the id. For actors this is true for all
   investigated implementations, for posts only Mastodon includes an
   "url", but it is also the only one with a differing display URL.
   2bafd7daf5 (diff-943bbb02c8ac74ac5dc5d20807e561dcdfaebdc3b62b10730f643a20ac23c24fR222)

Albeit Mastodon’s refetch offers higher compatibility with theoretical
implmentations using either multiple different display URL or not
denoting any of them as "url" at all, for now we chose to adopt a
GTS-like refetch-free approach to avoid additional implementation
concerns wrt to whether redirects should be allowed when fetching a
canonical AP id and potential for accidentally loosening some checks
(e.g. cross-domain refetches) for one of the fetches.
This may be reconsidered in the future.
2024-03-25 14:05:05 -01:00
f07eb4cb55 Sanity check fetched user data
In order to properly process incoming notes we need
to be able to map the key id back to an actor.
Also, check collections actually belong to the same server.

Key ids of Hubzilla and Bridgy samples were updated to what
modern versions of those output. If anything still uses the
old format, we would not be able to verify their posts anyway.
2024-03-25 14:05:05 -01:00
59a142e0b0 Never fetch resource from ourselves
If it’s not already in the database,
it must be counterfeit (or just not exists at all)

Changed test URLs were only ever used from "local: false" users anyway.
2024-03-25 14:05:05 -01:00
7825798e32 Add XML matcher 2023-08-07 11:12:14 +01:00
mae
d868348fac Completely disable xml entity resolution
Some checks are pending
ci/woodpecker/pr/build-amd64 Pipeline is pending
ci/woodpecker/pr/build-arm64 Pipeline is pending
ci/woodpecker/pr/docs Pipeline is pending
ci/woodpecker/pr/test Pipeline is pending
2023-08-05 12:32:05 +00:00
9c7409808f Add unit test for external entity loading
Some checks are pending
ci/woodpecker/push/build-amd64 Pipeline is pending
ci/woodpecker/push/build-arm64 Pipeline is pending
ci/woodpecker/push/docs Pipeline is pending
ci/woodpecker/push/test Pipeline is pending
2023-08-04 22:24:32 +01:00
3227ebf1e1 CommonFixes: more predictable context generation
`context` fields for objects and activities can now be generated based
on the object/activity `inReplyTo` field or its ActivityPub ID, as a
fallback method in cases where `context` fields are missing for incoming
activities and objects.
2023-06-14 16:22:26 +00:00
ilja
b4952a81fe Interpret \n as newline for MFM
Markdown doesn't generally consider `\n` a newline,
but Misskey does for MFM.

Now we do to for MFM (and not for Markdown) :)
2023-02-18 19:56:11 +01:00
ff5793198f add inbound language test
Some checks are pending
ci/woodpecker/push/woodpecker Pipeline is pending
2023-01-11 15:42:13 +00:00
f752126427 Remove quack, ensure adapter is finch
Some checks are pending
ci/woodpecker/push/woodpecker Pipeline is pending
2022-12-11 23:22:35 +00:00
e1e0d5d759 microblogpub federation fixes (#288)
Some checks are pending
ci/woodpecker/push/woodpecker Pipeline is pending
Co-authored-by: FloatingGhost <hannah@coffee-and-dreams.uk>
Reviewed-on: #288
2022-11-18 11:14:35 +00:00
Ilja
66a04cead3 Descriptions from exif data with only whitespeces are considered empty
Some checks are pending
ci/woodpecker/pr/woodpecker Pipeline is pending
I noticed that pictures taken with Ubuntu-Touch have whitespace in one of the fields
This should just be ignored imo
2022-10-23 14:46:22 +02:00
Ilja
338612d72b Use EXIF data of image to prefill image description
During attachment upload Pleroma returns a "description" field.

* This MR allows Pleroma to read the EXIF data during upload and return the description to the FE using this field.
    * If a description is already present (e.g. because a previous module added it), it will use that
    * Otherwise it will read from the EXIF data. First it will check -ImageDescription, if that's empty, it will check -iptc:Caption-Abstract
    * If no description is found, it will simply return nil, which is the default value
* When people set up a new instance, they will be asked if they want to read metadata and this module will be activated if so

There was an Exiftool module, which has now been renamed to Exiftool.StripLocation
2022-10-23 14:46:16 +02:00
b4261b0335 Use set of pregenerated RSA keys
All checks were successful
ci/woodpecker/push/woodpecker Pipeline was successful
ci/woodpecker/pr/woodpecker Pipeline was successful
Randomness is a huge resource sink, so let's just use
a some that we made earlier
2022-09-11 20:14:58 +01:00
aaf78e2b52 only put linked mfm in source (#171)
All checks were successful
ci/woodpecker/push/woodpecker Pipeline was successful
Reviewed-on: #171
2022-08-17 09:35:11 +00:00
c1e15ff6f8 Transmogrifier: fix reply context fixing
Some checks are pending
ci/woodpecker/push/woodpecker Pipeline is pending
ci/woodpecker/pr/woodpecker Pipeline is pending
Incoming Pleroma replies to a Misskey thread were rejected due to a
broken context fix, which caused them to not be visible until a
non-Pleroma user interacted with the replies.

This fix properly sets the post-fix object context to its parent Create
activity as well, if it was changed.
2022-08-04 12:57:48 +01:00
e26388a01c Support reaching user@sub.domain.tld at user@domain.tld (#134)
All checks were successful
ci/woodpecker/push/woodpecker Pipeline was successful
Reviewed-on: #134
Co-authored-by: Joel Beckmeyer <joel@beckmeyer.us>
Co-committed-by: Joel Beckmeyer <joel@beckmeyer.us>
2022-08-02 13:54:22 +00:00
a3501cab86 ensure quote fetching obeys max thread distance (#119)
All checks were successful
ci/woodpecker/push/test Pipeline was successful
ci/woodpecker/push/docs Pipeline was successful
ci/woodpecker/push/release Pipeline was successful
Reviewed-on: #119
2022-07-26 17:28:47 +00:00
0a55c37182 don't error out if the featured collection has a string ID
All checks were successful
ci/woodpecker/push/test Pipeline was successful
ci/woodpecker/push/docs Pipeline was successful
ci/woodpecker/pr/release Pipeline was successful
ci/woodpecker/pr/docs Pipeline was successful
ci/woodpecker/push/release Pipeline was successful
ci/woodpecker/pr/test Pipeline was successful
2022-07-26 15:08:35 +01:00
1419eee5df Quote posting (#113)
Some checks are pending
ci/woodpecker/push/docs Pipeline is pending
ci/woodpecker/push/release Pipeline is pending
ci/woodpecker/push/test Pipeline is pending
Reviewed-on: #113
2022-07-25 16:30:06 +00:00
c4e9c4bc95 extend custom runtime system (#108)
All checks were successful
ci/woodpecker/push/test Pipeline was successful
ci/woodpecker/push/docs Pipeline was successful
ci/woodpecker/push/release Pipeline was successful
Reviewed-on: #108
2022-07-24 16:42:43 +00:00
0f132b802d purge chat and shout endpoints
All checks were successful
ci/woodpecker/push/docs Pipeline was successful
ci/woodpecker/push/release Pipeline was successful
ci/woodpecker/push/test Pipeline was successful
ci/woodpecker/pr/release Pipeline was successful
ci/woodpecker/pr/docs Pipeline was successful
ci/woodpecker/pr/test Pipeline was successful
2022-07-21 11:29:28 +01:00
8215434c65 also resolve @full@tags in mfm (#62)
Some checks failed
ci/woodpecker/push/lint Pipeline was successful
ci/woodpecker/push/test Pipeline failed
ci/woodpecker/push/release Pipeline was successful
Reviewed-on: #62
2022-07-11 13:48:43 +00:00
ff6c8455fb [#58] ensure all users are linked in MFM content (#61)
All checks were successful
ci/woodpecker/push/lint Pipeline was successful
ci/woodpecker/push/test Pipeline was successful
ci/woodpecker/push/release Pipeline was successful
Reviewed-on: #61
2022-07-11 11:48:29 +00:00
5ad256f170 [#58] pre-link MFM content (#59)
All checks were successful
ci/woodpecker/push/lint Pipeline was successful
ci/woodpecker/push/test Pipeline was successful
ci/woodpecker/push/release Pipeline was successful
Reviewed-on: #59
2022-07-10 17:06:25 +00:00
sfr
058bf96798 implement Move activities (#45)
Some checks failed
ci/woodpecker/push/lint Pipeline was successful
ci/woodpecker/push/test Pipeline failed
ci/woodpecker/push/release Pipeline was successful
Reviewed-on: #45
Co-authored-by: sfr <sol@solfisher.com>
Co-committed-by: sfr <sol@solfisher.com>
2022-07-04 16:29:39 +00:00
05081cd81b Add collection fetching module 2022-07-03 19:20:59 +01:00
0a3a552696 Add support for a first reference in pinned objects 2022-07-03 17:25:20 +01:00
4da9a12bf8 Add test for friendica featured collection 2022-07-03 16:59:12 +01:00
3f7c64f30b Fix broken attachments from owncast (#31)
Some checks failed
ci/woodpecker/push/lint Pipeline was successful
ci/woodpecker/push/test Pipeline failed
ci/woodpecker/push/release Pipeline was successful
Reviewed-on: #31
2022-07-01 11:14:55 +00:00
142646426e fix emoji tests 2022-06-11 14:08:54 +01:00
25b4395830 just drop unknown tags 2022-01-07 20:14:04 +00:00
680c5d8d89 Add compatibility with bookwyrm's weird entities 2022-01-07 16:51:04 +00:00
Haelwenn
173e977e28 Merge branch 'features/ingestion-page' into 'develop'
Pipeline Ingestion: Page

See merge request pleroma/pleroma!3097
2021-07-12 05:05:49 +00:00
1c4c73c6a0 Add test for AnalyzeMetadata upload filter fetching dimensions from a video 2021-06-08 14:02:56 -05:00
Alex Gleason
fe4c4a7178
MRF: create MRF.Policy behaviour separate from MRF module
Speeds up recompilation by reducing compile-time deps
2021-06-07 14:22:08 -05:00
Haelwenn (lanodan) Monnier
eb7313b0d3
Pipeline Ingestion: Page 2021-06-04 20:06:33 +02:00
Haelwenn
c4b4258374 Merge branch 'features/validators-note' into 'develop'
Pipeline Ingestion: Note

Closes #290

See merge request pleroma/pleroma!2984
2021-06-01 01:51:38 +00:00
fe40f6f291 Switch from the deprecated "use Mix.config" to "import Config" 2021-05-20 14:02:58 -05:00