From 24b5a75d096387ac63b30a09c1a6d8a986eceb61 Mon Sep 17 00:00:00 2001 From: "Haelwenn (lanodan) Monnier" Date: Thu, 12 Jul 2018 19:52:17 +0200 Subject: [PATCH 1/3] Add test for Plume Articles --- .../baptiste.gelex.xyz-article.json | 1 + .../httpoison_mock/baptiste.gelex.xyz-user.json | 1 + test/support/httpoison_mock.ex | 16 ++++++++++++++++ test/web/activity_pub/activity_pub_test.exs | 9 +++++++++ 4 files changed, 27 insertions(+) create mode 100644 test/fixtures/httpoison_mock/baptiste.gelex.xyz-article.json create mode 100644 test/fixtures/httpoison_mock/baptiste.gelex.xyz-user.json diff --git a/test/fixtures/httpoison_mock/baptiste.gelex.xyz-article.json b/test/fixtures/httpoison_mock/baptiste.gelex.xyz-article.json new file mode 100644 index 000000000..3f3f0f4fb --- /dev/null +++ b/test/fixtures/httpoison_mock/baptiste.gelex.xyz-article.json @@ -0,0 +1 @@ +{"@context":["https://www.w3.org/ns/activitystreams","https://w3id.org/security/v1",{"Emoji":"toot:Emoji","Hashtag":"as:Hashtag","atomUri":"ostatus:atomUri","conversation":"ostatus:conversation","featured":"toot:featured","focalPoint":{"@container":"@list","@id":"toot:focalPoint"},"inReplyToAtomUri":"ostatus:inReplyToAtomUri","manuallyApprovesFollowers":"as:manuallyApprovesFollowers","movedTo":"as:movedTo","ostatus":"http://ostatus.org#","sensitive":"as:sensitive","toot":"http://joinmastodon.org/ns#"}],"attributedTo":["https://baptiste.gelez.xyz/@/BaptisteGelez"],"cc":[],"content":"

It has been one month since the last \"This Month in Plume\" article, so it is time for another edition of our monthly changelog!

\n

Bug Fixes and Security

\n

Let's start with the hidden, but still (very) important changes: bug fixes and security patches.

\n

First of all, @Trinity protected us against two major security flaws, called XSS and CSRF. The first one allows the attacker to run malicious code if you visit a Plume page where some of their personal data is present. The second one lets them post data with your Plume account by visiting one of their own website. It is two very common attack, and it is great we are now protected against them!

\n

The other big change in this area, is that we are now validating the data you are sending before doing anything with it. It means that, for instance, you will no longer be able to register with an empty username and to break everything.

\n

On the federation side, many issues were reported by @kaniini and redmatrix (respectively contributing to Pleroma and Hubzilla). By fixing some of them, we made it possible to federate Plume articles to Pleroma!

\n

@Trinity hopefully noticed that there was a bug in our password check code: we were not checking that your password was correct, but only that the verification process went without errors. Concretely, it means that you could login to any account with any password. I wrote this part of the code when I was still the only contributor to the project, so nobody could review my work. We will now be trying to check every change, especially when it deals with critical parts of Plume, to avoid similar issues in the future, and we I'm really sorry this happened (even if I think nobody exploited it).

\n

Zanfib and stephenburgess8 also commited some small bugfixes, improving the general experience.

\n

New Features

\n

Let's now talk about the features that we introduced during this month.

\n

One of the most easy to spot is the redesign of Plume, made by @Madeorsk. I personaly love what he did, it really improved the readability and gave Plume a bit more of identity than the previous design. And he is still improving it.

\n

We also enabled Mardown in comment, to let you write more structured and nicely formatted responses.

\n

As you may have noticed, I have used mentions in this post. Indeed, it is now possible to mention someone in your articles or in comments. It works exactly the same way as in other apps, and you should receive a notification if someone mentionned you.

\n

A dashboard to manage your blogs has also been introduced. In the future it may be used to manage your drafts, and eventually to show some statistics. The goal is to have a more specific homepage for authors.

\n

The federation with other ActivityPub softwares, like Mastodon or Pleroma is starting to work quite well, but the federation between Plume instances is far from being complete. However, we started to work on it, and it is now possible to view a distant user profile or blog from your instance, even if only basic informations are fetched yet (the articles are not loaded for instance).

\n

Another new feature that may not be visible for everyone, is the new NodeInfo endpoint. NodeInfo is a protocol allowing to get informations about a specific federated instance (whatever software it runs). It means that Plume instances can now be listed on sites like fediverse.network.

\n

Maybe you wanted to host a Plume instance, but you don't like long install process during which you are just copy/pasting commands that you don't really understand from the documentation. That's why we introduced a setup script: the first you'll launch Plume, it will ask you a few questions and automatically setup your instance in a few minutes. We hope that this feature will help to host small instances, run by non-professional adminsys. You can see a demo of this tool on asciinema.

\n

Last but not least, Plume is now translatable! It is already available in English, French, Polish (thanks to @m4sk1n)) and German (thanks to bitkeks). If your browser is configured to display pages in these languages, you should normally see the interface in your language. And if your language is not present yet, feel free to add your translation.

\n

Other Changes

\n

We also improved the code a lot. We tried to separate each part as much as possible, making it easier to re-use for other projects. For instance, our database code is now isolated from the rest of the app, which means it will be easier to make import tools from other blogging engines. Some parts of the code are even shared with another project, Aardwolf a federated Facebook alternative. For instance, both of our projects use the same internationalization code, and once Aardwolf will implement federation, this part of the code will probably be shared too. Since the WebFinger module (used to find new users and blogs) and the CSRF protection code (see the \"Bug fixes and Security\" section) have been isolated in their own modules, they may be shared by both projects too.

\n

We also worked a lot on documentation. We now have articles explaining how to setup your Plume instance on various operating systems, but also documenting the translation process. I want to thank BanjoFox (who imported some documentation from their project, Aardwolf, as the setup is quite similar), Kushal and @gled@plume.mastodon.host for working on this.

\n

As you can see, there were many changes this month, but there still a lot to do. Your help will of course be welcome. If you want to contribute to the code, translate Plume in your language, write some documentation, or anything else (or even if you're just curious about the project), feel free to join our Matrix room: #plume:disroot.org. Otherwise, as BanjoFox said on the Aardwolf Team Mastodon account, talking about the project around you is one of the easiest way to help.

\n","id":"https://baptiste.gelez.xyz/~/PlumeDevelopment/this-month-in-plume-june-2018/","likes":null,"name":"This Month in Plume: June 2018","published":"2018-07-10T20:16:24.087622Z","shares":null,"source":null,"tag":[{"href":"https://baptiste.gelez.xyz/@/Trinity","name":"@Trinity","type":"Mention"},{"href":"https://baptiste.gelez.xyz/@/kaniini/","name":"@kaniini","type":"Mention"},{"href":"https://baptiste.gelez.xyz/@/Trinity","name":"@Trinity","type":"Mention"}],"to":["https://unixcorn.xyz/users/Bat","https://mastodon.host/users/federationbot","https://social.tcit.fr/users/tcit","https://framapiaf.org/users/qwerty","https://mastodon.social/users/lthms","https://eldritch.cafe/users/Nausicaa","https://imaginair.es/users/Elanndelh","https://framapiaf.org/users/Drulac","https://mastodon.partipirate.org/users/NicolasConstant","https://aleph.land/users/Madeorsk","https://maly.io/users/Troll","https://hostux.social/users/superjey","https://mamot.fr/users/Phigger","https://mastodon.social/users/wakest","https://social.coop/users/wakest","https://unixcorn.xyz/users/Ce_lo","https://social.art-software.fr/users/Electron","https://framapiaf.org/users/Quenti","https://toot.plus.yt/users/Djyp","https://mastodon.social/users/brainblasted","https://social.mochi.academy/users/Ambraven","https://social.hacktivis.me/users/lanodan","https://mastodon.eliotberriot.com/users/eliotberriot","https://edolas.world/users/0x1C3B00DA","https://toot.cafe/users/zack","https://manowar.social/users/zatnosk","https://eldritch.cafe/users/fluffy","https://mastodon.social/users/david_ross","https://kosmos.social/users/xiroux","https://mastodon.art/users/EmergencyBattle","https://mastodon.social/users/trwnh","https://octodon.social/users/pybyte","https://anticapitalist.party/users/Trinity","https://mstdn.mx/users/xavavu","https://baptiste.gelez.xyz/@/m4sk1n","https://eldritch.cafe/users/milia","https://mastodon.zaclys.com/users/arx","https://toot.cafe/users/sivy","https://mastodon.social/users/ortegacmanuel","https://mastodon.observer/users/stephen","https://octodon.social/users/chloe","https://unixcorn.xyz/users/AmauryPi","https://cybre.space/users/rick_777","https://mastodon.social/users/wezm","https://baptiste.gelez.xyz/@/idlesong","https://mamot.fr/users/dr4Ke","https://imaginair.es/users/Phigger","https://mamot.fr/users/dlink","https://anticapitalist.party/users/a000d4f7a91939d0e71df1646d7a48","https://framapiaf.org/users/PhieLaidMignon","https://mastodon.social/users/y6nH","https://crazynoisybizarre.town/users/FederationBot","https://social.weho.st/users/dvn","https://mastodon.art/users/Wolthera","https://diaspodon.fr/users/dada","https://pachyder.me/users/Lanza","https://mastodon.xyz/users/ag","https://aleph.land/users/yahananxie","https://mstdn.io/users/chablis_social","https://mastodon.gougere.fr/users/fabien","https://functional.cafe/users/otini","https://social.coop/users/bhaugen","https://octodon.social/users/donblanco","https://chaos.social/users/astro","https://pachyder.me/users/sibear","https://mamot.fr/users/yohann","https://social.wxcafe.net/users/Bat","https://mastodon.social/users/dansup","https://chaos.social/users/juh","https://scifi.fyi/users/paeneultima","https://hostux.social/users/Deuchnord","https://mstdn.fr/users/taziden","https://mamot.fr/users/PifyZ","https://mastodon.social/users/plantabaja","https://mastodon.social/users/gitzgrog","https://mastodon.social/users/Syluban","https://masto.pt/users/eloisa","https://pleroma.soykaf.com/users/notclacke","https://mastodon.social/users/SiegfriedEhret","https://writing.exchange/users/write_as","https://mstdn.io/users/shellkr","https://mastodon.uy/users/jorge","https://mastodon.technology/users/bobstechsite","https://mastodon.social/users/hinterwaeldler","https://mastodon.xyz/users/mgdelacroix","https://mastodon.cloud/users/jjatria","https://baptiste.gelez.xyz/@/Jade/","https://edolas.world/users/pfm","https://mstdn.io/users/jort","https://mastodon.social/users/andreipetcu","https://mastodon.technology/users/0xf00fc7c8","https://mastodon.social/users/khanate","https://mastodon.technology/users/francois","https://mastodon.social/users/glherrmann","https://mastodon.host/users/gled","https://social.holdmybeer.solutions/users/kemonine","https://scholar.social/users/bgcarlisle","https://mastodon.social/users/oldgun","https://baptiste.gelez.xyz/@/snoe/","https://mastodon.at/users/switchingsocial","https://scifi.fyi/users/BrokenBiscuit","https://dev.glitch.social/users/hoodie","https://todon.nl/users/paulfree14","https://mastodon.social/users/aadilayub","https://social.fsck.club/users/anarchosaurus","https://mastodonten.de/users/GiantG","https://mastodon.technology/users/cj","https://cybre.space/users/sam","https://layer8.space/users/silkevicious","https://mastodon.xyz/users/Jimmyrwx","https://fosstodon.org/users/danyspin97","https://mstdn.io/users/cristhyano","https://mastodon.social/users/vanyok","https://hulvr.com/users/rook","https://niu.moe/users/Lucifer","https://mamot.fr/users/Thibaut","https://mastodont.cat/users/bgta","https://mstdn.io/users/hontoni","https://niu.moe/users/lionirdeadman","https://functional.cafe/users/phoe","https://mastodon.social/users/toontoet","https://mastodon.social/users/danipozo","https://scholar.social/users/robertson","https://mastodon.social/users/aldatsa","https://elekk.xyz/users/maloki","https://kitty.town/users/nursemchurt","https://neigh.horse/users/commagray","https://mastodon.social/users/hirojin","https://mastodon.xyz/users/mareklach","https://chaos.social/users/benthor","https://mastodon.social/users/djperreault","https://mastodon.art/users/eylul","https://mastodon.opportunis.me/users/bob","https://tootplanet.space/users/Shutsumon","https://toot.cat/users/woozle","https://mastodon.social/users/StephenLB","https://sleeping.town/users/oct2pus","https://mastodon.indie.host/users/stragu","https://social.coop/users/gilscottfitzgerald","https://icosahedron.website/users/joeld","https://mastodon.social/users/hellion","https://cybre.space/users/cooler_ranch","https://mastodon.social/users/kelsonv","https://mastodon.lat/users/scalpol","https://writing.exchange/users/hnb","https://hex.bz/users/Horst","https://mastodon.social/users/weddle","https://maly.io/users/sonya","https://social.coop/users/medusa","https://mastodon.social/users/DystopianK","https://mstdn.io/users/d_io","https://fosstodon.org/users/brandon","https://fosstodon.org/users/Cando","https://mastodon.host/users/panina","https://floss.social/users/tuxether","https://social.tchncs.de/users/suitbertmonz","https://mastodon.social/users/jrt","https://mastodon.social/users/sirikon","https://mstdn.io/users/yabirgb","https://mastodon.cloud/users/FerdiZ","https://mastodon.social/users/carlchenet","https://social.polonkai.eu/users/calendar_social","https://social.polonkai.eu/users/gergely","https://mastodon.social/users/Jelv","https://mastodon.social/users/srinicame","https://cybre.space/users/mastoabed","https://mastodon.social/users/tagomago","https://lgbt.io/users/bootblackCub","https://niu.moe/users/Nopplyy","https://mastodon.social/users/bpugh","https://www.w3.org/ns/activitystreams#Public"],"type":"Article","uploadMedia":null,"url":"https://baptiste.gelez.xyz/~/PlumeDevelopment/this-month-in-plume-june-2018/"} \ No newline at end of file diff --git a/test/fixtures/httpoison_mock/baptiste.gelex.xyz-user.json b/test/fixtures/httpoison_mock/baptiste.gelex.xyz-user.json new file mode 100644 index 000000000..b226204ba --- /dev/null +++ b/test/fixtures/httpoison_mock/baptiste.gelex.xyz-user.json @@ -0,0 +1 @@ +{"@context":["https://www.w3.org/ns/activitystreams","https://w3id.org/security/v1",{"Emoji":"toot:Emoji","Hashtag":"as:Hashtag","atomUri":"ostatus:atomUri","conversation":"ostatus:conversation","featured":"toot:featured","focalPoint":{"@container":"@list","@id":"toot:focalPoint"},"inReplyToAtomUri":"ostatus:inReplyToAtomUri","manuallyApprovesFollowers":"as:manuallyApprovesFollowers","movedTo":"as:movedTo","ostatus":"http://ostatus.org#","sensitive":"as:sensitive","toot":"http://joinmastodon.org/ns#"}],"endpoints":{"oauthAuthorizationEndpoint":null,"oauthTokenEndpoint":null,"provideClientKey":null,"proxyUrl":null,"sharedInbox":"https://baptiste.gelez.xyz/inbox/","signClientKey":null},"followers":null,"following":null,"id":"https://baptiste.gelez.xyz/@/BaptisteGelez","inbox":"https://baptiste.gelez.xyz/@/BaptisteGelez/inbox","liked":null,"likes":null,"name":"Baptiste Gelez","outbox":"https://baptiste.gelez.xyz/@/BaptisteGelez/outbox","preferredUsername":"BaptisteGelez","publicKey":{"id":"https://baptiste.gelez.xyz/@/BaptisteGelez#main-key","owner":"https://baptiste.gelez.xyz/@/BaptisteGelez","publicKeyPem":"-----BEGIN PUBLIC KEY-----\nMIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIBCgKCAQEA56vPlCAyxZDDy8hNiT1p\n0cdFKnUK/51LiP4nTAxGf5Eb8NmsB2ftDgiDWZfg3LiHkjNcfTDpmN0aZyRxnTg9\nZ4JiQagfynVEbMkcOQhO64OFZpB47GpLtxrb49IcUes/p4ngp/Wkp+arYZSpoSs6\n3I995mZp3ZJ78pNQf1/lV0VIdDe6SqvRj1GmBDXXcecxF0O7rN/WYNO7Jag4i/XA\nU1ToDAMeUFeijRioSNoD3CHkMIu7AN+gqAWzZ21H/ZUvmfxh3WqQi/MDNcUhhA+0\nXv7/dv4S20EGnHadtE7OrBC1IwiHEuRM41zZq0ze9cKpoXg3VK2fiSNrCHlYrA18\n2wIDAQAB\n-----END PUBLIC KEY-----\n"},"shares":null,"source":null,"streams":null,"summary":"Main Plume developer","type":"Person","uploadMedia":null,"url":"https://baptiste.gelez.xyz/@/BaptisteGelez"} \ No newline at end of file diff --git a/test/support/httpoison_mock.ex b/test/support/httpoison_mock.ex index befebad8a..a52d44ed6 100644 --- a/test/support/httpoison_mock.ex +++ b/test/support/httpoison_mock.ex @@ -736,6 +736,22 @@ defmodule HTTPoisonMock do }} end + def get("https://baptiste.gelez.xyz/~/PlumeDevelopment/this-month-in-plume-june-2018/", _, _) do + {:ok, + %Response{ + status_code: 200, + body: File.read!("test/fixtures/httpoison_mock/baptiste.gelex.xyz-article.json") + }} + end + + def get("https://baptiste.gelez.xyz/@/BaptisteGelez", _, _) do + {:ok, + %Response{ + status_code: 200, + body: File.read!("test/fixtures/httpoison_mock/baptiste.gelex.xyz-user.json") + }} + end + def get(url, body, headers) do {:error, "Not implemented the mock response for get #{inspect(url)}, #{inspect(body)}, #{ diff --git a/test/web/activity_pub/activity_pub_test.exs b/test/web/activity_pub/activity_pub_test.exs index bc33b4dfc..90c0bd768 100644 --- a/test/web/activity_pub/activity_pub_test.exs +++ b/test/web/activity_pub/activity_pub_test.exs @@ -476,6 +476,15 @@ defmodule Pleroma.Web.ActivityPub.ActivityPubTest do end end + test "it can fetch plume articles" do + {:ok, object} = + ActivityPub.fetch_object_from_id( + "https://baptiste.gelez.xyz/~/PlumeDevelopment/this-month-in-plume-june-2018/" + ) + + assert object + end + describe "update" do test "it creates an update activity with the new user data" do user = insert(:user) From 8472fba2a729f7764ffaf2a4744533c8a95adadb Mon Sep 17 00:00:00 2001 From: "Haelwenn (lanodan) Monnier" Date: Thu, 12 Jul 2018 23:09:42 +0200 Subject: [PATCH 2/3] [Pleroma.Web.ActivityPub.Transmogrifier]: Fix actor key outside of object The code here is copied from feature/peertube by lain. Co-authored-by: lain --- .../web/activity_pub/transmogrifier.ex | 26 +++++++++---------- 1 file changed, 13 insertions(+), 13 deletions(-) diff --git a/lib/pleroma/web/activity_pub/transmogrifier.ex b/lib/pleroma/web/activity_pub/transmogrifier.ex index 0d2166196..6080303b6 100644 --- a/lib/pleroma/web/activity_pub/transmogrifier.ex +++ b/lib/pleroma/web/activity_pub/transmogrifier.ex @@ -13,6 +13,14 @@ defmodule Pleroma.Web.ActivityPub.Transmogrifier do require Logger + def get_actor(%{"actor" => actor}) when is_binary(actor) do + actor + end + + def get_actor(%{"actor" => actor}) when is_list(actor) do + Enum.at(actor, 0) + end + @doc """ Modifies an incoming AP object (mastodon format) to our internal format. """ @@ -28,16 +36,8 @@ defmodule Pleroma.Web.ActivityPub.Transmogrifier do end def fix_actor(%{"attributedTo" => actor} = object) do - # attributedTo can be a list in the case of peertube or plume - actor = - if is_list(actor) do - Enum.at(actor, 0) - else - actor - end - object - |> Map.put("actor", actor) + |> Map.put("actor", get_actor(%{"actor" => actor})) end def fix_in_reply_to(%{"inReplyTo" => in_reply_to_id} = object) @@ -137,12 +137,12 @@ defmodule Pleroma.Web.ActivityPub.Transmogrifier do # - emoji def handle_incoming(%{"type" => "Create", "object" => %{"type" => objtype} = object} = data) when objtype in ["Article", "Note"] do + actor = get_actor(data) + data = Map.put(data, "actor", actor) + with nil <- Activity.get_create_activity_by_object_ap_id(object["id"]), %User{} = user <- User.get_or_fetch_by_ap_id(data["actor"]) do - # prefer the activity's actor instead of attributedTo - object = - fix_object(data["object"]) - |> Map.put("actor", data["actor"]) + object = fix_object(data["object"]) params = %{ to: data["to"], From 7501481db4be56cf7b5babeeebeb7b96273ae4db Mon Sep 17 00:00:00 2001 From: "Haelwenn (lanodan) Monnier" Date: Thu, 12 Jul 2018 23:25:44 +0200 Subject: [PATCH 3/3] [Pleroma.Web.ActivityPub.Transmogrifier] Add Person finding --- lib/pleroma/web/activity_pub/transmogrifier.ex | 5 +++++ 1 file changed, 5 insertions(+) diff --git a/lib/pleroma/web/activity_pub/transmogrifier.ex b/lib/pleroma/web/activity_pub/transmogrifier.ex index 6080303b6..2ebc526df 100644 --- a/lib/pleroma/web/activity_pub/transmogrifier.ex +++ b/lib/pleroma/web/activity_pub/transmogrifier.ex @@ -21,6 +21,11 @@ defmodule Pleroma.Web.ActivityPub.Transmogrifier do Enum.at(actor, 0) end + def get_actor(%{"actor" => actor_list}) do + Enum.find(actor_list, fn %{"type" => type} -> type == "Person" end) + |> Map.get("id") + end + @doc """ Modifies an incoming AP object (mastodon format) to our internal format. """