add changelog entry

Fixes #431

CHANGELOG.md

@@ -14,6 +14,12 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/).
 - Ability to "verify" links in profile fields via rel=me
 - Mix tasks to dump/load config to/from json for bulk editing
 - Followed hashtag list at /api/v1/followed\_tags, API parity with mastodon
+- Ability to set posting language in the post form, API parity with mastodon
+- Ability to match domains in MRF by a trailing wildcard
+  - Currently supported formats:
+    - `example.com` (implicitly matches `*.example.com`)
+    - `*.example.com`
+    - `example.*` (implicitly matches `*.example.*`)
 
 ### Removed
 - Non-finch HTTP adapters

docs/docs/installation/openbsd_en.md

@@ -1,6 +1,6 @@
 # Installing on OpenBSD
 
-This guide describes the installation and configuration of akkoma (and the required software to run it) on a single OpenBSD 6.6 server.
+This guide describes the installation and configuration of akkoma (and the required software to run it) on a single OpenBSD 7.2 server.
 
 For any additional information regarding commands and configuration files mentioned here, check the man pages [online](https://man.openbsd.org/) or directly on your server with the man command.
 
@@ -12,11 +12,10 @@ For any additional information regarding commands and configuration files mentio
 To install them, run the following command (with doas or as root):
 
 ```
-pkg_add elixir gmake git postgresql-server postgresql-contrib cmake ffmpeg ImageMagick erlang-wx-25
+pkg_add elixir gmake git postgresql-server postgresql-contrib cmake ffmpeg erlang-wx libmagic
+pkg_add erlang-wx # Choose the latest version as package version when promted
 ```
 
-(Note that the erlang version may change, it was 25 at the time of writing)
-
 Akkoma requires a reverse proxy, OpenBSD has relayd in base (and is used in this guide) and packages/ports are available for nginx (www/nginx) and apache (www/apache-httpd). Independently of the reverse proxy, [acme-client(1)](https://man.openbsd.org/acme-client) can be used to get a certificate from Let's Encrypt.
 
 #### Optional software
@@ -29,32 +28,35 @@ Per [`docs/installation/optional/media_graphics_packages.md`](../installation/op
 To install the above:
 
 ```
-pkg_add ImageMagick ffmpeg p5-Image-ExifTool
+pkg_add ffmpeg p5-Image-ExifTool
 ```
 
 #### Creating the akkoma user
-Akkoma will be run by a dedicated user, \_akkoma. Before creating it, insert the following lines in login.conf:
+Akkoma will be run by a dedicated user, `_akkoma`. Before creating it, insert the following lines in `/etc/login.conf`:
 ```
 akkoma:\
 	:datasize-max=1536M:\
 	:datasize-cur=1536M:\
 	:openfiles-max=4096
 ```
-This creates a "akkoma" login class and sets higher values than default for datasize and openfiles (see [login.conf(5)](https://man.openbsd.org/login.conf)), this is required to avoid having akkoma crash some time after starting.
+This creates a `akkoma` login class and sets higher values than default for datasize and openfiles (see [login.conf(5)](https://man.openbsd.org/login.conf)), this is required to avoid having akkoma crash some time after starting.
 
-Create the \_akkoma user, assign it the akkoma login class and create its home directory (/home/\_akkoma/): `useradd -m -L akkoma _akkoma`
+Create the `_akkoma` user, assign it the akkoma login class and create its home directory (`/home/_akkoma/`): `useradd -m -L akkoma _akkoma`
 
 #### Clone akkoma's directory
-Enter a shell as the \_akkoma user. As root, run `su _akkoma -;cd`. Then clone the repository with `git clone https://akkoma.dev/AkkomaGang/akkoma.git`. Akkoma is now installed in /home/\_akkoma/akkoma/, it will be configured and started at the end of this guide.
+Enter a shell as the `_akkoma` user. As root, run `su _akkoma -;cd`. Then clone the repository with `git clone https://akkoma.dev/AkkomaGang/akkoma.git`. Akkoma is now installed in `/home/_akkoma/akkoma/`, it will be configured and started at the end of this guide.
 
 #### PostgreSQL
-Start a shell as the \_postgresql user (as root run `su _postgresql -` then run the `initdb` command to initialize postgresql:
-You will need to specify pgdata directory to the default (/var/postgresql/data) with the `-D <path>` and set the user to postgres with the `-U <username>` flag. This can be done as follows:
+Create `_postgresql`'s user directory (it hasn't been created yet): `mdir var/postgresql/data`. To set it as home
+directory for user `_postgresql` run `usermod -d  /var/postgresql/data _postgresql`.
+
+Start a shell as the `_postgresql` user (as root run `su _postgresql -` then run the `initdb` command to initialize postgresql.
+You will need to specify pgdata directory to the default (`/var/postgresql/data`) with the `-D <path>` and set the user to postgres with the `-U <username>` flag. This can be done as follows:
 
 ```
 initdb -D /var/postgresql/data -U postgres
 ```
-If you are not using the default directory, you will have to update the `datadir` variable in the /etc/rc.d/postgresql script.
+If you are not using the default directory, you will have to update the `datadir` variable in the `/etc/rc.d/postgresql` script.
 
 When this is done, enable postgresql so that it starts on boot and start it. As root, run:
 ```
@@ -70,7 +72,7 @@ httpd will have three fuctions:
   * serve a robots.txt file
   * get Let's Encrypt certificates, with acme-client
 
-Insert the following config in httpd.conf:
+Insert the following config in `/etc/httpd.conf`:
 ```
 # $OpenBSD: httpd.conf,v 1.17 2017/04/16 08:50:49 ajacoutot Exp $
 
@@ -93,13 +95,10 @@ server "default" {
 	location "/robots.txt" { root "/htdocs/local/" }
 	location "/*" { block return 302 "https://$HTTP_HOST$REQUEST_URI" }
 }
-
-types {
-}
 ```
 Do not forget to change *<IPv4/6 address\>* to your server's address(es). If httpd should only listen on one protocol family, comment one of the two first *listen* options.
 
-Create the /var/www/htdocs/local/ folder and write the content of your robots.txt in /var/www/htdocs/local/robots.txt.
+Create the `/var/www/htdocs/local/` folder and write the content of your robots.txt in `/var/www/htdocs/local/robots.txt`.
 Check the configuration with `httpd -n`, if it is OK enable and start httpd (as root):
 ```
 rcctl enable httpd
@@ -108,7 +107,7 @@ rcctl start httpd
 
 #### acme-client
 acme-client is used to get SSL/TLS certificates from Let's Encrypt.
-Insert the following configuration in /etc/acme-client.conf:
+Insert the following configuration in `/etc/acme-client.conf`:
 ```
 #
 # $OpenBSD: acme-client.conf,v 1.4 2017/03/22 11:14:14 benno Exp $
@@ -129,7 +128,7 @@ domain <domain name> {
 }
 ```
 Replace *<domain name\>* by the domain name you'll use for your instance. As root, run `acme-client -n` to check the config, then `acme-client -ADv <domain name>` to create account and domain keys, and request a certificate for the first time.
-Make acme-client run everyday by adding it in /etc/daily.local. As root, run the following command: `echo "acme-client <domain name>" >> /etc/daily.local`.
+Make acme-client run everyday by adding it in `/etc/daily.local`. As root, run the following command: `echo "acme-client <domain name>" >> /etc/daily.local`.
 
 Relayd will look for certificates and keys based on the address it listens on (see next part), the easiest way to make them available to relayd is to create a link, as root run:
 ```
@@ -140,7 +139,7 @@ This will have to be done for each IPv4 and IPv6 address relayd listens on.
 
 #### relayd
 relayd will be used as the reverse proxy sitting in front of akkoma.
-Insert the following configuration in /etc/relayd.conf:
+Insert the following configuration in `/etc/relayd.conf`:
 ```
 # $OpenBSD: relayd.conf,v 1.4 2018/03/23 09:55:06 claudio Exp $
 
@@ -198,7 +197,7 @@ rcctl start relayd
 
 #### pf
 Enabling and configuring pf is highly recommended.
-In /etc/pf.conf, insert the following configuration:
+In `/etc/pf.conf`, insert the following configuration:
 ```
 # Macros
 if="<network interface>"
@@ -222,31 +221,30 @@ pass in quick on $if inet6 proto icmp6 to ($if) icmp6-type { echoreq unreach par
 pass in quick on $if proto tcp to ($if) port { http https } # relayd/httpd
 pass in quick on $if proto tcp from $authorized_ssh_clients to ($if) port ssh
 ```
-Replace *<network interface\>* by your server's network interface name (which you can get with ifconfig). Consider replacing the content of the authorized\_ssh\_clients macro by, for exemple, your home IP address, to avoid SSH connection attempts from bots.
+Replace *<network interface\>* by your server's network interface name (which you can get with ifconfig). Consider replacing the content of the `authorized_ssh_clients` macro by, for example, your home IP address, to avoid SSH connection attempts from bots.
 
 Check pf's configuration by running `pfctl -nf /etc/pf.conf`, load it with `pfctl -f /etc/pf.conf` and enable pf at boot with `rcctl enable pf`.
 
 #### Configure and start akkoma
-Enter a shell as \_akkoma (as root `su _akkoma -`) and enter akkoma's installation directory (`cd ~/akkoma/`).
+Enter a shell as `_akkoma` (as root `su _akkoma -`) and enter akkoma's installation directory (`cd ~/akkoma/`).
 
 Then follow the main installation guide:
 
   * run `mix deps.get`
   * run `MIX_ENV=prod mix pleroma.instance gen` and enter your instance's information when asked
-  * copy config/generated\_config.exs to config/prod.secret.exs. The default values should be sufficient but you should edit it and check that everything seems OK.
+  * copy `config/generated_config.exs` to `config/prod.secret.exs`. The default values should be sufficient but you should edit it and check that everything seems OK.
   * exit your current shell back to a root one and run `psql -U postgres -f /home/_akkoma/akkoma/config/setup_db.psql` to setup the database.
-  * return to a \_akkoma shell into akkoma's installation directory (`su _akkoma -;cd ~/akkoma`) and run `MIX_ENV=prod mix ecto.migrate`
+  * return to a `_akkoma` shell into akkoma's installation directory (`su _akkoma -;cd ~/akkoma`) and run `MIX_ENV=prod mix ecto.migrate`
 
-As \_akkoma in /home/\_akkoma/akkoma, you can now run `LC_ALL=en_US.UTF-8 MIX_ENV=prod mix phx.server` to start your instance.
+As `_akkoma` in `/home/_akkoma/akkoma`, you can now run `LC_ALL=en_US.UTF-8 MIX_ENV=prod mix phx.server` to start your instance.
 In another SSH session/tmux window, check that it is working properly by running `ftp -MVo - http://127.0.0.1:4000/api/v1/instance`, you should get json output. Double-check that *uri*'s value is your instance's domain name.
 
 ##### Starting akkoma at boot
 An rc script to automatically start akkoma at boot hasn't been written yet, it can be run in a tmux session (tmux is in base).
 
-
 #### Create administrative user
 
-If your instance is up and running, you can create your first user with administrative rights with the following command as the \_akkoma user.
+If your instance is up and running, you can create your first user with administrative rights with the following command as the `_akkoma` user.
 ```
 LC_ALL=en_US.UTF-8 MIX_ENV=prod mix pleroma.user new <username> <your@emailaddress> --admin
 ```

docs/docs/installation/otp_en.md

@@ -5,7 +5,7 @@
 This guide covers a installation using an OTP release. To install Akkoma from source, please check out the corresponding guide for your distro.
 
 ## Pre-requisites
-* A machine running Linux with GNU (e.g. Debian, Ubuntu) or musl (e.g. Alpine) libc and `x86_64`, `aarch64` or `armv7l` CPU, you have root access to. If you are not sure if it's compatible see [Detecting flavour section](#detecting-flavour) below
+* A machine running Linux with GNU (e.g. Debian, Ubuntu) or musl (e.g. Alpine) libc and an `x86_64` CPU you have root access to. If you are not sure if it's compatible see [Detecting flavour section](#detecting-flavour) below
 * For installing OTP releases on RedHat-based distros like Fedora and Centos Stream, please follow [this guide](./otp_redhat_en.md) instead.
 * A (sub)domain pointed to the machine
 

lib/pleroma/emoji/pack.ex

@@ -252,7 +252,7 @@ defmodule Pleroma.Emoji.Pack do
 
     with :ok <- validate_shareable_packs_available(uri),
          {:ok, remote_pack} <-
-           uri |> URI.merge("/api/v1/pleroma/emoji/pack?name=#{name}") |> http_get(),
+           uri |> URI.merge("/api/v1/pleroma/emoji/pack?name=#{URI.encode(name)}") |> http_get(),
          {:ok, %{sha: sha, url: url} = pack_info} <- fetch_pack_info(remote_pack, uri, name),
          {:ok, archive} <- download_archive(url, sha),
          pack <- copy_as(remote_pack, as || name),
@@ -593,7 +593,9 @@ defmodule Pleroma.Emoji.Pack do
         {:ok,
          %{
            sha: sha,
-           url: URI.merge(uri, "/api/v1/pleroma/emoji/packs/archive?name=#{name}") |> to_string()
+           url:
+             URI.merge(uri, "/api/v1/pleroma/emoji/packs/archive?name=#{URI.encode(name)}")
+             |> to_string()
          }}
 
       %{"fallback-src" => src, "fallback-src-sha256" => sha} when is_binary(src) ->

lib/pleroma/web/activity_pub/mrf.ex

@@ -161,10 +161,17 @@ defmodule Pleroma.Web.ActivityPub.MRF do
   # - https://extra.baddomain.net/
   # Does NOT match the following:
   # - https://maybebaddomain.net/
+
+  # *.baddomain.net
   def subdomain_regex("*." <> domain), do: subdomain_regex(domain)
 
+  # baddomain.net
   def subdomain_regex(domain) do
-    ~r/^(.+\.)?#{Regex.escape(domain)}$/i
+    if String.ends_with?(domain, ".*") do
+      ~r/^(.+\.)?#{Regex.escape(String.replace_suffix(domain, ".*", ""))}\.(.+)$/i
+    else
+      ~r/^(.+\.)?#{Regex.escape(domain)}$/i
+    end
   end
 
   @spec subdomains_regex([String.t()]) :: [Regex.t()]

lib/pleroma/web/static_fe/static_fe_controller.ex

@@ -25,7 +25,7 @@ defmodule Pleroma.Web.StaticFE.StaticFEController do
          true <- Visibility.is_public?(activity.object),
          {_, true} <- {:visible?, Visibility.visible_for_user?(activity, _reading_user = nil)},
          %User{} = user <- User.get_by_ap_id(activity.object.data["actor"]) do
-      meta = Metadata.build_tags(%{activity_id: notice_id, object: activity.object, user: user})
+      meta = Metadata.build_tags(%{url: activity.data["id"], object: activity.object, user: user})
 
       timeline =
         activity.object.data["context"]

priv/static/schemas/litepub-0.1.jsonld

@@ -41,7 +41,11 @@
                 "@type": "@id"
             },
             "vcard": "http://www.w3.org/2006/vcard/ns#",
-            "formerRepresentations": "litepub:formerRepresentations"
+            "formerRepresentations": "litepub:formerRepresentations",
+            "contentMap": {
+                "@id": "as:content",
+                "@container": "@language"
+            }
         }
     ]
 }

test/instance_static/emoji/test with spaces/pack.json

@@ -0,0 +1,12 @@
+{
+    "files": {
+        "blank": "blank.png",
+        "blank2": "blank2.png"
+    },
+    "pack": {
+        "description": "Test description",
+        "homepage": "https://pleroma.social",
+        "license": "Test license",
+        "share-files": true
+    }
+}

test/pleroma/web/activity_pub/mrf_test.exs

@@ -48,6 +48,31 @@ defmodule Pleroma.Web.ActivityPub.MRFTest do
       refute MRF.subdomain_match?(regexes, "sub.unsafe.tldanother")
     end
 
+    test "wildcard on the tld" do
+      regexes = MRF.subdomains_regex(["somewhere.*"])
+
+      assert regexes == [~r/^(.+\.)?somewhere\.(.+)$/i]
+
+      assert MRF.subdomain_match?(regexes, "somewhere.net")
+      assert MRF.subdomain_match?(regexes, "somewhere.com")
+      assert MRF.subdomain_match?(regexes, "somewhere.somewherelese.net")
+      refute MRF.subdomain_match?(regexes, "somewhere")
+    end
+
+    test "wildcards on subdomain _and_ tld" do
+      regexes = MRF.subdomains_regex(["*.somewhere.*"])
+
+      assert regexes == [~r/^(.+\.)?somewhere\.(.+)$/i]
+
+      assert MRF.subdomain_match?(regexes, "somewhere.net")
+      assert MRF.subdomain_match?(regexes, "somewhere.com")
+      assert MRF.subdomain_match?(regexes, "sub.somewhere.net")
+      assert MRF.subdomain_match?(regexes, "sub.somewhere.com")
+      assert MRF.subdomain_match?(regexes, "sub.sub.somewhere.net")
+      assert MRF.subdomain_match?(regexes, "sub.sub.somewhere.com")
+      refute MRF.subdomain_match?(regexes, "somewhere")
+    end
+
     test "matches are case-insensitive" do
       regexes = MRF.subdomains_regex(["UnSafe.TLD", "UnSAFE2.Tld"])
 

test/pleroma/web/activity_pub/object_validators/article_note_page_validator_test.exs

@@ -40,11 +40,17 @@ defmodule Pleroma.Web.ActivityPub.ObjectValidators.ArticleNotePageValidatorTest
     end
 
     test "a note with a language validates" do
-      user = insert(:user, %{ap_id: "https://mastodon.social/users/akkoma_ap_integration_tester"})
+      insert(:user, %{ap_id: "https://mastodon.social/users/akkoma_ap_integration_tester"})
       note = File.read!("test/fixtures/mastodon/note_with_language.json") |> Jason.decode!()
-      %{valid?: true, changes: %{ contentMap: %{
-        "ja" => "<p>tag</p>",
-      }}} = ArticleNotePageValidator.cast_and_validate(note)
+
+      %{
+        valid?: true,
+        changes: %{
+          contentMap: %{
+            "ja" => "<p>tag</p>"
+          }
+        }
+      } = ArticleNotePageValidator.cast_and_validate(note)
     end
 
     test "a note from factory validates" do

test/pleroma/web/pleroma_api/controllers/emoji_pack_controller_test.exs

@@ -40,11 +40,11 @@ defmodule Pleroma.Web.PleromaAPI.EmojiPackControllerTest do
       |> get("/api/v1/pleroma/emoji/packs")
       |> json_response_and_validate_schema(200)
 
-    assert resp["count"] == 4
+    assert resp["count"] == 5
 
     assert resp["packs"]
            |> Map.keys()
-           |> length() == 4
+           |> length() == 5
 
     shared = resp["packs"]["test_pack"]
     assert shared["files"] == %{"blank" => "blank.png", "blank2" => "blank2.png"}
@@ -61,7 +61,7 @@ defmodule Pleroma.Web.PleromaAPI.EmojiPackControllerTest do
       |> get("/api/v1/pleroma/emoji/packs?page_size=1")
       |> json_response_and_validate_schema(200)
 
-    assert resp["count"] == 4
+    assert resp["count"] == 5
 
     packs = Map.keys(resp["packs"])
 
@@ -74,7 +74,7 @@ defmodule Pleroma.Web.PleromaAPI.EmojiPackControllerTest do
       |> get("/api/v1/pleroma/emoji/packs?page_size=1&page=2")
       |> json_response_and_validate_schema(200)
 
-    assert resp["count"] == 4
+    assert resp["count"] == 5
     packs = Map.keys(resp["packs"])
     assert length(packs) == 1
     [pack2] = packs
@@ -84,7 +84,7 @@ defmodule Pleroma.Web.PleromaAPI.EmojiPackControllerTest do
       |> get("/api/v1/pleroma/emoji/packs?page_size=1&page=3")
       |> json_response_and_validate_schema(200)
 
-    assert resp["count"] == 4
+    assert resp["count"] == 5
     packs = Map.keys(resp["packs"])
     assert length(packs) == 1
     [pack3] = packs
@@ -94,7 +94,7 @@ defmodule Pleroma.Web.PleromaAPI.EmojiPackControllerTest do
       |> get("/api/v1/pleroma/emoji/packs?page_size=1&page=4")
       |> json_response_and_validate_schema(200)
 
-    assert resp["count"] == 4
+    assert resp["count"] == 5
     packs = Map.keys(resp["packs"])
     assert length(packs) == 1
     [pack4] = packs
@@ -221,6 +221,24 @@ defmodule Pleroma.Web.PleromaAPI.EmojiPackControllerTest do
           url: "https://nonshared-pack"
         } ->
           text(File.read!("#{@emoji_path}/test_pack_nonshared/nonshared.zip"))
+
+        %{
+          method: :get,
+          url: "https://example.com/api/v1/pleroma/emoji/pack?name=test%20with%20spaces"
+        } ->
+          conn
+          |> get("/api/v1/pleroma/emoji/pack?name=test%20with%20spaces")
+          |> json_response_and_validate_schema(200)
+          |> json()
+
+        %{
+          method: :get,
+          url: "https://example.com/api/v1/pleroma/emoji/packs/archive?name=test%20with%20spaces"
+        } ->
+          conn
+          |> get("/api/v1/pleroma/emoji/packs/archive?name=test%20with%20spaces")
+          |> response(200)
+          |> text()
       end)
 
       assert admin_conn
@@ -261,6 +279,18 @@ defmodule Pleroma.Web.PleromaAPI.EmojiPackControllerTest do
              |> json_response_and_validate_schema(200) == "ok"
 
       refute File.exists?("#{@emoji_path}/test_pack_nonshared2")
+
+      assert admin_conn
+             |> put_req_header("content-type", "multipart/form-data")
+             |> post("/api/v1/pleroma/emoji/packs/download", %{
+               url: "https://example.com",
+               name: "test with spaces",
+               as: "test with spaces"
+             })
+             |> json_response_and_validate_schema(200) == "ok"
+
+      assert File.exists?("#{@emoji_path}/test with spaces/pack.json")
+      assert File.exists?("#{@emoji_path}/test with spaces/blank.png")
     end
 
     test "nonshareable instance", %{admin_conn: admin_conn} do