From 5c9f65e8caea91191763125256b1cd0bd3bc3792 Mon Sep 17 00:00:00 2001 From: Oneric Date: Sat, 23 Nov 2024 23:12:50 +0100 Subject: [PATCH 16/22] stats: use cheaper peers query MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit This query is one of the top cost offenders during an instances lifetime. For small instances it was shown to take up 30-50% percent of the total database query time, while for bigger isntaces it still held a spot in the top 3 — alost as or even more expensive overall than timeline queries! The good news is, there’s a cheaper way using the instance table: no need to process each entry, no need to filter NULLs and no need to dedupe. EXPLAIN estimates the cost of the old query as 13272.39 and the cost of the new query as 395.74 for me; i.e. a 33-fold reduction. Results can slightly differ. E.g. we might have an old user predating the instance tables existence and no interaction with since or no instance table entry due to failure to query nodeinfo. Conversely, we might have an instance entry but all known users got deleted since. However, this seems unproblematic in practice and well worth the perf improvment. Given the previous query didn’t exclude unreachable instances neither does the new query. --- lib/pleroma/stats.ex | 8 ++++---- .../mastodon_api/controllers/instance_controller_test.exs | 4 ++++ 2 files changed, 8 insertions(+), 4 deletions(-) diff --git a/lib/pleroma/stats.ex b/lib/pleroma/stats.ex index 88740d155..7bbe089f8 100644 --- a/lib/pleroma/stats.ex +++ b/lib/pleroma/stats.ex @@ -10,6 +10,7 @@ defmodule Pleroma.Stats do alias Pleroma.CounterCache alias Pleroma.Repo alias Pleroma.User + alias Pleroma.Instances.Instance @interval :timer.seconds(300) @@ -66,14 +67,13 @@ def get_peers do } } def calculate_stat_data do + # instances table has an unique constraint on the host column peers = from( - u in User, - select: fragment("distinct split_part(?, '@', 2)", u.nickname), - where: u.local != ^true + i in Instance, + select: i.host ) |> Repo.all() - |> Enum.filter(& &1) domain_count = Enum.count(peers) diff --git a/test/pleroma/web/mastodon_api/controllers/instance_controller_test.exs b/test/pleroma/web/mastodon_api/controllers/instance_controller_test.exs index 7b400d1ee..39e9629eb 100644 --- a/test/pleroma/web/mastodon_api/controllers/instance_controller_test.exs +++ b/test/pleroma/web/mastodon_api/controllers/instance_controller_test.exs @@ -61,7 +61,9 @@ test "get instance stats", %{conn: conn} do {:ok, _user2} = User.set_activation(user2, false) insert(:user, %{local: false, nickname: "u@peer1.com"}) + insert(:instance, %{domain: "peer1.com"}) insert(:user, %{local: false, nickname: "u@peer2.com"}) + insert(:instance, %{domain: "peer2.com"}) {:ok, _} = Pleroma.Web.CommonAPI.post(user, %{status: "cofe"}) @@ -81,7 +83,9 @@ test "get instance stats", %{conn: conn} do test "get peers", %{conn: conn} do insert(:user, %{local: false, nickname: "u@peer1.com"}) + insert(:instance, %{domain: "peer1.com"}) insert(:user, %{local: false, nickname: "u@peer2.com"}) + insert(:instance, %{domain: "peer2.com"}) Pleroma.Stats.force_update() -- 2.39.5