stats: use cheaper peers query

This query is one of the top cost offenders during an instances
lifetime. For small instances it was shown to take up 30-50% percent of
the total database query time, while for bigger isntaces it still held
a spot in the top 3 — alost as or even more expensive overall than
timeline queries!

The good news is, there’s a cheaper way using the instance table:
no need to process each entry, no need to filter NULLs
and no need to dedupe. EXPLAIN estimates the cost of the
old query as 13272.39 and the cost of the new query as 395.74
for me; i.e. a 33-fold reduction.

Results can slightly differ. E.g. we might have an old user
predating the instance tables existence and no interaction with since
or no instance table entry due to failure to query nodeinfo.
Conversely, we might have an instance entry but all known users got
deleted since.
However, this seems unproblematic in practice
and well worth the perf improvment.

Given the previous query didn’t exclude unreachable instances
neither does the new query.
This commit is contained in:
Oneric 2024-11-23 23:12:50 +01:00
parent 9bf1460a40
commit b33ad1d8bd
2 changed files with 8 additions and 4 deletions

View file

@ -10,6 +10,7 @@ defmodule Pleroma.Stats do
alias Pleroma.CounterCache
alias Pleroma.Repo
alias Pleroma.User
alias Pleroma.Instances.Instance
@interval :timer.seconds(300)
@ -66,14 +67,13 @@ def get_peers do
}
}
def calculate_stat_data do
# instances table has an unique constraint on the host column
peers =
from(
u in User,
select: fragment("distinct split_part(?, '@', 2)", u.nickname),
where: u.local != ^true
i in Instance,
select: i.host
)
|> Repo.all()
|> Enum.filter(& &1)
domain_count = Enum.count(peers)

View file

@ -61,7 +61,9 @@ test "get instance stats", %{conn: conn} do
{:ok, _user2} = User.set_activation(user2, false)
insert(:user, %{local: false, nickname: "u@peer1.com"})
insert(:instance, %{domain: "peer1.com"})
insert(:user, %{local: false, nickname: "u@peer2.com"})
insert(:instance, %{domain: "peer2.com"})
{:ok, _} = Pleroma.Web.CommonAPI.post(user, %{status: "cofe"})
@ -81,7 +83,9 @@ test "get instance stats", %{conn: conn} do
test "get peers", %{conn: conn} do
insert(:user, %{local: false, nickname: "u@peer1.com"})
insert(:instance, %{domain: "peer1.com"})
insert(:user, %{local: false, nickname: "u@peer2.com"})
insert(:instance, %{domain: "peer2.com"})
Pleroma.Stats.force_update()