[bug] Indexing silently fails with Elasticsearch #549

New Issue

Open

opened 2023-05-26 21:35:39 +00:00 by Kris · 1 comment

Kris commented

2023-05-26 21:35:39 +00:00

Your setup

Docker

Extra details

Slightly modified Docker setup via Podman

Version

3.9.3-28-g9d7c877d

PostgreSQL version

Latest docker container

What were you trying to do?

I am trying to get search working with Elasticsearch. I got a single node setup up and running that I am using with Forgejo successfully and though it might be nice to add Akkoma to the same one.

This is to avoid running an extra Meilisearch, which seems to work fine, but uses a lot of resources without actually speeding up the search much.

What did you expect to happen?

Following the official documentation and previous guidance when I tried the same with Zincsearch (similar error actually) I expected there to be some on screen feedback of the indexing failing or so, but it just takes a long time without any result and seems to time out or only work in RAM and not actually write to the Elasticsearch instance.

What actually happened?

When running mix pleroma.search import activities (with MIX_ENV=prod set) it just gives a blinking cursor for a long time (>30 minutes) and the RAM use of the Akkoma container shoots up, but nothing seems to happen Elasticsearch side and after a while it times out and RAM use returns to normal.

Logs

When doing a search in Akkoma-FE without or during the failing indexing I get this in the logs. It mentions Elasticsearch, but seems unrelated to the failing indexing process?

```
21:21:56.338 [error] [__exception__: true, __struct__: HTTPoison.Error, id: nil, reason: :nxdomain]

warning: passing {:error, %HTTPoison.Error{reason: :nxdomain, id: nil}} to Logger is deprecated, expected a map, a keyword list, a string, a list of strings, or a zero-arity anonymous function

  (logger 1.14.3) lib/logger.ex:898: Logger.__do_log__/4

  (pleroma 3.9.3-28-g9d7c877d) lib/pleroma/search/elasticsearch/store.ex:47: Pleroma.Search.Elasticsearch.Store.search/2

  (pleroma 3.9.3-28-g9d7c877d) lib/pleroma/search/elasticsearch.ex:62: anonymous fn/4 in Pleroma.Search.Elasticsearch.search/3

  (elixir 1.14.3) lib/task/supervised.ex:89: Task.Supervised.invoke_mfa/2

  (elixir 1.14.3) lib/task/supervised.ex:34: Task.Supervised.reply/4

  (stdlib 4.3) proc_lib.erl:240: :proc_lib.init_p_do_apply/3


21:21:56.339 [error] Task #PID<0.2422.0> started from #PID<0.2420.0> terminating

** (Protocol.UndefinedError) protocol String.Chars not implemented for {:error, %HTTPoison.Error{reason: :nxdomain, id: nil}} of type Tuple. This protocol is implemented for the following type(s): Atom, BitString, Date, DateTime, Decimal, Float, Floki.Selector, Floki.Selector.AttributeSelector, Floki.Selector.Combinator, Floki.Selector.Functional, Floki.Selector.PseudoClass, Hex.Solver.Assignment, Hex.Solver.Constraints.Empty, Hex.Solver.Constraints.Range, Hex.Solver.Constraints.Union, Hex.Solver.Incompatibility, Hex.Solver.PackageRange, Hex.Solver.Term, Integer, List, NaiveDateTime, OpenApiSpex.Cast.Error, Phoenix.LiveComponent.CID, Postgrex.Copy, Postgrex.Query, Regex, RemoteIp.Block, Time, URI, Version, Version.Requirement

    (elixir 1.14.3) lib/string/chars.ex:3: String.Chars.impl_for!/1

    (elixir 1.14.3) lib/string/chars.ex:22: String.Chars.to_string/1

    (logger 1.14.3) lib/logger.ex:903: Logger.__do_log__/4

    (pleroma 3.9.3-28-g9d7c877d) lib/pleroma/search/elasticsearch/store.ex:47: Pleroma.Search.Elasticsearch.Store.search/2

    (pleroma 3.9.3-28-g9d7c877d) lib/pleroma/search/elasticsearch.ex:62: anonymous fn/4 in Pleroma.Search.Elasticsearch.search/3

    (elixir 1.14.3) lib/task/supervised.ex:89: Task.Supervised.invoke_mfa/2

    (elixir 1.14.3) lib/task/supervised.ex:34: Task.Supervised.reply/4

    (stdlib 4.3) proc_lib.erl:240: :proc_lib.init_p_do_apply/3

Function: #Function<1.78122218/0 in Pleroma.Search.Elasticsearch.search/3>

    Args: []


21:21:56.341 [error] Ranch protocol #PID<0.2416.0> of listener Pleroma.Web.Endpoint.HTTP (connection #PID<0.2415.0>, stream id 1) terminated

an exception was raised:

    ** (Protocol.UndefinedError) protocol String.Chars not implemented for {:error, %HTTPoison.Error{reason: :nxdomain, id: nil}} of type Tuple. This protocol is implemented for the following type(s): Atom, BitString, Date, DateTime, Decimal, Float, Floki.Selector, Floki.Selector.AttributeSelector, Floki.Selector.Combinator, Floki.Selector.Functional, Floki.Selector.PseudoClass, Hex.Solver.Assignment, Hex.Solver.Constraints.Empty, Hex.Solver.Constraints.Range, Hex.Solver.Constraints.Union, Hex.Solver.Incompatibility, Hex.Solver.PackageRange, Hex.Solver.Term, Integer, List, NaiveDateTime, OpenApiSpex.Cast.Error, Phoenix.LiveComponent.CID, Postgrex.Copy, Postgrex.Query, Regex, RemoteIp.Block, Time, URI, Version, Version.Requirement

        (elixir 1.14.3) lib/string/chars.ex:3: String.Chars.impl_for!/1

        (elixir 1.14.3) lib/string/chars.ex:22: String.Chars.to_string/1

        (logger 1.14.3) lib/logger.ex:903: Logger.__do_log__/4

        (pleroma 3.9.3-28-g9d7c877d) lib/pleroma/search/elasticsearch/store.ex:47: Pleroma.Search.Elasticsearch.Store.search/2

        (pleroma 3.9.3-28-g9d7c877d) lib/pleroma/search/elasticsearch.ex:62: anonymous fn/4 in Pleroma.Search.Elasticsearch.search/3

        (elixir 1.14.3) lib/task/supervised.ex:89: Task.Supervised.invoke_mfa/2

        (elixir 1.14.3) lib/task/supervised.ex:34: Task.Supervised.reply/4

        (stdlib 4.3) proc_lib.erl:240: :proc_lib.init_p_do_apply/3


21:21:56.345 [info] Postgrex.Protocol (#PID<0.962.0>) disconnected: ** (DBConnection.ConnectionError) client #PID<0.2418.0> exited
```

Severity

I cannot use it as easily as I'd like

Have you searched for this issue?

I have double-checked and have not found this issue mentioned anywhere.

### Your setup Docker ### Extra details Slightly modified Docker setup via Podman ### Version 3.9.3-28-g9d7c877d ### PostgreSQL version Latest docker container ### What were you trying to do? I am trying to get search working with Elasticsearch. I got a single node setup up and running that I am using with Forgejo successfully and though it might be nice to add Akkoma to the same one. This is to avoid running an extra Meilisearch, which seems to work fine, but uses a lot of resources without actually speeding up the search much. ### What did you expect to happen? Following the official documentation and previous guidance when I tried the same with Zincsearch (similar error actually) I expected there to be some on screen feedback of the indexing failing or so, but it just takes a long time without any result and seems to time out or only work in RAM and not actually write to the Elasticsearch instance. ### What actually happened? When running `mix pleroma.search import activities` (with `MIX_ENV=prod` set) it just gives a blinking cursor for a long time (>30 minutes) and the RAM use of the Akkoma container shoots up, but nothing seems to happen Elasticsearch side and after a while it times out and RAM use returns to normal. ### Logs ````shell When doing a search in Akkoma-FE without or during the failing indexing I get this in the logs. It mentions Elasticsearch, but seems unrelated to the failing indexing process? ``` 21:21:56.338 [error] [__exception__: true, __struct__: HTTPoison.Error, id: nil, reason: :nxdomain] warning: passing {:error, %HTTPoison.Error{reason: :nxdomain, id: nil}} to Logger is deprecated, expected a map, a keyword list, a string, a list of strings, or a zero-arity anonymous function (logger 1.14.3) lib/logger.ex:898: Logger.__do_log__/4 (pleroma 3.9.3-28-g9d7c877d) lib/pleroma/search/elasticsearch/store.ex:47: Pleroma.Search.Elasticsearch.Store.search/2 (pleroma 3.9.3-28-g9d7c877d) lib/pleroma/search/elasticsearch.ex:62: anonymous fn/4 in Pleroma.Search.Elasticsearch.search/3 (elixir 1.14.3) lib/task/supervised.ex:89: Task.Supervised.invoke_mfa/2 (elixir 1.14.3) lib/task/supervised.ex:34: Task.Supervised.reply/4 (stdlib 4.3) proc_lib.erl:240: :proc_lib.init_p_do_apply/3 21:21:56.339 [error] Task #PID<0.2422.0> started from #PID<0.2420.0> terminating ** (Protocol.UndefinedError) protocol String.Chars not implemented for {:error, %HTTPoison.Error{reason: :nxdomain, id: nil}} of type Tuple. This protocol is implemented for the following type(s): Atom, BitString, Date, DateTime, Decimal, Float, Floki.Selector, Floki.Selector.AttributeSelector, Floki.Selector.Combinator, Floki.Selector.Functional, Floki.Selector.PseudoClass, Hex.Solver.Assignment, Hex.Solver.Constraints.Empty, Hex.Solver.Constraints.Range, Hex.Solver.Constraints.Union, Hex.Solver.Incompatibility, Hex.Solver.PackageRange, Hex.Solver.Term, Integer, List, NaiveDateTime, OpenApiSpex.Cast.Error, Phoenix.LiveComponent.CID, Postgrex.Copy, Postgrex.Query, Regex, RemoteIp.Block, Time, URI, Version, Version.Requirement (elixir 1.14.3) lib/string/chars.ex:3: String.Chars.impl_for!/1 (elixir 1.14.3) lib/string/chars.ex:22: String.Chars.to_string/1 (logger 1.14.3) lib/logger.ex:903: Logger.__do_log__/4 (pleroma 3.9.3-28-g9d7c877d) lib/pleroma/search/elasticsearch/store.ex:47: Pleroma.Search.Elasticsearch.Store.search/2 (pleroma 3.9.3-28-g9d7c877d) lib/pleroma/search/elasticsearch.ex:62: anonymous fn/4 in Pleroma.Search.Elasticsearch.search/3 (elixir 1.14.3) lib/task/supervised.ex:89: Task.Supervised.invoke_mfa/2 (elixir 1.14.3) lib/task/supervised.ex:34: Task.Supervised.reply/4 (stdlib 4.3) proc_lib.erl:240: :proc_lib.init_p_do_apply/3 Function: #Function<1.78122218/0 in Pleroma.Search.Elasticsearch.search/3> Args: [] 21:21:56.341 [error] Ranch protocol #PID<0.2416.0> of listener Pleroma.Web.Endpoint.HTTP (connection #PID<0.2415.0>, stream id 1) terminated an exception was raised: ** (Protocol.UndefinedError) protocol String.Chars not implemented for {:error, %HTTPoison.Error{reason: :nxdomain, id: nil}} of type Tuple. This protocol is implemented for the following type(s): Atom, BitString, Date, DateTime, Decimal, Float, Floki.Selector, Floki.Selector.AttributeSelector, Floki.Selector.Combinator, Floki.Selector.Functional, Floki.Selector.PseudoClass, Hex.Solver.Assignment, Hex.Solver.Constraints.Empty, Hex.Solver.Constraints.Range, Hex.Solver.Constraints.Union, Hex.Solver.Incompatibility, Hex.Solver.PackageRange, Hex.Solver.Term, Integer, List, NaiveDateTime, OpenApiSpex.Cast.Error, Phoenix.LiveComponent.CID, Postgrex.Copy, Postgrex.Query, Regex, RemoteIp.Block, Time, URI, Version, Version.Requirement (elixir 1.14.3) lib/string/chars.ex:3: String.Chars.impl_for!/1 (elixir 1.14.3) lib/string/chars.ex:22: String.Chars.to_string/1 (logger 1.14.3) lib/logger.ex:903: Logger.__do_log__/4 (pleroma 3.9.3-28-g9d7c877d) lib/pleroma/search/elasticsearch/store.ex:47: Pleroma.Search.Elasticsearch.Store.search/2 (pleroma 3.9.3-28-g9d7c877d) lib/pleroma/search/elasticsearch.ex:62: anonymous fn/4 in Pleroma.Search.Elasticsearch.search/3 (elixir 1.14.3) lib/task/supervised.ex:89: Task.Supervised.invoke_mfa/2 (elixir 1.14.3) lib/task/supervised.ex:34: Task.Supervised.reply/4 (stdlib 4.3) proc_lib.erl:240: :proc_lib.init_p_do_apply/3 21:21:56.345 [info] Postgrex.Protocol (#PID<0.962.0>) disconnected: ** (DBConnection.ConnectionError) client #PID<0.2418.0> exited ``` ```` ### Severity I cannot use it as easily as I'd like ### Have you searched for this issue? - [x] I have double-checked and have not found this issue mentioned anywhere.

Kris added the

bug

label 2023-05-26 21:35:39 +00:00

Kris commented

2023-06-06 10:02:49 +00:00

Author

I am somewhat suspecting that the Elasticsearch library might be actively checking for alternative implementations and preventing the use of them :(

At least for a different Python project I am trying with Opensearch that has more verbose out-put that is the case...

I am somewhat suspecting that the Elasticsearch library might be actively checking for alternative implementations and preventing the use of them :( At least for a different Python project I am trying with Opensearch that has more verbose out-put that is the case...

No Milestone

No project

No Assignees

1 Participants

Notifications

Due Date

The due date is invalid or out of range. Please use the format 'yyyy-mm-dd'.

No due date set.

Dependencies

No dependencies set.

Reference: AkkomaGang/akkoma#549

No description provided.