Elasticsearch: Search doesn't work with OpenSearch #935

New issue

Open

opened 2025-05-21 17:36:57 +00:00 by akira · 0 comments

akira commented

2025-05-21 17:36:57 +00:00

Since OpenSearch is a fork of Elasticsearch, they share very similar API interface. So I want to give OpenSearch a shot. Long story short, I spined off an OpenSearch instance on my home server and followed the documentation to set it up as Elasticsearch. Indexing works fine. However, searching returns an error line in the log:

Error log

May 20 10:42:18 r730-akkoma pleroma[17129]: warning: passing {:error, %Elasticsearch.Exception{status: 400, line: nil, col: nil, message: "[1:59] [field_sort] unknown field [format]", type: "x_content_parse_exception", query: nil, raw: %{"error" => %{"reason" => "[1:59] [field_sort] unknown field [format]", "root_cause" => [%{"reason" => "[1:59] [field_sort] unknown field [format]", "type" => "x_content_parse_exception"}], "type" => "x_content_parse_exception"}, "status" => 400}}} to Logger is deprecated, expected a map, a keyword list, a string, a list of strings, or a zero-arity anonymous function

With this hint: [field_sort] unknown field [format], I found the Akkoma code which generates the Elasticsearch query:

       sort: [
         "_score",
         %{"_timestamp" => %{order: "desc", format: "basic_date_time"}}
       ],

This format field in the sort object is an Elasticsearch 7.13 feature introduced after the OpenSearch fork based on Elasticsearch 7.10. Therefore, it is expected to complain about the unknown field.

Looking at Akkoma code, it seems that Akkoma doesn't really require the _timestamp field to be formatted in a certain way, because only the _id field is used:

     with {:ok, results} <- search(:raw, "activities", q) do
       results
       |> Enum.map(fn result -> result["_id"] end)
       |> Pleroma.Activity.all_by_ids_with_object()

Therefore, it should be safe to remove the format field from the sort object in the Elasticsearch query. And this simple change would make it compatible with OpenSearch. I am running it on my server, and it works.

I know this is not a bug because Akkoma doesn't officially support OpenSearch. However, given the current license status of the Elasticsearch and OpenSearch, I am sure some folk might want to go with the community-driven OpenSearch, instead of Elasticsearch as a private company. More back stories [here](https://devclass.com/2025/05/07/opensearch-3-0-hits-first-major-release-under-linux-foundation-as-it-battles-elasticsearch-for-mindshare/). Since OpenSearch is a fork of Elasticsearch, they share very similar API interface. So I want to give OpenSearch a shot. Long story short, I spined off an OpenSearch instance on my home server and followed the documentation to set it up as Elasticsearch. Indexing works fine. However, searching returns an error line in the log: <details> <summary>Error log</summary> <blockquote>May 20 10:42:18 r730-akkoma pleroma[17129]: warning: passing {:error, %Elasticsearch.Exception{status: 400, line: nil, col: nil, message: "[1:59] [field_sort] unknown field [format]", type: "x_content_parse_exception", query: nil, raw: %{"error" => %{"reason" => "[1:59] <b><i>[field_sort] unknown field [format]</i></b>", "root_cause" => [%{"reason" => "[1:59] [field_sort] unknown field [format]", "type" => "x_content_parse_exception"}], "type" => "x_content_parse_exception"}, "status" => 400}}} to Logger is deprecated, expected a map, a keyword list, a string, a list of strings, or a zero-arity anonymous function</blockquote> </details> With this hint: `[field_sort] unknown field [format]`, I found the Akkoma code which generates the Elasticsearch query: https://akkoma.dev/AkkomaGang/akkoma/src/commit/3791b695f486908c3ea0705c0ece5694eb2ce281/lib/pleroma/search/elasticsearch.ex#L21-L24 This `format` field in the `sort` object is an Elasticsearch 7.13 feature [introduced](https://github.com/elastic/elasticsearch/issues/69192) after the OpenSearch fork based on Elasticsearch 7.10. Therefore, it is expected to complain about the unknown field. Looking at Akkoma code, it seems that Akkoma doesn't really require the `_timestamp` field to be formatted in a certain way, because only the `_id` field is used: https://akkoma.dev/AkkomaGang/akkoma/src/commit/3791b695f486908c3ea0705c0ece5694eb2ce281/lib/pleroma/search/elasticsearch/store.ex#L41-L44 Therefore, it should be safe to remove the `format` field from the `sort` object in the Elasticsearch query. And this simple change would make it compatible with OpenSearch. I am running it on my server, and it works.

Oneric added the

feature request

label

2025-05-28 15:45:41 +00:00

No milestone

No project

No assignees

1 participant

Notifications

Due date

The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference

AkkomaGang/akkoma#935

No description provided.

Rows
Columns