Improve README

README.md

@@ -4,6 +4,8 @@ A simple, no-nonsense Elasticsearch library for Elixir. Highlights include:
 
 - **No DSLs.** Interact directly with the `Elasticsearch` JSON API.
 - **Zero-downtime index (re)building.** Via `Mix.Tasks.Elasticsearch.Build` task.
+- **Dev Tools**. Helpers for runnig Elasticsearch as part of your supervision
+  tree during development.
 
 ## Installation
 
@@ -41,30 +43,28 @@ config :elasticsearch,
   # Elasticsearch time to catch up.
   bulk_wait_interval: 15_000, # 15 seconds
 
-  # This loader module must implement the Elasticsearch.DataLoader
-  # behaviour. It will be used to fetch data for each source in each
-  # indexes' `sources` list, below:
-  loader: MyApp.ElasticsearchLoader,
-
   # If you want to mock the responses of the Elasticsearch JSON API
   # for testing or other purposes, you can inject a different module
   # here. It must implement the Elasticsearch.API behaviour.
   api_module: Elasticsearch.API.HTTP,
 
   # You should configure each index which you maintain in Elasticsearch here.
+  # This configuration will be read by the `mix elasticsearch.build` task,
+  # described below.
   indexes: %{
-    # `:cities` becomes the Elixir name for this index, which you'll use in
-    # queries, etc.
-    cities: %{
-      # This is the base name of the Elasticsearch index. Each index will be
-      # built with a timestamp included in the name, like "cities-5902341238".
-      # It will then be aliased to "cities" for easy querying.
-      alias: "cities",
-
+    # This is the base name of the Elasticsearch index. Each index will be
+    # built with a timestamp included in the name, like "posts-5902341238".
+    # It will then be aliased to "posts" for easy querying.
+    posts: %{
       # This file describes the mappings and settings for your index. It will
       # be posted as-is to Elasticsearch when you create your index, and
       # therefore allows all the settings you could post directly.
-      settings: "priv/elasticsearch/cities.json",
+      settings: "priv/elasticsearch/posts.json",
+
+      # This loader module must implement the Elasticsearch.DataLoader
+      # behaviour. It will be used to fetch data for each source in each
+      # indexes' `sources` list, below:
+      loader: MyApp.ElasticsearchLoader,
 
       # This is the list of data sources that should be used to populate this
       # index. The `:loader` module above will be passed each one of these
@@ -72,40 +72,168 @@ config :elasticsearch,
       #
       # Each piece of data that is returned by the loader must implement the
       # Elasticsearch.Document protocol.
-      sources: [Type1]
+      sources: [Post]
     }
   }
 ```
 
+## Protocols & Behaviours
+
+#### Elasticsearch.DataLoader
+
+Your app must provide a `Loader` module, which will fetch data to upload to
+Elasticsearch. This module must implement the `Elasticsearch.DataLoader`
+behaviour.
+
+```elixir
+defmodule MyApp.ElasticsearchLoader do
+  @behaviour Elasticsearch.DataLoader
+
+  @impl Elasticsearch.DataLoader
+  def load(MyApp.Post, offset, limit) do
+    # Return MyApp.Posts, restricted by offset and limit
+  end
+end
+```
+
+#### Elasticsearch.Document
+
+Each result returned by your loader must implement the `Elasticsearch.Document`
+protocol.
+
+```elixir
+defimpl Elasticsearch.Document, for: MyApp.Post do
+  def id(post), do: post.id
+  def type(_post), do: "post"
+  def parent(_post), do: false
+  def encode(post) do
+    %{
+      title: post.title,
+      author: post.author
+    }
+  end
+end
+```
+
+#### Elasticsearch.API
+
+You can plug in a different module to make API requests, as long as it
+implements the `Elasticsearch.API` behaviour.
+
+This can be used in test mode, for example:
+
+```elixir
+# config/test.exs
+config :elasticsearch,
+  api_module: MyApp.ElasticsearchMock
+```
+
+Your mock can then stub requests and responses from Elasticsearch.
+
+```elixir
+defmodule MyApp.ElasticsearchMock do
+  @behaviour Elasticsearch.API
+
+  def get("/posts/1", _headers, _opts) do
+    {:ok, %HTTPoison.Response{
+      status_code: 404,
+      body: %{
+        "status" => "not_found"
+      }
+    }}
+  end
+end
+```
+
+## Indexing
+
+#### Bulk
+
+Use the `mix elasticsearch.build` task to build indexes using a zero-downtime,
+hot-swap technique with Elasticsearch aliases.
+
+```bash
+# This will read the `indexes[posts]` configuration seen above, to build
+# an index, `posts-123123123`, which will then be aliased to `posts`.
+$ mix elasticsearch.build posts
+```
+
+See the docs on `Mix.Tasks.Elasticsearch.Build` and `Elasticsearch.Builder`
+for more details.
+
+#### Individual Documents
+
+Use `Elasticsearch.put_document/2` to upload a document to a particular index.
+
+```elixir
+# MyApp.Post must implement Elasticsearch.Document
+Elasticsearch.put_document(%MyApp.Post{}, "index-name")
+```
+
+To remove documents, use `Elasticsearch.delete_document/2`:
+
+```elixir
+Elasticsearch.delete_document(%MyApp.Post{}, "index-name")
+```
+
 ## Querying
 
-You can query Elasticsearch using raw requests, or with the help of 
-the `Elasticsearch.Query` struct.
+You can query Elasticsearch the `post/2` function:
 
 ```elixir
 # Raw query
-Elasticsearch.post("/cities/city/_search", '{"query": {"match_all": {}}}')
+Elasticsearch.post("/posts/post/_search", '{"query": {"match_all": {}}}')
 
 # Using a map
-Elasticsearch.post("/cities/city/_search", %{"query" => %{"match_all" => %{}}})
-
-# Using a query
-query = %Elasticsearch.Query{
-  indexes: [:cities],
-  types: [:city],
-  query: %{
-    "query" => %{
-      "match_all" => %{}
-    }
-  }
-}
-
-Elasticsearch.execute(query)
+Elasticsearch.post("/posts/post/_search", %{"query" => %{"match_all" => %{}}})
 ```
 
-TODOS:
+See the official Elasticsearch [documentation](https://www.elastic.co/guide/en/elasticsearch/reference/6.x/index.html)
+for how to write queries.
 
-- [ ] Write tests
+## Dev Tools
+
+This package provides two utilities for developing with Elasticsearch:
+
+- `mix elasticsearch.install`: A mix task to install Elasticsearch and Kibana
+  to a folder of your choosing.
+
+- `Elasticsearch.Executable`. Use this to start and stop Elasticsearch as part
+  of your supervision tree.
+
+  ```elixir
+  children = [
+    worker(Elasticsearch.Executable, [
+      "Elasticsearch",
+      "./vendor/elasticsearch/bin/elasticsearch", # assuming elasticsearch is in your vendor/ dir
+      9200
+    ]),
+    worker(Elasticsearch.Executable, [
+      "Kibana",
+      "./vendor/kibana/bin/kibana", # assuming kibana is in your vendor/ dir
+      5601
+    ])
+  ]
+  ```
+
+## Documentation
+
+Run `mix docs` to generate local documentation.
+
+## Contributing
+
+To contribute code to this project, you'll need to:
+
+1. Fork the repo
+2. Clone your fork
+3. Run `bin/setup`
+4. Create a branch
+5. Commit your changes
+6. Open a PR
+
+## Todos
+
+- [x] Write tests
 - [ ] Update documentation in `Elasticsearch` module
 - [ ] Update documentation in `mix elasticsearch.build` task
 - [ ] Document how to mock Elasticsearch for testing

lib/mix/elasticsearch.build.ex

@@ -1,6 +1,24 @@
 defmodule Mix.Tasks.Elasticsearch.Build do
   @moduledoc """
   Builds Elasticsearch indexes using a zero-downtime, hot-swap technique.
+
+  1. Build an index for the given `alias`, with a timestamp: `alias-12323123`
+  2. Bulk upload data to that index using `loader` and `sources`.
+  3. Alias the `alias` to `alias-12323123`.
+  4. Remove old indexes beginning with `alias`.
+  5. Refresh `alias-12323123`.
+
+  For a functional version of this approach, see 
+  `Elasticsearch.Builder.hot_swap_index/4`.
+
+  ## Example
+
+      $ mix elasticsearch.build posts {index2} {index3}
+
+  To build an index only if it does not exist, use the `--existing` option:
+      
+      $ mix elasticsearch.build posts --existing
+      Index posts already exists.
   """
 
   require Logger