Elixir Ecto: Wildcard search a Postgres JSONB field

This seemingly straightforward task made me do some digging. Imagine we have a table with JSONB Field and we want to search for specific field values using wildcard character to allow partial matches.

Ecto doesn’t have a JSONB support but this particular task is so common that it’s even mentioned in the documentation:

fragment("?->>? ILIKE ?", p.map, "key_name", ^some_value)

What we need is just add the wildcard characters around the value. We can’t do that directly (“?->>? ILIKE %?%”) though since it would trigger Ecto’s syntax error and the query wouldn’t compile.
Continue reading

Streaming Data From The Database In Elixir

Say we have a search feature and an index_article/1 function that knows what to do with the given article – can fetch all related data, concatenate it into a tsvector field and then save in the search cache table. With it we can index any article – so far so good.

Now what if we want to rebuild the whole search index? Looping over each entry in the database and indexing it with the existing function would work, but will require loading everything in RAM which will definitely break in the long run.

Usually we would create some kind of Task that loops as long as there are unprocessed entries in the Database, takes a number of those entries, processes each of them adjusts offset etc.

Well, Elixir can do better, using Repo.stream/2 we can create a stream that we can use with the Stream module, this way we can greatly simplify the above logic.

Here is the whole function that reindexes all articles of the given type:
Continue reading

Elixir – Testing Phoenix Flash Messages

At times in my integration tests I want to check if a user sees a flash message. It’s easy when re-rendering a template, just check the response body like that:

assert html_response(conn, 200) =~ "some message"

UPD as pointed out by Dave Lugg there is also a built in helper that can do that:

assert get_flash(conn, :info) == “some Message”
assert get_flash(conn, :info) =~ “part of some Message”

But often we set flash messages and redirect the user, in this case response body would not contain them and we’d have to check against the data that is saved on the conn struct, this is doable using Phoenix controller module:

assert(
    conn
    |> Phoenix.Controller.get_flash()
    |> Enum.any?(fn(item) -> String.contains?(elem(item, 1), "some message") end))

which works, but doesn’t look very nice and clutters the tests. So the cleaner solution would be to pack the whole thing in a test helper function and put it in test/support (anything in this directory will be compiled when running tests). Here is an example of such a helper (file test/support/helper.ex): Continue reading

Elixir – Nested Changesets With Phoenix

Say we have a user registration form where the user enters their name, email etc (User schema) as well as their password (Authorization schema). Both have to be validated, both require extra actions and belong together, so if anything fails the whole thing has to roll back and give meaningful feedback.

This is doable using Ecto.Multi (or nested case with transactions in controllers, but that would be ugly). Here is an alternative to Ecto.Multi that keeps it as minimalistic as possible while still allowing for much flexibility when dealing with complex user inputs: nested forms with custom changesets (note: Ecto 2 is needed). Continue reading

Deploying Elixir Apps

Recently we started playing with Elixir which is an awesome language bringing quite a few perks in terms of performance, reliability and scalability but also enforcing a mentality shift to functional programming (as compared to OOP as in modern PHP).

When it came to deployment we basically had two options: using exrm releases or just starting the app (that uses Phoenix framework) in production mode.

Internet didn’t tell us much about which option is more useful aside from exrm being the “cooler” one and the rolling update – “easier”, so we tried both, starting with the cool one 🙂 Continue reading

PHPUnit – Integration Tests Console Output

When writing integration tests we have special group of slow tests that do a lot of stuff like performing hundreds of downloads or editing thousands of entries. In some cases one of the actions can trigger fatal error and the message would not be specific enough to determine which of those actions broke everything.

Here is a neat trick to enable any test to show console output, it uses fwrite to write to standard I/O stream: Continue reading